Introduction
POUNCE is a general interior-point method, implemented in pure Rust — one
numerical backbone that now spans nonlinear, conic/quadratic, and polynomial
global optimization rather than a single problem class. Its
nonlinear-programming core began as a faithful port of the
Ipopt filter line-search method —
the algorithm, console output, and option semantics follow upstream Ipopt
closely enough that anyone used to reading ipopt logs can drop in
pounce without relearning where the numbers live — and it has since grown
into a family of solvers sharing that backbone:
-
Nonlinear programming — the filter line-search interior-point method (the Ipopt port) plus an active-set SQP path, for general smooth problems
min f(x) s.t. g_L <= g(x) <= g_U x_L <= x <= x_Uwhere
fandgare twice-continuously-differentiable. -
Conic & quadratic — LP, convex QP, second-order (SOCP), positive-semidefinite (SDP), and the non-symmetric exponential and power cones, each solved to the global optimum.
-
Global optimization — certified global optima for nonconvex polynomial problems via SOS / Lasserre relaxations. Nonconvex problems that are not polynomial get a local answer; POUNCE has no spatial branch-and-bound solver.
See Choosing a Solver for which solver fits which problem.
Pure Rust by default
The default build is pure Rust — no Fortran, no commercial solver, no system BLAS
required. The bundled FERAL backend provides a sparse symmetric LDLᵀ
factorization. The HSL MA57 backend is available behind the optional
ma57 feature for users who have a license for libcoinhsl and have it installed (see
Installation).
Status
Production-ready for the core IPM workflow. The algorithm-side core,
NLP interface, line search, filter, barrier update (monotone +
Mehrotra adaptive), KKT solve, restoration phase, AMPL .nl reader,
the C ABI (pounce-cinterface), the Python wrapper (pounce-solver),
and the CLI all solve a wide range of NLPs from the standard test
suites (Hock-Schittkowski, CUTEst, Mittelmann ampl-nlp, CHO parameter
estimation, gas/water network design). Sensitivity analysis (sIPOPT
port), reduced-Hessian computation, the auxiliary-equality + FBBT
presolve, and the active-set SQP path are all wired
in and available behind option keys. Existing PyIpopt / cyipopt / JuMP / AMPL clients
link against libpounce_cinterface in place of libipopt
unchanged.
The conic and global solvers are wired end-to-end alongside the NLP
core: the convex interior-point solver (pounce-convex) handles
LP / QP, SOCP, exponential / power cones, and small SDPs — with a Conic
Benchmark Format (.cbf) reader cross-checked against the CBLIB tier —
and adds SOS / Lasserre polynomial global optimization (sos_minimize).
These are reachable from the CLI, the Python package, and the JSON solve
report. There is no spatial branch-and-bound solver for general factorable
nonconvex problems — outside the polynomial case, nonconvex models are solved
locally.
License
EPL-2.0, the same license as upstream Ipopt.
Where to go next
- Installation — build and install POUNCE.
- Quick Start — solve your first problem.
- Running Solves — the command-line driver in depth.
- Acknowledgments — the papers behind the algorithm.
Installation
Three routes, in the order most people want them:
| Command | When | |
|---|---|---|
| pip | pip install pounce-solver | you just want to solve something |
| container | docker pull ghcr.io/jkitchin/pounce | clusters, or nothing installed on the host |
| source | make && make install | developing POUNCE, or you want the ma57 backend |
With pip
pip install pounce-solver
Prebuilt wheels for Linux, macOS, and Windows (CPython 3.9+). No Rust toolchain is involved. This installs both interfaces at once:
pounce problem.nl # the CLI
python -c "import pounce; print(pounce.__version__)"
For Pyomo models:
pip install pyomo-pounce
import pyomo.environ as pyo
results = pyo.SolverFactory("pounce").solve(model)
Optional extras — none needed for a normal solve:
pip install "pounce-solver[jax]" # pounce.jax autodiff frontend
pip install "pounce-solver[torch]" # pounce.torch autodiff frontend
pip install "pounce-solver[viz]" # debugger plots (pounce-dbg-viz)
pip install "pounce-solver[gams]" # GAMS solver link — see gams.md
If the CLI will not start (GLIBC_2.39 not found)
Releases up to and including 0.9.0 bundled a Linux CLI built against a
newer glibc than the wheel advertised, so pounce fails to exec on older
distributions (Debian 12, Ubuntu 22.04, RHEL/Rocky/Alma 8 and 9, and most
HPC images) with:
pounce/bin/pounce: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.39' not found
import pounce still works; only the CLI (and therefore Pyomo, which
shells out to it) is affected. This is fixed for releases after 0.9.0. If
you hit it, use the container or build from source below.
With a container
No toolchain and nothing installed on the host:
docker run --rm -v "$PWD:/work" ghcr.io/jkitchin/pounce:latest problem.nl
apptainer pull pounce.sif docker://ghcr.io/jkitchin/pounce:0.9.0
Both images carry the CLI, the Python API, and the Pyomo plugin. See Docker & Containers for tags, bind mounts, and a Slurm example.
From source
Prerequisites
A stable Rust toolchain. Nothing else is needed for the default pure-Rust build. Install Rust via rustup:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
Verify the install:
rustc --version && cargo --version
Build
From the repository root:
make # release build of the workspace
make test # run all tests
make clippy # lint
make doc # rustdoc for the Rust API
Install
make install # installs to $HOME/.local
sudo make install PREFIX=/usr/local # or system-wide
This drops the pounce binary into $PREFIX/bin and the
libpounce_cinterface shared library into $PREFIX/lib. Make sure
$HOME/.local/bin is on your PATH, then verify:
pounce --version
HSL MA57 backend (optional)
The default FERAL backend needs no external libraries. To build with
the HSL MA57 linear solver instead, you need a CoinHSL install whose
lib/ directory holds libcoinhsl. Point the COINHSL_DIR
environment variable at it and build with the ma57 feature:
export COINHSL_DIR=/path/to/CoinHSL
cargo build -p pounce-cli --release --features ma57
The feature makes MA57 available; selecting it is a separate step,
because linear_solver defaults to feral in every build:
pounce problem.nl linear_solver=ma57
Build CoinHSL from https://www.hsl.rl.ac.uk/ipopt/. MA57 is
primarily useful for benchmarking against upstream Ipopt; the FERAL
backend is the supported default for everyday use, and a build without
--features ma57 never touches COINHSL_DIR.
Using POUNCE as a Rust library
The workspace is a set of library crates (see Algorithm & Workspace for the layout). To browse the Rust API, build and open the rustdoc:
make doc # generates target/doc
Docker & Containers
Prebuilt images carry the whole POUNCE surface — the pounce CLI, the
Python API, and the Pyomo plugin — with no Rust toolchain and no pip install on your side. This is the path of least resistance on a cluster,
where you often cannot install a toolchain anyway, and the quickest way to
try POUNCE without touching your environment.
docker pull ghcr.io/jkitchin/pounce:latest
docker run --rm ghcr.io/jkitchin/pounce:latest --version
Which tag
| Tag | Contains | Use it when |
|---|---|---|
latest | the newest release | you just want POUNCE |
X.Y.Z (e.g. 0.9.0) | exactly that release, forever | reproducibility — papers, cluster job scripts, CI |
X.Y (e.g. 0.9) | newest patch of that minor series | you want fixes but not feature changes |
edge | tip of main, compiled from source | testing an unreleased fix |
sha-<short> | one specific commit | reproducing a bug report against a known commit |
Pin X.Y.Z for anything you intend to re-run months later. latest and
edge both move under you.
Running solves
The entrypoint is the pounce CLI, so arguments after the image name go
straight to it:
docker run --rm ghcr.io/jkitchin/pounce:latest --list-problems
docker run --rm ghcr.io/jkitchin/pounce:latest --problem rosenbrock
Your own problems live on the host, so mount a directory. The working
directory inside the image is /work:
docker run --rm -v "$PWD:/work" ghcr.io/jkitchin/pounce:latest \
problem.nl print_level=5 tol=1e-10
That writes problem.sol next to problem.nl in the mounted directory —
see Running Solves for the full option surface. The image runs as
UID 1000 rather than root, so files it creates in a bind mount are not
root-owned on the host. If your host UID differs, add --user "$(id -u):$(id -g)".
Python and Pyomo
Override the entrypoint to get a shell or an interpreter:
docker run --rm -it --entrypoint python ghcr.io/jkitchin/pounce:latest
docker run --rm -it --entrypoint bash -v "$PWD:/work" ghcr.io/jkitchin/pounce:latest
Both import pounce (with numpy and scipy) and Pyomo’s
SolverFactory('pounce') work out of the box:
docker run --rm -v "$PWD:/work" --entrypoint python \
ghcr.io/jkitchin/pounce:latest my_model.py
The optional extras are not installed — no JAX, no PyTorch, no plotly,
no GAMS bindings. Add what you need in a derived image:
FROM ghcr.io/jkitchin/pounce:0.10.0
USER root
RUN pip install --no-cache-dir "pounce-solver[jax]==0.9.0"
USER pounce
On a cluster (Apptainer / Singularity)
Most HPC sites run Apptainer (formerly Singularity) rather than Docker, because it needs no daemon and no root. It pulls Docker images directly:
apptainer pull pounce.sif docker://ghcr.io/jkitchin/pounce:0.10.0
apptainer run pounce.sif problem.nl
Two differences from docker run are worth knowing before you write a job
script:
- You are yourself. Apptainer runs the container as your own user, not
the image’s, so output files land with your ownership and no
--userflag is needed. $HOMEand$PWDare already there. Apptainer bind-mounts them by default, so a.nlfile in your submit directory is usually visible with no-Bflag at all. Add-B /scratch:/scratch(or your site’s equivalent) for anything outside them.
Build the .sif once on a login node and reuse it — pulling on every array
task hammers the registry and will get rate-limited. In a Slurm script:
#!/bin/bash
#SBATCH --job-name=pounce
#SBATCH --cpus-per-task=4
apptainer exec $HOME/images/pounce.sif \
pounce $SLURM_SUBMIT_DIR/problem.nl --json-output result.json
Pin the digest rather than the tag if the run has to be reproducible years later — a tag can be re-pushed, a digest cannot:
apptainer pull pounce.sif docker://ghcr.io/jkitchin/pounce@sha256:<digest>
Building your own
You do not need the images to be published — both Dockerfiles are in the repository and build from a clone. From the repository root:
make docker # compiles the current working tree -> pounce:dev
make docker-release # installs the released wheels -> pounce:<version>
make docker is the one to reach for when testing a branch: it compiles
whatever is checked out, and stamps the commit into pounce --about so the
image can say what it contains. make docker-release needs no Rust
toolchain and takes seconds. See docker/README.md
for build arguments and the .dockerignore caveat.
What is not in the image
The HSL MA57 backend is absent, and cannot be added by us: CoinHSL is
license-restricted and not redistributable. Passing linear_solver=ma57
will not work in a container. The pure-Rust FERAL backend is the default
everywhere and needs no external libraries — see
Installation if you have a
CoinHSL license and want a local build with it.
The GAMS link is likewise absent, since it needs your own GAMS install and license on the host. See GAMS.
Quick Start
This page assumes POUNCE is built and on your PATH
(see Installation).
Solve an AMPL .nl file
pounce problem.nl
This solves the problem and writes a sibling problem.sol next to the
input, following the AMPL solver convention. The console output
mirrors upstream ipopt’s banner, per-iteration table, and final
summary.
Append KEY=VALUE pairs to override options — the syntax and
semantics match the upstream Ipopt CLI:
pounce problem.nl print_level=8 max_iter=500 tol=1e-10
See Solver Options for details.
Try a built-in problem
POUNCE ships several self-contained test problems that exercise the
full pipeline without parsing a .nl file (run pounce --list-problems
for the full set):
pounce --list-problems
pounce --problem rosenbrock
pounce --problem quadratic
From Python
import numpy as np
from pounce import minimize
res = minimize(lambda x: ((x - 1) ** 2).sum(), x0=np.zeros(3))
print(res.fun, res.x)
See the Python API chapter for the full cyipopt-compatible interface.
From Pyomo
import pyomo_pounce # registers 'pounce'
from pyomo.environ import SolverFactory
SolverFactory('pounce').solve(model)
See the Pyomo chapter for details.
Full help
pounce --help
Choosing a Solver
POUNCE is not a single solver but a small family of them sharing one numerical backbone. This page is the map: what each solver is, when to reach for it, and how they fit together.
The one-sentence version: convex and conic problems are solved to the global
optimum; nonconvex problems are solved locally by default, or to a certified
global optimum via the SOS (polynomial) and spatial branch-and-bound (general)
paths. Every solver, whatever its flavor, ultimately factorizes a symmetric
KKT system through the shared pounce-linsol layer, which in turn drives a
pluggable backend (FERAL by default, HSL MA57 optionally).
The solvers at a glance
| Solver | Problem class | Optimum | Crate | Entry points |
|---|---|---|---|---|
| NLP filter-IPM | general smooth NLP (nonconvex OK) | local (KKT) | pounce-algorithm + pounce-nlp | CLI default; Python Problem/minimize; solver_selection=nlp |
| NLP active-set SQP | general smooth NLP | local | pounce-algorithm (subproblems via pounce-qp) | algorithm=active-set-sqp |
| Convex IPM (LP/QP) | LP, convex QP | global | pounce-convex | solve_qp_ipm; pounce.qp.solve_qp; solver_selection=lp-ipm/qp-ipm |
| Convex IPM (conic) | SOCP, exp/power/PSD cones, convex QCQP | global | pounce-convex | solve_socp_ipm; pounce.qp.solve_socp; minimize (convex QCQP); solver_selection=socp; pounce <file>.cbf |
| Active-set QP | QP, convex or indefinite | local | pounce-qp | ParametricActiveSetSolver; solver_selection=qp-active-set — opt-in only; auto never picks it (see note) |
| SOS / Lasserre | polynomial (nonconvex) | global | pounce-convex | sos_minimize; pounce.sos_minimize |
When to reach for the active-set QP.
autonever selects it: a cold, one-shot convex QP goes to the interior-point path, which is materially more robust on that workload (137 of the 138 Maros-Mészáros problems, against substantially fewer for a cold active-set solve). That is the character of the method rather than a defect — an active-set iteration count is combinatorial in the size of the active set, while an interior-point count is nearly independent of problem size. Choosesolver_selection=qp-active-setwhen you want an exact vertex solution, or when you are solving a sequence of similar QPs — MPC steps, branch-and-bound nodes, continuation — where the working set carries across solves andsolve_parametriccan trace the homotopy from the previous solution instead of starting over.
POUNCE has no spatial branch-and-bound solver for general factorable nonconvex NLPs — no
solver_selection=globalCLI route, nominimize_globalPython entry point. The only certified-global path for a nonconvex problem is SOS / Lasserre, and it covers polynomials only.
When to choose each
General nonlinear program (the common case) → NLP filter-IPM
If your model has nonlinear objective or constraints and you don’t know (or can’t assume) convexity, this is the default and the most mature path. It is POUNCE’s port of Ipopt’s filter line-search interior-point method: robust on nonconvex problems, with a feasibility restoration phase for hard starts and exact or limited-memory Hessians. It returns a local KKT point — for a nonconvex problem there is no global guarantee.
- CLI:
pounce model.nl(or a built-in problem). - Python: the cyipopt-style
Problemclass, or the scipy-styleminimizefacade. - Reach for limited-memory Hessians (
hessian_approximation=limited-memory) when second derivatives are unavailable or expensive.
A sequence of related NLPs, or a stable active set → NLP active-set SQP
Selected with algorithm=active-set-sqp. It solves the NLP as a sequence
of quadratic subproblems (handed to pounce-qp), which warm-starts
extremely well when the active set is stable across solves — e.g. a
parametric sweep or a control loop. For a single cold solve of a general
NLP, prefer the filter-IPM.
Linear or convex quadratic program → Convex IPM (LP/QP)
If P ⪰ 0 (or P = 0 for an LP), use the convex interior-point solver:
it returns the global optimum, detects primal/dual infeasibility, and
offers warm-starting, batched and multiple-RHS solving, a build-once /
solve-many QpFactorization handle, and post-optimal sensitivity
(QpSensitivity — the sIPOPT analog). The CLI’s auto routing classifies
an .nl and sends LP/convex-QP problems here automatically.
- Python:
pounce.qp.solve_qp(andsolve_qp_batch,solve_qp_multi_rhs).
Second-order, exponential, or power cones → Convex IPM (conic)
The same convex solver handles conic programs: second-order cones, the
exponential and power cones that express geometric programming,
entropy / log-sum-exp, logistic models, and p-norm constraints, and the
positive-semidefinite cone for small dense SDPs. Also global. This
is the path to use when you can cast a nominally-nonconvex problem into a
convex cone — you trade modeling effort for a global guarantee. (The PSD
cone is self-scaled and runs on the symmetric driver; the exp/power cones
run on the non-symmetric HSDE driver, so the two families can’t yet be
mixed in one problem.)
A common special case routes here automatically: a convex
quadratically-constrained QP (QCQP). When auto routing finds a
convex-quadratic inequality ½xᵀHx + aᵀx + b ≤ 0 (H ⪰ 0), it reformulates
each such constraint to one second-order cone (H = FᵀF) and sends the whole
problem to the conic solver — no .cbf and no manual cone bookkeeping needed.
This works from a .nl/Pyomo model on the CLI and from minimize() in Python
(which probes each constraint’s Hessian and only routes when it can prove the
feasible set is convex). See LP / QP Solver Routing.
- Python:
pounce.qp.solve_socp(..., cones=[("exp", 3), ("pow", 0.5), ...])for an explicit cone program, or justminimize(...)for a convex QCQP. - CLI: a Conic Benchmark Format file,
pounce model.cbf(see the CBLIB benchmark tier), or any convex-QCQP.nlunderautorouting.
Nonconvex problem, global optimum required → SOS (polynomials only)
When the problem is genuinely nonconvex and a local optimum is not good enough, the one path to a certified global optimum is for polynomials:
- Polynomial objective/constraints → SOS / Lasserre (
sos_minimize, orpounce.sos_minimize). A single semidefinite program certifies the global minimum (the largestγwithp − γin the Putinar cone), and the global minimizers are recovered from the moment matrix — even multiple ones, via a facial-reduction step. Best for modest degree and dimension; the SDP grows with the relaxation order.
If the problem is nonconvex and not polynomial (exp/ln/trig), POUNCE
cannot certify a global optimum. Reformulate into the convex cone library if
you can; otherwise multistart the local NLP solver and accept that the result
is uncertified.
See Global Optimization for the SOS path in depth, and for the multistart fallback.
Indefinite QP, or a QP inner-solver → Active-set QP
pounce-qp is a sparse parametric active-set solver that accepts an
indefinite Hessian (via inertia control), with two-sided bounds and
factorization-reuse across a homotopy. It is the engine behind the
active-set SQP path, and is the right choice for MPC-style problems or any
setting where you re-solve a slowly-changing QP many times. Use the convex
IPM instead when P ⪰ 0 and you want a single robust solve with
infeasibility certificates.
How to override the automatic routing
The CLI classifies each .nl problem and picks a solver, but you can force
the choice:
pounce model.nl solver_selection=auto # default: classify, then route
pounce model.nl solver_selection=nlp # filter-IPM (or active-set-sqp via algorithm=)
pounce model.nl solver_selection=lp-ipm # convex LP interior-point
pounce model.nl solver_selection=qp-ipm # convex QP interior-point
pounce model.nl solver_selection=socp # conic interior-point (convex QCQP)
pounce model.nl solver_selection=qp-active-set # active-set QP
solver_selection is an ordinary POUNCE option, not a command-line flag:
it is passed as a trailing KEY=VALUE pair (the ipopt CLI convention), and
so also works from an options file, the pounce_options environment
variable, or Pyomo’s solver.options. Forcing a value the problem class
does not support is rejected with a message rather than silently ignored.
See LP / QP Solver Routing for how classification works and when it falls back to the more general solver.
The shared backbone
Every interior-point and active-set solver above assembles a symmetric KKT
system and factorizes it through pounce-linsol. That trait layer is
backend-agnostic:
- FERAL (
pounce-feral) — a pure-Rust sparse symmetric LDLᵀ factorization. The default; no external dependencies. - HSL MA57 (
pounce-hsl) — the well-known Harwell solver vialibcoinhsl, enabled with thema57build feature for large or ill-conditioned systems.
Because the backend is pluggable, the same solver code runs on either without change.
Cross-cutting layers
These are not solvers you select, but stages and tools the solvers share:
- Presolve (
pounce-presolve) — an optional front-end that tightens bounds (feasibility-based bound tightening), removes redundant rows, and repairs LICQ degeneracies before the solve. - Restoration (
pounce-restoration) — the feasibility-recovery phase the filter-IPM enters when a step cannot reduce both infeasibility and the objective;pounce-l1penaltyoffers an ℓ₁-exact penalty reformulation for degenerate / LICQ-violating problems. - Sensitivity —
pounce-sensitivitygives sIPOPT-style parametric steps and reduced Hessians for the NLP;QpSensitivitydoes the same for the convex QP. See Sensitivity Analysis. - Cone library (
pounce-convex) — nonnegative, second-order, exponential, power, and (for small dense problems) positive-semidefinite cones, so small SDPs solve as a convex class. The PSD cone cannot yet be mixed with the exponential/power cones in one problem (they use different drivers). - Solve report — every path can emit the machine-readable
pounce.solve-report/v1JSON (status, iterations, residuals, timing). See JSON Solve Report.
Global vs. local — the honest summary
POUNCE settles a problem globally along two routes, and locally along one:
- Global by convexity — LP, convex QP, SOCP, and the exponential / power / PSD cone classes. Local is global, so a convex or conic reformulation buys the guarantee outright.
- Global by certificate (polynomials) — the SOS / Lasserre optimizer certifies the global minimum of a nonconvex polynomial from a single SDP; see Global Optimization.
- Local for general NLP — the filter-IPM and SQP paths converge to a KKT point, which for a nonconvex problem carries no global guarantee.
There is no third route: a nonconvex, non-polynomial problem gets a local answer, and no certificate.
Two practical levers for a “global” answer: modeling (cast as much as you can into the convex cone library) and, when that is not possible, the SOS / Lasserre optimizer for polynomials.
Running Solves
The pounce command-line driver solves built-in TNLPs and AMPL .nl
files. Its console output mirrors upstream ipopt’s banner,
per-iteration table, and final summary, so anyone used to reading
ipopt logs can read pounce logs unchanged.
Basic usage
pounce problem.nl
pounce problem.nl print_level=8 max_iter=500 tol=1e-10
pounce problem.nl linear_solver=ma57 # with --features ma57
pounce problem.nl --options-file tuned.opt # upstream-format options file
pounce problem.nl option_file_name=tuned.opt # the Ipopt spelling; same thing
pounce problem.nl --no-options-file # ignore ./pounce.opt, ./ipopt.opt
Trailing KEY=VALUE pairs follow the same syntax and semantics as the
upstream Ipopt CLI; they override values loaded from the options file.
With no options file named, ./pounce.opt or ./ipopt.opt is read if
present, as ipopt reads ./ipopt.opt. See
Solver Options.
Built-in problems
pounce --list-problems
pounce --problem quadratic
pounce --problem rosenbrock
quadratic—min (x[0]-3)² + (x[1]-4)²(unconstrained, optimum(3, 4)).rosenbrock—min 100·(x[1]-x[0]²)² + (1-x[0])²(unconstrained, optimum(1, 1)).bounded-quadratic—quadraticwith box bounds0 ≤ x ≤ 2(optimum at the upper corner(2, 2)).eq-quadratic—min x[0]² + x[1]²s.t.x[0] + x[1] = 1(a single equality).circle—min x[0]s.t.x[0]² + x[1]² = 1(a nonlinear equality).infeasible-eq— two contradictory equalities (x[0]+x[1]=1and=2); exercises the infeasibility-detection path.
Run pounce --list-problems for the authoritative list.
Built-in problems have no .nl stub, so they only write a .sol file
when --sol-output is given explicitly.
Degenerate / MPCC NLPs — the ℓ₁-exact penalty-barrier wrapper
For problems where the standard IPM thrashes in restoration because LICQ fails at the iterate (degenerate equalities, MPCC-like complementarity), enable the Thierry–Biegler ℓ₁-exact penalty-barrier wrapper:
pounce problem.nl l1_exact_penalty_barrier=yes
The wrapper turns every equality row c_i(x) = g_i into a
slack-relaxed c_i(x) − p_i + n_i = g_i with (p_i, n_i) ≥ 0,
augments the objective by ρ · Σ(p + n), and runs a
Byrd–Nocedal–Waltz outer loop that escalates ρ until the slacks
collapse (constraints satisfied) or saturate (locally infeasible
problem detected). The user-visible (x*, λ*) are reported in the
original variable space.
For everyday use, the simpler form is an auto-fallback:
pounce problem.nl l1_fallback_on_restoration_failure=yes
POUNCE first runs the standard solve. If it terminates in
Restoration_Failed, Infeasible_Problem_Detected,
Solved_To_Acceptable_Level, Maximum_Iterations_Exceeded, or
Not_Enough_Degrees_Of_Freedom, the wrapper is invoked transparently
and the result is promoted to Solve_Succeeded only if the retry
succeeds. Otherwise the original status is preserved.
The tuning knobs are listed under Solver Options.
AMPL / Pyomo solver mode
AMPL drivers — and Pyomo’s ASL interface — invoke a solver as
solver problem.nl -AMPL. Pass -AMPL to run pounce that way:
pounce problem.nl -AMPL
It changes nothing about the solve itself; it switches the process to
the AMPL exit-code contract (see below), so the driver reads the
termination from the .sol file rather than the exit status. The
pyomo-pounce package builds on top of this mode.
Dual sign conventions
Two different quantities are in play, and they differ by a sign. Getting them confused is easy, so pounce names them differently:
| Surface | Quantity | Meaning |
|---|---|---|
.sol dual block, Pyomo model.dual | marginal | d obj / d b — the shadow price |
mult_g (Python API, JSON solution.lambda) | Lagrange multiplier | the λ of L = f + λ'g − z_L(x−x_L) + z_U(x−x_U) |
They are related by marginal = −λ. The .sol writer performs this
negation, so a .sol (and therefore Pyomo’s model.dual) carries shadow
prices with the same sign Ipopt, glpk, CBC and CONOPT report. For
min x₀²+x₁² s.t. x₀+x₁ = b the optimum is b²/2, so a .sol written at
b = 2 reports +2.
The Python API’s mult_g keeps the Lagrange-multiplier convention, which
matches cyipopt and satisfies the stationarity identity above directly.
If you are comparing a mult_g against a model.dual for the same solve,
expect them to differ by a sign — that is by design, not a bug.
Before v0.9.1 the
.solwriter emitted the Lagrange multiplier without converting it, so every AMPL/Pyomo dual came back negated relative to every other solver (#271). Objectives and primal solutions were never affected.
Exit codes
0—Solve_Succeeded(orSolved_To_Acceptable_Level).- non-zero — any other
ApplicationReturnStatus.
In AMPL solver mode (-AMPL) the exit code instead follows the AMPL
contract: 0 for any solve that ran and produced a .sol file —
limit-reached, infeasible, even a failed solve — since the termination
is carried by the file’s solve_result_num. Genuine startup failures
(unreadable .nl, bad option) still exit non-zero.
Diagnostics & introspection
pounce --about # version, build info, features, backends
pounce problem.nl --dump kkt:5-10 --dump iterate # dump per-iteration diagnostics
pounce problem.nl --dump kkt --dump-dir /tmp/d # override the dump root
--about— print version, build info, enabled features, and linear-solver backends, then exit.--dump <cat>[:<spec>]— write the diagnostic category to per-iteration files (JSONL). Wired categories arekktanditerate; an optional:<spec>selects iterations (e.g.kkt:5,kkt:2-10,iterate:all).--dump-dir <path>— override the dump root (default./pounce-dump-<timestamp>).--dump-format <fmt>— dump format (defaultjsonl).
Help
pounce --help
pounce --version # also -v, -V
Solver Options
POUNCE accepts options the same way upstream Ipopt does. Option names
and semantics follow Ipopt’s, so an existing Ipopt options file or
KEY=VALUE invocation works unchanged.
Setting options
On the command line — append KEY=VALUE pairs after the input:
pounce problem.nl tol=1e-10 max_iter=500 print_level=8
From an options file — upstream ipopt.opt format (one name value
pair per line, # comments):
pounce problem.nl --options-file tuned.opt
pounce problem.nl option_file_name=tuned.opt # the Ipopt spelling; same thing
From an options file nobody named. With neither of the above, POUNCE
looks in the working directory for pounce.opt, then ipopt.opt, and
reads the first one it finds — the way ipopt picks up ./ipopt.opt:
printf 'max_iter 5\n' > ipopt.opt
pounce problem.nl # → Using option file "ipopt.opt".
The run says which file configured it (on the same sb gate as the
banner). If both default names are present, pounce.opt wins and the
other is named in a warning rather than left to look applied.
--no-options-file skips the lookup entirely — the escape hatch for a
directory holding an options file written for some other run.
Command-line KEY=VALUE pairs — and $pounce_options, AMPL’s
<solver>_options channel — override values loaded from the options
file, never the reverse.
An options file that was named but cannot be read is an error, not a shrug:
$ pounce problem.nl option_file_name=typo.opt
pounce: failed to load options file: options file "typo.opt" does not exist. …
Upstream opens a named file with a bare ifstream and reads nothing if
that fails, so a typo there runs at stock defaults without a word. That
silence is the one thing this path exists to remove
(#518): a run configured
entirely through an options file that quietly ran at defaults still
reported success, which invalidates any benchmark set up that way.
One upstream quirk is inherited: option_file_name set inside an
options file chains nowhere, because the file has already been chosen by
the time it is read. POUNCE warns about that rather than ignoring it.
Commonly used options
| Option | Meaning |
|---|---|
tol | Overall convergence tolerance on the KKT error. |
max_iter | Maximum number of outer iterations. |
print_level | Console verbosity, 0 (silent) – 12 (maximum debug). |
linear_solver | KKT linear-solver backend: feral (default) or ma57 (needs a --features ma57 build). Any other registered name is refused. See below. |
mu_strategy | Barrier-parameter update strategy (monotone / adaptive). |
solver_selection | Route LP/convex-QP to the specialized convex IPM. See LP/QP Routing. |
qp_presolve | Presolve on the convex LP/QP path (yes / no, default yes). See LP/QP Routing. |
obj_scaling_factor | Constant multiplier on the objective; negative maximizes. See below. |
bound_relax_factor | Relaxation applied to variable/constraint bounds before the solve (default 1e-8). |
honor_original_bounds | Project the reported point back into the un-relaxed bounds (yes / no, default no). See below. |
For the full upstream option catalogue, see the Ipopt options reference; POUNCE reuses those names.
For scaling-specific options (nlp_scaling_method, target-gradient
overrides, linear_system_scaling), see the Scaling
reference page. For nonlinear bound tightening (presolve_fbbt,
fbbt_tol, fbbt_max_iter, fbbt_max_constraints), see the
FBBT reference page.
Options POUNCE does not implement
POUNCE’s option registry is a faithful port of Ipopt’s: every name Ipopt
registers is registered here, so an ipopt.opt written for Ipopt parses
unchanged. Registering an option is not the same as implementing it,
though — and for a long time setting an unimplemented one did nothing at
all, silently.
Options naming a feature POUNCE does not have now fail the solve, naming the option, the feature, and what to use instead:
$ pounce model.nl dependency_detector=mumps
pounce: `dependency_detector` configures linear-dependency detection on the
equality constraints, which pounce does not implement. It is registered so an
ipopt.opt written for Ipopt still parses, but setting it used to do nothing at
all — silently — so it is refused instead. Instead: pounce's presolve removes
structurally redundant rows; see `presolve`. Remove it to run.
The features in question: the Chen-Goldfarb (CG-penalty) / inexact-Newton
line search, derivative approximation by finite differences,
linear-dependency detection, the per-iteration NaN/Inf derivative check,
multiplier recalculation by least squares, a selectable
constraint-violation norm, magic steps, bound replacement, the L-BFGS
augmented-system variants, skipping the finalize callback, the dynamic
HSL loader, and suppress_all_output / debug_print_level.
option_file_name was on that list until
#518 implemented it;
refusing an option is the cheap half of “implement it or fail loudly”,
and an entry leaves this table by getting the other half.
Two deliberate exceptions:
- Setting an option to its registered default is allowed. A generated
ipopt.optspells out defaults, anddependency_detector=noneasks for nothing. Only a value that differs from the default is a request POUNCE cannot honour. - Caching hints warn instead of failing.
grad_f_constant,hessian_constant,jac_c_constantandjac_d_constanttell the solver a quantity does not change between iterations. POUNCE re-evaluates regardless, so ignoring them costs evaluations and never correctness — failing the solve would be a worse trade.
Options whose feature runs and whose value simply is not read yet are not in this category; they still solve, with the default in effect. Wiring those is tracked on #191 and #483.
Derivative checker
Wrong analytic derivatives are the most common cause of an NLP that
stalls, cycles, or converges to something that is not a solution — and
they are invisible from the iteration log. derivative_test compares
what your TNLP returns against finite differences at the (bound-projected)
starting point, before the solve:
pounce problem.nl derivative_test=first-order
| Option | Default | Effect |
|---|---|---|
derivative_test | none | none / first-order / second-order / only-second-order. |
derivative_test_perturbation | 1e-8 | Relative finite-difference step: `perturbation · max(1, |
derivative_test_tol | 1e-4 | Flag an entry when |analytic − fd| > tol · max(1, |fd|). |
derivative_test_first_index | -2 (all) | First variable for the first-order test; first constraint for the second-order one, where -1 is the objective’s Hessian. |
derivative_test_print_all | no | List every entry, not just the suspicious ones. |
first-order checks eval_grad_f and eval_jac_g; second-order adds
eval_h; only-second-order checks the Hessian alone. The Hessian is
checked one multiplier block at a time — obj_factor = 1, λ = 0 against
differences of eval_grad_f, then obj_factor = 0, λ = eⱼ against
differences of row j of eval_jac_g.
Entries that look wrong are marked *:
Derivative checker: first derivatives at the starting point (perturbation 1.0e-8, tolerance 1.0e-4).
* grad_f[ 1] = 3.5000000000000000e0 ~ 3.0000000119209290e0 [ 1.667e-1]
1 suspicious derivative(s) and 0 missing sparsity entrie(s) out of 6 checked (8 evaluations).
Two checks beyond upstream Ipopt’s, because both catch a class of bug no value-by-value comparison can:
- A Jacobian or Hessian entry whose finite difference is nonzero but
which the sparsity structure omits (
!in the report). A missing structural entry is not a wrong number — it is a derivative the solver can never see. - The perturbation is taken downward when stepping up would leave a
variable’s box, so a model using
sqrt,log, or1/xis not evaluated outside its own domain by the checker.
The test is advisory: it reports and the solve continues. It is written
to stderr, so it survives print_level=0 and never mixes into
--json-output’s stdout. It is slow — the second-order test costs
roughly (m+1)·n evaluations — so leave it off for production runs.
check_derivatives_for_naninfis a separate upstream option, for a per-iteration NaN/Inf guard, and is not implemented.
Choosing a linear solver
POUNCE implements two KKT backends:
feral— pure-Rust sparse symmetric indefinite solver. The effective default; no Fortran toolchain, no HSL licence.ma57— HSL MA57, available only in acargo build --features ma57build.
The option’s registered value list is a faithful port of upstream
Ipopt’s (ma27, ma77, ma86, ma97, mumps, pardiso,
pardisomkl, spral, wsmp, custom), so an ipopt.opt written for
Ipopt parses here unchanged. Selecting one of those fails the solve
with a message naming it. They used to fall through to FERAL silently,
which meant linear_solver=ma97 “worked” and a benchmark comparing
backends compared FERAL with itself.
The registered default is feral, which diverges from upstream’s
ma57 on purpose: a default has to name a solver the binary actually
contains. Under the upstream default a pure-Rust build advertised MA57 to
every print_user_options dump while running FERAL, and an
HSL-enabled build used MA57 without being asked. If you build
--features ma57 and want it, select it explicitly — that is the one
behavioural change here.
Not a failure: explicit ma57 on a build without the feature falls
back to FERAL and says so in the banner (FERAL (ma57 requested but not compiled)). That substitution is reported rather than hidden, and
failing a portable ipopt.opt over a build flag would cost more than it
buys.
The per-backend tuning options (ma97_scaling, mumps_pivtolmax,
pardiso_*, wsmp_*, spral_*, …) remain registered for the same
ipopt.opt-compatibility reason. They are unreachable now that their
backend cannot be selected.
Bound relaxation and honor_original_bounds
Before the solve, POUNCE widens every variable and constraint bound by
bound_relax_factor (default 1e-8, capped by constr_viol_tol),
exactly as upstream Ipopt does — it keeps the interior-point iterates
strictly feasible without the user’s bounds becoming numerically
degenerate. The consequence is that a solution pinned to a bound is
reported just past it:
min (x − 3)² s.t. 0 ≤ x ≤ 1 → x = 1.00000000937
honor_original_bounds=yes projects the reported point back into the
bounds you declared, so that solve returns exactly x = 1. Reach for it
whenever the value flows somewhere that cares about the domain — a
sqrt(1 − x), a domain assertion, or a Pyomo Var the value is loaded
back into.
The default is no, matching upstream. As upstream also documents, the
constraint-violation and complementarity figures in the end-of-run
summary are for the non-projected point; only the reported x (and
the objective and constraint values evaluated at it) move.
Large constraint values and primal_noise_floor_kappa
On a model whose constraint values run to ~1e7 and beyond, a converged
solve could exit Search_Direction_Becomes_Too_Small while holding the
correct optimum. The cause is arithmetic, not the model.
The KKT error the convergence test compares against tol is
max( ‖∇L‖∞ / s_d , max(‖c‖∞, ‖d − s‖∞) , ‖compl‖∞ / s_c )
The dual and complementarity terms are normalised; the primal one — like
upstream Ipopt’s — is a bare absolute residual. But c_i = g_i(x) − b_i
and d_i − s_i are each a difference of quantities the row’s own size,
so they are quantised in units of eps · |b_i|. At |b| ~ 1e8 the
smallest nonzero value the primal term can take is one ulp,
1.5e-8 — already larger than the default tol = 1e-8. Asking for
nlp_err <= tol there is asking the residual to land on a bitwise-exact
0 rather than on one ulp, which is arithmetic luck rather than a
property of the iterate.
POUNCE therefore judges the primal term in the strict test against
each row’s own floating-point resolution: a row’s residual counts only
where it exceeds max(placement floor, kappa · eps · |row magnitude|),
with kappa = primal_noise_floor_kappa (default 64). Verdicts are flat
across kappa from 8 to 1024 on the measured set.
Three things bound what this can do:
- Only the strict test reads it.
constr_violis still checked againstconstr_viol_tol(default1e-4) on the full, unfloored residual, so nothing the floor forgives can exceed the feasibility tolerance you set — however large your data grows. - The acceptable-level band keeps the raw error. It sits two decades
above
tol, clear of any realistic quantum. - It cannot rescue an infeasible model. On a model with no feasible point the filter and restoration phase reach a verdict on their own criteria; the floor only ever participates at a point the rest of the algorithm already believes is converged.
Set primal_noise_floor_kappa = 0 to switch the floor off and restore
upstream Ipopt’s bare-absolute primal term exactly.
When the floor changes the reported picture, the end-of-run summary says
so — a large-|b| solve prints the tested value under the raw one:
Overall NLP error.......: 2.3841857910156250e-07 2.3841857910156250e-07
...above the per-row floating-point noise floor: 0.0000000000000000e+00
Solves where the two agree — every model whose data is O(1) — print the
usual block unchanged.
One case this does not paper over: tightening constr_viol_tol below
a row’s own ulp (say 1e-8 on data at 1e8) still will not certify. That
is the tolerance gate doing what you asked — the residual you requested is
not representable at that scale.
Solved_To_Acceptable_Level and acceptable_progress_kappa
Solved_To_Acceptable_Level is the fallback verdict for a solve that
cannot reach tol: after acceptable_iter (default 15) consecutive
iterates with an NLP error under acceptable_tol (default 1e-6), the
solver stops and hands back the point it has. That criterion is a count of
iterates inside a band, and on its own it asks only is the error small —
never has anything stopped moving.
Those come apart. An interior-point iterate can be near-stationary for the
current barrier subproblem — a much weaker statement than near-KKT for
the NLP — for fifteen iterations running while the solve is still
descending. Two measured cases: the kissing model stopped with objective
1.00000108 where continuing reaches 0.84544259 and a strict
certificate, 18% lower; NARX_CFy stopped with both residuals near 1e-7
where sixty more iterations collapse them by five orders.
POUNCE therefore also requires the streak to have flattened. Across the
acceptable_iter iterates that made it up:
- the spread (
max − min) of the NLP error must be withinacceptable_progress_kappa · acceptable_tol; and - the spread of the objective within the same fraction of
acceptable_tol · max(1, |f|).
acceptable_progress_kappa defaults to 0.1, so at default tolerances
both quantities must have stayed inside a tenth of the acceptable band over
the whole streak.
It is a spread, not a trend, and either signal alone is enough to keep
solving. Both choices are deliberate: kissing’s error was an order of
magnitude worse at the iterate it stopped on than at one it had already
reached inside the same streak — it was wandering across the band, not
converging inside it — while its objective was flat to all eight printed
figures over the same iterates.
Three things bound what this can do:
- It cannot lose a verdict. The refused termination is recorded, and
a run that fails to do better ends at exactly that iterate under exactly
that status. A misfire costs iterations, never the answer — you will not
see
Maximum_Iterations_Exceededwhere the count alone would have saidSolved_To_Acceptable_Level. - It never looks at a solve that converges. A solve that reaches
tolnever completes an acceptable-level streak, so nothing here runs. - A genuine stall flattens. When the iterate, the objective and the error are all pinned — the case the acceptable-level exit exists for — the window is flat and termination happens as before.
Set acceptable_progress_kappa = 0 to switch the progress test off and
restore upstream Ipopt’s bare consecutive-count criterion. Widening
acceptable_tol widens the flat bar with it, so asking for a looser band
still gets you the early exit.
Big models that start feasible and the theta_max ceiling
The filter has a hard ceiling. Any trial iterate whose constraint
violation θ exceeds
theta_max = theta_max_fact · max(1, θ₀)
is rejected outright, before any of the filter’s usual tests run. It is a global-convergence safeguard: it keeps the line search from wandering arbitrarily far from feasibility.
The trouble is the 1. POUNCE’s θ is a 1-norm over constraint rows —
‖c‖₁ + ‖d − s‖₁, a sum of m residuals — so a ceiling of T really
says “a mean per-row violation of T/m”, and that allowance shrinks as the
model grows. And on a problem started at a feasible point, θ₀ = 0,
the max collapses and the ceiling is the bare constant theta_max_fact
however large the model is.
robot_a is the measured case: 52 013 constraint rows, a feasible start,
so theta_max locked at 1e4 — a mean per-row allowance of 0.19 — while
the route to the optimum passes through θ ≈ 9.4e7. Every step toward the
solution was refused at the gate, and the solve ground to its iteration
limit at objective 8.173304 instead of the true 1.0431952.
POUNCE’s answer is theta_max_adaptive_trigger (default 3), described
below. theta_max_row_scale_kappa (default 0, off) is an earlier,
static attempt at the same problem, kept because it is occasionally the
more direct lever; it floors the reference at the row count instead:
theta_max = theta_max_fact · max(θ₀, theta_max_row_scale_kappa · rows, 1)
so the ceiling means a mean per-row violation of theta_max_fact
regardless of m. Measured under defaults otherwise, against Ipopt 3.14
on the same machine:
| model | POUNCE default | POUNCE kappa = 1 | Ipopt (default) |
|---|---|---|---|
robot_a | Maximum_CpuTime_Exceeded, 8.173304 | Optimal, 1.0431952, 112 it | Maximum_Iterations_Exceeded, 8.173304 |
robot_b | Maximum_CpuTime_Exceeded, 15.484684 | Optimal, 2.3330990, 252 it | Maximum_Iterations_Exceeded, 15.484684 |
robot_c | Maximum_CpuTime_Exceeded, 29.039906 | Optimal, 1.4059756, 109 it | Maximum_Iterations_Exceeded, 29.039906 |
Ipopt has the same defect and no correction for it; all three solve under
theta_max_fact = 1e8 set by hand, which is the blunt version of the same
move.
The adaptive rule: theta_max_adaptive_trigger
On by default. Rather than guessing from problem size whether a model needs headroom, POUNCE measures whether the ceiling is what is refusing the line search, and raises it only then.
A trial refused because θ_trial > theta_max takes a distinct early exit,
before the filter and Armijo tests run at all. So the acceptor can count
those refusals and compare them against the number of trials attempted.
When every trial of a line search was refused at the gate, for
theta_max_adaptive_trigger consecutive line searches, the ceiling is
demonstrably the binding constraint — not the filter — and it is
multiplied by theta_max_adaptive_factor (default 100), at most
theta_max_adaptive_max_raises times per solve (default 4).
| option | default | meaning |
|---|---|---|
theta_max_adaptive_trigger | 3 | consecutive fully gate-refused line searches before a raise; 0 disables |
theta_max_adaptive_factor | 100 | geometric factor per raise |
theta_max_adaptive_max_raises | 4 | cap on raises per solve |
Three properties follow, and they are what the static floor could not offer:
- A converging model cannot trip it. Converging means trials are
getting past the gate; a model accepting steps never accumulates the
streak.
brainpc1/3/5/7— the family the static floor damaged at everykappa— are untouched by construction, not by a lucky constant. - A blocked model trips it immediately.
robot_ais refused at the gate from its first line search onward. - The ceiling stays finite. Wächter–Biegler’s global-convergence
argument (Thm. 2) needs
theta_maxfinite, not fixed. A bounded number of bounded raises keeps it finite, so a solve cannot ratchet the safeguard away one line search at a time.
Requiring a streak rather than a single line search is deliberate: one
Newton direction that overshoots into a huge θ can legitimately have
all its trials refused, and backtracking is the right response to that.
Only a model that cannot get past the gate repeatedly is one whose route
needs the headroom.
Measured, defaults otherwise:
| model | rule off (trigger = 0) | rule on (default) |
|---|---|---|
robot_a | Maximum_Iterations_Exceeded, 14.23 | Optimal, 1.0432009, 190 it |
robot_b | max time, 15.484684 | Optimal, 2.3330990, 269 it |
robot_c | max time, 29.039906 | Optimal, 1.4059756, 222 it |
brainpc1 | Optimal, 64 it | Optimal, 64 it — identical |
brainpc3 | Optimal, 43 it | Optimal, 43 it — identical |
brainpc5 | Optimal, 982 it | Optimal, 982 it — identical |
brainpc7 | Optimal, 43 it | Optimal, 43 it — identical |
bt4 | Optimal, 9 it, −3.7047681836394486 | identical |
Across the whole Vanderbei corpus (733 problems) the rule changes four
outcomes: britgas goes from its iteration limit to Optimal in 16
iterations, catenary solves to the same objective in 50 iterations
instead of 56, and coshfun and brainpc0 fail either way — coshfun
now reporting diverging iterates, which is what Ipopt 3.14 also does on
it. Net Optimal count 702 → 703.
Note brainpc0 does trip the rule while brainpc1/3/5/7 do not,
despite identical row counts. That is precisely the distinction a
size-based floor cannot draw.
The restoration sub-IPM always runs with the rule disabled. Upstream
already corrects the resto phase’s instance of this degeneracy by
hard-coding resto.theta_max_fact = 1e8 (IpRestoMinC_1Nrm.cpp:91), so a
rule that ratchets further would be compounding a correction already made.
Set theta_max_adaptive_trigger = 0 to restore upstream Ipopt’s fixed
ceiling exactly.
When to reach for the static floor instead
Symptoms, all three together:
- a large number of constraint rows (thousands upward);
- a feasible or near-feasible starting point — the iteration log’s
first
inf_pris0or very small; - the solve stalls with
inf_prflat andalpha_prtiny, and raisingmax_iterdoes not help.
The quick confirmation is to set theta_max_fact = 1e8 by hand. If that
unsticks the model, theta_max_row_scale_kappa = 1 is the principled
version of it — it scales the ceiling to the model rather than to a
constant you picked.
Why the static floor is off by default
Because raising the ceiling unconditionally is not free. It relaxes a global-convergence
safeguard, and a model that was not being blocked by it can wander
instead. On the Vanderbei corpus, brainpc1/3/5/7 (m = 6900,
θ₀ = 1e-2) all regress — brainpc1 from Optimal in 64 iterations to
divergent, objective 3.7e3 against the correct 4.4e-04.
A scan over kappa showed why this cannot be tuned away. The damage is a
step function, not a gradient:
| kappa | robot_a | brainpc1 | brainpc3 | brainpc7 |
|---|---|---|---|---|
| 0 (default) | max time, 616 it | Optimal, 64 it | Optimal, 43 it | Optimal, 43 it |
| 0.01 | Optimal, 287 it | max time, 3.8e8 | Acceptable, 149 it | Acceptable, 552 it |
| 0.05 | Optimal, 153 it | max time | Acceptable, 149 it | Acceptable, 552 it |
| 0.2 | Optimal, 127 it | max time | Acceptable, 149 it | Acceptable, 552 it |
| 1.0 | Optimal, 112 it | max time | Acceptable, 149 it | Acceptable, 552 it |
brainpc3 and brainpc7 land on the identical worse answer at every
nonzero kappa, even 0.01 — where the ceiling moves only from 1e4 to
6.9e5. The instant it rises at all, they break. robot_a meanwhile
improves monotonically all the way to kappa = 1. There is no separating
value.
That is a verdict on the design, not on the tuning: the real question is whether a model’s route to the optimum needs the extra headroom, and the row count does not answer it. A static floor cannot know — which is why the adaptive rule above, which asks the question directly, is the default and this one is not.
What still bounds the option when you do turn it on:
- It only ever raises the reference. A model whose own
θ₀already exceedskappa · rowsgets exactly upstream’s ceiling. - It reduces to upstream on a single-row problem, where the floor is
max(kappa · 1, 1) = 1— upstream’s constant. theta_maxis still finite, and still fixed for the whole solve after its first line search. This rescales the safeguard; it does not remove it.- The restoration sub-IPM is untouched, at any
kappa. Upstream already fixes its own instance of this by hard-codingresto.theta_max_fact = 1e8(IpRestoMinC_1Nrm.cpp:91) — the resto NLP is also initialised feasible, so it hit the same degeneracy. Stacking the row floor on top would push that inner ceiling to1e8 · m, i.e. remove it, so the sub-IPM always runs withkappa = 0.
Large gradients and dual_inf_scale_kappa
The dual side of the same story. dual_inf_tol (default 1.0) is a bare
absolute bound on ‖∇L‖∞ — but the aggregate above normalises
that quantity, dividing it by s_d, which grows with the mean magnitude
of the multipliers. On a model whose gradients live at 1e10 the two
gates are judging one number by standards ten orders apart.
Vanderbei’s orthrds2 is the reported case: s_d ≈ 1.6e10 with
‖∇L‖∞ = 89.7, so the aggregate’s dual term is 5.6e-09 — comfortably
inside the default tol = 1e-8, i.e. stationary to nine digits relative
to the size of the gradients involved — while the component gate refused
it against 1.0. The solve exited Solved_To_Acceptable_Level holding
the answer, and dual_inf_tol=1e3 alone turned it into
Optimal Solution Found at the same objective.
The simplest statement of the defect: multiply an objective by a positive
constant. Same feasible set, same solution, same active set, same Newton
step — and every multiplier, s_d and ‖∇L‖∞ scale with it, so a large
enough constant costs the certificate.
The strict test therefore judges the unscaled dual infeasibility against
max( dual_inf_tol , kappa · tol · dual_scale )
with kappa = dual_inf_scale_kappa (default 1) and dual_scale the
magnitude of the largest single term ∇L is assembled from (∇f,
Jᵀy, the bound multipliers). Since ∇L is the sum of those terms,
‖∇L‖∞ / dual_scale is the fraction of them that failed to cancel — a
scale-invariant statement of stationarity, and the thing the absolute
bound was standing in for.
What bounds it:
- It cannot forgive non-stationarity. A point where nothing cancelled
has
‖∇L‖∞ ≈ dual_scale, a ratio of1against a bar of1e-8.min −exp(x) s.t. x >= 0running away toinf_du = 8.8e+47is refused by eight orders, because its∇fruns away by exactly the same factor. - The aggregate still has to pass.
nlp_err <= tolis tested on the same iterate; this only removes the second, inconsistent standard. - It is inert on ordinary models. At the defaults the floor does not
rise above
dual_inf_toluntildual_scaleexceedsdual_inf_tol / tol = 1e8, so every model withO(1)gradients keeps upstream’s comparison bit for bit. - Only the strict gate reads it.
acceptable_dual_inf_tol(1e10) is untouched.
Set dual_inf_scale_kappa = 0 to switch the floor off and restore
upstream Ipopt’s bare-absolute bound. That is also the setting to reach
for if you tighten dual_inf_tol and want that absolute standard
honoured unconditionally — the floor is a floor, so it can override a
tightened dual_inf_tol on a large-gradient model.
Objective sense and obj_scaling_factor
obj_scaling_factor multiplies the objective the IPM minimizes, so a
negative value maximizes — upstream’s documented spelling for a
maximization problem stated as a minimization. Because it changes what is
being optimized rather than just its conditioning, it is honored only by
the general NLP interior-point path: a model that would otherwise route
to the specialized convex solvers (LP / convex QP / SOCP, see
LP/QP Routing) is re-routed under
solver_selection=auto, and an explicit convex solver_selection is
refused rather than silently answering with the minimizer.
A positive factor is a pure conditioning knob; the convex path reports natural units either way, so it keeps the fast path.
Barrier-parameter (μ) strategy
The barrier parameter μ controls the inner subproblem’s relaxation of
complementarity. The two strategies are monotone (default — geometric
schedule) and adaptive (quality-function oracle picks each μ from the
current iterate’s complementarity). See
μ-strategy for when to switch.
| Option | Default | Meaning |
|---|---|---|
mu_strategy | monotone | monotone (Fiacco–McCormick schedule) or adaptive (oracle-driven). |
mu_oracle | quality-function | Adaptive oracle: quality-function / loqo / probing. |
mu_init | 0.1 | Seed value for μ at the first iterate. |
mu_min | 1e-11 | Floor on μ; the solver stops decreasing past this. In both μ strategies the effective floor is capped at `compl_inf_tol· |
mu_max | 1e5 | Cap on μ (adaptive mode). When set explicitly it overrides the mu_max_fact initialization. |
mu_max_fact | 1e3 | Initializes mu_max as mu_max_fact · curr_avrg_compl at the first iterate (adaptive mode). |
mu_target | 0.0 | Stop target for μ in monotone mode. |
mu_linear_decrease_factor | 0.2 | κ_μ in μ ← min(κ_μ · μ, μ^θ_μ). |
mu_superlinear_decrease_power | 1.5 | θ_μ in the same formula. |
barrier_tol_factor | 10.0 | Inner-subproblem tolerance scales as barrier_tol_factor · μ. |
sigma_max | 1e2 | Upper clamp on σ chosen by the quality-function oracle. |
sigma_min | 1e-6 | Lower clamp on σ (raising this to 1e-2 can break a stair-stepping stall on some problems). |
adaptive_mu_globalization | obj-constr-filter | Adaptive-mode globalization: kkt-error, obj-constr-filter, or never-monotone-mode. |
Quality-function oracle (adaptive-μ details)
These are only consumed when mu_strategy=adaptive and
mu_oracle=quality-function. Defaults mirror upstream
IpQualityFunctionMuOracle::RegisterOptions.
| Option | Default | Meaning |
|---|---|---|
quality_function_norm_type | 2-norm-squared | Norm used to aggregate KKT components inside q(σ): 1-norm, 2-norm, 2-norm-squared, max-norm. |
quality_function_centrality | none | Centrality penalty term: none, log, reciprocal, cubed-reciprocal. |
quality_function_balancing_term | none | Balancing penalty when complementarity ≪ infeasibilities: none or cubic. |
quality_function_max_section_steps | 8 | Cap on golden-section iterations when picking σ. |
quality_function_section_sigma_tol | 1e-2 | Width tolerance in σ-space terminating the golden-section search. |
quality_function_section_qf_tol | 0.0 | Relative flatness tolerance on q(σ) terminating golden section. |
Adaptive-μ globalization
Tuning the safeguards that fall back to monotone-μ mode when the
adaptive oracle stops making progress. Defaults mirror upstream
IpAdaptiveMuUpdate::RegisterOptions.
| Option | Default | Meaning |
|---|---|---|
adaptive_mu_safeguard_factor | 0.0 | LOQO safeguard floor on the oracle’s μ candidate. |
adaptive_mu_monotone_init_factor | 0.8 | Multiplier on avrg_compl when seeding monotone mode after a bailout. |
adaptive_mu_restore_previous_iterate | no | Restore the latest free-mode iterate when switching to fixed mode. |
adaptive_mu_kkterror_red_iters | 4 | Window length for the kkt-error globalization history. |
adaptive_mu_kkterror_red_fact | 0.9999 | Required relative KKT-error reduction over that window. |
adaptive_mu_kkt_norm_type | 2-norm-squared | Norm used to score the iterate in adaptive globalization decisions. |
ℓ₁ penalty-barrier wrapper options
These tune the degenerate-NLP wrapper described in Running Solves. All are default-tuned and rarely need overriding:
| Option | Default | Meaning |
|---|---|---|
l1_exact_penalty_barrier | no | Run the ℓ₁-exact penalty-barrier wrapper unconditionally. |
l1_fallback_on_restoration_failure | no | Retry with the wrapper only when the standard solve fails. |
l1_penalty_init | 1.0 | Initial penalty weight ρ. |
l1_penalty_max | 1e6 | Maximum penalty weight before declaring infeasibility. |
l1_penalty_increase_factor | 8.0 | Multiplier applied to ρ each outer iteration. |
l1_penalty_max_outer_iter | 8 | Maximum penalty outer iterations. |
l1_slack_tol | 1e-6 | Slack tolerance for “constraints satisfied”. |
l1_steering_factor | 10.0 | Steering-rule factor for ρ escalation. |
NLP Presolve
POUNCE’s TNLP-wrapper presolve pipeline runs before the IPM starts. It tightens variable bounds, drops redundant rows, and (optionally) eliminates square auxiliary-equality sub-systems structurally. All are off by default — set the master switch first:
presolve=yes applies equally to CLI solves and to every
IpoptApplication::optimize_tnlp library solve; callers no longer need to
wrap a callback TNLP manually.
The wrapper postsolves before finalize_solution, so callback payloads remain
in the submitted TNLP’s original variable and constraint space. Bare callback
TNLPs do not expose an expression provider, so presolve_fbbt=yes remains a
no-op for this library entry point.
| Option | Default | Meaning |
|---|---|---|
presolve | no | Master switch for the whole presolve layer. Off → wrapper is a no-op. |
presolve_bound_tightening | yes | Phase 1 — Andersen-style bound propagation from linear rows. |
presolve_redundant_constraint_removal | yes | Phase 2 — drop linear constraints already implied by current bounds. |
presolve_linear_eq_reduction | no | Phase 6 — eliminate variables determined by linear equality rows (see below). |
presolve_licq_check | yes | Phase 3 — detect rank-deficient equality blocks before the IPM starts. |
presolve_licq_action | warn | What to do on degeneracy: warn (just report) or auto_l1 (turn on ℓ₁). |
presolve_warm_z_bounds | yes | Phase 4 — warm-start bound multipliers when bounds get tightened by Phase 1. |
presolve_bound_mult_init_val | 1.0 | Value used by Phase 4 for those warm-start hints. |
presolve_max_passes | 3 | Fixed-point iteration cap across the bound-tightening passes. |
presolve_print_level | 0 | Per-pass verbosity (0 silent, 5 per-pass, 8 per-transformation). |
Linear-equality variable elimination (Phase 6)
presolve_linear_eq_reduction=yes is the only pass that removes
columns. It reads the model’s linear equality rows and eliminates the
variables they determine, iterating to a fixed point so chains propagate:
- a variable whose declared bounds are equal becomes a constant;
- a singleton row
a·x = bpins its variable atb/a; - a two-variable row
a₁·x + a₂·y = bsubstitutes one variable for the other,x := α·y + β. There is no anchoring requirement: a row linking two otherwise-free interior variables — an arc equality, aReferencealias, a unit-conversion link — aggregates away, which is the case the auxiliary-equality pass cannot reach because it only solves determined square blocks.
Rows that collapse to 0 = 0 under the accumulated substitutions are
dropped as structurally redundant.
A row written with a constant on the left — x0 − 2·x1 + 3 = 3 — is
eligible on the same terms as x0 − 2·x1 = 0. The .nl reader folds a
constant row body into the row’s bounds when the file is read, so the pass
sees an ordinary linear equality.
Every eliminated variable’s bounds are transferred onto its survivor, so
the reduced box is never looser than the original. finalize_solution
lifts the primal back to the original variable order and recovers a
multiplier for each consumed row, so .sol / JSON solution blocks keep the
original model’s shape and can still be read positionally by AMPL or Pyomo.
Three things to know before turning it on:
-
Dual attribution. A transferred bound’s multiplier comes back on the variable that declared the bound, not on the survivor that inherited it. The plan records where each reduced bound came from, and postsolve rescales the multiplier by the substitution’s coefficient
α— and moves it to the other side of the box whenα < 0, since a negative coefficient turns a lower bound into an upper one. On a model with a single active transferred bound the reported duals match a no-presolve solve exactly.Where the survivor’s own bound and a transferred bound are active at the same point, the split between the two multipliers is genuinely non-unique — the reduced problem has one where the full problem has two — and the pass leaves the whole multiplier on the survivor. That is a valid KKT point, but it is not the split a no-presolve solve happens to report. The same holds for a variable pinned by a singleton row
a·x = bwhose value lands on one of its own bounds: the row multiplier absorbs it.A practical consequence for
.solreaders, unchanged by any of the above: the writer omits exact zeros from suffix blocks (it always has), so a variable whose bound multiplier is zero gets noipopt_zL_out/ipopt_zU_outentry at all rather than an entry of zero. Code that indexes those suffixes must treat a missing index as zero — as it already must for any variable whose bound multiplier lands exactly on zero. Row multipliers are unaffected: the dual block is dense and comes back at the original row count. -
Bounds the reduced problem never saw. Re-attribution can only move a multiplier the solver reported, and sometimes there is none. The transfers can leave a survivor’s reduced box as a single point, and a variable with equal bounds is a fixed variable, which the solver drops — so it comes back with no bound multiplier at all even though the cluster it stands for is sitting on a bound that needs one. Postsolve fills that in: whatever stationarity residual the recovered row multipliers cannot close is a bound multiplier that was never reported, and it goes on the declared bound the point is actually resting on — the survivor’s own where that is the active one, otherwise the column the bound was borrowed from, through the same
αrescale as above. A residual with no active declared bound to carry it is left alone rather than parked somewhere that would break complementarity.One column is deliberately outside this: one the model declares fixed (
x_l == x_u). The solver drops those as parameters whether or not the reduction runs, and reports no multiplier for them either way, so nothing here changes what they report. -
Failing closed. If the equality system is contradictory, the pass stands down entirely and hands the model to the solver untouched, rather than being the first and only voice to call a model infeasible. The same goes for a model whose every column is determined: a zero-variable problem is not a shape worth handing the IPM.
It is off by default because it changes the variable count, which the
sensitivity and reduced-Hessian paths index against the original .nl.
(The CLI already disables presolve entirely when those are requested.)
LP and convex QP take a different route to the same reduction. Those
models never reach Phase 6 — the CLI dispatches them to pounce-convex
before any presolve wrapper is built — but they are not left unreduced.
pounce-convex has its own presolve, on by default, and it now performs
the two-variable aggregation as part of that catalog, sharing this
planner rather than restating it. So the reduction is the same; only the
switch differs (qp_presolve=no / presolve=no turns it off there, and
presolve_linear_eq_reduction does not apply). See
LP/QP routing.
The two agree on dual attribution as well: a transferred bound’s multiplier is reported on the column that declared the bound, not on the survivor that inherited it. They get there differently — Phase 6 records during planning where each reduced bound came from, while the convex path reads the leftover reduced cost at postsolve and hands it to whichever column is sitting on its own bound, because it also has inequality rows and its own bound-tightening layer to account for. Where the survivor’s own bound is active as well, both leave the multiplier on the survivor; the split is genuinely non-unique there and either answer is a valid KKT point.
Feasibility-based bound tightening (Phase 1b)
Interval-arithmetic propagation through nonlinear constraint
expression DAGs (see FBBT). Available today for
.nl-loaded problems via NlTnlp; other TNLP sources opt out
silently.
| Option | Default | Meaning |
|---|---|---|
presolve_fbbt | no | Master switch. Requires presolve=yes and an ExpressionProvider. |
fbbt_tol | 1e-6 | Minimum per-variable bound improvement to keep iterating. |
fbbt_max_iter | 10 | Outer-sweep cap. |
fbbt_max_constraints | 0 | Per-sweep cap on constraints inspected (0 = unlimited). |
Auxiliary-equality preprocessing (Phase 0)
A separate set of options controls the structural elimination pass documented in Auxiliary-Equality Preprocessing:
| Option | Default | Meaning |
|---|---|---|
presolve_auxiliary | no | Master switch for the Phase-0 structural elimination pass. |
presolve_auxiliary_coupling | safe | Which coupling classes are eligible: none / safe / aggressive. |
presolve_auxiliary_tol | 1e-8 | Residual tolerance for accepting a candidate block solve. |
presolve_auxiliary_max_block_dim | 8 | Largest block the lightweight Newton solver will attempt (larger blocks rejected in v1). |
presolve_auxiliary_wall_time_fraction | 0.1 | Fraction of the solver’s wall-time budget the pass is allowed to spend. |
presolve_auxiliary_diagnostics | no | Emit the diagnostics summary via the journalist after Phase 0 runs. |
FERAL backend tuning
linear_solver=feral (the default — see
Commonly used options) is configurable
through seven feral_* options. Defaults are tuned for the IPM
workload and rarely need changing; reach for these when profiling a
specific problem. Each also falls back to a matching POUNCE_FERAL_*
environment variable when left unset on the OptionsList (see
Environment overrides).
| Option | Default | Meaning |
|---|---|---|
feral_ordering | auto | Fill-reducing ordering method (see table below). auto lets feral’s adaptive dispatcher pick per-matrix; auto_race measures the actual symbolic outcome and keeps the best. |
feral_pivtol | 1e-8 | Relative Bunch-Kaufman partial-pivoting threshold u. Analog of ma27_pivtol / ma57_pivtol. Smaller → sparser L, faster, less stable; larger → more 2×2 blocks, denser, more stable. LAPACK’s textbook maximum-stability value is 0.5. |
feral_refine | yes | Iterative refinement on every back-solve. Closes the residual floor from cascade-break’s L-factor perturbation; disable only when timing the bare factor + back-solve in isolation. |
feral_cascade_break | (unset) | Tri-state. Unset → inherit feral’s Phase B default (CB on with bounded delayed-pivot catchment). yes records explicit intent (no behavioural change). no reproduces pre-Phase-B behaviour by surfacing DelayBudgetExceeded on non-root cascade victims. |
feral_fma | no | Dispatch dense kernels through fused multiply-add intrinsics. Roughly 2× throughput on aarch64 / x86_v3, at the cost of per-pivot rounding drift that trips more WrongInertia checks. Turn on when kernel throughput dominates and the IPM tolerates a noisier inertia signal. |
feral_singular_pivot_floor | 1e-20 | Pounce’s analog of MA57’s CNTL(2). After a successful factor, the smallest accepted D-block pivot magnitude (scaled space) is compared against this absolute floor; if it falls below, the factor is reported Singular so the IPM bumps δ_w. 0 disables. |
feral_inertia_pivot_floor | 1e-12 | Pivot magnitude below which a mismatching inertia count is treated as noise rather than as evidence (#540). Consulted only once the negative-eigenvalue count already disagrees with what the IPM asked for: if the smallest accepted pivot (scaled space) is under this floor, the factor is reported Singular instead of WrongInertia, so δ_c — the perturbation that repairs a rank-deficient constraint block — is applied before the δ_w ladder starts multiplying by 8 per retry. Because it only ever fires on a factor the caller was already going to reject, it cannot turn a usable factorization into a failure. Necessarily larger than feral_singular_pivot_floor, which governs factors that are unusable outright. 0 disables. |
feral_min_par_flops | 1e8 | Flop threshold above which a supernode subtree is dispatched to a parallel worker (feral#19). Lower → dispatch more aggressively (0 fires on every multi-child tree at/above N_PAR_MIN supernodes); a very large value rejects all tree-level parallelism. Only matters when feral’s internal parallelism is active; no effect on a serial factor. |
feral_static_pivoting | (unset) | Tri-state. Factor with static pivoting (SSIDS-style delayed pivots disabled). Unset → inherit feral’s delayed-pivot default. yes runs every supernode as the root does — a failing pivot is force-accepted in place with iterative refinement recovering the residual — breaking the delayed-pivot cascade that can turn one factorization into tens of seconds (feral#8; the emfl050 case in #254). feral’s analog of MA57’s cntl[4]. no keeps delayed pivoting on. Deliberately not coupled to max_wall_time — the accuracy/speed trade is the caller’s to set per solve. |
feral_ordering variants
All six concrete and adaptive options live under the same string
option. feral_ordering also falls back to the
POUNCE_FERAL_ORDERING environment variable when not set on the
OptionsList.
| Value | Strategy |
|---|---|
auto | Default. Adaptive dispatcher: picks a concrete method per matrix from cheap pattern features. Branches: very-large-and-sparse (n > 100 000, avg degree < 5) → AMD; n ≤ 10 000 → AMF; otherwise → MetisND. One symbolic pass; right when the heuristic shape rules apply (the common case). |
auto_race | Race-based dispatcher: runs full symbolic factorization on AMD, MetisND, ScotchND, KahipND and keeps the smallest factor_nnz. ~4× a single symbolic pass, paid once per problem (symbolic factorization is cached across numeric refactorizations with the same pattern). Use when the cheap dispatcher’s guess is suspect — e.g. pinene_3200_0009, where auto picks MetisND (88 s numeric factor) but amd factors in 19.5 s on the same matrix. |
amd | Approximate Minimum Degree (Amestoy/Davis/Duff). Pins AMD regardless of problem shape; robust default for IPM workloads. Best for very-large-and-sparse cases that the adaptive dispatcher already routes here. |
amf | Approximate Minimum Fill (HAMF4 variant of Amestoy 1999). Strong on small-and-sparse populations (n ≤ 10 000); aggregate fill ≈ 0.87× AMD on feral’s IPM small-sparse inventory. |
metis | feral-metis multilevel nested dissection. Tends to produce squarer fronts than AMD on banded / nearly-1D structure; preferred for large structured matrices. |
scotch | feral-scotch nested dissection. Similar regime to METIS; alternative when METIS is unavailable or for cross-validation. |
kahip | feral-kahip flow-based nested dissection with K1 preprocessing. Ties METIS on fill geomean at 4–6× per-call symbolic cost. Reach for it only when ND fill matters and per-call cost is amortized. |
When in doubt: leave feral_ordering at the default. When a hard
problem looks linear-solver-bound, try feral_ordering auto_race
before per-variant manual sweeping — it’s the safe choice when the
per-problem winner is uncertain.
Caller-supplied ordering (External)
Beyond the string variants above, a structure-aware caller can inject a
precomputed permutation the generic AMD/METIS pass cannot see — a
block-triangular / Schur ordering (Parker, Garcia & Bent,
arXiv:2602.17968) or a tearing ordering from equation-oriented
decomposition. Because a permutation is a vector it cannot travel through
the string feral_ordering option; supply it programmatically instead:
- Python:
Problem.set_ordering(perm)(andget_ordering()/clear_ordering()) — see the Python guide. - Rust:
IpoptApplication::set_external_ordering(perm).
perm is a 0-based, new-to-old permutation (perm[k] is the original
index that becomes index k) whose length must equal the augmented KKT
system dimension (variables + slacks + constraint duals), not the
problem’s n. FERAL validates it as a bijection and fails the
factorization with an error on a wrong length or duplicate — a valid but
poor ordering only costs fill/time, never correctness. This maps to
FERAL’s OrderingMethod::External (feral#107) and honors only the default
FERAL backend.
Environment overrides (FERAL and debug gates)
A handful of knobs are reachable through environment variables. The
feral_* numerics knobs read their POUNCE_FERAL_* variable only as a
fallback when the matching option is left unset on the OptionsList — set
the option (per solve, recordable, discoverable via the debugger’s opt
command) in preference to the env var (process-wide, invisible to the solve
report). The debug gates below have no option equivalent; they exist purely
to switch on extra diagnostic output.
FERAL numerics fallbacks
Each maps one-to-one to a registered option in FERAL backend tuning. Prefer the option; the env var is the fallback for callers with no OptionsList (some tests, legacy embeddings).
| Variable | Option |
|---|---|
POUNCE_FERAL_ORDERING | feral_ordering |
POUNCE_FERAL_SCALING | feral_scaling |
POUNCE_FERAL_PIVTOL | feral_pivtol (deprecated bare FERAL_PIVTOL also accepted) |
POUNCE_FERAL_REFINE | feral_refine |
POUNCE_FERAL_CASCADE_BREAK | feral_cascade_break |
POUNCE_FERAL_FMA | feral_fma |
POUNCE_FERAL_SINGULAR_PIVOT_FLOOR | feral_singular_pivot_floor |
POUNCE_FERAL_INERTIA_PIVOT_FLOOR | feral_inertia_pivot_floor |
POUNCE_FERAL_MIN_PAR_FLOPS | feral_min_par_flops |
POUNCE_FERAL_STATIC_PIVOTING | feral_static_pivoting |
FERAL_PARALLEL (legacy, no POUNCE_ prefix) forces feral’s internal
factor serial or parallel process-wide — 0/off/false/no to force
serial, 1/on/true/yes to force parallel, and unset to leave
feral’s own platform-derived default alone. The force-on direction is the
only override available to CLI, Python and NL callers on a host where
that autodetection is wrong (feral falls back to sequential when the
rayon pool fails to build); the first-class per-backend lever,
FeralConfig.parallel, is the Rust solver API, not an option.
Debug and diagnostic gates
These switch on extra diagnostic emission for a specific subsystem. Most
emit at debug level under a pounce::* tracing
target, so setting the gate alone is not enough — pair it with a matching
RUST_LOG (e.g. RUST_LOG=pounce::mu=debug) or the output stays filtered.
Presence-only unless a value is noted; they are diagnostic aids, not part
of the stable interface, and may change between releases.
| Variable | Subsystem (RUST_LOG target) | Emits |
|---|---|---|
POUNCE_DBG_AMU | pounce::mu | Adaptive-μ per-iteration state (θ, f, oracle inputs). |
POUNCE_DBG_ORACLE | pounce::mu | μ-oracle probe-guard decisions (probe-Newton → restoration requests). |
POUNCE_DBG_QF | pounce::mu | Quality-function μ-oracle σ search (floor, current μ). |
POUNCE_DBG_QF_AGGR | pounce::mu | Quality-function aggregate step/complementarity terms per σ. |
POUNCE_DBG_QF_SWEEP=<iter> | pounce::mu | Dumps the full quality-function σ sweep at the given iteration number. |
POUNCE_DBG_DELTA | pounce::algorithm | Primal-dual search direction δ per iteration. |
POUNCE_DBG_LS=1 | pounce::linesearch | Filter line-search / backtracking acceptance trace (must equal 1). |
POUNCE_DBG_PERT | pounce::linsol | Inertia-perturbation handler decisions (WRONG_INERTIA, δ_w escalation). |
POUNCE_DBG_PD_TAGS | pounce::linsol | Primal-dual full-space solver dependent-block tag changes. |
POUNCE_DBG_KKT_DUMP=<path> | pounce::linsol | Writes the tagged KKT matrix to <path>. |
POUNCE_DBG_KKT_DUMP_SKIP=<n> | — | Skip the first <n> factorizations before honoring POUNCE_DBG_KKT_DUMP. |
POUNCE_DUMP_KKT=<path> | pounce::linsol | Writes the standard augmented-system KKT matrix to <path>. Deprecated — prefer --dump kkt:<iter-spec> (see pounce --help). |
POUNCE_DBG_RESTO | pounce::algorithm, pounce::restoration | Restoration entry trace and the augmented restoration-system stats. Canonical spelling; the legacy POUNCE_RESTO_DBG (restoration-system stats only) is a deprecated alias. |
POUNCE_DBG_RESTO_CYCLE | pounce::algorithm | Restoration no-progress cycle-detector relative-step metrics. |
POUNCE_DBG_RESTO_INIT | pounce::restoration | Restoration initial-point vectors. |
POUNCE_DBG_RESTO_KAPPA | pounce::restoration | Restoration κ_resto convergence-guard evaluation. |
POUNCE_DBG_RESTO_LOCINF | pounce::restoration | Restoration local-infeasibility verdict inputs. |
POUNCE_DBG_TAPE_STATS | — (stderr) | AD tape counts after parsing an .nl model. Printed straight to stderr; no RUST_LOG needed. |
POUNCE_SIMPLEX_DEBUG | — (stderr) | Convex/LP-QP simplex pivoting trace. Printed straight to stderr; no RUST_LOG needed. |
Two already-documented gates round out the set: POUNCE_DBG_LLM and
POUNCE_DBG_VIEWER (see the debugger guide).
Logging and colored output
POUNCE emits structured logs and a colored iteration table through the
tracing ecosystem. Behavior is governed by
environment variables (not solver options), so they apply to the pounce
CLI, the C/Python frontends, and anything embedding the library.
| Variable | Values | Effect |
|---|---|---|
RUST_LOG | e.g. info, debug, pounce::restoration=debug | Log verbosity / per-target filtering. Default info. Logs go to stderr. |
POUNCE_LOG_FORMAT | text (default) · json | json emits line-delimited JSON on stderr (incl. the per-iteration pounce::iteration stream) for Studio / CI ingestion. |
NO_COLOR | set to any value | Disables ANSI color in the iteration table and logs (see https://no-color.org). |
CLICOLOR_FORCE | set to any value | Forces color even when stdout is not a terminal. |
Filtering by subsystem. Solver internals log under namespaced targets
— pounce::algorithm, pounce::linsol, pounce::mu, pounce::sqp,
pounce::linesearch, pounce::restoration, pounce::presolve,
pounce::py. For example, to trace only the restoration phase:
RUST_LOG=pounce::restoration=debug pounce problem.nl
Program output vs. logs. The iteration table, the final summary, and
--dump diagnostics are program output on stdout; diagnostic and
progress messages are logs on stderr. Redirecting one does not
affect the other:
pounce problem.nl > result.txt 2> solve.log
Color. The iteration table is colored with a tiger/rust theme:
restoration lines take a background that varies by restoration kind
(soft-stay → tan, soft-exit → amber, hard → deep rust), and the row text
shades from black toward red as the primal step length alpha shrinks
(stalling). Color is emitted only when stdout is a terminal; redirected
output and NO_COLOR get plain text with identical column alignment.
Machine-readable iterations. POUNCE_LOG_FORMAT=json turns the
per-iteration records into JSON on stderr:
POUNCE_LOG_FORMAT=json pounce problem.nl 2> iters.jsonl
LP / QP Solver Routing
POUNCE can route linear programs (LP), convex quadratic
programs (QP), and convex quadratically-constrained QPs (QCQP) to a
specialized interior-point solver (pounce-convex) instead of the general
nonlinear (NLP) filter-IPM. The specialized path uses Mehrotra
predictor-corrector and reaches the solution in materially fewer iterations
on these problem classes — typically 30–50% fewer than the general NLP path
on bound- or inequality-constrained convex QPs.
Routing is automatic and transparent: you do not change how you
call POUNCE. The same pounce problem.nl, the same
SolverFactory('pounce') in Pyomo, and the same AMPL solve all work
unchanged — POUNCE inspects the problem and picks the solver.
How routing works
When POUNCE loads a problem it classifies it into one of:
| Class | Routed to |
|---|---|
| LP | convex IPM (pounce-convex) |
| convex QP | convex IPM (pounce-convex) |
| convex QCQP | conic IPM (pounce-convex, SOCP) |
| nonconvex QP | NLP filter-IPM (finds a local minimum) |
| NLP | NLP filter-IPM |
The classifier is conservative: a problem is sent to the convex
solver only when POUNCE can prove it is convex — an LP or convex QP
(degree-≤2 objective with a positive-semidefinite Hessian, linear
constraints), or a convex QCQP (additionally allowing convex-quadratic
inequality constraints, each with a positive-semidefinite Hessian and a
one-sided ≤ bound, which are reformulated to second-order cones).
Anything it cannot prove convex — transcendental terms, an indefinite
objective Hessian, a quadratic equality, or a quadratic inequality whose
feasible set is nonconvex — falls back to the general NLP solver, which
always produces a correct (locally optimal) answer. You never get a wrong
“optimum” from a misclassification.
Note on QP detection. The AMPL
.nlformat has no dedicated quadratic section: a QP’s quadratic terms are written into the nonlinear expression tree. POUNCE walks that tree to recover the Hessian and test convexity, the same way QP-capable AMPL solvers do.
Note on row constants. A
.nlwriter may leave a constant on the left of a constraint —x0 + x1 + 3 <= 6rather thanx0 + x1 <= 3— and it too lands in the nonlinear expression tree. The reader folds such a constant into the row’s bounds when the file is read, so a model that is otherwise an LP still classifies as one. The shift is exact: body and bound move together, so the solution and every multiplier are the same as for the hand-folded model.
Choosing the solver explicitly
The solver_selection option overrides the automatic choice. It is a
normal POUNCE option, so it works on the command line, in an options
file, or through Pyomo’s solver.options.
| Value | Behavior |
|---|---|
auto | Default. Route by detected class (table above). |
nlp | Always use the NLP filter-IPM, regardless of class. |
lp-ipm | Force the convex IPM; errors if the problem is not an LP. |
qp-ipm | Force the convex IPM; errors if the problem is not LP/convex-QP. |
socp | Force the conic IPM; errors if the problem is not a convex QCQP. |
qp-active-set | Force the active-set SQP engine; errors if the problem is not LP/convex-QP. |
# Let POUNCE decide (default):
pounce model.nl
# Force the NLP path even on a convex QP (e.g. to compare):
pounce model.nl solver_selection=nlp
# Insist the problem is a convex QP — fail loudly if it is not:
pounce model.nl solver_selection=qp-ipm
# Solve that same QP with the active-set engine instead of the IPM:
pounce model.nl solver_selection=qp-active-set
A forced value that does not match the detected class is rejected with a clear message rather than silently ignored:
pounce: problem class NLP does not match forced solver qp-ipm
(expected an LP or convex QP)
qp-active-set hands the QP directly to pounce-qp’s
ParametricActiveSetSolver, through the same convex driver the IPM uses —
so it inherits presolve, postsolve, dual recovery, .sol writing, timing
and the convex status vocabulary. It is not the same route as
algorithm=active-set-sqp, which wraps the QP in the full SQP outer loop;
that option still exists and is the right one for a genuine NLP.
Choose it deliberately. For a cold, one-shot convex QP the
interior-point path (qp-ipm, and what auto selects) is materially more
robust: on the 138-problem Maros-Mészáros set the IPM solves 137 while the
active-set engine solves substantially fewer, mostly by exhausting its
iteration budget on large degenerate instances. That is the expected
character of a cold active-set method rather than a defect — its iteration
count is combinatorial in the size of the active set, where an
interior-point count is nearly independent of problem size. The active-set
engine earns its keep on warm-started sequences — MPC steps,
branch-and-bound nodes, continuation — where consecutive QPs differ little
and the working set carries over; see solve_parametric.
What it will not do is lie: it reports Maximum_Iterations_Exceeded rather
than a wrong answer, and every claimed optimum is re-verified against the
original problem’s KKT conditions before being reported.
From Pyomo
solver = SolverFactory('pounce')
solver.options['solver_selection'] = 'qp-ipm' # or 'auto', 'nlp', ...
solver.solve(model)
What you get back
Before solving, POUNCE prints a one-line routing banner naming the
detected class, the solver it selected, and the effective
solver_selection — so it is always clear which of POUNCE’s solvers ran
and why:
Problem class: LP. Selected solver: convex QP interior-point (pounce-convex) [solver_selection=auto].
(The banner is suppressed alongside the startup banner — sb yes or
JSON-debug protocol mode — to keep stdout clean for machine consumers.)
The convex IPM then reports the same way as the NLP path: an
optimal-status line, the objective value (in your original sense — a
maximize objective and any constant term are reported correctly), and a
.sol file with the primal solution when one is requested.
POUNCE (LP IPM, pounce-convex): Optimal Solution Found.
obj=2.00000000 iters=2
Driver. The convex path uses the homogeneous self-dual embedding (HSDE) interior-point driver — the same self-dual formulation Clarabel/ECOS use. It is self-starting, returns verified infeasibility/unboundedness certificates, and conditions the KKT system internally through its per-cone scaling, so it solves even badly-scaled LPs (e.g. NETLIB
nl,‖c‖ ~ 1e6) without external pre-scaling.
Presolve
Before the convex interior-point solve, POUNCE runs a presolve pass that shrinks the problem and can detect trivial infeasibility or unboundedness without solving. It removes empty, duplicate, and activity-redundant rows; fixes and substitutes structural columns (singleton-row fixings, free columns, free column singletons); folds away two-variable equality rows (below); and recovers both the primal and dual of the eliminated pieces so the reported solution is for your original problem. When it reduces the model, it logs a one-line summary:
Presolve: 40 → 24 vars, 12 → 4 rows (fixed 3, free-fixed 2, substituted 3, aggregated 8, ...)
Two-variable equality rows (aggregation)
A row a₁·x + a₂·y = b linking two variables says one of them is the
other, up to a scale and a shift — an arc equality between two units, a
Reference alias, a unit conversion. Neither variable is determined by
it, so nothing in the older catalog could act on it, and on a flowsheet
these rows are most of the model. POUNCE now substitutes one variable
for the other and drops the row, iterating to a fixed point so chains
of aliases collapse to a single column. Any bound on the eliminated
variable is carried across onto the one that survives, so the reduced
problem describes exactly the same feasible set.
Two things this deliberately does not do:
- It never calls your model infeasible. A contradictory alias system —
x = yandx = y + 1— makes the pass stand down and hand the model over untouched, for the rest of presolve or the solver itself to judge. - It does not run on the conic path (SOCP, exponential/power cones, SDP, SOS). Those rows are structurally coupled in fixed-size blocks that a substitution would rewrite.
The aggregation shares its planner with the NLP path’s Phase 6, so the two agree on what can be eliminated (see NLP Presolve).
Infeasibility verdicts are re-derived before they are reported
A presolve infeasibility comes back in milliseconds with no iteration behind it, so when it is wrong it is the most expensive answer the solver can give. Two reductions — forcing constraints and dominated columns — fix a variable at a value they choose from a tolerance judgment, and a fixing that is wrong is substituted into every row that variable appears in until some row reads as contradictory: a false infeasibility, reported against a row nowhere near the reduction that caused it.
So presolve does not report an infeasibility on the strength of the pass
that found it. It re-derives the verdict from your original model with
those two reductions switched off, and reports Infeasible_Problem_Detected
only if that pass reaches the same conclusion on its own. If it does not,
the model is solved normally and presolve says so:
Presolve: discarded an unconfirmed infeasibility claim — <screen> (<detail>); solving normally
A confirmed verdict now names the screen that proved it and the row, column, or bound it tripped on, rather than exiting silently:
Presolve: proved primal infeasible — empty equality row (equality row 7 is `0 = 3e0`)
Nothing that only reports is withheld from the re-derivation — empty rows, activity ranges, parallel rows, and emptied-row residuals all still apply — so no infeasibility presolve could detect before goes undetected now. What the guard costs, in the rare case it fires, is a handful of eliminations.
When the reduction is truncated
The reductions are iterated to a fixpoint — each one can expose work for the next, so presolve keeps going until nothing fires. It also carries a cap on how many layers that may take, and on a model with a long bound-propagation chain the cap is what stops it. When that happens the summary line says so:
Presolve: 315 → 128 vars, 233 → 77 rows (fixed 61, ..., tightened 158, cap-truncated after 32 layers)
This is common and it is not a problem. Measured across the LP and QP suites, the cap binds on 46% of LP models and 25% of QP models — and on every one of the 394 models that presolve at all, it changed only how tightly variable boxes were narrowed, never the structural reduction: same variables, same rows, same fixings, aggregations, forcing rows and dominated columns as running the iteration to convergence. Bound propagation is the one reduction that can keep going indefinitely, so it is what the cap ends up trimming.
What you get is still a correct problem — every reduction applied is a sound transform with its own dual recovery, and your solution is postsolved back to the original either way. The suffix is there so a reduction that came out of a truncated loop is distinguishable from one that converged, which matters when you are comparing two runs or reporting a bug against presolve. There is no option to turn it up.
Presolve is on by default. Turn it off with qp_presolve=no (e.g. to
compare timings or isolate a solver issue):
pounce model.nl qp_presolve=no
Scope and limitations
- Convex problems only. Nonconvex (indefinite-Hessian) QPs, quadratic equalities, and quadratic inequalities whose feasible set is nonconvex are solved by the NLP path to a local minimum; POUNCE does not do global optimization.
- Convex QCQP (convex-quadratic constraints) routes to the conic IPM:
each convex-quadratic inequality
½xᵀQx + aᵀx + b ≤ 0(withQ ⪰ 0) is reformulated to one second-order cone (Q = FᵀF, so‖Fx‖² = xᵀQx) and solved alongside the QP objective and linear constraints.
Both the primal solution and the constraint duals are written to the
.sol file, in the same sign convention as POUNCE’s NLP path (so Pyomo
and AMPL read them identically regardless of which solver ran).
Requests the convex path does not implement
The convex solvers are a specialized fast path, not a drop-in for every option the NLP path honors. Where a request would be dropped rather than merely unused, routing gives way rather than answering a different question:
| Request | Under auto | Under an explicit solver_selection |
|---|---|---|
obj_scaling_factor < 0 (maximize) | re-routes to the NLP path | refused (exit 2) — running would report the minimizer |
nlp_scaling_method=user-scaling with scaling_factor suffixes | re-routes to the NLP path | warns; the scaling is skipped |
sIPOPT sens_* suffixes, --compute-red-hessian | re-routes to the NLP path | warns; the step is skipped |
A positive obj_scaling_factor is not in this table: it only rescales
conditioning, and the convex path reports natural units either way, so
both paths give the same answer.
When the convex path cannot certify an LP
Routing gives way one more time, and this one is decided after the solve
rather than before it. Under auto, an LP whose convex solve finishes
without a KKT certificate — Solved to acceptable level (reduced accuracy)
or Maximum iterations exceeded — is re-solved on the general NLP
interior-point path, which owns the whole verdict. Nothing from the
declined convex solve is printed or written, so a rerouted run still
reports exactly one status.
The case this exists for is the NETLIB gen / gen1 family. They are
highly degenerate and rank-deficient, strict complementarity fails, and a
pure interior-point method cannot certify the optimal vertex: the convex
IPM spends its whole 200-iteration budget (190.8 s) and stops at a primal
residual of 1.4e-7 against tol = 1e-8. The NLP filter-IPM — the same
binary, the default for every other class — solves the same model in 19
iterations and 0.98 s to a strict certificate, matching Ipopt-3.14.20/MA57
to four figures. Rerouting is also the faster answer here: a second solve
of one second is nothing against the three minutes the first one costs.
The fallback is narrow by construction, and does not fire when:
| why | |
|---|---|
the class is not LP (P ≠ 0) | a stalling convex QP is a different, unmeasured population |
the solve certified (Optimal Solution Found) | there is nothing to improve, and a second solve would double the cost of every LP |
| the status is infeasible or unbounded | those verdicts carry a verified certificate (see below); a second solve must not overwrite a proof |
solver_selection names an engine | a named engine keeps its verdict — that is what makes the stall observable |
max_iter was set explicitly | a user-set budget is the question being asked; max_iter=0 in particular must stop without a solve |
| the interactive debugger is attached | you are stepping this engine |
A tightened tol is deliberately not in that list: that is an accuracy
request, so trying the engine that can meet it is the right response.
Infeasible and unbounded problems
The convex solver detects infeasibility and unboundedness directly, reporting a clean status instead of exhausting the iteration budget:
- Primal infeasible — no point satisfies the constraints. Reported
with AMPL
solve_result_num200. - Unbounded (dual infeasible) — the objective decreases without
bound along a feasible direction. Reported with
solve_result_num300.
Each verdict is backed by a verified certificate (a Farkas
infeasibility proof or an unbounded recession direction that is checked,
not merely inferred), so these statuses are never reported in error; a
problem the solver cannot certify simply runs to the iteration limit —
and, if it is an LP under auto, is then handed to the NLP path (above).
solver_selection=qp-active-set follows the same contract. Its inner QP
certifies the recession ray of the linearization, which on a nonlinear
model is not yet a statement about the problem, so the ray is re-tested
against the true objective and constraints before the 300 is reported;
a ray that does not survive yields
Search_Direction_Becomes_Too_Small, never an unboundedness claim.
The design and roadmap live in
dev-notes/lp-qp-routing.md.
Convex Solver: LP, QP, and SOCP
POUNCE ships a specialized convex conic interior-point solver
(pounce-convex) alongside the general NLP filter-IPM. It solves the
standard-form convex program
minimize ½ xᵀP x + cᵀx
subject to A x = b
G x ⪯_K h
lb ≤ x ≤ ub
where P ⪰ 0 and the inequality block lies in a product cone K of
nonnegative orthants and second-order cones. P = 0 is an LP; an
all-orthant K is an LP/QP; second-order blocks make it an SOCP.
The method is a Mehrotra predictor–corrector primal–dual interior-point
algorithm with Nesterov–Todd scaling for the cones, sharing the pure-Rust
feral sparse LDLᵀ backend with the NLP path. It reaches
optimality in materially fewer iterations than routing the same problem
through the general NLP solver (≈30–50% fewer on bound/inequality QPs).
Inspiration. The conic interior-point design follows Clarabel (Goulart & Chen) — handling a quadratic objective directly and a product of symmetric cones — and the presolve follows PaPILO (the presolving library of SCIP). POUNCE does not wrap either (the pure-Rust guarantee) but ports their ideas; see Acknowledgments.
This chapter covers the Python API (pounce.qp and the differentiable
pounce.jax layers). For automatic CLI/Pyomo routing of .nl LPs/QPs, see
LP / QP Solver Routing. Runnable, progressive notebooks
live in python/notebooks/:
15_convex_qp.ipynb, 16_socp.ipynb, 17_differentiable_convex.ipynb.
Quadratic programs
import numpy as np
from pounce.qp import solve_qp
# min ½·2‖x‖² − 3x₀ − 4x₁ s.t. x₀ + x₁ ≤ 1, 0 ≤ x ≤ 1
r = solve_qp(
P=np.diag([2.0, 2.0]),
c=[-3.0, -4.0],
G=[[1.0, 1.0]], h=[1.0],
lb=[0, 0], ub=[1, 1],
)
r.status # 'optimal'
r.x # primal solution
r.y, r.z # equality / inequality multipliers
r.z_lb, r.z_ub # bound multipliers (≥ 0)
r.obj, r.iters
P (lower triangle used, assumed symmetric), A, and G accept dense
arrays or scipy-sparse matrices; any of them may be omitted. The result is
a QpResult dataclass with a .success property. The solver reports
verified infeasibility / unboundedness ('primal_infeasible' /
'dual_infeasible') backed by a Farkas / recession certificate rather than
an iteration-limit guess.
Second-order cone programs
A second-order (Lorentz) cone is { (t, x) : t ≥ ‖x‖₂ }. Partition the
inequality rows of Gx ⪯_K h with cones — a list of (kind, dim) specs
("nonneg" or "soc"; a bare int means a second-order cone). Each slack
block s = h − Gx must lie in its cone.
from pounce.qp import solve_socp
# minimize ‖x − x*‖ ⇔ min t s.t. (t, x − x*) ∈ SOC
r = solve_socp(
c=[1.0, 0.0, 0.0], # minimize t
G=-np.eye(3), h=[0.0, -2.0, 1.0], # s = (t, x₀−2, x₁+1) ∈ SOC(3)
cones=[("soc", 3)],
)
r.x # ≈ [0, 2, -1]: t* = 0, x = x*
Mixed cones compose — e.g. cones=[("nonneg", 1), ("soc", 2)] puts the
first slack in ℝ₊ and the next two in a 2-D second-order cone. Large
cones use a sparse diagonal-plus-rank-1 KKT representation (one
auxiliary variable per cone, the ECOS/Clarabel “sparse SOC” trick) so the
factorization stays sparse.
Warm starting
Feed a previous (or nearby) solution back to seed the interior-point iteration — useful for parametric sweeps, receding-horizon MPC, and branch-and-bound subproblems:
base = solve_qp(P=P, c=c, G=G, h=h, lb=lb, ub=ub)
nxt = solve_qp(P=P, c=c2, G=G, h=h, lb=lb, ub=ub, warm_start=base)
The warm start only affects the iteration count, never the solution (a mismatch is ignored). The recentering is adaptive for the orthant (sized to the warm point’s KKT residual, so it exploits a nearby problem’s duals yet self-corrects when the active set moves) and re-centers the cone duals for second-order blocks (a converged conic point sits on the cone boundary, where the scaling is singular).
The step length is what makes it pay off
A warm start lowers the starting duality measure μ₀; whether that turns into
fewer iterations depends on how much of each Newton step the solver is
allowed to take. With a static fraction-to-boundary parameter τ, every
step covers at most a τ fraction of the distance to the cone boundary, so μ
falls by a fixed factor per iteration and the count is log₁/₍₁₋τ₎(μ₀/tol)
however good the start was — a logarithm of the perturbation, not the one or
two Newton steps a nearby problem deserves.
So on orthant blocks the step follows the Mehrotra tail
τ = clamp(1 − μ, tau, tau_max): as the solve converges τ approaches 1 and a
near-optimal iterate takes a near-full Newton step. On the QP families in the
warm-start benchmark this is worth 35–60% of the warm iterations. Both ends
are tunable, and both are method="ipm" only:
r = solve_qp(P=P, c=c2, G=G, h=h, warm_start=base,
tau=0.95, # floor: the flat τ far from the solution
tau_max=0.999) # ceiling on the tail (default: just under 1)
Passing tau_max=tau pins τ flat — the most conservative setting, and the
one to reach for if a badly-conditioned sequence starts producing
numerical_failure. Two scopes are deliberate and not tunable: second-order
and PSD blocks always keep the static tau (their boundary is curved, and an
iterate that close to it breaks the Nesterov–Todd scaling), and cold
solves are unaffected because they run the homogeneous self-dual embedding,
a different loop.
Batching and factorization reuse
from pounce.qp import solve_qp_batch, QpFactorization
# Solve many independent QPs in parallel (rayon, across instances).
results = solve_qp_batch([dict(P=P, c=c_k, G=G, h=h) for c_k in cs])
# Build the KKT symbolic factor once, solve many same-structure problems.
fac = QpFactorization(P=P, c=c0, G=G, h=h, lb=lb, ub=ub)
for c_k in cs:
rk = fac.solve(P=P, c=c_k, G=G, h=h, lb=lb, ub=ub) # reuses the factor
solve_qp_batch parallelizes across instances (outer-parallel /
inner-serial) and QpFactorization reuses the AMD ordering and symbolic
factorization across solves that share a structure — the two compose with
warm starting.
Presolve (PaPILO-inspired)
Before the interior-point solve, POUNCE can apply a transaction-stack presolve with full primal and dual postsolve, modeled on PaPILO. The catalog:
- empty / duplicate / parallel (scalar-multiple) rows,
- fixed-variable elimination (singleton equalities),
- free columns and free-column singletons,
- activity-based redundancy and infeasibility detection,
- forcing constraints (a row at its activity extreme pins its variables),
- dominated columns (sign-definite columns optimal at a bound),
- bound tightening (domain propagation), with the active-bound multiplier re-attributed to its source row in postsolve,
iterated to a fixpoint so reductions cascade. Each reduction carries
the data to reverse itself, and the postsolve reconstructs a valid KKT
point of the original problem — the dual recovery is the contract, and is
verified by KKT-residual tests. A cone-aware variant (presolve_conic)
gates the ≤-row reductions off second-order-cone blocks (which are
coupled) and recovers the reduced cone partition.
The iteration also carries a layer cap, and on a model with a long bound-propagation chain — commonly, on roughly half the LP corpus — the cap is what stops it rather than the fixpoint. That distinction is visible: presolve reports which of the two happened and the CLI says so on its summary line (see LP / QP Solver Routing). A truncated reduction is still correct — every reduction it did apply is a sound transform with its own dual recovery — and measured across the LP and QP suites the truncation costs only box tightness, never a structural reduction.
Presolve is applied automatically on the CLI LP/QP route; it lives in
pounce-convex::presolve for Rust callers. See
LP / QP Solver Routing.
Differentiable convex layers (JAX)
pounce.jax exposes the solve as a differentiable JAX op via the
implicit-function theorem on the KKT system at the optimum (Amos & Kolter,
OptNet, 2017). The forward calls the solver; the backward is a single
linear solve through the same KKT matrix.
import jax, jax.numpy as jnp
from pounce.jax import solve_qp, solve_socp, QpLayer
# x*(c) for a parametric QP, differentiable w.r.t. all of P, c, G, h, A, b.
def loss(c):
x = solve_qp(P=P, c=c, G=G, h=h)
return jnp.sum((x - target) ** 2)
grad_c = jax.grad(loss)(c0) # exact gradient via implicit diff
J = jax.jacrev(lambda c: solve_qp(P=P, c=c, G=G, h=h))(c0)
- Gradients are provided w.r.t. every parameter that enters through the
optimum:
c,b,h, and the matricesP,G,A(the full OptNet matrix derivatives;∇Pis the symmetric gradient). solve_socpdifferentiates SOCPs too — the complementarity row uses the cones’ arrow operators in place of the orthant’s diagonal.QpLayercaptures a fixedP/G/Astructure for use inside a larger JAX model, withjax.grad/jacrev/vmapand a parallel.batch.- A warm start may be passed through (non-differentiated — it cannot change the solution or its gradients, only the iteration count).
All gradients are validated against finite differences in the test suite.
Global Optimization
Most of POUNCE settles a problem at a local optimum (the NLP filter-IPM and SQP) or exploits convexity so that local is global (the convex/conic IPM). For a genuinely nonconvex problem, POUNCE offers one certified-global route, and it is for polynomials:
- The SOS / Lasserre hierarchy (
pounce-convex) — for polynomial problems, via a single semidefinite program. Callable from Rust (sos_minimize) and Python (pounce.sos_minimize).
It returns a result that is certified: a lower bound together with a moment certificate that, when exact, pins the global minimum and recovers its minimizer(s).
There is no general-purpose spatial branch-and-bound solver in POUNCE. For a nonconvex problem that is not polynomial — anything with
exp/ln/trig — POUNCE has no certified-global path. Use the local NLP solver from several starting points (see the multistart notebooks below), or reformulate into the convex cone library. Apounce-globalcrate was prototyped and removed frommainbefore release; its design is recorded indev-notes/spatial-bnb-design.md.
The SOS / Lasserre path (polynomials)
When the objective and constraints are polynomials, the
sum-of-squares / moment approach in pounce-convex certifies the global
minimum from a single semidefinite program — no branching — by searching for
the largest γ such that p(x) − γ lies in the Putinar cone (a sum of squares
plus constraint multipliers). The SDP is solved by POUNCE’s own convex conic
interior-point method; flat truncation of the resulting moment matrix certifies
when the bound is exact, and a facial-reduction step recovers every global
minimizer — even when the optimum is attained at several points.
From Python, a polynomial is a dict mapping an exponent tuple to its coefficient (the all-zeros key is the constant term):
from pounce.sos import sos_minimize
# x**4 - 2 x**2 + 3 -> global minimum 2, attained at BOTH x = +1 and x = -1
r = sos_minimize({(4,): 1.0, (2,): -2.0, (0,): 3.0})
r.lower_bound # ≈ 2.0
r.is_exact # True — flat-truncation certificate: the bound is the minimum
r.minimizers # both x = +1 and x = -1
Constraints are polynomials too, passed as inequalities (g_i(x) ≥ 0) and
equalities (h_j(x) = 0); raise the relaxation order to tighten the bound
(the Lasserre hierarchy) at the cost of a larger SDP. A runnable walkthrough —
double well, a constrained problem, and a 2-D example — is in
18_sos_global_optimization.ipynb.
The same solver from Rust, via the pounce-rs facade with the convex
feature on (pounce-rs = { version = "0.9", features = ["convex"] }):
#![allow(unused)]
fn main() {
use pounce_rs::convex::{sos_minimize, PolyProblem, Polynomial};
use pounce_rs::linsol::backend; // the sparse LDLᵀ factory the solver takes
// x⁴ − 2x² + 3 → global minimum 2 at x = ±1.
let p = Polynomial::new(1, vec![(vec![4], 1.0), (vec![2], -2.0), (vec![0], 3.0)]);
let sol = sos_minimize(&PolyProblem::new(p), None, backend);
// sol.lower_bound ≈ 2; when the moment matrix is flat, sol.minimizers holds
// the global minimizer(s) — here both x = +1 and x = −1.
}
The full treatment lives in the pounce_convex::sos module documentation —
reachable without a second dependency, since pounce_rs::convex re-exports
the pounce_convex crate itself for anything outside its curated surface.
When SOS fits: polynomials of modest degree and dimension — one SDP, recovers all global minimizers, but the SDP grows with the relaxation order.
When SOS does not fit
For a general factorable problem (exp/ln/trig), or a polynomial whose SDP
would be too large, the textbook tool is spatial branch-and-bound — and POUNCE
does not have one. Two things you can do:
- Reformulate into the cone library. If the model can be cast as an LP, convex QP, SOCP, or an exponential / power / PSD cone program, local is global and the guarantee comes for free. See Choosing a Solver.
- Multistart the local solver. Running the NLP filter-IPM from many
starting points finds the low minima in practice, but certifies nothing —
there is no bound proving you have the global one. Three notebooks work
through the tactics: repulsion-based sampling
(
19_find_minima_repulsion.ipynb), random restarts (20_find_minima_restart.ipynb), and basin hopping (21_find_minima_hopping.ipynb).
Solution Output
The .sol file
Following the AMPL solver convention, solving a positional .nl file
writes a sibling <stub>.sol next to it — pounce problem.nl
produces problem.sol. The file carries the primal x and dual
lambda blocks plus an objno line with the AMPL solve_result_num,
so AMPL (or any .sol reader) can pull the solution back:
pounce problem.nl # writes problem.sol
pounce problem.nl --sol-output out.sol # write to an explicit path
pounce problem.nl --no-sol # skip the .sol write
A .sol is written even when the solve fails, so the
solve_result_num is always recoverable. Built-in problems
(--problem …) have no .nl stub, so they only produce a .sol
when --sol-output is given explicitly.
Reading solve_result_num
The objno line carries an AMPL solve_result_num (Gay 2005, Hooking Your
Solver to AMPL §5). Consumers key on the band, not the exact number:
| Band | Meaning |
|---|---|
0–99 | solved |
100–199 | solved, with a warning |
200–299 | infeasible |
300–399 | unbounded |
400–499 | limit reached (iterations, time) |
500–599 | failure |
Pyomo maps each band to a TerminationCondition, so anything in 200–299
arrives as TerminationCondition.infeasible.
Infeasible: proved vs. local
Within the infeasible band POUNCE distinguishes how it knows:
| Code | Verdict | What it means |
|---|---|---|
200 | InfeasibleProblemDetected | The solver converged to a point of local infeasibility — a stationary point of the constraint violation with the violation bounded away from zero. |
201 | ... (detected by presolve: …) | Presolve’s bound propagation / interval arithmetic found the feasible region empty before any iteration. |
The difference is real, not cosmetic. 201 is a structural detection made on
the model’s bounds before iterating, not a certified proof — it is subject to
the same floating-point limits as any interval computation, and is withheld
whenever the violation is smaller than the feasibility tolerance. 200 is
different in kind — on a nonconvex problem a positive local minimum of the
violation does not rule out a feasible point elsewhere, which is why the
console message says “Problem may be infeasible.”
Because 200 is an inference rather than a proof, it is withdrawn when POUNCE
holds a point that contradicts it. Before any numerical path reports 200, the
model’s own starting point is evaluated against every constraint; if it
satisfies them all, the feasible set is demonstrably non-empty and the verdict
becomes Error_In_Step_Computation (500) — an honest “the solve broke down”
rather than a wrong answer. This can only ever withdraw a verdict: a model
with no feasible point cannot produce such a point, so a correct 200 is
unaffected. Supplying a feasible starting point is therefore worth doing on a
model you believe is feasible but POUNCE reports otherwise.
When the region is found empty the solve is skipped entirely and the message names how it was found, so the claim is checkable:
POUNCE 0.9.0: InfeasibleProblemDetected (detected by presolve: bound propagation)
objno 0 201
201 requires presolve to be enabled (presolve=yes);
it is off by default. A presolve-derived infeasibility is only reported when the
contradiction holds on the original box — one produced by presolve’s own
auxiliary elimination is re-checked after rollback and never certified.
One more route to 200: over-determined systems
An over-determined model — more equality rows than free variables, such as
x == 0.2 with x == 0.8 — cannot be solved at all: it fails a structural gate
before the first iteration. That used to be reported as
Not_Enough_Degrees_Of_Freedom (504, the failure band), which says “cannot
attempt this” for a model whose answer is already decided.
POUNCE now checks such a model for a bound-propagation contradiction on that
failure path and reports 200 when it finds one. This does not need
presolve=yes — nothing is transformed and no solve runs through the check — so
it is the one way to reach the infeasible band with the default options and no
iterations. A consistent over-determined system is unaffected and still
reports 504.
Because the solve provably cannot run here, this route measures constraint residuals against each row’s declared magnitude rather than an absolute tolerance, so the verdict does not change when every row is multiplied by a constant. Elsewhere — wherever a solve can run — an infeasibility smaller than the feasibility tolerance is still withheld, as described above.
Choosing an output format
| You want… | Use |
|---|---|
| AMPL / Pyomo to read the result back | the .sol file (default) |
| A structured, schema-versioned report for tooling | --json-output (see JSON Solve Report) |
| Just the console summary | --no-sol |
The .sol and JSON outputs are not exclusive — you can request both
in the same run.
JSON Solve Report
Pass --json-output PATH to write a structured solve report alongside
the regular console output:
pounce problem.nl --json-output result.json
pounce problem.nl --json-output result.json --json-detail full
The report carries everything an AMPL .sol file holds — status,
primal x, dual lambda, suffix blocks — plus FAIR-aligned
provenance metadata (Wilkinson et al. 2016, DOI
10.1038/sdata.2016.18) and,
optionally, the per-iteration trajectory.
Detail levels
| Level | Emits |
|---|---|
summary (default) | FAIR metadata, problem dimensions, final solution, aggregate statistics. |
full | The above plus the per-iteration trajectory (iter, objective, inf_pr, inf_du, mu, step norms, alphas, line-search trials) and sensitivity / suffix blocks. |
Choose summary for production logs and batch runs; full for
debugging — it is the JSON equivalent of upstream’s print_level=8.
Reproducibility: recorded environment overrides
Solve-affecting environment variables — the POUNCE_FERAL_* linear-solver
knobs and the legacy FERAL_PIVTOL / FERAL_PARALLEL — are captured into
fair_metadata.environment when set, so a run that differs because one was
exported in a shell profile says so instead of differing silently:
"environment": [
{ "name": "POUNCE_FERAL_PIVTOL", "value": "1e-6" }
]
The block is omitted entirely when no such variable is set (the common
case). Debug-only gates (POUNCE_DBG_*) are not captured. See
the schema reference
for the full field contract.
Schema stability
The schema is versioned (pounce.solve-report/v1) so downstream
tooling can pin against a major version:
- Adding fields is non-breaking — consumers must tolerate unknown fields.
- Removing or renaming a field bumps the major version (
v1→v2).
The Schema v1 Reference documents every field, the FAIR mapping, and the versioning policy in full.
POUNCE solve-report schema, v1
Schema tag: pounce.solve-report/v1
This document is the canonical reference for the JSON solve report
emitted by pounce --json-output PATH and pounce_sens --json-output PATH. The report carries everything an AMPL .sol file holds —
status, primal x, dual lambda, suffix blocks — plus FAIR-aligned
provenance metadata and (optionally) the per-iteration trajectory.
Implementation: the serde structs live in crates/pounce-solve-report/src/lib.rs (per-iteration IterRecord in crates/pounce-nlp/src/solve_statistics.rs); crates/pounce-cli/src/solve_report.rs wires them to the CLI.
Why a structured solve report?
Production NLP workflows often need to (a) capture which solve
produced which numbers for audit / reproducibility, (b) feed solver
output into downstream tooling (notebooks, dashboards, ML pipelines)
that don’t want to parse a free-form .sol file, and (c) compare
runs across versions of pounce. Both upstream Ipopt’s stdout summary
and AMPL’s .sol were designed for human consumption and AMPL’s
reader respectively — neither carries provenance metadata, neither is
schema-versioned, and neither is trivially machine-parseable across
ecosystems.
A versioned JSON schema with FAIR-aligned provenance solves all three.
FAIR alignment
The fair_metadata block maps onto the four FAIR principles
(Wilkinson et al. 2016, “The FAIR Guiding Principles for scientific
data management and stewardship”, Scientific Data 3, 160018, DOI
10.1038/sdata.2016.18; citation
verified via Crossref on 2026-05-14):
| Principle | Mapping in this schema |
|---|---|
| Findable | result_id (<unix_nanos>-<pid>, globally unique and time-ordered), created_at_iso, created_at_unix_nanos. |
| Accessible | Plain-text JSON on disk; no protocol gating; UTF-8. Same trust model as the .sol file. |
| Interoperable | Schema-versioned (pounce.solve-report/v1); JSON primitives only (no binary blobs); units documented per-field below; solution.status is the enum-variant string for cross-language consumption. |
| Reusable | solver (name + version + git commit + target triple), license, input (kind + path + size), and environment (solve-affecting env-var overrides in force) capture enough provenance to reproduce a solve. |
Versioning policy
schema is the version tag. Compatibility rules:
- Adding fields is non-breaking. Consumers MUST tolerate unknown fields. New optional fields land between versions; the major version doesn’t bump.
- Removing or renaming fields bumps the major version (
v1→v2). Consumers should pin against a major version (schema starts_with "pounce.solve-report/v1"). - Changing field semantics without a rename is forbidden. If semantics need to change, add a new field and deprecate the old.
The pre-1.0 phase of POUNCE itself does NOT relax this rule for the schema. Once a solve-report version ships, its field set is frozen even while the rest of the solver is under churn.
Top-level shape
{
"schema": "pounce.solve-report/v1",
"fair_metadata": { ... },
"problem": { ... },
"solution": { ... },
"statistics": { ... },
"iterations": [ ... ], // optional, omitted when empty
"linear_solver": { ... } // optional, omitted when backend did not report
}
Fields
schema (string, required)
Identifier for this schema version. Always
"pounce.solve-report/v1" for v1. Major-version bumps change the
prefix; minor / patch (additive) changes do not.
fair_metadata (object, required)
| Field | Type | Notes |
|---|---|---|
result_id | string | Format: <unix_nanos>-<process_id>. Monotonically ordered within a process, globally unique across processes. No external UUID library needed. |
created_at_iso | string | Solve start time as ISO-8601 UTC: YYYY-MM-DDTHH:MM:SS.sssZ. |
created_at_unix_nanos | integer | Same instant as Unix nanoseconds since 1970-01-01 UTC. Provided alongside the ISO string for consumers that prefer integer arithmetic. |
elapsed_seconds | float | Wallclock seconds the solve took (matches statistics.total_wallclock_time_secs modulo float precision). |
solver | object | See below. |
license | string | SPDX identifier. Always "EPL-2.0" for this version. |
input | object | See Input descriptor below. |
environment | array | omitted | Solve-affecting environment overrides in force. Omitted when none are set. See Environment overrides below. |
solver sub-object
| Field | Type | Notes |
|---|---|---|
name | string | Always "pounce". |
version | string | Crate version (e.g. "0.1.0"). Read from CARGO_PKG_VERSION at build time. |
git_commit | string | omitted | Build-time git revision. Omitted when the build environment did not set POUNCE_GIT_COMMIT (e.g. development builds). Set via POUNCE_GIT_COMMIT=$(git rev-parse HEAD) cargo build to populate. |
target_triple | string | Build target triple (e.g. "x86_64-apple-darwin"); falls back to "unknown" when Cargo did not expose TARGET at build time. |
Input descriptor (input)
Tagged enum keyed on kind. Possible shapes:
{ "kind": "nl-file", "path": "/path/to/foo.nl", "size_bytes": 366 }
{ "kind": "builtin", "name": "rosenbrock" }
{ "kind": "tnlp-direct" }
nl-file— the input came from.nlfile atpath.size_bytesis present when the file’s metadata is readable; consumers that want bit-exact provenance can hash the file themselves.builtin— the input was a built-in problem named byname(e.g.pounce --problem rosenbrock).tnlp-direct— used by library callers building a TNLP in-process without a.nlround-trip.
Environment overrides (environment)
An array of { "name", "value" } objects, one per solve-affecting
environment variable set in the process at report time:
"environment": [
{ "name": "POUNCE_FERAL_PIVTOL", "value": "1e-6" }
]
The whole array is omitted when none are set (the common case). Only the
variables that change pounce’s numerics or parallelism are captured — the
POUNCE_FERAL_* linear-solver knobs and the legacy FERAL_PIVTOL /
FERAL_PARALLEL. These alter the factorization and can otherwise silently
differ a run between two machines (e.g. one with POUNCE_FERAL_PIVTOL
exported in a shell profile) with nothing in the report saying so. The
POUNCE_DBG_* debug gates are deliberately not captured — they only
add diagnostic output and never change the result.
Presence records that the variable was set, not that it took effect: an
explicit OptionsList setting (e.g. feral_pivtol in an options file)
takes precedence over the env fallback. See
Options › Environment overrides
for the option each variable maps to.
problem (object, required)
Problem dimensions reported by the TNLP at get_nlp_info().
| Field | Type | Notes |
|---|---|---|
n_variables | integer | Number of primal variables. |
n_constraints | integer | Number of constraints (equalities + inequalities). |
n_objectives | integer | Number of objectives. The IPM uses objective 0; extras are read but ignored. |
minimize | boolean | true for minimization (the AMPL default). |
nnz_jac_g | integer | omitted | Number of declared non-zeros in the constraint Jacobian. |
nnz_h_lag | integer | omitted | Number of declared non-zeros in the lower triangle of the Lagrangian Hessian. |
solution (object, required)
| Field | Type | Notes |
|---|---|---|
status | string | ApplicationReturnStatus enum variant name verbatim (e.g. "SolveSucceeded", "MaximumIterationsExceeded"). |
solve_result_num | integer | AMPL-style solve-result code (Gay 2005, “Hooking Your Solver to AMPL” §5, p. 23 table): 0 = solved, 100-range = warning, 200-range = infeasible, 400-range = limit reached, 500-range = failure. |
objective | float | Final unscaled objective value. 0.0 (not NaN) when the solve never completed; check statistics.iteration_count > 0 to distinguish. |
x | array of float | empty | Primal vector, length problem.n_variables. Empty when the binary doesn’t capture the final iterate (currently: pounce on the newton_driver fast-path). Omitted from JSON when empty. |
lambda | array of float | empty | Constraint multipliers, length problem.n_constraints. Same omission convention as x. |
suffixes | array of object | empty | sIPOPT-style suffix blocks; emitted only at --json-detail full. See below. |
Suffix entries
{
"name": "sens_sol_state_1",
"target": "var",
"kind": "real",
"values": [0.576..., 0.378..., -0.046..., 4.5, 1.0]
}
| Field | Type | Notes |
|---|---|---|
name | string | AMPL suffix name. |
target | string | One of "var", "con", "obj", "problem". Matches AMPL’s Sufkind_* enum. |
kind | string | "real" or "int". Selects which payload array is populated. |
values | array of float | Dense values (length = target dimension). Present when kind = "real". |
int_values | array of integer | Present when kind = "int". |
statistics (object, required)
Projection of pounce_nlp::solve_statistics::SolveStatistics minus
the per-iteration history (which lives at the top level when present).
| Field | Type | Notes |
|---|---|---|
iteration_count | integer | Number of accepted outer iterations. |
final_objective | float | null | Unscaled. Matches solution.objective. null if never computed — see below. |
final_scaled_objective | float | null | Scaled by the IPM’s internal NLP scaling. Equal to final_objective when no scaling is in effect. null if never computed. |
final_dual_inf | float | null | ` |
final_constr_viol | float | null | ` |
final_compl | float | null | Max complementarity over the four bound blocks. null if never computed. |
final_kkt_error | float | null | Overall KKT error reported by the convergence check. null if never computed. |
nullvalues. The four residuals are produced by the convergence check at the end of a solve. A solve the solver refused — rejected during setup (NotEnoughDegreesOfFreedom,InvalidProblemDefinition), aborted, or caught by the batch panic handler — never reaches it, and these slots are emitted asnullrather than0.0. A zero there is indistinguishable from a perfect solve, and consumers acted on it: it was enough to makepounce.minimizereportsuccess=Truefor a problem the solver had declined to attempt.The two objective fields follow the same rule for the same reason:
0.0is an ordinary objective value, so it cannot signal “never evaluated”. They are seeded from the current iterate whenever one exists, so they arenullonly when the solve produced no point at all.Consumers should treat
nullas “not computed”, not as zero. pounce’s own readers map it to NaN, which fails closed against anyvalue <= toltest. |num_obj_evals| integer |eval_fcall count. | |num_constr_evals| integer |eval_gcall count. | |num_obj_grad_evals| integer |eval_grad_fcount. | |num_constr_jac_evals| integer |eval_jac_gcount. | |num_hess_evals| integer |eval_hcount. | |total_wallclock_time_secs| float | Wall time spent insideoptimize_*. | |restoration_calls| integer | Number of restoration-phase entries (pounce#12). | |restoration_inner_iters| integer | Cumulative inner-IPM iterations across all restoration calls. | |restoration_outer_iters| integer | Outer iterations that ran in restoration mode (R-line equivalents). | |restoration_wall_secs| float | Wall time spent insideperform_restoration. |
Eval counters (num_*_evals) populate only on the .nl-file path
because the pounce binary’s CountingTnlp wrapper tracks them.
Library callers using IpoptApplication::optimize_tnlp directly see
zeros there; the underlying counts are still available through
upstream’s IpoptCalculatedQuantities if needed.
iterations (array of object, optional)
Per-iteration trajectory. Emitted only at --json-detail full (when
IpoptApplication::enable_iter_history() was called). Omitted from
JSON entirely when empty.
Each row maps to one line of the upstream-formatted console iter table. Fields:
| Field | Type | Notes |
|---|---|---|
iter | integer | 0-based iteration index. |
objective | float | f(x_k) at the start of iter k (unscaled). |
inf_pr | float | Primal infeasibility ` |
inf_du | float | Dual infeasibility ` |
mu | float | Barrier parameter μ_k (not log10; consumers can take log10 if they want the console format). |
d_norm | float | ` |
regularization | float | Hessian regularization δ_w applied this iter; 0.0 when none was needed. |
alpha_dual | float | Dual step length. |
alpha_primal | float | Primal step length. |
alpha_primal_char | string (1 char) | Single-character tag (f, h, r, etc.) matching the alpha-primal column of upstream’s iter table. |
ls_trials | integer | Number of backtracking line-search trials this iter. |
linear_solver (object, optional)
Aggregate post-mortem from the symmetric-indefinite linear backend
that solved the KKT systems. Populated only when the backend
self-instruments (the default FERAL backend does; HSL MA57 and
custom backends plugged through set_linear_backend_factory do not).
Omitted from JSON when no backend reported.
| Field | Type | Notes |
|---|---|---|
solver_name | string | Backend identifier (e.g. "feral"). |
n_factors | integer | Total numeric factorizations performed. |
n_pattern_reuse | integer | Factor calls that reused the existing symbolic pattern. |
n_pattern_changes | integer | Factor calls that triggered a re-analysis. |
max_fill_ratio | float | omitted | Peak nnz(L) / nnz(A) observed across all factorizations. |
min_abs_pivot | float | omitted | Smallest absolute pivot magnitude seen across all factorizations (diagnostic for near-singularity). |
max_abs_pivot | float | omitted | Largest absolute pivot magnitude. |
last_inertia | [int, int, int] | omitted | (positive, negative, zero) inertia of the final factor. Should match (n, m, 0) at a regular KKT optimum. |
last_nnz_a | integer | omitted | Non-zero count of the assembled KKT matrix at the final factor. |
last_nnz_l | integer | omitted | Non-zero count of the L-factor at the final factor. |
Detail levels
The --json-detail LEVEL flag selects how much detail is emitted.
Levels map to verbosity in the same spirit as upstream’s print_level
(0 silent → 12 maximum debug):
| Level | What’s emitted | What’s omitted |
|---|---|---|
summary (default) | FAIR metadata, problem, solution scalars + arrays, aggregate statistics | iterations, solution.suffixes |
full | All of the above plus per-iteration trajectory and suffix blocks | nothing — full detail |
summary is the right choice for production logs and batch runs.
full is the debugging equivalent of upstream’s print_level=8.
Worked example
pounce_sens crates/pounce-cli/tests/fixtures/parametric.nl out.sol --json-output result.json --json-detail full produces (truncated for brevity):
{
"schema": "pounce.solve-report/v1",
"fair_metadata": {
"result_id": "1778777029606881000-76543",
"created_at_iso": "2026-05-14T16:43:49.606Z",
"created_at_unix_nanos": 1778777029606881000,
"elapsed_seconds": 0.011,
"solver": {
"name": "pounce",
"version": "0.1.0",
"target_triple": "x86_64-apple-darwin"
},
"license": "EPL-2.0",
"input": {
"kind": "nl-file",
"path": "crates/pounce-cli/tests/fixtures/parametric.nl",
"size_bytes": 366
}
},
"problem": { "n_variables": 5, "n_constraints": 4, "n_objectives": 1, "minimize": true },
"solution": {
"status": "SolveSucceeded",
"solve_result_num": 0,
"objective": 0.5510204081632656,
"x": [0.6326530575201161, 0.3877551079678144, 0.020408165487930466, 5.0, 1.0],
"lambda": [-0.16326530000405073, -0.28571431357898697, -0.16326530000405073, 0.18075803406303625],
"suffixes": [{
"name": "sens_sol_state_1",
"target": "var",
"kind": "real",
"values": [0.5765305974643309, 0.3775510440570709, -0.04591835847859835, 4.5, 1.0]
}]
},
"statistics": { "iteration_count": 9, "final_dual_inf": 2.89e-14, "...": "..." },
"iterations": [
{ "iter": 0, "objective": 0.0451, "inf_pr": 5.0, "inf_du": 0.407, "mu": 0.1,
"d_norm": 0.0, "regularization": 0.0, "alpha_dual": 0.0, "alpha_primal": 0.0,
"alpha_primal_char": " ", "ls_trials": 0 },
{ "iter": 1, "objective": 0.957, "inf_pr": 0.212, "...": "..." }
]
}
Consumer guidance
- Pin the major version. Check
schema.startswith("pounce.solve-report/v1")before consuming. - Tolerate unknown fields. New optional fields will land between minor versions of pounce. Use
serde(default)/ equivalent. - Distinguish “no solve” from “solve produced zero”. Pre-solve, scalar fields are
0.0(notNaN, because JSON has no NaN literal).statistics.iteration_count == 0is the signal that no solve occurred. solution.x/solution.lambdamay be empty. When the binary couldn’t capture the final iterate (currently: thepouncebinary on itsnewton_driverfast-path form=0, n≤1000problems), the arrays are empty and the keys are omitted from JSON entirely.pounce_sensalways populates them.
References
- Wilkinson et al. (2016). “The FAIR Guiding Principles for scientific data management and stewardship.” Scientific Data 3, 160018. DOI 10.1038/sdata.2016.18. (Verified via Crossref 2026-05-14.)
- Gay (2005). “Hooking Your Solver to AMPL.” https://ampl.com/REFS/hooking2.pdf. §5 (Returning Results to AMPL) for the
.solbaseline this schema is structured around. - SPDX license identifiers: https://spdx.org/licenses/.
Verifying Solutions
pounce verify <problem.nl> <claim.sol> [OPTIONS]
pounce verify independently checks that the solution in a .sol file
actually satisfies the constraints and bounds of a .nl problem. It
re-derives feasibility from the model itself — it does not trust the
.sol’s status line, and it does not rerun the solver. This makes it the
trust anchor when pounce is a tool an agent calls: the agent proposes a
solution, and a small, deterministic checker disposes.
Optimization is unusually well-suited to this because a solution is far
cheaper to verify than to produce: a claimed x* is just numbers, and
feasibility is a single constraint evaluation — g_l ≤ g(x*) ≤ g_u,
x_l ≤ x* ≤ x_u — O(nnz) work with no resolve and no dense linear
algebra.
Status. The
verifycheck itself — recompute feasibility against the canonical model, with a content-addressed receipt — is solid and ready to use; it needs no secrets and is the recommended default. The signing and remote-service trust layer layered on top of it (HMAC receipts, thesigner_service.pyreference, running the MCP server as a remote authority) is a proof of concept: it demonstrates the architecture but is not hardened for production. If you want to rely on the signed/remote path for real, see Status and hardening at the end for the checklist of what that would take.
What it defends against
In an agent workflow, three things can go wrong with “here is a solution”:
| Failure mode | How verify catches it |
|---|---|
Fabrication — a .sol that looks like a pounce result but wasn’t solved | invented numbers fail the residual check against the real model |
| Ignoring the solver — claiming success without actually solving | a consumer gates on the receipt’s verified: true + the problem hash, not on prose |
| Solving the wrong problem — dropping or relaxing a constraint to dodge infeasibility | the check runs against the canonical constraints/bounds, so a point that is only feasible for a relaxed model is rejected here |
The key design rule: always verify against the canonical problem, never
against whatever the agent claims it solved. If the agent loosened a bound
to manufacture feasibility, the returned x* still violates the canonical
bound, and verify reports it.
Output and exit codes
$ pounce verify gaslib40_steady.nl good.sol
pounce verify — independent solution check
problem : gaslib40_steady.nl (1694 vars, 1682 cons)
sha256:4bb435a3…
solution: good.sol
sha256:b77d9e7b…
claimed solve_result_num: 0
feasibility (tol 1.0e-6):
max constraint violation: 1.407e-12 at c[114] (value 1.4e-12, bounds [0, 0])
max bound violation : 9.775e-9 at x[24] (value 1.05, bounds [1.05, 2.0])
objective at x*: 1.2899875310e0
optimality (tol 1.0e-6, duals + bound multipliers supplied):
KKT stationarity residual (bound-projected) : 2.675e-3 (dual sign +1)
dual infeasibility (with z_L/z_U suffixes) : 6.248e-14
constraint complementarity (rows, |λ|·slack) : 0.000e0
bound complementarity (vars, |z|·slack) : 9.091e-10
VERDICT: VERIFIED — solution is feasible for the canonical problem
| Exit code | Meaning |
|---|---|
0 | VERIFIED — every violation within tolerance |
20 | REJECTED — a constraint or bound violation exceeds tolerance |
2 | usage / I/O error (missing file, malformed .sol, dimension mismatch) |
A consumer (CI step, agent harness, Makefile) gates on the exit code.
Options
| Flag | Default | Meaning |
|---|---|---|
--feas-tol <t> | 1e-6 | feasibility tolerance for constraints and bounds |
--opt-tol <t> | 1e-6 | stationarity tolerance for the optimality check |
--require-optimal | off | also fail (exit 20) if the KKT stationarity residual exceeds --opt-tol — the exact one when the .sol carries bound multipliers, otherwise the bound-projected one |
--json-output <path> | — | write a JSON verification receipt |
Feasibility is the gate; optimality is reported
By default only feasibility gates the exit code. Feasibility is rigorous and sign-convention-independent — it is the guarantee that matters when the claim is “this solution meets the constraints.”
When the .sol carries constraint duals, verify also reports a KKT
stationarity residual (the bound-projected “dual infeasibility”: the part
of ∇f + Jᵀλ that a valid sign-constrained bound multiplier cannot absorb)
and the complementarity residuals below. These are informational unless
you pass --require-optimal. The AMPL dual-sign convention can differ from
pounce’s, so verify computes the residual for both signs and reports the
better one plus the sign it used.
The two complementarity residuals
Two distinct quantities answer to “complementarity”, and at the same point
they can disagree by many orders of magnitude. verify names both by what
they range over, and never prints an unqualified complementarity residual:
| Line | Quantity | Needs |
|---|---|---|
constraint complementarity (rows, |λ|·slack) | max_i |λ_i| · dist(g_i, nearest finite side) over rows | the .sol’s constraint duals |
bound complementarity (vars, |z|·slack) | max_j max(|z_L·(x−x_L)|, |z_U·(x_U−x)|) over variables | the ipopt_zL_out / ipopt_zU_out suffixes |
The bound one is what a solver prints as Complementarity — Ipopt’s and
pounce’s own end-of-solve report alike. Do not compare a solver’s
Complementarity against the row line; they measure different things and
neither is wrong for what it measures.
When the .sol carries no ipopt_zL_out / ipopt_zU_out suffixes, bound
complementarity is reported as not checked, never as 0.0.
Bound multipliers sharpen the stationarity check
Without the suffixes, z_L and z_U are inferred from which bounds are
active, and the reported stationarity residual is bound-projected: it
projects out exactly the component a bound multiplier would carry, so it
reads 0.0 on a point whose bound multiplier is missing or wrong.
When the .sol does carry them (pounce always writes them; so does Ipopt’s
AMPL interface), verify additionally reports the exact dual
infeasibility ‖∇f + Jᵀλ − (zL_out + zU_out)‖∞, using the multipliers the
file actually claims. That number is directly comparable to the solver’s
Dual infeasibility, and it is what --require-optimal gates on whenever
it is available — the projected residual can only understate it.
The JSON receipt
--json-output writes a machine-readable receipt that content-addresses
both inputs by SHA-256 — so a downstream consumer can confirm exactly
which problem and which solution were checked:
{
"pounce_verify_version": 1,
"solver": "pounce 0.4.0",
"problem": { "path": "…", "sha256": "4bb435a3…", "n_vars": 1694, "n_cons": 1682 },
"solution": { "path": "…", "sha256": "b77d9e7b…", "duals_present": true },
"tolerances": { "feasibility": 1e-6, "optimality": 1e-6 },
"feasibility": {
"max_constraint_violation": 1.4e-12,
"worst_constraint": { "index": 114, "name": "c[114]", "value": 1.4e-12,
"lower": 0.0, "upper": 0.0, "violation": 1.4e-12 },
"max_bound_violation": 9.77e-9,
"worst_bound": { "index": 24, "name": "x[24]", … },
"feasible": true
},
"optimality": {
"available": true,
"stationarity_residual": 2.6e-3,
"stationarity_residual_with_bound_multipliers": 6.2e-14,
"constraint_complementarity_residual": 0.0,
"bound_complementarity_residual": 9.1e-10,
"bound_multipliers_present": true,
"complementarity_residual": 0.0,
"optimal": true, "note": "…"
},
"verdict": "VERIFIED",
"verified": true
}
bound_complementarity_residual and
stationarity_residual_with_bound_multipliers are null when the .sol
carries no ipopt_zL_out / ipopt_zU_out suffixes — null means not
checked, not zero. complementarity_residual is a deprecated alias of
constraint_complementarity_residual, kept so v1 consumers keep parsing;
its bare name is the one that invited the wrong comparison, so read the
qualified field instead.
A consumer should accept a solution iff:
verified == true, andproblem.sha256equals the SHA-256 of its own canonical.nl, and- (when signing is used) the signature validates — see below.
Checking the hash in step 2 is what closes the “solved the wrong problem” gap at the receipt layer: the receipt is only meaningful for the exact problem bytes it names.
The default: recompute, don’t trust a receipt
The strongest and simplest design uses no key and no signature at all:
the consumer runs pounce verify itself, against its own copy of the
canonical .nl.
# the consumer does this — not the agent
pounce verify ./canonical/problem.nl ./from-agent/claim.sol || reject
Because verification is keyless, deterministic, and cheap (O(nnz), no
resolve), the consumer can afford to just do it rather than trust someone
else’s word. In this design the agent is never in the trust path: it
hands over x*, and the consumer believes its own arithmetic. There is no
key to steal, so the question “what if the agent gets the key?” does not
arise. Forgery is impossible because nothing is being trusted on faith —
feasibility is decided by evaluating g(x*), not by matching fields in a
document.
This is the recommended default. Prefer it whenever the consumer can run
pounce verify (or call a verifier it controls). Reach for signatures only
when it genuinely cannot — see below.
Signed receipts — trust transport, conditional on key isolation
Signing addresses a narrower situation: the consumer won’t or can’t recompute — a remote or expensive verifier, or an audit log you want to trust later without re-solving — and instead wants to trust a receipt produced elsewhere. A signature lets that receipt be checked without redoing the work.
When the POUNCE_VERIFY_KEY environment variable is set (non-empty), the
receipt gains:
"signature_alg": "HMAC-SHA256",
"signed_fields": ["verify_version","nl_sha256","sol_sha256",
"n_vars","n_cons","feasible","verified","verdict"],
"signature": "5bdcc146bf60754e…"
The signature is HMAC-SHA256(key, preimage), where preimage is a
deliberately float-free byte string — only hex hashes, integer counts,
and the verdict — so any language reproduces it byte-for-byte without
float-formatting parity problems. The exact preimage is:
pounce-verify-receipt/v1
verify_version=1
nl_sha256=<hex>
sol_sha256=<hex>
n_vars=<int>
n_cons=<int>
feasible=<true|false>
verified=<true|false>
verdict=<VERIFIED|REJECTED>
(eight lines, \n-joined, with a trailing newline; booleans lowercase.)
A holder of the key recomputes the HMAC over this preimage and compares it
to signature.
What the signature does and does not guarantee
HMAC gives existential unforgeability under chosen-message attack — but only while the key stays secret. That single condition carries the entire weight of the scheme:
- A party without the key cannot mint a receipt that validates. It cannot produce a correct MAC for any message it has not already seen one for. This is the real guarantee.
- A party with the key can sign anything — including a hand-written
receipt that falsely says
verified: true. If the agent obtains the key, signing buys you nothing, and you must fall back to recomputation.
So the security reduces to a question that is not cryptographic: can the
agent reach the key? If the keyholder and the agent share a process, a
user account, an environment, or a filesystem, the agent simply reads it —
echo "$POUNCE_VERIFY_KEY", cat keyfile, or introspecting shared memory —
and the signature becomes theater. An agent with a shell on the same
user/host as the signer is not kept out by HMAC.
Setting
POUNCE_VERIFY_KEYfor apounce verifythat runs in the agent’s own environment is the weakest form and provides no real protection against that agent. It is only meaningful when the key lives behind a privilege boundary the agent cannot cross. See Out-of-process signing below.
Two further limits HMAC does not address, even with a perfectly isolated key:
- Replay. A previously, legitimately-signed receipt can be re-presented.
Here that is benign: a receipt attests “this specific
x*(bysol_sha256) is feasible for this specific problem (bynl_sha256),” and that statement stays true. Replay cannot manufacture a valid-looking attestation for a new (problem, solution) pair the signer never blessed. If you ever need freshness (“the work was done this time”), add a nonce/timestamp to the signed preimage — but for “does this solution meet the constraints,” freshness is irrelevant. - Wrong model. The signature says nothing about whether the
.nlencodes the problem you meant. That spec-ownership question must live outside the agent regardless of any crypto.verifycertifies feasibility against a given model; it does not certify that the model is the right one. Likewise, feasibility is fully checkable but global optimality is not — the stationarity residual certifies a first-order/KKT point, not a global minimum.
Out-of-process signing
For a signature to mean anything against a capable agent, the key must live in a separate trust boundary — a different process and user/container/ host that the agent cannot exec into or read. The signer exposes “verify-and-sign” as an RPC; the key never enters the agent’s address space.
┌─ agent boundary (no key) ────────────┐ ┌─ signer boundary (holds key) ─────┐
│ agent proposes x* │ │ POUNCE_VERIFY_KEY in THIS env only │
│ │ │ RPC │ 1. run `pounce verify` on the │
│ └── POST /verify {nl, sol} ─────────┼─────▶│ CANONICAL .nl (+ the key) │
│ │ │ 2. binary signs the receipt │
│ signed receipt ◀────────────────────┼──────┤ 3. return receipt JSON │
└───────────────────────────────────────┘ └────────────────────────────────────┘
│
└── relays receipt to the consumer
consumer: accept iff verified==true ∧ problem.sha256==canonical ∧ signature valid
What each party can do under this split:
| Party | Has key? | Can forge a verdict? |
|---|---|---|
| Agent (proposer) | no | no — it can only ask the signer to verify a real x* |
| Signer service | yes | yes, but it is the trusted authority — that’s the point |
| Consumer | shares key or recomputes | detects any tampering / can verify independently |
The boundary is only real if the agent cannot run code as the signer’s user or on its host. Running the signer as a separate user, container, or host (or behind a KMS/HSM that signs without exposing the key) is what turns “signed” from theater into a guarantee. An MCP server is already a separate process from the model, which helps — but only achieves isolation if the agent also lacks a shell on the same user/host.
A minimal reference signer is in
studio/mcp/examples/signer_service.py:
a stdlib HTTP service that holds the key in its own environment, shells out
to pounce verify, and returns the signed receipt. The agent calls it; the
agent’s environment never contains the key.
Use in an agent workflow
Putting it together — recompute by default, sign only to transport trust:
agent ── proposes x* ──▶ consumer / verifier-it-controls
1. pin + hash the canonical .nl
2. pounce verify .nl .sol (against the CANONICAL model)
◀─ accept iff verified==true ∧ problem.sha256==canonical
When the verifier must be remote and the consumer won’t recompute, insert an
out-of-process signer (above) and add ∧ signature valid to the consumer’s
acceptance test — remembering that the last clause is only as strong as the
signer’s key isolation.
The pounce-studio MCP server exposes verify_solution so an agent can
request a check but cannot fake its result. Deploy that server as a
distinct boundary from the agent (separate user/container) for the signature
to carry weight; otherwise rely on the consumer recomputing.
Status and hardening
What is ready to use as-is:
- The feasibility check (
pounce verify, and the consumer-recomputes pattern). It is deterministic, keyless, content-addressed, and rigorous — this is the part to build on.
What is a proof of concept — demonstrates the shape, not hardened:
- HMAC signing via
POUNCE_VERIFY_KEY, thesigner_service.pyreference, and treating a remotely-deployed MCP server as a signing authority.
If you ever want to depend on the signed/remote path in production, these are the gaps to close. None are implemented here.
Key management
- Don’t keep the key in a plain environment variable or file. Use a KMS/HSM (or sealed secret) that signs without exposing the key to the process — then even a compromised signer host can’t exfiltrate it.
- Add key rotation and a key id in the receipt (
kid) so a consumer knows which key to check against and old receipts stay verifiable across rotations. - Consider an asymmetric scheme (e.g. Ed25519) instead of HMAC when more than one party must verify without also being able to sign — HMAC’s symmetric key means every verifier is also a forger. Public-key signatures give public verifiability with a single private signer.
Transport / service (the moment it leaves stdio)
- TLS on the endpoint; never plaintext for a service that holds a key.
- Authn/authz — bearer token or OAuth on every request (MCP’s HTTP transport supports this). An unauthenticated endpoint that runs solves and shells out is effectively remote code execution.
- Resource limits — request-size caps, solve timeouts (there is a
timeout_seconds, but also wall/CPU/memory limits at the OS level), concurrency caps, and rate limiting. - Sandbox the solve — treat every
.nlas untrusted input. Parsing and evaluating an arbitrary model is attacker-controlled computation; run it in a locked-down container/user with no network and a constrained filesystem.
Input handling
- Over a network the path-based tools (
nl_file/sol_file) assume a shared filesystem. Prefer content upload (the server receives and hashes the exact.nl/.solbytes) so there’s no path-traversal surface and the receipt binds what was actually sent. The reference signer’sPOUNCE_SIGNER_ROOTallowlist is a stopgap, not a substitute.
Freshness / replay
- The current preimage has no nonce or timestamp, so a signed receipt is
replayable. That is benign for “is this
x*feasible” (a timeless fact), but if a consumer needs “this was checked recently” or “in response to my request,” add a nonce/timestamp (and a receipt expiry) to the signed preimage and bump thepounce-verify-receiptversion.
Auditability
- Log every verification (problem hash, solution hash, verdict, key id, caller identity) to an append-only store, so a disputed result can be reconstructed. Keep the key out of the logs.
Standing non-goals (true regardless of hardening)
verifycertifies feasibility against a given model — it does not certify the model is the right one. Model/spec correctness must be owned outside the agent.- Feasibility is fully checkable; global optimality is not. The stationarity residual certifies a first-order/KKT point, not a global minimum.
Sensitivity Analysis
POUNCE includes a parametric sensitivity capability compatible with
upstream Ipopt’s contrib/sIPOPT/ (Pirnay, López-Negrete & Biegler
2012, DOI
10.1007/s12532-012-0043-2).
It computes the first-order change in the optimal primal solution with
respect to a problem parameter, reusing the KKT factorization from the
converged solve. Four entry points cover the common workflows.
AMPL CLI
The main pounce driver auto-detects the sIPOPT suffixes
(sens_state_1, sens_state_value_1, sens_init_constr) in an input
.nl, runs a post-optimal sensitivity step after the solve, and
writes the perturbed primal back as a sens_sol_state_1 suffix — no
separate binary or flag needed:
pounce problem.nl # writes problem.sol
pounce problem.nl out.sol --json-output result.json --json-detail full
pounce_sens is retained as a thin backward-compatibility alias:
pounce_sens in.nl out.sol is identical to pounce in.nl out.sol, so
existing AMPL / solver scripts keep working unchanged.
Related flags:
--sens-boundcheck/--sens-bound-eps EPS— clamp the perturbed primalx* + Δxonto the declared[x_l, x_u]box.--compute-red-hessian/--rh-eigendecomp— compute the reduced Hessian (and its eigendecomposition) over the variables tagged by thered_hessianinteger var-suffix.
Rust library
Reach the sensitivity path through the pounce-rs facade, with the
sensitivity feature on:
[dependencies]
pounce-rs = { version = "0.9", features = ["sensitivity"] }
SensSolve is a builder that wraps the on_converged callback
plumbing into a single call:
#![allow(unused)]
fn main() {
use pounce_rs::sensitivity::SensSolve;
let result = SensSolve::new(vec![2, 3])
.with_deltas(vec![0.05, 0.0])
.with_reduced_hessian()
.run(&mut app, tnlp);
// result.dx, result.reduced_hessian, result.status
}
with_reduced_hessian_eigen() adds the eigendecomposition;
with_boundcheck(eps) enables the bound projection.
Eigenvector sign convention
Every eigendecomposition POUNCE hands back — the reduced Hessian’s
here and through the CLI and Python wrappers, the QP one from
QpSensitivity.reduced_hessian, and covariance().eigen() /
information().eigen() in pyomo-pounce — returns sign-pinned
eigenvectors: the largest-magnitude component of each column is
positive, ties broken by the earliest row. v and -v are equally
valid eigenvectors, so without a convention the direction you read
back depends on the arithmetic that produced it and is not
reproducible across builds or machines.
The sign is all that is pinned. A repeated eigenvalue leaves the basis within its eigenspace arbitrary — any rotation of those columns diagonalizes equally well — so read a degenerate block as a subspace, not column by column.
Python
solve_with_sens exposes the same capability from the
cyipopt-compatible Python wrapper:
# pin_constraint_indices is required; pass deltas=..., compute_reduced_hessian=True,
# or both. Returns (x, info) — sensitivity outputs live in the info dict.
x, info = prob.solve_with_sens(x0, pin_constraint_indices=[2, 3],
deltas=[0.05, 0.0], sens_boundcheck=True)
# info["dx"], info["reduced_hessian"], info["reduced_hessian_eigenvalues"], ...
compute_reduced_hessian=True returns the reduced Hessian in
info["reduced_hessian"]; rh_eigendecomp=True adds its
eigendecomposition; sens_bound_eps=… tunes the bound projection. See
python/notebooks/04_sensitivity.ipynb
for a walkthrough.
Pyomo
pyomo_pounce wraps the same machinery in a declare-then-query
interface: flag the parameters that matter while building the model
(no perturbed values required), solve normally, then ask for
derivatives. Parameters are declared with declare_sens_param
(mutable Param or fixed Var, scalar or indexed); when declarations
are present, SolverFactory("pounce").solve(m) runs in-process and
keeps the converged KKT factorization, so every query afterwards is a
single backsolve.
import pyomo.environ as pyo
import pyomo_pounce
from pyomo_pounce import declare_sens_param, gradient, estimate
m.p = pyo.Param(initialize=2.0, mutable=True)
declare_sens_param(m.p) # a flag, not a perturbation
pyo.SolverFactory("pounce").solve(m) # ordinary solve
gradient(m.x, wrt=m.p) # dx*/dp (float)
gradient(m.con, wrt=m.p) # d(multiplier of con)/dp
G = gradient(m.z, wrt=m.r) # containers -> Gradient object
G[m.z[1], m.r[2]]; G.to_dataframe() # element access / full Jacobian
estimate(m, [(m.p, 2.5)]) # first-order solution estimate at
# new values, clamped to bounds
gradient returns exact first-order derivatives (unit-perturbation
backsolves, no finite differencing); estimate combines the stored
derivative columns for arbitrary perturbed values after the fact. Its
perturbation is measured from the solve point, not the Param’s current
value, so writing a measurement into the Param before asking (the
receding-horizon pattern) does not change the answer. It also
warns when the linear step leaves the variable bounds (a single-pass
projection analogous to the CLI’s --sens-boundcheck) — with one
exception, a bound written on a declared Param, covered in
Declared Params in variable bounds
below. Multiplier sensitivities are available for equality constraints.
Models without declarations solve through the ordinary AMPL/CLI path,
unchanged. See
python/notebooks/25_pyomo_sensitivity.ipynb
for a worked optimal-control example (initial conditions as
parameters; the first-move gradient IS the NMPC feedback gain).
Declared Params in variable bounds
A limit is often most naturally written as a bound rather than a constraint:
m.u_max = pyo.Param(initialize=1.0, mutable=True)
declare_sens_param(m.u_max)
m.u = pyo.Var(m.t, bounds=(0, m.u_max)) # the cap, as a bound
pyomo.contrib.sensitivity_toolbox, which supplies the expression
surgery underneath, substitutes declared Params in constraint
expressions only. A Param left in a bound is written to the .nl file
as a constant at its pre-perturbation value, so the bound never moves
and gradient(m.u[t], wrt=m.u_max) reads exactly 0.0 — a wrong
answer that is indistinguishable from a legitimate insensitivity.
POUNCE rewrites such a bound as a constraint over the substituted
variable before the solve, so both spellings of the same limit give the
same derivative. Expression bounds work too, e.g.
bounds=(0, 2 * m.p + 1). Two kinds of variable are deliberately left
alone: fixed Vars, whose bounds the solver never enforces, and Vars
on deactivated Blocks.
This is a deliberate divergence from sensitivity_calculation, which
still reports zero for the same model. Four things follow from it:
- The bound is dropped on the clone that is solved.
m.x.ubreadsNonethere and the NL row carries the reader’s no-bound sentinel1e19— finite, so anisinf()test will not catch it. The model you wrote is never modified. estimate()does not clamp against a rewritten bound, and raises no clamp warning for it. That is correct rather than an oversight: the bound now moves with the perturbation, so the linear step already respects it to first order.covariance()’s bound-active projection still fires. The value the bound held at the solve point is recorded and read back for the activity test, so adeclare_fittedvariable capped by a declared Param is still projected and still warns.- It costs a row. A simple bound is handled directly in the barrier; a general inequality costs a slack and a Jacobian row. A model with many Param-dependent bounds trades roughly one row per bound. Only models that write a bound in terms of a declared Param pay this.
Solver options and warm starts
Solver options reach the in-process path the same two ways they reach an
ordinary solve: factory-level (SolverFactory("pounce", options={...})
or solver.options[...]) and per-call (solve(m, options={...})), with
the per-call mapping winning on conflict. Everything the CLI accepts
works here: tolerances, max_iter, scaling, warm-start knobs.
With warm_start_init_point=yes (Python True works too) among the
options, the initial multipliers come from the model’s suffixes, the
same ones the ASL path uses: dual for equality multipliers,
ipopt_zL_in / ipopt_zU_in for bound multipliers, matched by
component name (a constraint rewritten by the declared-parameter
surgery is reached through its internal alias). Sign conventions are
handled: dual holds the AMPL marginal and ipopt_zU_in Ipopt’s
negative-at-upper value, and both are translated to the solver’s
internal conventions on the way in.
One deliberate improvement over the ASL path: entries you do not
supply take the solver’s own default initialization rather than zero.
Through a dense ASL array an absent entry is indistinguishable from a
zero multiplier, and a zero bound multiplier on an active bound is a
contradictory KKT certificate the solver must first recover from. A
suffix knows which entries exist, so an explicit zero is honored
(then floored at warm_start_mult_bound_push, exactly as a
round-tripped inactive multiplier is) and absence means “initialize as
you normally would”: the solver’s own bound_mult_init_val for bound
multipliers, and for equality duals the warm path’s 0, which is not
the cold path’s least-squares estimate. Seed everything from a prior
solve and the two paths behave identically; seed partially and the
in-process path degrades gracefully.
Watching the solve (tee=True)
SolverFactory("pounce").solve(m, tee=True) streams the solver’s full
Ipopt-style log — banner, problem statistics, iteration table, and
end-of-run summary — live to standard output, including inside a Jupyter
notebook cell. The log is emitted by the engine itself (the same blocks the
pounce CLI prints), so the in-process path just tails it: a long solve
shows its iteration table as it runs rather than as one block at the end.
Without tee=True the solve is silent, matching the Pyomo convention.
Parameter covariance and identifiability
For a parameter-estimation model whose objective is a plain sum of squared residuals, the factorization from ONE ordinary solve yields the asymptotic covariance of the fitted parameters. Declare the fitted variables (they stay free) and the residual container while building the model, solve, and ask:
from pyomo_pounce import covariance, declare_fitted, declare_residual
m.A = pyo.Var(); m.k = pyo.Var() # the fitted parameters, free
declare_fitted(m.A, m.k)
m.r = pyo.Var(m.I) # residuals, one per data point
m.res = pyo.Constraint(m.I, rule=...) # r[i] == y[i] - model(A, k, t[i])
declare_residual(m.r)
m.obj = pyo.Objective(expr=sum(m.r[i]**2 for i in m.I))
pyo.SolverFactory("pounce").solve(m) # one solve
cov = covariance(m) # no further information needed
cov[m.A, m.k] # covariance entry (either order)
cov.std_err[m.k] # standard error of one parameter
cov.correlation[m.A, m.k] # correlation matrix entry
cov.matrix # dense numpy array, ordered like cov.params
w, V = cov.eigen() # eigendecomposition, for identifiability
The recipe: the parameter block of the inverse KKT matrix, one
backsolve per parameter against the held factor, equals the inverse
reduced Hessian of the eliminated problem, and for a sum-of-squares
objective cov = 2 sigma^2 (K^-1)_pp. The factor 2 belongs to the
unscaled sum of squares (a Gaussian negative log-likelihood objective,
SSR / (2 sigma^2), would drop it). The scaling is pinned by test
against the analytical linear-regression covariance
sigma^2 inv(X^T X) (pyomo-pounce/tests/test_covariance.py).
The noise variance comes from, in order of precedence: sigma_sq=
(known measurement variance); the declared residuals (estimated as
SSR / (n - n_params), with both numbers derived from the container);
or the n_data= fallback for models without explicit residuals, whose
SSR is the objective value at the solve — like estimate()’s
baseline, writing into the model afterwards (a measurement, a warm
start for the next horizon) does not move the answer. The
solve warns if the declared residuals do not reproduce the objective
value (weights or regularization terms would silently corrupt the
estimate).
Groups. declare_residual(m.r_conc, group="conc") partitions
residuals into noise groups by arbitrary user strings: containers
sharing a group (or all ungrouped containers) pool into one estimated
variance; distinct groups get their own (cov.sigma_sq becomes a
dict), and the covariance switches to the heteroscedastic sandwich
form, whose per-group pieces come from the same backsolves. When
groups genuinely differ, weighting the objective itself (dividing each
group’s residuals by its sigma) is the statistically efficient fix;
the sandwich is the truthful report on the unweighted fit.
cov.eigen() returns ascending eigenvalues and matching eigenvectors.
An eigenvalue much larger than the rest flags a poorly identified
problem: its eigenvector is the parameter combination the data cannot
pin down, and the corresponding cov.correlation entries approach
+/-1. The returned signs follow the project-wide
eigenvector sign convention —
the largest-magnitude component of each eigenvector is positive,
ties broken by the earliest position in cov.params — so the
direction reproduces across machines instead of coming back as
whatever LAPACK’s build chose. information().eigen() is the same.
covariance warns when the held factor carries
inertia-correction perturbations (typically an exactly unidentifiable
parameterization) and when the covariance diagonal comes out negative
(not a least-squares minimum).
Bound and constraint activity is classified from the solve’s own barrier geometry, not a slack threshold. A STRONGLY ACTIVE bound pins its parameter: zero variance, correlations 0, conditional on the bound, warned. A WEAKLY ACTIVE bound (slack and multiplier vanish together) is KEPT at its full finite variance, corrected for the barrier weight the held factor carries; AMBIGUOUS (loosely converged) and UNIDENTIFIED (curvature below the model’s own noise scale) stay in the free block, each with a warning. A strongly active inequality CONSTRAINT over the fitted parameters pins a combination rather than a coordinate: the matrix is projected on the constraint’s null space, going singular by one per binding row, and the warning names the constraint, the pinned combination, and its conditional information. The same limit written as a bound or as a row returns the same matrix. A binding row that reaches the fitted parameters through free eliminated variables cannot be represented by a restricted normal and is kept unprojected with an explicit warning.
To classify honestly, the declaration-triggered solve sets
bound_relax_factor = 0 (slacks must measure distance to your own
bounds). This applies to every solve routed through the sensitivity
session, not only ones that end in covariance(). If you need the
relaxation, pass bound_relax_factor explicitly in options=: your
value wins, and covariance() then refuses with a clear error rather
than classifying against shifted slacks.
Relation to pounce.curve_fit. This uses the same
scale-and-invert-the-reduced-Hessian recipe as
pounce.curve_fit — both read a reduced-Hessian
block from the held KKT factor and scale it by 2 sigma^2 with
sigma^2 = SSR / (n - p) — but with one substantive difference for
nonlinear models: curve_fit factors the Gauss-Newton Hessian
(pcov = 2 sigma^2 (J^T J)^-1, the expected-information / scipy /
pycse.nlinfit convention, always positive semidefinite), while
covariance() here feeds the exact Lagrangian Hessian through the
.nl bridge, so it reports the observed-information covariance —
the full reduced Hessian including the residual-curvature term that
Gauss-Newton drops. The two are identical for linear models and in the
small-residual / large-n limit, and differ by O(residual x model curvature) otherwise (a few percent on a strongly-curved fit). Neither
is uniquely “correct”: Gauss-Newton is the conventional, robust default
(it cannot produce a negative variance); observed information is the
honest local curvature of the objective you actually solved (Efron &
Hinkley 1978) but can go indefinite — which is what the negative-variance
warning above is telling you. covariance() offers both: the default
hessian="lagrangian" inverts the exact reduced Hessian of the
Lagrangian, and covariance(m, hessian="gauss-newton") rebuilds the
expected-information form from the residual Jacobian, recovered from
the same backsolves at no extra solve (declared residuals required).
Reach for it when the numbers must match scipy/nls, when
covariance() warns about a negative diagonal, or when the covariance
must stay positive semidefinite by construction, e.g. feeding an
arrival-cost update in moving horizon estimation.
The other difference is the input surface.
curve_fit(f, xdata, ydata, ...) is the batteries-included fitter for a
callable model f(x, *params) and data arrays: it chooses a starting
point, offers robust losses, per-point sigma weights, confidence
intervals, prediction bands, dpopt/ddata, and out-of-core streaming,
and it projects the covariance onto the active-constraint nullspace
when a parameter sits on a bound. covariance() is the post-solve
primitive for a model you have already written in Pyomo — residuals
as constraints, arbitrary surrounding structure — where you want the
covariance of the fit as posed without re-expressing it as
f(x, *params). Use curve_fit when the fit is naturally a
model-plus-data call; use covariance() to interrogate an existing
Pyomo estimation model. Both project a bound-active fitted parameter
onto the active-constraint nullspace: covariance() reports the
covariance conditional on the active bound (zero variance in the
pinned direction, computed by inverting the free block of the
information matrix) and still warns, since boundary asymptotics are
nonstandard. Only variable bounds on the fitted parameters themselves
are detected; a parameter held at the same value by an active
constraint row is treated as free
(#362). A bound
rewritten into a constraint by the rule in
Declared Params in variable bounds
is the one exception: the value it held at the solve point is recorded,
so it is still detected and still projected.
Relation to pyomo.contrib.parmest. parmest is an estimation
workflow: multi-experiment data management, bootstrap resampling, and
likelihood-ratio confidence regions, at the price of restructuring the
problem into its experiment framework, with covariance computed by
finite differences or an ipopt re-solve. covariance() is a
post-solve primitive: the model as written, one declaration per
component, the asymptotic covariance and identifiability diagnostics
from the factorization the solve already produced. Use parmest for
multi-experiment campaigns and non-asymptotic intervals; use this to
interrogate the fit you already have.
See
python/notebooks/26_parameter_covariance.ipynb
for a worked example with a Monte Carlo validated confidence ellipse
and an identifiability diagnosis.
Activity classification
Which bounds and constraint rows are actually holding the solution
is a question the converged iterate answers only ambiguously: at a
weakly active bound the slack and its multiplier are both O(√μ), so
no fixed threshold on either one alone separates “just touching” from
“not binding”. Solver.classify_activity() keys on the ratio of
barrier curvature to the model’s own curvature instead, which is
O(μ), O(1) and O(1/μ) in the three regimes:
solver = pounce.Solver(problem) # problem.add_option("bound_relax_factor", 0.0)
x, info = solver.solve(x0=x0)
rep = solver.classify_activity()
rep["var_status"] # ["inactive", "unbounded", "fixed", "strongly_active"]
rep["row_status"] # ["equality", "strongly_active"]
rep["var_ratio"] # the ratio behind each call (NaN where nothing was classified)
rep["mu"] # the barrier parameter the calls were made at
Statuses are inactive, weakly_active, strongly_active,
ambiguous (the ratio fell in a gap where this μ cannot decide —
re-solve tighter), and unidentified (the curvature is below noise
scale, so the question does not arise). unbounded, fixed and
equality mark entries with no barrier geometry to classify.
Both arrays are indexed in user space: var_* follows your n
variables and row_* your m constraints, in your order. A variable
that fixed_variable_treatment = make_parameter removed from the
solve (lb == ub) reports fixed at its own index rather than
shifting everything after it.
Two per-entry flags report on the assumptions rather than the
geometry: off_central_path (s·z differs from μ by more than 10×
on some side) and contaminated (classified inactive yet carrying
barrier curvature well above the O(μ) an inactive bound should have
— typically a bound that sits close enough to the optimum to bend it).
Inequality rows classify through the same rule, via the curvature
along the constraint normal. That is the point of classifying rows at
all: move a bound off a variable and onto a row and the activity
disappears from the bound-multiplier view entirely, while any
covariance or identifiability heuristic keyed on z alone silently
stops seeing it
(#362).
The call requires the solve to have run with bound_relax_factor=0
(the Ipopt default is 1e-8) and raises ValueError otherwise:
relaxed bounds shift the very slacks the classifier reads. The guard
tests the value that solve ran under, so setting the option after the
fact does not change the answer — set it on the Problem and solve
again.
The information matrix
information(model) is the un-inverted sibling of covariance(): the
reduced Hessian over the declared fitted block, from the same single
solve, in natural units with no sigma^2 anywhere. For a homoscedastic
Lagrangian fit, covariance() equals 2*sigma^2*inv(information()) on
the free block. hessian= selects the observed ("lagrangian",
default) or expected ("gauss-newton") form exactly as in
covariance().
The Lagrangian form is built by tangent recovery against the held
factorization rather than by inverting the covariance back: the
K-inverse columns’ x-blocks are T*M, so T = Zx*inv(M) exactly and
R = T'HT with the exact Lagrangian Hessian. The barrier weight
cancels multiplicatively, so equality and variable-bound activity
carries machine precision at any barrier parameter, including on
pinned parameters where a subtract-the-barrier route loses
log10(Sigma/q) digits. A binding inequality row is the one
exception: it couples through its slack barrier and leaves ~1e-6
relative residue at practical barrier parameters.
Membership and warnings follow covariance(). One disposition is
opposite by design: a strongly active (pinned) parameter’s entry is
S, the reduction onto the pinned set, NOT a zero row — zero
information is the opposite of what a pinned parameter carries —
conditional on the rest of the pinned set, with zero cross blocks to
the free parameters. Binding constraint rows project the free block on
both sides (the pseudo-inverse of the projected covariance). An
indefinite Lagrangian block is returned as computed with a warning
naming Gauss-Newton as the PSD alternative: refusing would withhold
the finding that the point is not a minimum or the model is
over-parameterized. eigen() reads identifiability directly: a
near-zero eigenvalue is a direction the data does not inform; its
eigenvector’s sign follows the project-wide
convention.
Choosing the block: wrt=
Both accessors take wrt= to reduce onto any block of the solve’s
variables off the held factor, post-solve; the declared fitted block is
the default, so omitting it is exactly the prior behavior. Accepted
forms: a Var (scalar or indexed, every member), an indexed slice
(m.x[2, :]), a (Var, iterable) pair, data objects, or a list mixing
these.
cov = covariance(m) # the fitted block, as before
cov_a = covariance(m, wrt=[m.a]) # one parameter's marginal
band = covariance(m, wrt=m.r) # a predicted trajectory
info_a = information(m, wrt=[m.a])
Each call re-reduces onto its own argument, so one solve serves as many blocks as are asked about, and each block gets its MARGINAL: everything outside it is profiled out, not held fixed. Sigma estimation always divides by the fit’s own degrees of freedom (a property of the solve, not of the question being asked), so a sub-block’s numbers agree exactly with the corresponding entries of the default answer.
A rank-deficient block, one with more coordinates than the fit has
degrees of freedom or with linearly dependent coordinates (a
duplicated design point), is the trajectory-band case: covariance()
returns its (rank-deficient) marginal, 2 sigma^2 M, the confidence
band on the fitted trajectory (add the observation noise for a
prediction band), with the membership handling bypassed, and
information() raises an error pointing to covariance(), since such
a block carries no information matrix. For information(),
a block that parameterizes the constraint manifold (size equal to the
degrees of freedom) gets the exact tangent construction; a sub-block of
the fitted set gets its marginal as a Schur complement of the exact
tangent R over the fitted block (never inverting a covariance, so a
pinned member costs no digits); other blocks reduce off the held factor
with the item-1 corrections, which is benign for free coordinates.
One exception is returned rather than hidden: a strongly active
variable OUTSIDE the block is not deleted from the factor, so the
block’s numbers are the values conditional on that bound, not the
marginal over it. The result carries the list as .conditioned_on
(empty when there is none); inside-block activity is membership, not
conditioning, and is handled as before. The list is decided by the
same classification the block members get, applied per candidate as a
singleton block, so it is scale-invariant; only near-bound variables
pay the extra backsolve.
Keeping and releasing the factor: retain_kkt(), release_kkt()
The solve factors the KKT matrix to solve the NLP; the only question is
whether that factor is kept for post-solve queries. Any declaration
keeps it. retain_kkt(model) keeps it with no declaration at all,
which is what wrt= queries with nothing declared need: the MHE case,
where the arrival state and the parameters are each queried by wrt=
and neither is THE fitted set. It defaults off, so a solve with no
sensitivity pays nothing.
retain_kkt(m)
SolverFactory("pounce").solve(m)
arrival = covariance(m, sigma_sq=s2, wrt=m.x[:, t0])
params = information(m, wrt=[m.k1, m.k2])
release_kkt(m) # done asking: give the memory back now
| setup | factor kept | covariance(model) | covariance(model, wrt=T) |
|---|---|---|---|
| nothing | no | error | error |
declare_fitted(S) | yes | over S | over T |
retain_kkt() only | yes | error, no default | over T |
retain_kkt() + declare_fitted(S) | yes | over S | over T |
The retention policy in one place: the factor is kept if anything is
declared or retain_kkt() was called, and a Covariance or
Information result whose lazy conditioned_on has not been read
keeps the session alive through its pending computation until first
access. release_kkt(model) is the exit: it drops the model’s hold
on the factor immediately, freeing the memory, while declarations and
the retain flag still apply to the next solve. Release drops the
model’s hold, not a result’s: a Covariance or Information with a
pending conditioned_on, and a Gradient (which reads the factor on
every lookup), each hold their own reference, so they keep working
across the release and keep the factor in memory until they are
discarded. Noise is a separate question: retain_kkt() keeps the
factor, not a noise model, and with nothing declared fitted the
degrees of freedom for a noise ESTIMATE are unknown, so
covariance() under retain-only needs sigma_sq=; the estimation
routes (declared residuals, n_data=) raise an error saying so.
Like any declaration, retain_kkt() routes the solve through the
in-process sensitivity path, whose solve() surface is not
keyword-identical to the ordinary subprocess path (for example,
load_solutions=False is not honored there). Adding it to an
existing script changes how the solve runs, not just what is kept.
Units and NLP scaling
All sensitivity outputs are in natural (unscaled) units. The IPM
holds its converged KKT factor in an internally scaled space whenever
NLP scaling is active (the default nlp_scaling_method = "gradient-based" fires when an objective gradient or constraint row
exceeds nlp_scaling_max_gradient = 100 at the starting point);
pounce undoes that scaling in every held-factor back-solve, so dx,
kkt_solve, and the reduced Hessian are independent of how the
problem was scaled internally
(#128).
That covers user scaling too, on all three of its axes. A
per-variable scaling_factor is applied as a change of variables
x̃ = d ⊙ x below the algorithm, so the held factor is the scaled
problem’s; the factors are carried into the same translation, and
every accessor answers in your units
(#486). The factors a
solve ran under are readable back from Solver.nlp_scaling["x_scaling"]
(Python) / Solver::variable_scaling (Rust) — diagnostic rather than a
correction to apply, since the outputs already carry it.
classify_activity() is scale-invariant for the same reason, and
mostly by construction rather than by undoing anything: its ratios are
formed so that rescaling a constraint row or the objective leaves them
fixed. Writing a constraint as 1000·x ≥ 0 instead of x ≥ 0 does
not move a status, and neither does the solver’s own per-row
d_scale. A change of variables is the one case the ratios do not
absorb on their own — the identification floor is a single number
shared across entries, so a non-uniform d would move entries
across it — and there the factors are divided out of the geometry
before anything is classified, which keeps a status from depending on
the conditioning you asked for. The values the report exports follow
the natural-units contract like everything else: var_sigma and
row_sigma are the barrier diagonals in the model’s own units,
row_normal(j) is the constraint gradient with the solver’s per-row
scale divided out, and hessian_vec(v) is the exact Lagrangian
Hessian times a user-space vector with the objective scale divided
out; classification happens on the scaled quantities internally, the
report never shows them.
Variable indices are user-space, factor rows are not. Everything
the sensitivity API reports or accepts — the .col file’s order, the
activity report’s var_* arrays, row_normal(j)’s entries — indexes
the variables you wrote. The converged factor does not: a variable
whose bounds are equal is removed from the solve
(fixed_variable_treatment = make_parameter, the default), so its
column is absent and every later variable sits one row earlier. The
two orders coincide exactly when the model has no fixed variable,
which makes the difference easy to miss. Translate with
Solver.primal_rows(indices) — None marks a removed variable —
before indexing a kkt_solve or parametric_step_full result, just
as multiplier_rows has always been required for the y_c block.
In particular, for a parameter-estimation NLP with the parameters
pinned by equality constraints, -inv(info["reduced_hessian"]) is
directly the parameter covariance — no per-problem scale factor, no
need to set nlp_scaling_method = "none". (Sign convention: over pin
constraint rows, B K⁻¹ Bᵀ equals the multiplier sensitivity
∂λ/∂p = −∂²f*/∂p², hence the minus in the covariance recipe.)
For callers that calibrated against the pre-#128 behavior, the solver-space value and the factors that relate the two are exposed:
- Python:
info["reduced_hessian_scaled"],info["obj_scaling_factor"],info["pin_g_scaling"];Solver.reduced_hessian(pins, scaled=True),Solver.kkt_solve(rhs, scaled=True), and theSolver.nlp_scalingdict ({"obj": df, "c_scale": …, "d_scale": …, "x_scaling": …}). - Rust:
SensResult::{reduced_hessian_scaled, obj_scaling_factor, pin_g_scaling},Solver::{compute_reduced_hessian_scaled, kkt_solve_scaled, nlp_scaling, pin_g_scaling}, andPdSensBacksolver::solve_scaled_space.
The relation is H_scaled[i,j] = df / (dc_i·dc_j) · H[i,j], where
df is the objective scaling factor and dc_i the pin rows’
constraint scaling factors.
One caveat: the IPM’s inertia-correction perturbations (δ_x, δ_s,
δ_c, δ_d) are added to the factor in scaled space, so on a
problem whose final factorization needed regularization (e.g.
linearly dependent pin rows) the unscaling maps a slightly different
perturbed system per scaling method. The perturbations are reported —
info["kkt_perturbations"] / Solver.kkt_perturbations (Python),
SensResult::kkt_perturbations / Solver::kkt_perturbations (Rust)
— so a covariance workflow can assert they are all zero before
trusting -inv(reduced_hessian); on well-posed estimation problems
the final factor is unregularized and the invariance is exact.
Verification
All three entry points are verified against upstream sIPOPT 3.14.19’s
parametric_cpp golden output to within roughly 6e-9 per component.
The bound projection is a single-pass clamp; upstream’s iterative
Schur refinement (re-factorize on each violation) is intentionally not
ported.
Beyond one perturbation
Everything above answers “how does x* move for this \(\Delta\theta\)”
— a first-order step off one converged factor. Repeat it and you are
tracing a path, at which point the questions become where the linear
prediction stops being good enough, when the active set changes under
you, and what to do where \(\partial x^*/\partial\theta\) goes singular.
The Python frontend answers those with PathFollower, which turns the
same held factor into a predictor–corrector continuation loop (and a
pseudo-arclength mode that traces through folds), plus inverse_map_rhs
for running the map backwards as an ODE. See
Path Following & Inverse Mapping.
Sessions: Factor-Once / Solve-Many
POUNCE’s IPM converges to a KKT linear system that, once factored, answers a number of useful follow-up questions cheaply: parametric steps, reduced Hessians, custom back-solves. The session APIs let you hold that factor alive between operations, rather than rebuilding it on every call. The same machinery serves two workloads:
- Sensitivity / many-RHS. After one solve, issue many cheap operations against the converged factor — parametric steps for several parameter perturbations, reduced Hessians over several pinned-row sets, raw KKT back-solves.
- Factor-only. For non-IPM uses (shift-invert eigensolves, custom
Newton iterations) the underlying [
Factorization] handle inpounce-linsolexposes factor / refactor / back-solve directly, without the IPM in the loop.
Which layer do I want?
| You want… | Use |
|---|---|
| One solve plus a few sensitivity queries, from Python | pounce.Solver (Python) |
| The same, from C | IpoptSolver (C ABI) |
| The same, from Rust | pounce_rs::sensitivity::Solver |
| Just a sparse symmetric factor — no IPM involved | pounce_rs::linsol::Factorization |
| A one-shot sensitivity computation with a fluent builder | pounce_rs::sensitivity::SensSolve (Rust) or Problem.solve_with_sens (Python) |
The session API does not rebuild the IPM. Each solve() call runs
the full barrier method from scratch. What it reuses is the factor
that exists at convergence: KKT back-solves and sensitivity
operations skip the symbolic factor, AMD ordering, and numeric
factorization.
Python
import pounce
problem = pounce.Problem(...)
solver = pounce.Solver(problem)
x, info = solver.solve(x0=x0)
assert solver.converged
# Parametric step ∂x*/∂p · Δp, with p pinned by g(x) row indices.
dx = solver.parametric_step([2, 3], [-0.5, 0.0])
# Reduced Hessian B K⁻¹ Bᵀ over the same pinned-row set.
hr = solver.reduced_hessian([2, 3])
# Raw KKT back-solve, useful for custom workflows.
dim = solver.kkt_dim
rhs = np.zeros(dim)
lhs = solver.kkt_solve(rhs)
# Which bounds and rows actually hold the solution, in user index order.
rep = solver.classify_activity() # needs bound_relax_factor=0
rep["var_status"], rep["row_status"]
# Constraint-row gradient in user space, natural units.
a = solver.row_normal(j)
# Exact Lagrangian Hessian times a user-space vector, natural units.
hv = solver.hessian_vec(v)
The KKT compound vector is laid out as
x || s || y_c || y_d || z_l || z_u || v_l || v_u. pin indices in
parametric_step / reduced_hessian are 0-based row indices into
g(x); they are mapped internally to the matching y_c rows (through
the equality/inequality split, so inequalities may precede the pins).
All back-solves are in natural (unscaled) units — any NLP scaling
the IPM applied internally is undone, so results are independent of
nlp_scaling_method
(#128). The
solver-space values remain available via
reduced_hessian(pins, scaled=True) / kkt_solve(rhs, scaled=True),
and the factors via the Solver.nlp_scaling dict — see
Sensitivity Analysis.
pounce.Problem.solve() and Problem.solve_with_sens() still work
unchanged — each internally builds a fresh session — but new code that
issues more than one sensitivity query per solve should prefer
pounce.Solver to skip rebuilding the application.
C
IpoptProblem prob = CreateIpoptProblem(...);
AddIpoptStrOption(prob, "linear_solver", "feral");
/* Consumes prob — the IpoptSolver is now the sole owner.
prob is NULLed; calling FreeIpoptProblem(prob) on the now-null
pointer is harmless. */
IpoptSolver sol = IpoptCreateSolver(&prob);
double x[n], obj;
IpoptSolverSolve(sol, x, NULL, &obj, NULL, NULL, NULL, user_data);
Index dim = IpoptSolverGetKktDim(sol); /* compound KKT dim */
double rhs[dim], lhs[dim]; /* memset rhs as needed */
IpoptSolverKktSolve(sol, rhs, lhs);
Index pins[2] = {2, 3};
double deltas[2] = {-0.5, 0.0};
double dx[n];
IpoptSolverParametricStep(sol, 2, pins, deltas, dx);
double hr[2 * 2]; /* column-major dense */
IpoptSolverReducedHessian(sol, 2, pins, 1.0, hr);
IpoptFreeSolver(sol);
The classic IpoptSolve API is unchanged and unaffected; the session
handle lives alongside it.
Rust
Both session APIs come through the pounce-rs facade. Solver needs the
sensitivity feature; the bare Factorization below needs convex or qp,
whichever you are already using — either one enables pounce_rs::linsol.
#![allow(unused)]
fn main() {
use pounce_rs::sensitivity::Solver;
let mut solver = Solver::new(app, tnlp);
solver.solve();
assert!(solver.converged().is_some());
let dx = solver.parametric_step(&[2, 3], &[-0.5, 0.0])?;
let hr = solver.compute_reduced_hessian(&[2, 3], 1.0)?;
let mut lhs = vec![0.0; solver.kkt_dim().unwrap()];
solver.kkt_solve(&rhs, &mut lhs)?;
}
For purely linear-algebra uses with no IPM in the loop:
#![allow(unused)]
fn main() {
use pounce_rs::linsol::{Factorization, backend};
let mut fact = Factorization::new(dim, ia, ja, values, backend())?;
fact.solve(&mut rhs, 1)?; // back-substitute in place
fact.refactor(&new_values)?; // pattern preserved; numeric reuse
fact.solve_one(&mut another_rhs)?;
}
What’s preserved across operations
- Symbolic factor / AMD ordering. Owned by the linear-solver
backend; reused on every back-solve and on
refactor(). - Numeric factor. Reused on every back-solve until you refactor.
- The converged primal-dual state (
x*, multipliers,g(x*), iteration stats).
What’s not preserved across solve() calls
The session is currently a factor-and-query value: one solve,
many follow-up operations. A separate resolve() that re-runs the
IPM while reusing the symbolic factor + AMD ordering across top-level
solves (for MPC / B&B / warm-start workloads) is planned but not yet
implemented. Each solve() call today runs a fresh IPM.
Verification
All session entry points are tested for numerical equivalence with the corresponding one-shot APIs:
pounce.Solver.solve≡Problem.solve(1e-12).pounce.Solver.parametric_step≡Problem.solve_with_sens(deltas=…)['dx'](1e-10).pounce.Solver.reduced_hessian≡Problem.solve_with_sens(compute_reduced_hessian=True)['reduced_hessian'](1e-10).pounce_rs::sensitivity::Solver::parametric_step≡SensSolve::with_deltas(1e-10).
See python/tests/test_solver_session.py and
crates/pounce-sensitivity/tests/solver_session.rs for the full test
matrix.
Differentiable Solves & the DiffHandoff Contract
POUNCE solves are differentiable: a solve can sit inside a JAX or PyTorch
model and pass gradients with respect to the problem parameters. This
page documents the handoff contract — the well-defined bundle of
post-convergence data every solve produces — so that any consumer (the
built-in JAX/Torch layers, a downstream tool such as discopt, or your
own autodiff code) can differentiate a POUNCE solve from one stable
surface rather than from solver internals.
Design notes: dev-notes/diff-handoff-contract.md.
What a differentiable backward needs
The gradient of an optimal solution x*(p) with respect to a parameter
p comes from the implicit-function theorem applied to the KKT
conditions at the solution. To assemble it, a backward pass needs:
- the primal solution
x*and the constraint / bound multipliers; - the active set — which variable bounds bind and which constraint rows are active — so inactive directions drop out correctly;
- (for performance) the converged KKT factorization, reused as a back-solve rather than rebuilt.
POUNCE produces all three. The first two ride out in the solve info
dict; the third is reused automatically by JaxProblem (see
Sessions).
The active-set masks (the DiffHandoff core)
Every NLP solve’s info dict carries a precomputed active set, derived
once on the Rust side (pounce_sensitivity::DiffHandoff) so no consumer
re-derives it under its own tolerance:
info key | Type | Meaning |
|---|---|---|
pinned_vars | bool[n] | Variable i has an active bound — its sensitivity is zero (dx_i/dp = 0). True when mult_x_L[i] > active_tol or mult_x_U[i] > active_tol. |
active_constraints | bool[m] | Constraint row i is active: an equality (g_l[i] == g_u[i]) or a binding inequality (abs(mult_g[i]) > active_tol). |
active_tol | float | The activity threshold used to derive the two masks above (default 1e-6). |
pinned_vars is the seam used for mixed-integer problems: a
branch-and-bound leaf fixes integer variables at their optimal values,
and those variables differentiate exactly like an active bound
(dx/dp = 0). A producer of a fixed-integer leaf adds them to the mask
(DiffHandoff::pin on the Rust side).
Multiplier conventions (canonical mapping)
The same dual quantity is named differently across POUNCE’s solver
surfaces — deliberately, because each surface preserves an external
contract. The canonical field is DiffHandoff.lambda (general
constraint multipliers); this table maps every surface onto it so a
consumer knows the correspondence:
| Surface | Problem form | General-constraint dual | Bound duals | Why this naming |
|---|---|---|---|---|
NLP (Problem, C ABI) | min f(x) s.t. g_l ≤ g(x) ≤ g_u, x_l ≤ x ≤ x_u | mult_g | mult_x_L, mult_x_U | cyipopt-compatible — drop-in for cyipopt / JuMP / AMPL clients. |
Convex QP/SOCP (solve_qp) | min ½xᵀPx + cᵀx s.t. Gx ≤ h, Ax = b | z (inequality G), y (equality A) | z_lb, z_ub | OptNet / convex-solver convention (Amos & Kolter 2017). |
DiffHandoff (canonical) | general | lambda | mult_x_lower, mult_x_upper | one name for the contract. |
Caution. The internal symbol
lamis not a single quantity: in the NLP backward (jax/_diff.py) it is all constraint multipliers (= mult_g); in the QP backward (jax/_qp.py) it is the inequality-only duals (= z). Always map through the table above rather than assuming a shared name means a shared quantity.
These names are stable: the NLP keys are an external cyipopt contract and will not be renamed.
Consuming the contract
JAX / PyTorch (built in)
pounce.jax and pounce.torch already differentiate solves; you do not
touch the masks directly. Use pounce.jax.solve / JaxProblem (or the
torch equivalents) and call jax.grad / .backward() as usual. For
batched and repeated solves, JaxProblem reuses the converged KKT factor
in the backward (factor_reuse=True, default) — see Sessions.
Across a language / tool boundary (e.g. discopt)
A downstream tool that drives POUNCE as its NLP backend and composes its
own autodiff reads the contract straight from the info dict returned by
Problem.solve:
x, info = problem.solve(x0=...)
# primal + duals
lam = info["mult_g"] # general-constraint multipliers (the canonical λ)
z_L = info["mult_x_L"]
z_U = info["mult_x_U"]
# precomputed active set — do NOT re-derive |mult| > tol yourself
pinned = info["pinned_vars"] # bool[n]: dx/dp = 0 on these
active = info["active_constraints"] # bool[m]: rows in the KKT block
tol = info["active_tol"]
Because the active set is computed once in the producer, every consumer sees the same masks under the same tolerance — which is what makes a gradient assembled on one side of the boundary agree with one assembled on the other.
Verification
The contract is exercised by the test suite:
python/tests/test_problem.py::test_diff_handoff_masks_in_infoasserts the masks against a problem with a known active set (HS071: one variable on its lower bound, a binding inequality, and an equality).python/tests/test_jax.py(85 finite-difference gradient checks) andpython/tests/test_parity_jax_torch.py(JAX↔Torch gradient agreement) confirm the backward passes that rest on this data are correct and frontend-independent.
Interactive Solver Debugger
POUNCE ships an interactive debugger for the interior-point loop — a pdb for the IPM. You can pause the solve at well-defined points, inspect and mutate the live mathematical state (the iterate, multipliers, the barrier parameter μ), set breakpoints (by iteration, on a numeric condition, or on a solver event), step through an iteration’s internal phases, rewind to an earlier iterate, re-solve from a saved point with new options, and drop in automatically when a solve fails.
It has two front ends sharing one command engine:
- a human REPL (
--debug) with history, Ctrl-R search, and Tab completion, and - a newline-delimited JSON protocol (
--debug-json) that an LLM agent, a script, or a visual debugger (e.g. a VS Code Debug Adapter) can drive programmatically.
No production NLP solver ships anything like this; if you have used
ipopt you have had print_level and a log. This is a live debugger.
The same debugger spans every POUNCE solver — the NLP filter-IPM and the convex / conic interior-point solver share one command engine and one REPL. See Beyond the interior-point loop for the small set of commands whose availability is backend-conditional.
The debugger has zero effect on the solve when it is not attached. The checkpoint fire-sites short-circuit when no debugger is installed, so the standard regression suite is bit-for-bit identical with and without the feature compiled in.
Quick start
pounce problem.nl --debug # human REPL, pauses at iteration 0
pounce problem.nl --debug-json # JSON protocol on stdin/stdout
pounce problem.nl --debug-on-error # run freely; drop in only if it fails
pounce problem.nl --debug-on-interrupt # run; Ctrl-C drops you in
A 30-second session (human REPL):
$ pounce --problem rosenbrock --debug
── pounce-dbg ── iter 0 @iter_start mu=1.000e-1 obj=2.420000e1 inf_pr=0.00e0 inf_du=1.00e2
pounce-dbg> info
iter = 0
mu = 1.000000e-1
objective = 2.42000000e1
...
pounce-dbg> print x
x = [-1.200000e0, 1.000000e0]
pounce-dbg> break if inf_du<1e-6
conditional breakpoint: inf_du<1e-6
pounce-dbg> continue
... solver runs ...
── pounce-dbg ── iter 21 @iter_start mu=... inf_du=8.7e-7
↳ inf_du<1e-6
pounce-dbg> quit
The prompt is on stderr; the solver’s own iteration table stays on stdout, so a redirected log is unaffected.
The two front ends
--debug (REPL) | --debug-json | |
|---|---|---|
| Audience | human at a terminal | agent / script / GUI |
| Channel | prompt + output on stderr | pure JSON on stdout |
| Line editing | rustyline: history (~/.pounce_dbg_history), Ctrl-R, Tab completion | n/a (caller supplies UI) |
| Solver table | shown on stdout | suppressed (print_level 0) |
| Commands | bare strings | bare strings or {"cmd":…,"args":[…],"id":…} |
On a non-TTY stdin (a pipe), the REPL falls back to a plain line reader (no history/Tab) but otherwise behaves identically — handy for scripted tests.
The JSON protocol is documented in full below.
Pausing and flow control
Checkpoints
The loop fires the debugger at these points (a pause reports which one
via its checkpoint field):
| Checkpoint | Fires | What’s fresh |
|---|---|---|
iter_start | top of each outer iteration | the accepted iterate from the previous step |
after_mu | μ updated for this iteration | the new barrier parameter |
after_search_dir | Newton step δ solved | the step (dx …), regularization, KKT inertia |
after_step | trial accepted | the step lengths α, the new iterate |
step_rejected | line search gave up (tiny step / all backtracks failed), before restoration | the search direction δ and the un-accepted iterate |
pre_restoration_entry | just before restoration | the iterate that tripped restoration |
post_restoration_exit | restoration returned | what restoration produced |
terminated | once, before the solve returns | the final / failing iterate + status |
By default the debugger only stops at iter_start (and terminated).
The sub-iteration checkpoints fire every iteration but resume immediately
unless you ask to stop at them.
Stepping into restoration. The same debugger drives the restoration
inner IPM: when the solve enters restoration, the inner solve’s
checkpoints fire too. A step/stepi that lands on an inner iteration
pauses there with in_restoration: true (REPL banner shows
[restoration]), and print x shows the restoration sub-NLP iterate.
stop-at resto (pre_restoration_entry) is the easy way to catch the
hand-off and then step inward.
Stepping
| Command | Effect |
|---|---|
step / s / n | run to the next iter_start |
step sub / stepi / si | run to the next checkpoint of any kind (walk an iteration’s phases) |
continue / c | run to the next breakpoint (or to completion) |
run N / r N | run until iteration N |
stop-at <cp> | always pause at checkpoint <cp> |
detach | stop pausing; run to completion |
quit / q | stop the solve now |
stop-at takes a checkpoint name or a friendly alias:
stop-at after_search_dir # or: stop-at kkt
stop-at pre_restoration_entry # or: stop-at resto
stop-at # list active stop-at checkpoints
stop-at clear
Aliases: mu → after_mu, kkt/search_dir → after_search_dir,
step → after_step, resto → pre_restoration_entry, resto_exit →
post_restoration_exit.
Breakpoints
Three kinds, all reported in break and surfaced as the pause reason.
By iteration
break 12 # pause at iteration 12 (alias: b 12)
tbreak 12 # one-shot: pause at 12, then delete itself (alias: tb)
break # list all breakpoints
break del 12 # remove
break clear # remove everything (iters + conditions + events)
Watchpoints (data breakpoints)
watchpoint x[3] # pause when x[3] changes (alias: wp)
watchpoint x 1e-3 # pause when any x component moves by > 1e-3
watchpoint # list; watchpoint del x[3]; watchpoint clear
Distinct from watch (which only displays): a watchpoint pauses the
solve when the watched value changes by more than its threshold (default
0 = any change) between iterations. Useful for a component expected to
stay put (e.g. a variable pinned at a bound).
Breakpoint command lists
Attach commands to a breakpoint that run automatically when it hits — semicolon-separated, ending with a flow command to auto-resume:
break 5
commands 5 print kkt ; set mu 0.1 ; continue # at iter 5: inspect, tweak μ, go
commands 5 clear # remove
commands # list all
When iteration 5 is reached, the debugger emits the pause, runs the
attached commands (each result is reported), and if one of them
resumes/stops, honors it without dropping to the prompt — otherwise it
falls through to the interactive prompt as usual.
Conditional (with compound predicates)
break if inf_pr<1e-6
break if mu<1e-4 && inf_pr>1e-3
break if iter>10 && (inf_du>1e-2 || obj<0)
break clear cond
- Metrics:
mu,inf_pr,inf_du,obj,err(overall NLP error),iter. - Operators:
<,<=,>,>=,==(==is float-tolerant). - Compound:
&&and||, evaluated strictly left-to-right with no precedence; parentheses are accepted but stripped (they don’t group). For real grouping, register several conditions — any one that holds fires.
Conditions are evaluated at iter_start.
On a solver event
break on regularized
break on resto_entered
break clear events
| Event | Fires when |
|---|---|
resto_entered | the algorithm enters restoration |
resto_exited | restoration returns |
regularized | the KKT system needed regularization (δ_w > 0 — inertia correction) |
tiny_step | the primal step is numerically negligible (‖dx‖∞ < 1e-10) |
ls_rejected | the line search tried more than one trial point |
mu_stalled | μ held (to tolerance) for 3 consecutive iterations |
nan | the NLP error or objective became non-finite |
Events fire at whatever checkpoint makes them observable (e.g.
regularized at after_search_dir), and pause with
reason: "event: <name>".
Inspecting state
info # one-line summary: iter, mu, obj, inf_pr, inf_du, nlp_error, dims
print x # a primal/dual block (alias: p x)
print dx # a search-direction block (d + block name)
print mu # a scalar: mu|obj|inf_pr|inf_du|err|compl|iter
print kkt # KKT inertia + regularization (see below)
print rank # SVD numerical rank of the equality Jacobian J_c (see below)
print active # which bound categories are near-active (small slack)
watch mu # auto-print a target at every pause (alias: display)
watch # list watches; watch del mu; watch clear
watch <target> registers any print target (block, dx, scalar,
kkt) to be shown automatically at every subsequent pause — the
debugger’s equivalent of gdb’s display. In JSON mode the values arrive
in the pause event’s watches array.
Blocks (the eight components of the primal-dual iterate):
| Name | Meaning |
|---|---|
x | primal variables |
s | inequality slacks |
y_c | equality-constraint multipliers |
y_d | inequality-constraint multipliers |
z_l, z_u | bound multipliers on x |
v_l, v_u | bound multipliers on s |
Prefix any block with d (dx, dz_l, …) to print the corresponding
block of the most recent Newton step.
Model names (.col / .row)
A solver-internal diagnostic that says “variable 132 in equation 3 looks
singular” is far less actionable than one that says “T_reactor in
energy_balance”. Lee et al. (2024) identify this gap — between
detecting an issue numerically and tracing it back to a named equation
in the modeling environment — as a central roadblock for debugging
equation-oriented models.1
AMPL .nl files carry no names, but AMPL emits two optional sibling
files when the modeler sets option auxfiles rc;:
| File | Contents |
|---|---|
stub.col | one variable name per line, in column order |
stub.row | one constraint name per line, in row order |
When these sit next to the .nl, pounce captures them
(NlProblem::var_names / con_names) and exposes them through the
ExpressionProvider::variable_name / constraint_name seam. Missing or
malformed name files are non-fatal — names are a diagnostic aid, never
load-blocking, so the debugger simply falls back to index labels.
print residuals uses these names directly. Residual values live in the
solver’s split space (equalities and inequalities separated, fixed
variables removed), so a name only labels the right row if it is carried
through the same permutations. The TNLP publishes its .col/.row names
under the conventional idx_names metadata key, and OrigIpoptNlp
projects them into split space (x_not_fixed_map for variables, c_map
for equalities, d_map for inequalities) — the debugger reads the result
via DebugCtx::split_names. So a near-singular equality residual prints as
c[energy_balance] = +3.142e-04 |3.142e-04|
instead of c[3]. The same idx_names pool labels grad_x_L[...]
(variable names) and grad_s_L[...] / d-s[...] (inequality names). The
JSON payload keeps the numeric index and adds a name field.
Status. Capture, exposure, and
print residualslabeling are live on the AMPL.nlpath with names projected through the bound / c-d-split permutations. Presolve renumbers rows, soPresolveTnlpdeclinesidx_namesrather than risk mislabeling a permuted row — under presolve the debugger safely falls back to index labels. Carrying names through the presolve map and decoratingprint activeare the next steps built on this foundation.
print equation — the algebra of a named constraint
Naming a culprit row is only half the story; the next question is always
what does that equation actually say? Lee et al. (2024) make this the
core of actionable equation-oriented diagnostics — a debugger should
surface the named equation, not just a row index.1 print equation closes that loop: once print residuals points at, say,
c[energy_balance], you read the constraint’s source algebra directly.
(dbg) print equation energy_balance
energy_balance: T_reactor*flow - 300*flow - Q = 0
(dbg) print equation 14 # by original .nl row index
c[14]: x[3]^2 + x[7]^2 <= 1
A constraint is addressable by its model name (preferred, and robust
to row reordering) or its original .nl row index. With no argument,
print equation reports how many equations are available. The renderer
works from the faithful Expr DAG the .nl parser built — not the lossy
evaluation tape — so common-subexpressions, imported functions, and
piecewise/conditional forms render as written. The affine part is printed
with tidy signs (a - 2*b, not a + -2*b), zero-coefficient Jacobian
placeholders are suppressed, and bounds render in their natural relation
(= rhs, lo <= body <= hi, >= lo, <= hi). The JSON payload carries
{index, name, equation}.
Equations are static model data in original .nl row order, so unlike
residuals they need no split-space projection — print equation works
regardless of presolve. It is available whenever a model was loaded from
an .nl file; the JSON name field is present only when a .row auxfile
supplied one.
print kkt — inertia and regularization
Available at/after after_search_dir (use stop-at kkt). This is the
view a solver expert reaches for when a step looks wrong:
pounce-dbg> stop-at kkt
pounce-dbg> continue
── pounce-dbg ── iter 3 @after_search_dir ...
pounce-dbg> print kkt
dim = 3
inertia = n+=2 n-=1 (expected n-=1) → correct
delta_w = 0.000000e0 (primal regularization)
delta_c = 0.000000e0 (dual regularization)
status = Success
The augmented (KKT) system has expected inertia (n₊ = n, n₋ = m, n₀ = 0) where m is the number of equality + inequality multipliers.
A mismatch — or a nonzero delta_w/delta_c — is the classic signal
that the step is being stabilized (the solver added regularization to
fix the inertia).
For the matrix and factor themselves:
viz kkt # the assembled augmented-system matrix (triplets) + inertia
viz L # the LDLᵀ factor (strict-lower triplets + values)
viz kkt writes the KKT matrix as 1-based lower-triangle triplets
(dim, irn, jcn, vals) alongside the inertia summary — point
$POUNCE_DBG_VIEWER at a heatmap script. viz L writes the LDLᵀ
factor (n, fill-reducing perm, strict-lower l_irn/l_jcn/l_vals
in permuted coordinates), read out of the factor the solver actually
computed.
Both are read-only and always show the most recent factorization:
the current iteration’s system at an after_search_dir stop, or the
previous iteration’s at the default iter_start pause (the step
that produced where you’re standing). The matrix and factor are captured
every iteration while the debugger is stepping; once you detach (run
free) the capture is dropped — so on a large problem a free run doesn’t
pay the O(nnz) assembly. If you viz kkt/viz L right after a free run,
step once to re-capture.
print rank — numerical rank of the equality Jacobian
print kkt tells you that the dual system needed regularization
(delta_c > 0) or that the inertia was wrong; the structural_singularity
finding names equations that are dependent by sparsity pattern. print rank closes the last gap: a rank-revealing SVD of the equality Jacobian
J_c at the current iterate. It factors the matrix the solver actually
sees (constraint scaling already applied), so it localizes the dependency
to specific equations — including dependencies that are numerical only
(values that cancel over a full sparsity pattern), which the structural
Dulmage–Mendelsohn pass cannot detect.
It doesn’t just name the culprit equations — it prints them. When a
.nl model is loaded, each implicated row’s source algebra is rendered
directly beneath it (the same DAG-faithful text print equation shows), so
you read the dependency without a second command:
pounce-dbg> print rank
equality Jacobian J_c: 3 row(s) × 4 column(s)
numerical rank = 2 / 3 (deficiency 1)
σ_max = 3.162e0 σ_min = 0.000e0 cond = inf (σ_min = 0) (rank tol τ = 1.40e-15)
singular values: [3.162e0, 1.414e0, 0.000e0]
rank-deficient: 1 equation(s) lie in the near-null space (linearly dependent / redundant) — the source of δ_c regularization:
c[mass_balance] (participation 0.50)
x[0] + x[1] - 10 = 0
c[mass_balance_dup] (participation 0.50)
x[0] + x[1] - 10 = 0
The two equations print identically — that is the redundancy, now visible on its face.
For the SVD J_c = U Σ Vᵀ, the left singular vectors u_k whose singular
value σ_k ≈ 0 span the left null space — the row combinations u_kᵀ J_c ≈ 0 that vanish. Each row’s participation w_i = Σ_{k : σ_k ≤ τ} u[i,k]² ∈ [0, 1] localizes the dependency: a redundancy shared between two
equations splits ≈ 0.5/0.5, while w_i = 1 means row i lies entirely in
the null space. The numerical-rank threshold is the standard LAPACK/NumPy
τ = σ_max · max(m, n) · ε; the implicated rows are resolved to model
names through the same .row plumbing as print residuals / print equation.
The inline algebra is resolved by model name, so it appears for named
rows. The rank report’s row index is the split equality position, not the
original .nl row the equation source keys on, so an unnamed row can’t be
mapped — there print rank falls back to a print equation <name> hint
instead of guessing. When J_c has full row rank, that is reported as a
positive signal (J_c has full row rank at this iterate.) with the
σ_min/cond witnessing how far it is from degenerate — silence would be
ambiguous. The command is available whenever the iterate has an equality
block; a problem with no equality constraints returns a short explanatory
error. The JSON payload is {iter, n_rows, n_cols, rank, deficiency, rank_deficient, sigma_max, sigma_min, cond, tol, singular_values, culprits: [{row, kind, index, name, label, weight, equation}]} (equation is the
rendered source or null when unresolved; cond is null when σ_min = 0, since JSON has no infinity).
diagnose — a live, named health report
info, print residuals, and print kkt each expose one facet of the
current iterate. diagnose (alias diag) runs a panel of heuristics over
all of them at once and returns a ranked list of findings — and, crucially,
names the culprit equation or variable behind each numerical symptom.
That last step is the actionable-diagnostics path of Lee et al.
(2024):1 a report that says “mass_balance is the worst
constraint residual” is worth far more than “row 13 is infeasible.”
pounce-dbg> diagnose
[ error] primal_infeasible: Primal infeasibility 1.70e+02; worst constraint
residual is c[mass_balance] = +1.701e+02. Inspect this equation's
feasibility and scaling (`print equation mass_balance`).
[warning] dual_infeasible: Dual infeasibility 9.84e-01; largest stationarity
residual is grad_x_L[T_reactor] = -9.838e-01.
[warning] inertia_wrong: KKT inertia is wrong (n-=2 vs expected 1): the system
was indefinite/singular and the step had to be stabilized.
[ info] bounds_pinned: 3 variable bound(s) are active (slack < 1e-6).
This is the live counterpart to the pounce-studio diagnose tool,
which runs temporal heuristics over a finished solve report. The two
share a {severity, code, message} shape so
a client can treat them uniformly, but the live command sees what a saved
report cannot: the current KKT inertia and regularization, and the
named primal/dual residuals at this exact point. Findings are sorted
error → warning → info; a clean iterate yields a single healthy
finding. The checks:
| code | severity | fires when |
|---|---|---|
primal_infeasible | error/warning | inf_pr above tol → names the worst constraint residual |
dual_infeasible | warning | inf_du above tol → names the worst stationarity residual |
inertia_wrong | warning | KKT inertia ≠ expected (rank-deficient Jacobian / indefinite Hessian) |
heavy_regularization | info | primal δ_w applied (Hessian indefinite) |
dual_regularization | warning | dual δ_c applied (linearly dependent / redundant equalities) |
structural_singularity | warning | a subset of equalities is over-determined → names the dependent equations |
rank_deficient_jacobian | warning | SVD of J_c is numerically rank-deficient → names the equations in the near-null space (catches value-only dependencies too) |
large_multipliers | warning | a multiplier exceeds 1e8 (constraint-qualification / scaling) |
bounds_pinned | info | variables pressed against their bounds |
tiny_step | warning | accepted α_pr collapsed |
heavy_line_search | warning | ≥10 backtracking trials for the accepted step |
in_restoration | warning | currently inside feasibility restoration |
mu_stalled | warning | μ flat for ≥3 consecutive iterations |
KKT-derived findings (inertia_wrong, *_regularization) need a computed
search direction, so they appear at/after after_search_dir. Names follow
the same rule as print residuals: present on the .nl path with
.col/.row files, index labels (c[13]) under presolve. The JSON payload
is {iter, findings: [{severity, code, message}], n_findings}.
Structural rank: naming the dependent equations
inertia_wrong and dual_regularization detect a rank-deficient
Jacobian, but only as a scalar — they tell you a redundancy exists, not
which equations are redundant. structural_singularity closes that gap
with a Dulmage–Mendelsohn decomposition of the equality Jacobian’s
sparsity pattern (the same structural check at the heart of IDAES’s
DiagnosticsToolbox). A maximum bipartite matching between equality rows
and variables partitions the system; any over-determined block — more
equations than the variables they jointly touch — forces at least one of
those equations to be redundant or mutually inconsistent (LICQ fails). The
finding lists those equations by model name, e.g.:
pounce-dbg> diagnose
[warning] structural_singularity: Constraint Jacobian is structurally singular
(Dulmage–Mendelsohn): 2 equation(s) over-determine the 1 variable(s)
they jointly touch (flow_rate), so ≥1 of them must be redundant or
mutually inconsistent (LICQ fails on this block). Candidate dependent
equations: mass_balance, mass_balance_dup. Inspect them with
`print equation <name>`; this names the rows behind any δ_c
dual-regularization / wrong-inertia signal.
This is the named-culprit payoff of Lee et al. (2024):1
reporting “mass_balance and mass_balance_dup are linearly dependent”
rather than “the Jacobian is singular.” The check is iterate-independent
(it reads only the sparsity pattern), so unlike the KKT-derived findings it
fires from iteration 0 — it can flag a structurally broken model before the
solver ever stalls on it. It is suppressed for well-posed problems: an NLP
with more variables than equality constraints is the normal case (the spare
degrees of freedom are pinned by the objective, bounds, and inequalities),
so only the over-determined side is reported, never the under-determined
one. Available on the .nl path; names fall back to c[i]/x[i] when no
.col/.row auxiliary files were emitted.
Numerical rank: the value-dependency the structure can’t see
structural_singularity reads only the sparsity pattern, so it is blind
to a redundancy that lives in the values — three equations whose every
entry is nonzero (a structurally full-rank pattern) but whose rows satisfy
row₂ = row₀ + row₁ numerically. rank_deficient_jacobian is the
numerical complement: it runs the same SVD as print rank over J_c at
the current iterate and, when the numerical rank falls short, names the
equations in the near-null space:
pounce-dbg> diagnose
[warning] rank_deficient_jacobian: Equality Jacobian J_c is numerically
rank-deficient at this iterate: rank 2/3 (deficiency 1),
σ_min=0.00e0, cond=inf (σ_min = 0). Linearly dependent or redundant
equality constraints — the root cause behind δ_c regularization /
wrong inertia. Implicated equations: c[mass_balance],
c[mass_balance_dup].
Unlike the structural check, this one is iterate-dependent — it factors
J_c at the current x, so it reflects the matrix the solver is actually
regularizing and catches dependencies that only appear at certain points.
The two checks are deliberately layered: structural_singularity fires
from iteration 0 on the pattern alone; rank_deficient_jacobian confirms
it numerically and, more importantly, surfaces the value-only dependencies
the structural pass provably cannot. See print rank for the SVD
math and the per-equation participation weights.
Mutating state
Mutations feed straight back into the solve.
set mu 0.5 # overwrite the barrier parameter
set x[2] 1.5 # overwrite one component of a block
set x 1.0,2.0,3.0 # overwrite a whole block (comma-separated)
Setting any block works (set z_l[0] 1e-3, …). Iterate edits rebuild the
iterate with a fresh change-tag, so the cached derived quantities
(curr_f, slacks, σ, …) invalidate correctly and the next step is
computed from the new point — exactly as if the line search had produced
it.
Staging a solver option (validated against the registry):
set opt mu_strategy adaptive
set opt linear_solver ma57
Staged options are not applied to the strategies already built for
the running solve (they don’t re-read options mid-iteration). They take
effect on a resolve or the next solve.
The read-side counterpart is get opt <name>, which reports an option’s
current (or staged) value and its registry metadata — so you can confirm
what a set opt actually staged before you resolve:
get opt mu_strategy # → mu_strategy = adaptive (staged)
Discovering options
opt # list every registered option
opt mu # filter by name/category substring
complete pri # completion candidates for a prefix
opt <exact-name> also prints the long description. In the REPL, Tab
completes command verbs, block names, metric names (after break if),
checkpoint names (after stop-at), event names (after break on),
option names (after set opt / opt), and filesystem paths (after
load / sweep / save / source — directories get a trailing /).
The same contexts are available programmatically via the complete <prefix…> command (JSON complete), so an agent or GUI can offer the
same completions.
Time travel
Rewind (goto / restart)
The debugger snapshots the primal-dual state (x, s, multipliers, μ,
τ) every iteration. goto rewinds to a captured iteration and stays
paused so you can re-tune before resuming:
goto 3 # rewind to the start of iteration 3
restart # rewind to the earliest snapshot
Caveat — this is a soft rewind. Only the primal-dual state is restored; strategy history (the filter, the adaptive-μ oracle, the quasi-Newton memory) is not rolled back. So continuing from a rewound point is “resume from here,” not a bit-exact replay of the original run.
Re-solve from a saved point
resolve re-runs the solve from the current x with any
set opt edits applied — a primal warm start with new options. Use it
for “what if I change mu_strategy from here?”:
pounce-dbg> set opt mu_strategy adaptive
pounce-dbg> resolve
re-solving from current x with 1 staged option override(s)…
── pounce-dbg ── iter 0 @iter_start ... # fresh solve, seeded from the captured x
Because each solve rebuilds its strategies from the options, the changes do take effect on the re-solve. The seed is dropped (falling back to the problem’s own start) if presolve / fixed-variable elimination changed the coordinate count.
Saving and visualizing artifacts
save # write the current iterate + residuals to a temp JSON
save /tmp/iter3.json # explicit path
viz x # write a block and open it in an external viewer
viz dx # a search-direction block
viz kkt # the KKT inertia/regularization report
save writes every non-empty block, the search-direction blocks, and the
residual scalars (iter, mu, objective, inf_pr, inf_du,
nlp_error) — a self-contained artifact for external analysis.
load — the inverse of save
Typing a start point by hand is fine for a 2-variable toy and miserable for
anything real. load reads a block straight into the live iterate, so you
generate the point once (a prior solve, a surrogate, a sampler) and pull it
in:
load /tmp/it0.json # a `save` artifact: every block it contains is loaded
load start.csv # a plain numeric file → x (comma/space/newline sep)
load start.csv s # … into a named block instead of x
Two input shapes are accepted:
- A
saveartifact (JSON). Blocks are read from the top level or from aniterateobject; every block present (x,s, multipliers, …) is written, each validated against the current dimension. Sosave→loadround-trips a full point, and you can lift just the part that fits if dimensions changed. - A plain numeric file — values separated by commas, whitespace, or
newlines — written into the named block (default
x). This is the many-variable escape hatch:numpy.savetxt("start.csv", x0)thenload start.csv.
A loaded x becomes the seed for the next step (or for resolve — a
warm start from an externally-computed point with no typing).
Interactive figures (pounce-dbg-viz)
viz writes a JSON artifact and hands it to a viewer. The Python package
ships an interactive Plotly viewer that renders these properly —
a spy/heatmap for viz kkt (the augmented matrix, colored by value, with
the inertia/regularization in the title) and viz L (the LDLᵀ factor),
and a bar chart for vector blocks (viz x, viz dx):
pip install 'pounce-solver[viz]' # installs the `pounce-dbg-viz` script
When pounce-dbg-viz is on PATH, viz uses it automatically (opening
an interactive figure in your browser). The launch order is:
$POUNCE_DBG_VIEWER— a command template ({}← the artifact path), if set;pounce-dbg-viz— the bundled Plotly viewer, if installed;- the OS opener (
xdg-open/open) on the raw JSON.
So export POUNCE_DBG_VIEWER='python my_plot.py {}' overrides with your
own plotter, and with nothing set + the viz extra installed it just
works. The same pounce-dbg-viz <file.json> also renders a save
artifact (the full iterate).
Multi-start and initialization sensitivity
Interior-point methods find a local solution, and which one depends on
where you start. Two commands turn the debugger into an
initialization-sensitivity probe: they run many full solves — each from a
different start — and tabulate where each one ends up. Both build on the
same re-solve machinery as resolve (so
they need the restart cell the CLI wires by default; they error in
contexts without it), and both leave you at a normal prompt on the final
solve afterward.
sweep <file> — explicit starts
Run one solve per start point listed in a file (one start per line,
comma/whitespace-separated; #/// comments and blank lines skipped):
pounce-dbg> sweep starts.txt
sweep 1/4: Success iters=21 obj=3.743990e-21 inf_pr=0.00e0
sweep 2/4: Success iters=15 obj=1.233088e-28 inf_pr=0.00e0
sweep 3/4: Success iters=14 obj=1.328861e-28 inf_pr=0.00e0
sweep 4/4: Success iters=29 obj=2.982346e-18 inf_pr=0.00e0
── sweep complete ── 4 solves, 4 succeeded, 1 distinct minima
# status iters objective inf_pr
0 Success 21 3.743990e-21 0.00e0
1 Success 15 1.233088e-28 0.00e0
2 Success 14 1.328861e-28 0.00e0
3 Success 29 2.982346e-18 0.00e0
best: solve #2 obj=1.32886077e-28
Each start must have the same length as x (mismatches are reported with
the line number). The summary clusters successful objectives to a relative
1e-6 to count distinct minima and flags the best (lowest-objective)
solve. This is the “is this solve fragile to its start, and to which basins
does it fall?” diagnostic — and unlike a black-box global search it leaves
every solve’s trajectory observable: set a break on resto_entered or a
stop-at kkt first and the sweep will pause inside whichever solve trips
it.
multistart <N> [rel] — sampled restarts
When you don’t have a file of starts, multistart generates N of them:
pounce-dbg> multistart 8 # 8 starts
pounce-dbg> multistart 8 0.3 # wider jitter on any unbounded vars
Each variable that has a finite box [x_Lᵢ, x_Uᵢ] is sampled
uniformly inside it — a genuine box multistart. Variables that are
unbounded on either side fall back to a relative jitter ±rel·(|xᵢ|+1)
around the current point (rel default 0.1, with a floor so components at
zero still move). The command reports the split, e.g.
multistart 8 (box 5/7 vars; 2 unbounded → jitter rel=0.1).
Start 0 is always the unperturbed current x (so the run includes where
you already are), and the sampler is a fixed-seed PRNG, so a multistart
run reproduces exactly.
The bounds are the ones the algorithm sees — full-length, post-scaling,
after any bound_relax_factor — so every sampled start is a valid seed.
For a problem with no finite bounds (a pure unconstrained NLP) multistart
degrades to jitter around x; sweep an external sample if you want a
specific spread there.
Driving a sweep from a file with load
The pieces compose. To seed a sweep from points computed elsewhere, write
them with numpy.savetxt and sweep the file directly — or, for a single
externally-computed warm start, load it and resolve:
import numpy as np
np.savetxt("starts.txt", sampler(n=32), delimiter=",") # 32 starts, one per row
pounce-dbg> sweep starts.txt
sweep vs. find_minima
sweep/multistart are diagnostics: they show you how a handful of
starts behave, with full visibility into each solve’s path. For an
automated global search — Sobol sampling, deduplication, minimum
certification (PSD Hessian), redundant-descent avoidance — reach for the
Python pounce.find_minima, whose multistart and
mlsl methods are the production tools. Rule of thumb: debugger sweep
when you’re asking why a solve is start-sensitive; find_minima when
you want the minima themselves.
Ask an LLM about the state
ask [question] packages the current paused state — checkpoint,
residuals, step lengths, dimensions, and the KKT inertia/regularization —
into a prompt and runs it through an LLM CLI (by default Claude Code,
claude -p headless print mode), printing the reply inline. It’s
AI-assisted debugging without leaving the loop:
pounce-dbg> stop-at kkt
pounce-dbg> continue
pounce-dbg> ask why is the dual infeasibility stalling?
# → the model's analysis of the state + suggested options to try
With no question it defaults to “explain the current state and suggest what to try next.”
Choosing the LLM
Set $POUNCE_DBG_LLM to pick the backend. It accepts either a bare
provider keyword — which expands to that CLI’s correct non-interactive
invocation — or a full command template:
export POUNCE_DBG_LLM=claude # Claude Code → claude -p (default)
export POUNCE_DBG_LLM=codex # OpenAI Codex CLI → codex exec <prompt>
export POUNCE_DBG_LLM=gemini # Google Gemini → gemini -p <prompt>
export POUNCE_DBG_LLM=llm # simonw's llm → llm <prompt>
# …or a full template:
export POUNCE_DBG_LLM='llm -m claude-opus' # any prompt-on-stdin CLI
export POUNCE_DBG_LLM='mytool --ask {}' # prompt substituted into {}
For a template, the prompt is fed on the tool’s stdin unless it contains a
{} placeholder, in which case it is substituted as an argument. A bare
word that isn’t a known provider is treated as a program name with the
prompt on stdin.
Graceful when the CLI is absent. If the selected tool isn’t installed
or on PATH, ask returns an error naming the tool and listing the
provider keywords — the rest of the debugger (and the solve) is
unaffected. ask is the only command that shells out; nothing else
depends on an LLM being present.
In JSON mode the reply comes back in the result event’s data.reply.
Attaching to a run
You don’t have to single-step from iteration 0.
- Drop in on failure —
--debug-on-errorruns the solve freely and pauses at theterminatedcheckpoint only if the solve did not succeed, leaving you at the failing iterate for a post-mortem. (Plain--debugalso pauses atterminatedfor a final-point inspect.) - Attach with Ctrl-C —
--debug-on-interruptruns normally but installs a SIGINT handler; a first Ctrl-C drops you in at the next iteration (reason: "interrupt (Ctrl-C)"), a second Ctrl-C aborts. Ctrl-C also breaks into any other debug mode mid-continue.
Ctrl-C at the prompt. At a rustyline prompt Ctrl-C arrives as input,
not a signal, so it has its own analogous double-tap: the first Ctrl-C
cancels the current input line (readline convention), a second in a row
stops the solve (a clean UserRequestedStop, same as quit). So
whether you are running or sitting at the prompt, two Ctrl-Cs always get you
out; quit/q and Ctrl-D (EOF, which detaches and finishes) remain the
explicit exits.
Scripting
Run a sequence of debugger commands from a file — one per line, # and
// comments and blank lines skipped:
# warmup.pdbg
break if inf_pr<1e-6
watch mu
stop-at after_search_dir
continue
pounce problem.nl --debug-script warmup.pdbg # run at the first pause
pounce-dbg> source warmup.pdbg # or interactively
A script runs top-to-bottom and stops early if a command resumes or
stops the solve (so ending with continue hands control back at the
first breakpoint). --debug-script implies --debug when no --debug*
mode is given, and runs once at the first pause (not on a resolve).
Example: a scripted initialization-sensitivity run
Because load, sweep, and set opt are ordinary commands, a whole
diagnostic fits in a script file. This one watches each solve’s path and
sweeps a set of externally-generated starts:
# sensitivity.pdbg — generate starts.txt first (e.g. numpy.savetxt)
break on resto_entered # surface any start that falls into restoration
sweep starts.txt # one solve per row; tabulated at the end
pounce model.nl --debug-script sensitivity.pdbg
Or compare a baseline against a what-if on the same starts by staging an option before the sweep:
# adaptive-vs-monotone.pdbg
set opt mu_strategy adaptive
multistart 16 0.2 # 16 sampled restarts, all under adaptive μ
Example: drive a multistart from a program (JSON protocol)
For many variables and many starts, hold the x0s as arrays in a driver
program and let it assemble the commands — no point is ever typed. The
--debug-json protocol emits a sweep_result per solve and a final
sweep_summary:
import subprocess, json, numpy as np
p = subprocess.Popen(["pounce", "big.nl", "--debug-json"],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True)
send = lambda c, **k: (p.stdin.write(json.dumps({"cmd": c, **k}) + "\n"), p.stdin.flush())
recv = lambda: json.loads(p.stdout.readline())
recv() # hello
recv() # initial pause
# Option A — let the debugger sample: N restarts (uniform in finite boxes).
send("multistart", args=["32", "0.25"])
# Option B — supply your own starts via a file and sweep it:
# np.savetxt("starts.txt", my_sampler(n=32), delimiter=",")
# send("sweep", args=["starts.txt"])
results = []
for line in p.stdout:
ev = json.loads(line)
if ev.get("event") == "sweep_result":
results.append((ev["status"], ev["objective"]))
elif ev.get("event") == "sweep_summary":
print(f"{ev['succeeded']}/{ev['solves']} ok, "
f"{ev['distinct_minima']} distinct minima, "
f"best obj {ev['best_objective']:.6e}")
break
Each sweep_result carries index, status, iters, objective,
inf_pr, and the seed it started from; the sweep_summary adds
distinct_minima, best_index, and best_objective. A client can
feature-detect support via hello.capabilities.sweep.
Exit model
| Path | Result |
|---|---|
quit | stops now → UserRequestedStop |
| Ctrl-C ×2 at the prompt | cancel line, then stop → UserRequestedStop |
Ctrl-C ×2 mid-continue | break in, then abort (exit 130) |
continue / detach | run to natural completion |
| stdin EOF, REPL (Ctrl-D) | detach and finish (pdb convention) |
| stdin EOF, JSON (pipe closed) | abort — the controlling client is gone |
| external SIGKILL | process dies (no terminated event) |
Every non-kill path ends with a terminated event in JSON mode.
Command reference
| Command (aliases) | Summary |
|---|---|
help (h, ?) | list commands |
info (i) | current-iterate summary |
print <what> (p) | block, d-block, scalar, kkt, or residuals |
print equation <name|row> | source algebra of a constraint, by model name or .nl row |
step (s, n) | run to next iter_start |
step sub / stepi (si) | run to next checkpoint of any kind |
continue (c) | run to next breakpoint |
run N (r) | run until iteration N |
break … (b) | iteration / if / on breakpoints; list; clear; del N |
stop-at <cp> | always pause at a checkpoint |
set mu/x/<block>/opt … | mutate μ, the iterate, or stage an option |
get opt <name> (get <name>) | report an option’s current/staged value, source, and default |
opt [filter] | list/search registered options |
complete <prefix> | completion candidates |
viz <target> | open an artifact in a viewer |
save [path] | dump the iterate to JSON |
load <file> [block] | read a block (default x) from a save artifact / numeric file |
sweep <file> | one solve per start in <file>; tabulate outcomes |
multistart <N> [rel] | N restarts (uniform in each finite box; jitter elsewhere); tabulate |
watch <target> (display) | auto-print a target at every pause |
tbreak N (tb) | one-shot iteration breakpoint |
commands N <c>;<c>… | auto-run commands when iteration N’s breakpoint hits (commands N clear removes) |
watchpoint <blk>[<i>] [τ] (wp) | pause when a value changes by > τ |
diff | what changed in the iterate since the last iteration |
diagnose (diag) | live health report: named culprit residuals, KKT inertia, stalls |
source <file> | run debugger commands from a file |
goto N / restart | soft-rewind to a captured iteration |
resolve | re-solve from current x with staged options |
ask [question] | ask an LLM about the state (default Claude Code; $POUNCE_DBG_LLM=claude/codex/gemini/llm or a template) |
progress [on/off] | toggle JSON progress events |
detach | stop pausing; run to completion |
quit (q, exit) | stop the solve |
The JSON protocol
--debug-json makes stdout a pure stream of newline-delimited JSON
objects (the banner, problem stats, and final summary are routed to
stderr, and print_level is forced to 0). A program reads one JSON
object per line.
For an LLM agent: the whole contract
You do not need this page to drive the debugger — the protocol is self-describing. The contract is five lines:
- Launch
pounce <model> --debug-json(or--problem <name>), with the child’s stdin and stdout piped. - Read the first line —
hello. It enumerates everything you can do:commands(the verbs),events(breakpoint triggers),checkpoints(where you can pause),metrics(the scalar field names),blocks(the inspectable vectors), and acapabilitiesmap. Feature-detect off these lists, never off the version string. - Send commands, one JSON object (or bare string) per line, e.g.
{"cmd":"break if inf_pr<1e-6","id":1}then{"cmd":"continue","id":2}. Setidto correlate the matchingresult. - Read events until you see the one you want. Every
pause/progress/terminatedevent carries the same scalar metric fields, under the exact names listed inhello.metrics(objective,mu,inf_pr,inf_du,nlp_error,complementarity,iter) — so you can index them directly. - Finish with
{"cmd":"continue"}to run to completion (then readterminated), or{"cmd":"quit"}to stop early.
A complete minimal transcript (→ sent, ← received), eliding long lines:
← {"event":"hello","protocol":"pounce-dbg/1","commands":[…],"metrics":[…],…}
← {"event":"pause","checkpoint":"iter_start","iter":0,"objective":24.2,…}
→ {"cmd":"break if inf_du<1e-6","id":1}
← {"event":"result","request_id":1,"command":"break","ok":true,…}
→ {"cmd":"continue","id":2}
← {"event":"progress","iter":1,"objective":4.7,"inf_du":2.1e1,…}
… more progress events …
← {"event":"pause","checkpoint":"iter_start","iter":21,"inf_du":8.7e-7,"reason":"inf_du<1e-6"}
→ {"cmd":"continue","id":3}
← {"event":"terminated","status":"SolveSucceeded","iterations":21,…}
If you are wired in through the pounce-studio MCP server, you don’t
even spawn the CLI yourself: call debug_start to open a live session and
debug_command to step it (debug_state / debug_sessions /
debug_close round it out) — the server owns the child process and the
framing, and debug_start hands you the same hello handshake. Call
debug_session_guide for the contract and a launch snippet if you’d
rather drive --debug-json directly. The MCP analysis tools (diagnose,
find_stalls, …) are post-mortem over a finished report; the
debug_* tools and --debug-json are the live loop.
Session lifecycle
hello— emitted once, up front. The handshake.pause— at each stop.result— one per command, echoing the client’srequest_id.progress— one per iteration while running between pauses.sweep_result/sweep_summary— during asweep/multistart: onesweep_resultper completed solve, then asweep_summaryat the end.terminated— once, after the solve.
Commands
Write one per line to stdin, either a bare string or an object:
{"cmd": "print", "args": ["x"], "id": 7}
{"cmd": "break if inf_pr<1e-6", "id": 8}
"continue"
id (any JSON value) is echoed back as request_id on the matching
result, for async correlation.
hello
{"event":"hello","protocol":"pounce-dbg/1","pounce_version":"0.4.0",
"capabilities":{"inspect":true,"mutate_iterate":true,"mutate_mu":true,
"conditional_breakpoints":"compound","request_ids":true,
"viz":["block","delta","kkt","L"],"save":true,"load":true,"sweep":true,
"kkt_inspect":true,"diagnose":true,"llm_assist":true,
"pause_command":true,"equations":false,"structural_diagnose":false,
"rewind":"primal_dual","resolve":true,"terminal_checkpoint":true,
"interruptible":true,"progress_events":true,"async_pause":"checkpoint"},
"checkpoints":["iter_start","after_mu","after_search_dir","after_step",
"step_rejected","pre_restoration_entry",
"post_restoration_exit","terminated"],
"events":["resto_entered","resto_exited","regularized","tiny_step",
"ls_rejected","mu_stalled","nan"],
"commands":[…],"blocks":[…],"metrics":[…]}
A client should feature-detect off capabilities / checkpoints /
events rather than the protocol string — those lists are additive as
the debugger grows. A few capabilities are model-conditional: equations
and structural_diagnose are true only when the solve came from an
.nl file (which carries the source algebra and structural metadata) and
false for a built-in problem, as shown above.
The handshake above is the NLP filter-IPM’s. Capabilities are answered for
the backend that is actually running, so they agree with what the REPL will
do: on the convex / conic IPM the backend-conditional entries
(kkt_inspect, diagnose, mutate_mu, resolve, sweep, load,
structural_diagnose) are false, viz is ["block","delta"], and
blocks names that solver’s iterate blocks (x/s/y/z, plus
tau/kappa on the HSDE drivers) — see the capability matrix below.
pause
{"event":"pause","checkpoint":"iter_start","status":null,
"iter":3,"mu":2.0e-2,"objective":5.05,"inf_pr":0.0,"inf_du":2.7e-14,
"nlp_error":0.0237,"complementarity":1.9e-2,"dims":{"x":2,"s":0,"y_c":0,
"y_d":0,"z_l":2,"z_u":2,"v_l":0,"v_u":0},"breakpoints":[],"conditions":[],
"reason":"mu<0.05"}
status is non-null only at the terminated checkpoint. reason
carries the firing breakpoint / condition / event / interrupt.
result
{"event":"result","request_id":7,"command":"print x","ok":true,
"output":["x = [-1.18e0, 1.38e0]"],"data":{"name":"x","values":[-1.18,1.38]}}
output is human-readable lines; data is the structured payload
(present for inspection commands).
progress
{"event":"progress","iter":42,"mu":1.0e-5,"inf_pr":3.2e-7,"inf_du":1.1e-6,
"objective":12.34,"nlp_error":1.1e-6,"complementarity":9.0e-6}
Emitted once per outer iteration during a continue, so a UI can show
live progress instead of a hang. Carries the same scalar fields, under
the same names, as pause — so hello.metrics names index directly off
either event. Default on; toggle with the progress command.
terminated
{"event":"terminated","status":"SolveSucceeded",
"status_message":"Optimal Solution Found.","iterations":6,
"objective":4.9999999,"evals":{"obj":7,"obj_grad":7,"constr":1,
"constr_jac":12,"hess":6}}
Async pause
A running continue can be interrupted two ways, both pausing at the
next checkpoint with a reason:
- SIGINT —
process.kill(pid, "SIGINT")(or Ctrl-C). This is what a Debug Adapter’s pause button maps to. Reason:"interrupt (Ctrl-C)". - In-band command — send
{"cmd":"pause"}on stdin while the solve is running (JSON mode). No signals, so it works on Windows. Reason:"pause (requested)".
hello.capabilities.async_pause is "checkpoint", and
pause_command is true.
Tutorials
1. Why did this problem go to restoration?
$ pounce hard.nl --debug-json
{"cmd":"break on resto_entered"}
{"cmd":"continue"}
# → pause at checkpoint "pre_restoration_entry", reason "event: resto_entered"
{"cmd":"info"} # how infeasible is the iterate?
{"cmd":"print kkt"} # was the KKT singular / heavily regularized?
{"cmd":"print x"}
2. Catch a step that gets regularized
break on regularized
continue
# → pause at after_search_dir when delta_w > 0
print kkt # inertia n- vs expected; delta_w / delta_c
print dx # the (stabilized) Newton step
3. What-if: try a different μ strategy from here
break 5
continue # stop at iteration 5
set opt mu_strategy adaptive
resolve # re-solve from the iter-5 point with adaptive μ
4. Post-mortem on a failure
pounce maybe-infeasible.nl --debug-on-error
Runs unattended; if the solve returns anything but success you land at the final iterate:
── pounce-dbg ── TERMINATED (LocalInfeasibility) iter 11 obj=1.13e0 inf_pr=5.0e-1 inf_du=1.2e-8
pounce-dbg> print x
pounce-dbg> print kkt
5. Drive it from a program / agent
import subprocess, json
p = subprocess.Popen(["pounce", "hs071.nl", "--debug-json"],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True)
def send(cmd, **kw): p.stdin.write(json.dumps({"cmd": cmd, **kw}) + "\n"); p.stdin.flush()
def recv(): return json.loads(p.stdout.readline())
hello = recv() # capabilities / vocabulary
print(recv()) # initial pause
send("break if inf_du<1e-6", id=1)
print(recv()) # result, request_id=1
send("continue")
for line in p.stdout: # progress … pause … terminated
ev = json.loads(line)
if ev["event"] == "terminated": break
6. Is this solve sensitive to its start?
break on resto_entered # flag any start that falls into restoration
multistart 16 # 16 restarts (uniform in each finite box)
# → per-solve lines, then a table: succeeded / distinct minima / best
Swap multistart 16 for sweep starts.txt to run your own start
points (numpy.savetxt("starts.txt", X0, delimiter=",")). See
Multi-start and initialization sensitivity.
Beyond the interior-point loop
Everything above is the NLP filter-IPM. The same debugger — same command engine, same REPL — drives the other solvers too.
Convex and conic solves
The convex LP/QP interior-point solver and the HSDE conic drivers (SOCP,
the exponential / power cones, and small PSD cones) expose the same
checkpoints and commands as the NLP loop. The iterate blocks follow the QP
standard form — x (variables), s (cone slacks), y (equality
multipliers), z (inequality / cone multipliers) — and the HSDE drivers
additionally expose the homogenizing scalars tau / kappa as 1-element
blocks (print tau). set <block> and goto work as on the NLP path;
set mu is rejected, because the convex μ is derived from ⟨s, z⟩
(edit s/z to move it).
pounce model.nl --debug # LP / convex-QP (auto-routed) — IPM REPL
pounce_cblib model.cbf --debug # SOCP / exp / power / PSD (conic) — IPM REPL
pounce_cblib model.cbf --debug-script s.pdbg
Capability matrix
The flow-control core — checkpoints, stepping, breakpoints, watchpoints,
block/scalar inspection, diff, goto/restart, save, ask, and the
JSON protocol — works identically on every backend. The table below is
just the commands whose availability is backend- or model-conditional;
anything not listed is universal. A command that isn’t available on the
current backend returns an explicit “not available for this solver” error
(it never silently no-ops), and a JSON client should feature-detect off
hello.capabilities rather than this table.
| Command | NLP filter-IPM | Convex / conic IPM | Notes |
|---|---|---|---|
print kkt | ✅ | ➖ | convex IPM exposes no augmented-system inertia |
print rank | ✅ | ➖ | SVD rank of the equality Jacobian — NLP only |
print residuals | ✅ | ➖ | per-component primal/dual residuals — NLP only |
print active / inactive | ✅ | ➖ | needs a bound-slack notion |
print equation <name|row> | ⚠️ | ⚠️ | needs a source .nl model (capabilities.equations) |
viz kkt / viz L | ✅ | ➖ | depends on a captured KKT matrix / factor |
diagnose | ✅ | ➖ | live health report — NLP only |
resolve | ✅ | ➖ | warm re-solve from the current iterate — NLP only |
sweep / multistart / load | ✅ | ➖ | initialization-sensitivity tools — NLP only |
set opt <name> <val> | ✅ | ➖ | staged option edits — NLP only |
set mu | ✅ | ❌ | rejected on convex: μ is derived from ⟨s, z⟩ (edit s/z) |
set <block> / goto / restart | ✅ | ✅ | snapshots are supported on both |
✅ available · ⚠️ model-conditional · ➖ reports “not available for this solver” · ❌ explicitly rejected with an explanation
The streamed scalar metric vocabulary (iter, mu, objective,
inf_pr, inf_du, nlp_error, complementarity) is the same on every
backend — see hello.metrics. Each backend maps its native quantities onto
these NLP-centric names; the convex IPM, for instance, reports
nlp_error = max(pinf, dinf, μ). A backend that has no value for a metric
reports it as JSON null (never a dropped field), and a test pins the
emitted set to that single advertised vocabulary so it can’t drift.
A third backend — an interactive branch-and-bound tree debugger for a spatial global solver — is not part of this release.
Limitations
- Soft rewind only.
goto/restartrestore the primal-dual state, not strategy history (see the caveat above). set optis staged, not hot-applied to a running solve; it takes effect onresolve/ the next solve.
-
A. Lee, R. B. Parker, S. Poon, D. Gunter, A. W. Dowling, and B. Nicholson, “Model Diagnostics for Equation-Oriented Models: Roadblocks and the Path Forward,” Systems and Control Transactions 3:966–974 (2024). https://doi.org/10.69997/sct.147875 ↩ ↩2 ↩3 ↩4
Pyomo
Because POUNCE speaks the AMPL NL/SOL protocol, it drops into Pyomo through the AMPL Solver Library interface — exactly how Pyomo drives Ipopt.
The pyomo-pounce
package registers pounce as a Pyomo SolverFactory solver:
import pyomo_pounce # registers 'pounce'
from pyomo.environ import ConcreteModel, Var, Objective, SolverFactory
model = ConcreteModel()
model.x = Var(bounds=(-10, 10))
model.obj = Objective(expr=(model.x - 3) ** 2)
solver = SolverFactory('pounce')
solver.solve(model)
Options pass through the usual Pyomo mechanism:
solver.solve(model, options={'tol': 1e-10, 'max_iter': 500})
Under the hood, Pyomo writes the model to an AMPL .nl file, invokes
pounce problem.nl -AMPL, and reads the result back from the .sol
file. See Running Solves for the -AMPL solver mode.
Which pounce binary runs
import pyomo_pounce is required before SolverFactory('pounce').
Without it Pyomo does not know the solver and raises a clear
UnknownSolver / “plugin not registered” error — it does not silently run
some other pounce. With it imported, the plugin runs the binary bundled
in the pounce-solver wheel, independent of PATH; only a source/dev
install lacking that wheel falls back to a pounce on PATH (and the plugin
warns when it does).
Because two builds can report the same version string (X.Y.Z) while
behaving differently — a binary from before and after a fix does — a stale
pounce on PATH is otherwise hard to notice. To see exactly which
executable will run, its build (the git commit from pounce --about), and
whether a different pounce earlier on PATH would shadow it:
import pyomo_pounce
pyomo_pounce.check_binary() # prints a report; returns a dict
Which interface runs — and why it matters for timing
Pyomo has more than one way to drive an NL/SOL solver, and they are genuinely different code paths, not aliases. All of these reach POUNCE (verified against Pyomo 6.10.1):
| call | works | carries pyomo-pounce’s extras |
|---|---|---|
SolverFactory('pounce') | yes | yes |
contrib.solver SolverFactory('pounce') | yes | yes |
SolverFactory('pounce_v2') | yes | yes |
SolverFactory('ipopt_v2', executable=<pounce>) | yes | no |
SolverFactory('ipopt', executable=<pounce>) | yes | no |
SolverFactory('asl', executable=<pounce>, solver='pounce') | yes | no |
SolverFactory('appsi_ipopt', …) | no — takes no executable | — |
The first three are pyomo-pounce’s own registrations and the supported
routes; they are the only ones that bring the rest of this page with
them: the scaling_factor suffix handling, the sensitivity path, the
preflight/repair helpers, the guard against handing a model with live
integer variables to a continuous solver, and the bundled-binary
resolution above. The generic routes run the same solver and return the
same answer, but silently do without all of that.
ipopt and asl are Pyomo’s legacy solver interface; ipopt_v2 is
the newer pyomo.contrib.solver one. Driving POUNCE through ipopt_v2
needs a build carrying the two ASL-compatibility fixes noted in the
CHANGELOG under “Pyomo’s modern solver interface could not drive POUNCE
at all” — before them it failed on every model, because Pyomo v2 passes
options as key="value" in a single argv entry (quotes and all, since
no shell is involved) and because POUNCE’s .sol wrote an Options
count of 0, which the v2 .sol reader rejects.
Choosing between the legacy and v2 interfaces
Both of pyomo-pounce’s interfaces carry the same extras and return the
same numbers — a test in pyomo-pounce/tests/test_v2.py solves one model
through both and compares primals, objective, duals and reduced costs, so
this is checked on every CI run rather than asserted here.
import pyomo_pounce
from pyomo.environ import SolverFactory
from pyomo.contrib.solver.common.factory import SolverFactory as SolverFactoryV2
solver = SolverFactory('pounce') # legacy interface
solver = SolverFactoryV2('pounce') # v2 interface (a Results object)
solver = SolverFactory('pounce_v2') # v2 engine, legacy-style API
The v2 route needs Pyomo ≥ 6.10.1 (where the SolutionLoader /
get_vars API it builds on landed — pyomo.contrib.solver.common
exists from 6.9.2, but 6.9.2–6.10.0 ship the older
SolutionLoaderBase / get_primals) and pounce-solver > 0.9.0
(Pyomo’s asl_sol_reader is strict where the legacy reader is lenient
and needs the per-model .sol Options echo added after 0.9.0).
pip install pyomo-pounce[pyomo-v2] asks for both. Neither applies to
SolverFactory('pounce'): on an older Pyomo the legacy plugin works
exactly as before and pyomo_pounce.HAVE_V2_INTERFACE reports False.
They differ in API and in per-solve overhead. The v2 interface returns a
Results object and hands the solution back through a solution loader
(so load_solutions=False gives you the values without touching the
model); the legacy one returns a SolverResults and loads into the model
as a side effect. Options are solver_options={...} on v2 against
options={...} on the legacy route.
The v2 route can be materially faster outside the solve, and how much
depends on the model’s shape. Same POUNCE binary, wall clock around
solve() minus POUNCE’s own reported time:
| model | legacy remainder | v2 remainder |
|---|---|---|
plain pyomo.dae four-tank collocation, n = 3,010 | 0.109 s | 0.104 s |
drto/IDAES quad_tank N=100, n = 2,910 | 0.553 s | 0.301 s |
drto/IDAES cart_pole N=100, n = 2,810 | 0.566 s | 0.295 s |
On the plain model the two are indistinguishable; on the IDAES-shaped ones the legacy interface adds roughly 0.25 s per solve (~1.8×). If your models are of that kind and you are solving many of them, the v2 route is worth taking. (Figures from the #552 measurements; the first row was measured on Linux, the other two on Windows, so read down the columns rather than across the rows.)
If you are benchmarking, put both solvers on the same interface.
solver.solve(model) is not only the solve: it is Pyomo writing the
.nl, launching the process, reading the .sol back and loading it into
the model. Timing around that call and subtracting the solver’s own
reported time leaves a remainder that is mostly Pyomo’s work, and — as
the table above shows — it is not the same work on every interface. On
the 3,010-variable collocation model that remainder breaks down as
~0.082 s Pyomo writing the .nl, ~0.020 s process spawn plus POUNCE’s
own .nl read and setup, and ~0.008 s Pyomo reading the .sol and
loading it. So comparing SolverFactory('pounce') against
SolverFactory('ipopt_v2', …) compares two Pyomo interfaces as much as
two solvers, and attributing the remainder to either solver’s file
handling will mislead you. Use the same interface on both sides, or
compare the solvers’ own reported times.
User scaling with the scaling_factor Suffix
A badly conditioned model converges poorly, and often you know its
natural units better than the solver can infer from gradients at x0.
The standard Pyomo channel for saying so is the scaling_factor
Suffix, read exactly as Ipopt reads it:
model.scaling_factor = Suffix(direction=Suffix.EXPORT)
model.scaling_factor[model.obj] = 1e-3 # objective in MW, not W
model.scaling_factor[model.mass_balance] = 1e2 # one constraint
model.scaling_factor[model.energy_balance] = 1e2 # or a whole container
solver.solve(model, options={'nlp_scaling_method': 'user-scaling'})
Both halves are required: without nlp_scaling_method=user-scaling the
Suffix is inert (a scaling_factor Suffix also drives Pyomo’s own
core.scale_model transformation, which never involves the solver), and
without the Suffix the option has nothing to apply — pyomo-pounce warns
in that case rather than leaving you to wonder.
Rules, matching AMPL/Ipopt:
- Only an export-enabled Suffix counts (
Suffix.EXPORTorSuffix.IMPORT_EXPORT). - Components you do not list are unscaled, as are components listed with
a factor of
0. - An entry on a container applies to every member.
- Entries on inactive constraints/objectives and on fixed variables are skipped — none is a row or column of the problem the solver is handed.
- Scaling changes conditioning, never the answer: solutions, duals, and everything the sensitivity accessors report come back in your model’s units.
Variables can be scaled, and a factor on a Var is applied as a
change of variables inside the solver: the algorithm works in the
scaled coordinates and the solution, the duals, and the bound
multipliers come back in your model’s own units. No clone of the model
is made and no propagate_solution step is needed, which is what
distinguishes this from Pyomo’s core.scale_model transformation.
Factors must be positive and finite. A negative factor would reverse a
variable’s direction and swap its bounds, so it raises rather than
being applied.
This works on both solve paths: the ordinary ASL/subprocess solve and
the in-process path taken when the model carries sensitivity
declarations — including the accessors themselves.
covariance(), information(), gradient() and estimate() read the
solver’s KKT factorization directly rather than through the scaling
layer, so they carry the factors through their own natural-units
translation and answer in your model’s units on a variable-scaled solve
(issue #486).
Preflight and initialization
A Var whose .value was never set is written as 0 into the
.nl file, so an uninitialized model actually starts at the origin
(see Initialization and Warm Starts). The package
ships a preflight check plus an initialization pipeline for exactly
this:
import pyomo_pounce
report = pyomo_pounce.preflight(model) # what will POUNCE see at x0?
print(report) # unset vars, bound/constraint
if report.fatal: # violations, NaN/inf evaluations
...
# fill -> repair -> block-solve, with the decisions held constant:
rep = pyomo_pounce.initialize(model, decisions=[m.feed, m.reflux])
if not rep.block.square:
print(rep) # names of what you forgot to specify
preflight evaluates every active constraint and the objective at the
current values with unset values treated as 0 (exactly what the NL
writer sends), restores the model untouched, and reports what
iteration 0 will see; report.fatal means the solve would abort with
Invalid_Number_Detected.
initialize follows the workflow you would run by hand on, say, a
distillation column: set the decisions (feed, reflux, boilup), solve
for a physical profile with them held constant, then let the optimizer
move them. Its three stages are also available individually:
pyomo_pounce.initialize_missing_values(model) # bounds-aware fill
# (midpoint / one unit
# inside / zero)
pyomo_pounce.project_to_feasible(model) # min-norm repair: move the
# current point onto the
# model's own constraints
# (one POUNCE solve)
rep = pyomo_pounce.block_initialize( # solve the equality
model, decisions=[m.feed, m.reflux]) # system's square blocks
# in calculation order
initialize_missing_values fills each variable independently, so the
fill can be internally inconsistent (mole fractions that do not sum to
one); project_to_feasible repairs that by minimizing
sum((v - v0)**2) subject to the model’s active constraints and
bounds — the full nonlinear projection, solved with POUNCE, with the
original objective restored afterwards.
Both stages guarantee that a failed solve leaves variable values exactly as they were: a diverged projection restores the pre-projection point, and a failed block solve restores that block’s seeds and stops, so initialization can never make your starting point worse than it found it.
block_initialize is IDAES-flavored initialization without
hand-written routines. decisions= holds the listed variables at
their current values for the solve and releases them afterwards (each
must have a value). The active equality constraints are decomposed
(Dulmage-Mendelsohn, via pyomo.contrib.incidence_analysis); the
square part is solved block by block in topological order by Pyomo’s
solve_strongly_connected_components (1x1 blocks by Newton, larger
blocks by POUNCE), filling Var.value along the way. When the system
is not square, report.square is False and the offending
variables and constraints are reported by name —
underconstrained_variables is the list of things you forgot to
specify or flag as decisions, overconstrained_constraints the
redundant or conflicting specifications. Permanently-known inputs can
simply be fix()ed instead of listed as decisions.
The analysis half is also available on its own:
rep = pyomo_pounce.block_analyze( # the DM partition only:
model, decisions=[m.feed, m.reflux]) # nothing seeded or solved
rep.underconstrained_variables # VarData objects, uncapped
rep.n_extra_degrees_of_freedom # how many specs are missing
rep.variable_blocks # the calculation order
block_analyze runs the same decision handling and the same
Dulmage-Mendelsohn decomposition, but touches nothing: no values are
read or written (so, unlike block_initialize, the decisions do not
need values), and no solve happens. Where the initialization reports
cap their name lists for display, block_analyze returns the full
partition as the component objects themselves: the underconstrained
and overconstrained subsystems, the square part, and its
block-triangular calculation order. Use it to diagnose a large model’s
specification, or as the structural front end for tooling that decides
what to specify before calling initialize /
block_initialize to do the work.
Repairing a bad specification
Some specifications are structurally wrong, not just badly started. On
a distillation column at steady state, holding all the flow
controls leaves the drum levels undetermined while the holdup balances
become redundant — square by count, singular in structure, and no
starting point fixes that. block_repair_plan plans a valid
specification instead of failing on the broken one:
plan = pyomo_pounce.block_repair_plan(
model,
decision_candidates=[m.LT, m.VB, m.D, m.B]) # what you would like held
plan.decisions # candidates a square system can hold
plan.pruned # candidates the equalities claim: solved for instead
plan.pinned # what nothing determines: hold at values you choose
The candidates are pruned to the subset a valid specification can
hold: matching prefers plain variables over candidates, which provably
minimizes the number pruned, and among candidates earlier-listed
ones are preferentially kept, so the listing order acts as an
implicit priority when a pruning tie could go either way. The pins
need no user input: a
variable is pinned when every one of its edges is provably unusable —
the key case being an equation 0 == f/g, which cannot determine a
variable appearing only in the denominator g, since its sensitivity
there vanishes at every solution. That is exactly the shape
substituting d/dt = 0 into a dynamic balance produces, which is how
loose integrators (drum levels with no weir feedback) hide in
steady-state models. Like block_analyze it is a plan, not an action:
nothing is fixed, read, or written, and no values are needed.
loose_variables (undetermined, not repairable) and
redundant_constraints (satisfiable by no specification) are genuine
model defects.
initialize and block_initialize run the same check on their
decisions automatically (repair="auto", the default). A square
specification is used exactly as given, the shipped behavior. A broken
one is repaired: the decisions become the candidate pool, conflicting
ones are pruned (they need no values), pins are seeded bounds-aware
and never at zero when valueless (a pin lives in denominators, so zero
is the one forbidden seed), and report.repair records the plan (None
when nothing was needed). Pass repair="off" for the strict path:
decisions held exactly as given, and a non-square specification is
reported (report.square, the name lists) instead of repaired. The repair is call-scoped exactly like the
decisions themselves (fixed flags restored, values only), so it never
changes your model’s own specification. To apply a plan to a model
you intend to solve — a square simulation, say — fix plan.decisions
and plan.pinned and leave plan.pruned free; which variables to fix
is a modeling decision, so the plan leaves it to you.
GAMS
POUNCE plugs into GAMS as an NLP solver, so a model can hand its problem to POUNCE with:
option nlp = pounce;
solve mymodel using nlp minimizing obj;
There are two ways to make POUNCE available to GAMS. Pick one:
| Route | Install | What it is |
|---|---|---|
| pip (recommended) | pip install pounce-solver[gams] then pounce-gams register | A pure-Python solver link built on GAMS’s own gamsapi package. No compiler, no sudo, survives GAMS upgrades. |
| native C link | build + sudo make -C gams install | A C shared library installed into the GAMS system directory. Adds active-set-SQP working-set / state-file warm starts. See gams/README.md. |
Both register POUNCE under the same name (pounce) for NLP, DNLP, and
RMINLP models — POUNCE is a continuous local NLP solver, so mixed-integer and
conic model types are not offered here.
The pip route
1. Install
pip install pounce-solver[gams]
The [gams] extra pulls in
gamsapi[core] — GAMS’s own expert-level
GMO/GEV Python bindings — and PyYAML. The bindings dlopen the GAMS C
libraries from your local install, so gamsapi must match your GAMS
version. POUNCE itself redistributes nothing GAMS-owned. If your GAMS and
gamsapi versions disagree, install the matching one from your GAMS system
(GAMS ships a gamsapi wheel under apifiles/Python/), or:
pip install 'gamsapi[core]==<your GAMS X.Y.Z>'
2. Check the install
pounce-gams status
reports whether gamsapi imports, the config directory POUNCE will register
into, and whether POUNCE is already registered:
gamsapi: available
gamsapi 53.2.0 importable
config dir: /Users/you/Library/Preferences/GAMS
gamsconfig: /Users/you/Library/Preferences/GAMS/gamsconfig.yaml (missing)
POUNCE solver: not registered
3. Register
pounce-gams register
This writes a tiny launcher script and a solverConfig entry into your GAMS
per-user gamsconfig.yaml. It merges — any other solvers already in that
file (CONOPT overrides, discopt, …) are preserved — and is idempotent (re-running
just updates POUNCE in place). The per-user config directory GAMS searches is
OS-specific:
| OS | Directory |
|---|---|
| macOS | ~/Library/Preferences/GAMS |
| Linux | $XDG_CONFIG_HOME/GAMS (else ~/.config/GAMS) |
| Windows | %LOCALAPPDATA%\GAMS (else …\Documents\GAMS) |
Override the target with --config-dir <path> (e.g. to register into the GAMS
system directory instead). To undo, pounce-gams unregister.
No sudo is needed and nothing is written into the GAMS system directory, so a
GAMS upgrade does not wipe the registration.
4. Solve
option nlp = pounce;
solve mymodel using nlp minimizing obj;
GAMS invokes the launcher with a control file; the launcher runs the Python link, which reads the model through GMO/GEV, solves it with POUNCE, and writes the primal/dual solution and GAMS model/solve status back.
Marginals
Equation marginals (.M) follow the usual GAMS convention: they are stated
against the objective as you wrote it, for both minimizing and
maximizing models. POUNCE always minimizes internally, so for a
maximizing model it solves min(−f) and converts its multipliers back
via pi = −obj_sign · λ (see gams_pi in python/pounce/gams/link.py and
the matching block in gams/gams_pounce.c).
Variable marginals (.M on variables, i.e. reduced costs) are
z_L − z_U, carrying the same obj_sign factor.
Before v0.9.1 the conversion omitted the
obj_signfactor, so equation marginals onmaximizingmodels came back with the wrong sign (#272).minimizingmodels, objective values, variable marginals and status mapping were never affected. Both the pip link and the native C link were affected identically, so the install method made no difference.
Option files
If a model sets mymodel.optfile = 1, POUNCE reads pounce.opt (.op2,
.op3, … for higher optfile values). Each line is a keyword value pair
using POUNCE’s option names; lines starting with * or # are
comments. The GAMS iterlim and reslim are honored as max_iter and
max_wall_time.
* pounce.opt
tol 1e-10
max_iter 500
Machine-readable solve report
Set json_output in pounce.opt to also emit a structured
pounce.solve-report/v1 JSON report (identical to the CLI’s --json-output,
consumable by pounce-studio):
json_output my_solve.json
json_detail full * "summary" or "full"; default is "full"
See the JSON Solve Report schema for the format.
Notes & limitations
- Version match. The single most common failure is a
gamsapi↔ GAMS version mismatch;pounce-gams statusdiagnoses it. - Warm starts. The active-set-SQP working-set / state-file warm-start
features (
algorithm active-set-sqp,sqp_state_file) are currently only in the native C link, where eachsolvereuses an in-process state. The pip link runs each solve as a fresh process; full warm-start parity is a planned follow-up.
Python API
POUNCE ships a Python wrapper that is intentionally cyipopt-compatible: code written for cyipopt typically runs against POUNCE by changing only the import.
Install
cd python
pip install maturin
maturin develop --release # builds the native extension into your venv
Optional extras:
pip install -e .[jax] # JAX integration
pip install -e .[torch] # PyTorch integration
pip install -e .[dev] # tests + jax + torch + scipy
cyipopt-style interface
import numpy as np
import pounce
class HS071:
def objective(self, x):
return x[0]*x[3]*(x[0]+x[1]+x[2]) + x[2]
def gradient(self, x):
return np.array([
x[0]*x[3] + x[3]*(x[0]+x[1]+x[2]),
x[0]*x[3],
x[0]*x[3] + 1.0,
x[0]*(x[0]+x[1]+x[2]),
])
def constraints(self, x):
return np.array([np.prod(x), np.dot(x, x)])
def jacobianstructure(self):
return (np.repeat([0, 1], 4), np.tile([0, 1, 2, 3], 2))
def jacobian(self, x):
return np.array([
x[1]*x[2]*x[3], x[0]*x[2]*x[3], x[0]*x[1]*x[3], x[0]*x[1]*x[2],
2*x[0], 2*x[1], 2*x[2], 2*x[3],
])
prob = pounce.Problem(
n=4, m=2,
problem_obj=HS071(),
lb=[1]*4, ub=[5]*4,
cl=[25, 40], cu=[2e19, 40],
)
prob.add_option('tol', 1e-8)
x, info = prob.solve(x0=np.array([1.0, 5.0, 5.0, 1.0]))
print(info['status_msg'], info['obj_val'], x)
Verifying convergence / trustworthy duals
info carries the final KKT residuals so a consumer can independently
check how converged a returned point is — useful when the duals
(info["mult_g"], info["mult_x_L"], info["mult_x_U"]) feed a
downstream certificate (e.g. dual bound tightening). Two flavors:
final_kkt_error/final_dual_inf/final_constr_viol/final_compl— the residuals the convergence test saw, in the internally-scaled NLP space (thenlp_scaling_methodfactors).final_unscaled_kkt_error/final_unscaled_dual_inf/final_unscaled_constr_viol/final_unscaled_compl— the same residuals with the scaling divided back out, i.e. in your original problem units. Equal to the scaled values when no scaling activates.
On ill-conditioned problems nlp_scaling can deflate the scaled residual
enough that the default test reports Solve_Succeeded while the
unscaled duals have drifted. The robust guard is to read the residual
yourself rather than trust the status enum alone — info["status"] is a
coarse signal, and some callers treat Solve_Succeeded (0) and
Solved_To_Acceptable_Level (1) identically:
x, info = prob.solve(x0=...)
converged = info['final_unscaled_kkt_error'] <= 1e-6 # your own threshold
If you’re feeding the duals into a downstream certificate (e.g. a dual bound), prefer building a safe bound from the multipliers — valid for any dual-feasible point, so it doesn’t hinge on the solver’s exact convergence.
Two convenience knobs back this up:
- Tighten the (unscaled) component tolerances —
dual_inf_tol,constr_viol_tol,compl_inf_tolgate on the unscaled residuals, so the solver keeps iterating until it actually meets them (or exits non-success). kkt_fidelity_tol(default 0 = off) — a defensive post-solve relabel: aSolve_Succeededwhosefinal_unscaled_kkt_errorexceeds it is demoted toSolved_To_Acceptable_Level. Note this only helps a caller that distinguishes those two statuses; if yours doesn’t, gate on the residual directly as above.
Where the time went (info["timing"])
Every Problem.solve attaches a per-subsystem wall-clock breakdown so you
can attribute a solve’s runtime without patching or rebuilding the solver.
info["wall_time"] is the overall-algorithm total (seconds); info["timing"]
is a dict of the same total plus its components:
x, info = prob.solve(x0=...)
t = info["timing"]
print(t["overall_alg"]) # total solve wall time
print(t["linear_system_factorization"], # KKT factorization vs …
t["linear_system_back_solve"], # … back-solve, and their
t["linear_system_total"]) # sum (total linear algebra)
print(t["eval_objective"], t["eval_gradient"],
t["eval_constraints"], t["eval_constraint_jacobian"],
t["eval_lagrangian_hessian"]) # per-callback eval time
The scipy-style pounce.minimize facade mirrors these onto the result as
res.wall_time and res.timing (also in res.info). The callback split is
what lets you see, for example, that a reduced-space / variable-aggregation
solve converges in few iterations but spends most of its time in a densified
Lagrangian-Hessian evaluation — the func/Jacobian/Hessian story becomes a
direct measurement rather than an inference. All values are wall-clock
seconds; unused subsystems read 0.0.
Caller-supplied KKT ordering (set_ordering)
A structure-aware presolve can hand pounce a fill-reducing permutation for
the KKT linear solver that the built-in AMD/METIS pass cannot derive — a
block-triangular / Schur ordering (Parker, Garcia & Bent, arXiv:2602.17968)
or a tearing ordering from equation-oriented decomposition. Install it on
the low-level Problem before solving:
prob = pounce.Problem(n, m, problem_obj=...)
prob.set_ordering(perm) # 0-based new-to-old permutation (list / int array)
x, info = prob.solve(x0=...)
# prob.get_ordering() -> the installed permutation, or None
# prob.clear_ordering() -> restore the feral_ordering default
perm[k] is the original index that becomes index k. Its length must
equal the augmented KKT system dimension (variables + slacks + constraint
duals), not the problem’s n; for an unconstrained problem that is n,
but with constraints it is larger. The ordering is validated inside FERAL as
a bijection — a wrong length or a duplicate fails the factorization and the
solve returns a non-success status (e.g. Error_In_Step_Computation) rather
than crashing or returning a wrong answer, since a permutation only affects
fill and pivot order, never the computed solution. set_ordering is
persistent config (it applies to every subsequent solve() until
clear_ordering()) and is honored only by the default FERAL backend. This
maps to FERAL’s OrderingMethod::External (feral#107).
Block-triangular / Schur KKT solve (set_kkt_schur_block)
If a presolve can identify a reducible block of the KKT system — e.g. the nonsingular block-triangular submatrix a reduced-space / variable-aggregation analysis exposes (Parker, Garcia & Bent, arXiv:2602.17968) — it can hand that block to pounce, which Schur-complements it out and factorizes only the two diagonal blocks, recovering the full-system inertia a priori via Sylvester’s law:
prob = pounce.Problem(n, m, problem_obj=...) # needs an exact Hessian
prob.set_kkt_schur_block(indices) # KKT-space indices of the Schur block
x, info = prob.solve(x0=...)
# prob.get_kkt_schur_block() -> installed indices, or None
# prob.clear_kkt_schur_block()
indices are KKT-space indices into 0..dim where
dim = n + n_slack + n_eq + n_ineq, in the solver’s internal
x, slack, eq-dual, ineq-dual block order (e.g. for an all-equality problem
the constraint-dual block is range(n, n + n_eq), and the primal block is the
positive-definite eliminated block — the classic range/null-space split). The
method wins only when the Schur block is much smaller than the eliminated
block (the dense Schur complement is O(n_schur²) to store and O(n_schur³)
to factor). When the partition is unsuitable — too large a fraction of the
system, malformed, or a diagonal block turns out singular — the solver falls
back to the standard full-space path transparently, so the hook can never
break a solve; it only changes how the identical system is factored, never
the solution. Honored on the default feral + exact-Hessian path.
Building a model in memory (NlExpr / build_nl_problem)
pounce.read_nl("model.nl") gives you pounce’s native reverse-mode-AD
evaluators for an AMPL .nl file on disk. Two sibling entry points reach
the same machinery without a file:
pounce.parse_nl_text(text, var_names=None, con_names=None)— the same parser, fed a string. For a frontend that already generates.nl, this drops the temp file and its cleanup. There are no sibling.col/.rowfiles to read, so names are passed explicitly.pounce.build_nl_problem(...)— skip.nlentirely and hand over expression trees built frompounce.NlExpr.
Both return the same NlProblem class read_nl does, with the same
surface: objective, gradient, constraints, jacobian /
jacobian_structure, hessian / hessian_structure,
hessian_vector_product, and variant. They also feed solve_nlp_batch.
import pounce
x = pounce.NlExpr.vars(2) # [Var(0), Var(1)]
rosen = (1 - x[0]) ** 2 + 100 * (x[1] - x[0] ** 2) ** 2
p = pounce.build_nl_problem(
n=2,
objective=rosen,
constraints=[x[0] ** 2 + x[1] ** 2],
g_l=[0.0], g_u=[2.0],
x0=[-1.2, 1.0],
)
p.objective(p.x0) # float
p.gradient(p.x0) # ndarray[n]
(x_star, info), = pounce.solve_nlp_batch([p])
Bounds default to unbounded (±1e19, the .nl sentinel) and x0 to
zeros. minimize=False maximizes; as with a parsed maximize model, the
returned objective/gradient/Hessian are negated so that minimizing them
solves the model, and p.minimize records the original sense.
NlExpr supports the Python arithmetic operators (+ - * / ** -, abs,
with plain numbers accepted on either side) plus method-form
transcendentals: sqrt exp log log10 sin cos tan asin acos atan sinh cosh tanh asinh acosh atanh erf. Multi-argument and control-flow nodes are
static methods: NlExpr.sum(iterable), NlExpr.atan2(y, x),
NlExpr.min(*args), NlExpr.max(*args), NlExpr.compare(op, a, b),
NlExpr.select(cond, then_, else_), and NlExpr.logical_and /
logical_or / logical_not.
Comparison is spelled NlExpr.compare("<", a, b) rather than a < b:
overloading Python’s comparison operators would break every ordinary use
of an expression in a container. The result is piecewise constant (zero
derivative), and pairs with NlExpr.select.
Why not just write .nl? Because the round trip is lossy. .nl
writers commonly refuse atan2 (no two-argument funcall path) and
min/max (they force a DNLP model type), and AMPL has no erf opcode
at all — yet pounce’s tape differentiates all three natively. Built here,
they survive:
x = pounce.NlExpr.vars(2)
p = pounce.build_nl_problem(n=2, objective=pounce.NlExpr.sum([
pounce.NlExpr.atan2(x[0], x[1]),
pounce.NlExpr.min(x[0], x[1]),
x[0].erf(),
]))
Operands are shared, not copied. a * b references its operands
rather than deep-copying them, so building an expression costs the same
whether the pieces are two variables or two half-million-node models. Two
consequences worth knowing:
- Accumulating in a Python loop is linear in the number of terms. It still
nests one level deeper per term, though, and nesting is capped (below) —
so a many-term sum still belongs in one flat
NlExpr.sum(terms)node, which tapes better and is one level whatever its length. The same goes formin/max, flat in their argument count too. - Reusing a Python name reuses the subexpression.
t = x[0] * x[1]used in ten places is one shared body on the tape, evaluated once per sweep, with its adjoint summing the ten contributions — the same value and the same derivatives as writing it out ten times, off a tape a tenth the size. Expressions that are only tractable as a DAG work too:for _ in range(40): e = e * eis 40 nodes describingx ** 2**40, and it builds, tapes, and differentiates in under a millisecond.
e = pounce.NlExpr.const_(0.0)
for t in terms: # linear, but len(terms) levels deep
e = e + t
e = pounce.NlExpr.sum(terms) # linear and one level — prefer this
Nesting is capped at NlExpr.max_depth (10 000), and exceeding it
raises ValueError. Every consumer of an expression — the tape builder,
the problem assembler, freeing it, and the .nl parser that produces one
— recurses once per level, so a deep enough tree overflows the stack,
which is a hard crash rather than an exception. Two things keep that
unreachable: those walks run on a worker thread with a 64 MB stack, so
what is survivable does not depend on the calling thread (8 MB on a
macOS/Linux main thread, 1 MB on Windows, less on a threading.Thread),
and the cap then keeps the depth well inside it.
The same limit applies to read_nl and parse_nl_text, which enforce
it on what they parsed rather than as it is built — a model that arrives
already built cannot be capped during construction. A .nl file that
spells a long sum as an o0 (binary +) chain rather than o54 (n-ary
sum) is the way to hit it. The cap bounds nesting, not size: NlExpr.sum
and o54 are one level regardless of term count, so wide models are
unaffected. Each expression’s .depth is readable.
For checking a subexpression before wiring it into a model, NlExpr has
.eval(x), .gradient(x), and .variables(), which build a one-off tape
for that expression alone.
Two things NlExpr does not do: it cannot carry AMPL imported (external)
functions — build_nl_problem has nowhere to put the F-segment
declarations that bind them, so use read_nl / parse_nl_text for a
model that needs them — and it cannot be pickled. copy.copy and
copy.deepcopy do work.
Hessian-vector products
NlProblem.hessian_vector_product(x, v, lam=None, obj_factor=1.0) returns
(obj_factor·∇²f + Σᵢ lamᵢ·∇²gᵢ) · v without ever forming the Hessian —
one forward-over-reverse AD pass per tape, seeded with v directly.
hessian(...) instead runs one such pass per Hessian color and decodes
the compressed columns into the sparse lower triangle, so on a large model
the matrix-free call is cheaper by roughly the chromatic number of the
coloring. It is the operator a Newton–Krylov / truncated-CG step wants.
Hv = p.hessian_vector_product(x, v) # objective block only
Hv = p.hessian_vector_product(x, v, lam, 1.0) # full Lagrangian
Available on every NlProblem, however it was built — read_nl,
parse_nl_text, build_nl_problem, or variant.
Dense and sparse directions. v may be any of:
v | result |
|---|---|
dense length-n vector (ndarray of any dtype or stride, list, sequence) | (n,) |
dense (n, k) array of k directions | (n, k) |
SciPy sparse (n,) vector, (n, 1) column, or (n, k) matrix | matching its shape |
The shape rule is the same dense or sparse: (n,) or (n, k). A (1, n)
row vector raises rather than being guessed at — for a square-ish
block it is indistinguishable from k directions of the wrong length.
Watch for this with SciPy sparse matrices, which shape a 1-D input as a
row: csr_matrix(v) on a length-n v is (1, n) and will be refused.
Pass v[:, None], or use the 1-D sparse array API — coo_array(v) is
genuinely (n,), on SciPy >= 1.14.
import scipy.sparse as sp
p.hessian_vector_product(x, sp.csc_matrix(V)) # sparse block of directions
p.hessian_vector_product(x, np.eye(n)) # densify: H, in one call
A sparse v is densified on the way in, and an all-zero direction is
skipped, so a mostly-empty block costs only the columns that carry signal.
The sparsity that actually pays here is the model’s, not v’s: every
pass is O(tape ops), never O(n²), whichever way v arrives. On a
model with a tridiagonal Hessian — the usual IPM shape — that is the whole
game.
The block form is not just a loop: the forward sweep depends only on x,
so k directions share one sweep per tape where k separate calls would
repeat it. Only the forward-tangent and reverse-over-tangent passes are
per-direction.
The result is always dense, including for sparse input. ∇²L · v is
dense in general even when both ∇²L and v are sparse, so a sparse
return type would advertise an economy the product does not have. When you
want the sparse Hessian itself, hessian_structure() + hessian(x) give
it directly as a COO lower triangle:
hr, hc = p.hessian_structure()
lower = sp.coo_matrix((p.hessian(x), (hr, hc)), shape=(p.n, p.n)).tocsr()
H = lower + lower.T - sp.diags(lower.diagonal()) # full symmetric matrix
NaN and Inf do not spread through structural zeros. AD never
multiplies by an entry that is not in the tape, so a NaN in one component
of v stays confined to the variables actually coupled to it. A dense
H @ v computes 0 * nan and smears NaN across every row. On a
block-diagonal Hessian with v = [nan, 0, 1, 0], the dense product is
[nan nan nan nan] while the HVP is [nan nan 2.42 3.08]. Arguably the
better semantics, but it does mean the HVP is not bit-equivalent to a
dense product on non-finite input.
Sharing one NlProblem across threads
An NlProblem may be built on one thread and evaluated — or garbage
collected — on any other. Threaded hosts (a branch-and-bound worker pool,
a ThreadPoolExecutor) can hold one shared evaluator rather than one
tape per worker:
p = pounce.read_nl("model.nl")
with ThreadPoolExecutor(max_workers=8) as pool:
values = list(pool.map(p.objective, points)) # one tape, N workers
The evaluators do not release the GIL, so concurrent calls serialize
rather than overlap — the win is memory (one copy of the tapes) and the
absence of thread-affinity ceremony, not parallel throughput. For actual
parallelism across instances, use
solve_nlp_batch, which releases
the GIL and runs the whole batch on a Rayon pool.
DenseLU and SparseLU carry the same guarantee: factor on one thread,
back-solve on another.
Solver, QpFactorization and QpSensitivity are the exceptions —
their held Ipopt / KKT factorizations are genuinely thread-affine, so
each must be used and released on the thread that created it. Keep them
in a threading.local (not a dict keyed by thread id: CPython clears a
threading.local on the owning thread as it exits, so the object is both
built and dropped where it belongs), which is what pounce.jax’s
JaxProblem does internally. Using one from another thread raises a
PanicException — note that this derives from BaseException, so an
except Exception will not catch it.
Batched NLP solving (solve_nlp_batch)
pounce.solve_nlp_batch solves N independent NLPs and returns one
(x, info) pair per input, in input order — for parametric sweeps,
multi-start, MPC chains, or branch-and-bound node relaxations where
each sibling differs only in tightened bounds.
import numpy as np
import pounce
base = pounce.read_nl("model.nl") # native-Rust evaluators
# One parsed structure, many variations (cheap clones of the AD tapes):
rng = np.random.default_rng(0)
batch = [base.variant(x0=np.asarray(base.x0) + rng.normal(0, 0.01, base.n))
for _ in range(24)]
results = pounce.solve_nlp_batch(batch, options={"tol": 1e-8})
for x, info in results:
print(info["status_msg"], info["obj_val"])
NlProblem.variant(x0=, x_l=, x_u=, g_l=, g_u=) builds a sibling
instance with per-instance starting point / bounds; everything
structural (expression DAG, AD tapes, sparsity, coloring) is shared
work that is not redone.
Native vs. callback inputs — the GIL caveat. Both kinds solve in parallel, with different ceilings:
NlProbleminputs (fromread_nl/variant) are native-Rust reverse-mode-AD evaluators. The batch runs on a Rayon thread pool with the GIL fully released; each worker solves its instance end-to-end with an inner-serial factorization (outer-parallel / inner-serial, the same model assolve_qp_batch).- Callback-based
Probleminputs (passx0s=, one starting point per instance) also run one instance per worker, but everyobjective/gradient/constraints/jacobian/hessiancall re-acquires the GIL. The Python share of the work is therefore serialized: the speedup scales with the Rust/Python work ratio — medium and large problems whose factorizations dominate parallelize well (~4x on 4 cores for an n=800 banded NLP with vectorized NumPy callbacks); tiny problems whose callbacks dominate won’t. EachProblem’s ownadd_optionsettings are honored per instance, withoptions=as a batch-level overlay.
With parallel=False either path solves one instance at a time,
letting each factorization parallelize internally — better for a few
large instances. For the batch, print_level defaults to 0 (N workers
interleaving iteration tables is noise); pass an explicit
print_level to override.
Warm-start chaining (MPC / B&B). Feed one batch’s results into the next solve of a nearby batch:
results = pounce.solve_nlp_batch(batch_t) # cold
results = pounce.solve_nlp_batch(batch_t1, warms=results) # warm
Each instance is seeded with the previous x and duals, the converged
barrier parameter (info["mu"]) is threaded into mu_init, and
warm_start_init_point=yes is forced. A warm start changes iteration
counts, never solutions (re-solving the 24-instance gaslib sweep warm
drops 482 total iterations to 120). A dimension-mismatched warm entry
falls back to that instance’s cold start.
Partial multiplier seeds (Problem.solve / Solver.solve). The
lagrange=, zl=, zu= arguments take the solver’s internal
conventions (+λ with L = f + λᵀg, non-negative bound multipliers).
Under warm_start_init_point=yes, a NaN entry means “unseeded”: the
warm-start initializer substitutes its own resolved default
(bound_mult_init_val for bound multipliers, the warm path’s 0 for
equality duals) before its clamps, so a partial seed never turns into
a zero bound multiplier on an active bound, which is a contradictory
KKT certificate. This contract belongs to the warm-start initializer
only: the batch warms= hand-off above and the SQP working_set
arrays do not route through it and must not carry NaN.
Identical-sparsity batches (share_structure=True). When every
instance shares its KKT sparsity (parametric sweeps, multi-start, B&B
siblings), this opt-in keeps each worker’s factorization backend alive
across instances so the symbolic analysis (fill-reducing ordering,
supernode structure) runs once per worker rather than once per
instance. Always correct — a pattern change just triggers a fresh
analysis — but pooled solver state means results are within solver
tolerance of, not bit-identical to, the default fresh-backend solves.
The win scales with how expensive ordering is for your model (small
models: negligible; large sparse models: worth measuring).
scipy.optimize-style
import numpy as np
from pounce import minimize
res = minimize(lambda x: (x - 1) @ (x - 1) + 1, x0=np.zeros(5))
print(res.fun, res.x)
minimize is a thin facade over pounce.Problem shaped after
scipy.optimize.minimize, so SciPy code ports with few changes — including as a
method= callable handed to scipy.optimize.minimize itself. It returns a
genuine scipy.optimize.OptimizeResult (res.x, res.fun, res.success,
res.status, res.message, res.nit, and the res.nfev / res.njev /
res.nhev evaluation counters), with pounce-specific extras under res.info
and a back-compat shim so a key absent at the top level falls back to res.info.
Compatibility with scipy.optimize.minimize
minimize(fun, x0, args=(), jac=None, hess=None, bounds=None,
constraints=None, callback=None, **options)
| Argument | Status | Notes |
|---|---|---|
fun, x0 | ✅ | objective callable and start point |
args | ✅ | tuple of extra positional arguments forwarded to fun / jac |
jac | ✅ | callable, or jac=True (then fun returns (value, gradient), cached so the gradient is not recomputed); omitted → central finite differences (eps^(1/3) step) and a one-time UserWarning. Provide one (or use pounce.jax / pounce.torch) for production. |
hess | ⚠️ | used when there are no constraints or all constraints are linear (the constraint curvature is then zero, so the objective Hessian is the Lagrangian Hessian); with nonlinear constraints the solver falls back to L-BFGS (hessian_approximation=limited-memory) |
bounds | ✅ | a sequence of (lo, hi) pairs or a scipy Bounds object; a None element or endpoint means ±∞. A NaN bound is rejected (previously it slipped past the reversed-bound check and behaved as “no bound”); use ±∞ / None for an unbounded side |
constraints | ✅ | scipy dict(s) {"type": "eq"|"ineq", "fun": …, "jac": …} or scipy LinearConstraint object(s) (dense or sparse A); multiple are concatenated; dict "jac" optional (finite-diff fallback) |
callback | ✅ | called each iteration; both scipy signatures supported — callback(xk) and callback(intermediate_result) |
tol | ✅ | accepted directly (scipy gtol / ftol / xtol are synonyms) |
options / **options | ✅ | pass options as keyword args (legacy options={…} dict still works); keys are pounce/Ipopt names, with scipy synonyms mapped: maxiter→max_iter, gtol/ftol/xtol→tol, disp→print_level, maxcor→limited_memory_max_history |
method | ✅ | scipy.optimize.minimize(fun, x0, method=pounce.minimize, …) works — pounce satisfies scipy’s custom-method contract |
hessp | ❌ | no Hessian-vector-product mode |
Conventions that match SciPy (so constraints port directly):
- Inequalities use the SciPy sign convention
g(x) ≥ 0; equalities areg(x) = 0. ALinearConstraint(A, lb, ub)becomeslb ≤ A x ≤ ub. - The result object is a genuine
scipy.optimize.OptimizeResult(subset of fields + aninfomap).
Gaps worth knowing:
NonlinearConstraintobjects are not accepted — pass nonlinear constraints as the dict form{"type": …, "fun": …, "jac": …}. (BoundsandLinearConstraintobjects are accepted.)- A constraint dict’s Jacobian is dense; for large sparse Jacobians use the
Problemclass directly (aLinearConstraintmay carry a sparseA, which is honored). options={"maxiter": 100}now works (scipy synonyms are mapped), but the underlying pounce option is stillmax_iter; an unrecognized key is forwarded verbatim to the backend.
Solver routing in minimize
By default minimize uses the general NLP filter line-search interior-point
method and does no structure probing — an expensive fun pays nothing. Opt
in with solver_selection="auto" (the same key the CLI uses) and minimize
probes the callables: a problem that is provably a linear program or a
convex quadratic program is dispatched to the specialized convex
interior-point solver (pounce.solve_qp, the HSDE driver), and a provably
convex QCQP (convex-quadratic objective and/or constraints) is reformulated
to a second-order cone program and dispatched to the conic solver
(pounce.solve_socp). Both reach a global optimum in materially fewer
iterations; everything else falls through to the NLP solver.
The catch is that minimize only sees opaque callables — it cannot read a
.nl expression tree the way the CLI can. So instead of reading the
structure it probes it: it evaluates fun/jac/hess at several points,
fits a linear/quadratic model, and then validates that model against the
true callables at held-out points before trusting it. The two
misclassification directions are not symmetric, and the validation gates the
dangerous one:
- A convex LP/QP/QCQP mistakenly sent to the NLP solver is merely slower — the filter-IPM still solves it correctly.
- A genuinely nonlinear or nonconvex problem sent to the convex solver would return a silently wrong answer.
So any probe that raises, any model mismatch beyond route_tol, a
non-constant Hessian/Jacobian, an indefinite objective Hessian (a nonconvex
QP), a quadratic equality, or a quadratic inequality whose feasible set is
nonconvex (a non-PSD constraint Hessian) all fall back to the NLP solver.
You never get a wrong “optimum” from a misclassification.
Forcing the solver
The solver_selection option (passed in options=) overrides the automatic
choice — mirroring the CLI option of the same name:
solver_selection=… | Behavior |
|---|---|
"nlp" | Default. Skip routing entirely; always use the NLP solver — no probe overhead. |
"auto" | Probe-and-validate; route provable LP/convex-QP to solve_qp, a convex QCQP to solve_socp, else NLP. |
"lp-ipm" | Force the convex solver; raise ValueError if the problem is not detected as an LP. |
"qp-ipm" | Force the convex solver; raise ValueError if it is not detected as a convex LP/QP. |
"socp" | Force the conic solver; raise ValueError if it is not detected as a convex QCQP. |
"qp-active-set" | Run the pounce-qp active-set engine on a detected LP/convex QP — the same engine and route the CLI uses. Raises ValueError if the problem is not a convex LP/QP; for the active-set SQP outer loop on a general NLP, pass algorithm="active-set-sqp". |
Any other value raises ValueError. These are the same six selectors the CLI
accepts, and matching is case-insensitive, as on the CLI.
Two differences from the CLI are worth knowing, both because minimize is a
library consumer with no .nl file to classify:
"qp-active-set"is class-validated here, unlike the other library-side differences below — it takes the same Python-side convex extraction as"qp-ipm"and dispatches to the same engine the CLI uses, so a given problem gets the same algorithm from either surface. It previously forwarded to the backend and ran the SQP outer loop, which meant one selector named two different solvers depending on how you called POUNCE; that is fixed, and the SQP outer loop is now reached only by its own name,algorithm="active-set-sqp".- The convex selectors (
"lp-ipm","qp-ipm","socp") work becauseminimizedoes its own Python-side structure detection. The equivalent Rust library API rejects them withInvalid_Option.
# Default: the general NLP solver, no probing.
res = minimize(fun, x0, bounds=bounds)
# Opt into routing: a convex QP goes to the fast convex IPM automatically.
res = minimize(fun, x0, bounds=bounds, solver_selection="auto")
print(res.info.get("solver")) # 'qp-ipm' / 'socp' when routed; None on the NLP path
# Insist the problem is a convex QP; fail loudly if the probe disagrees:
res = minimize(fun, x0, solver_selection="qp-ipm")
# A convex QCQP (e.g. a quadratic ball constraint) routes to the conic solver
# under `solver_selection="auto"`. Give the objective and constraint analytic
# `jac`s: derivative-free detection recovers the constraint Hessian from a
# finite-difference-of-finite-difference Jacobian, which is too noisy to confirm
# the quadratic, so without `jac` the probe conservatively defers to NLP (still
# the correct answer, just slower).
ball = {"type": "ineq",
"fun": lambda x: 1.0 - x @ x, # x·x ≤ 1
"jac": lambda x: -2.0 * np.asarray(x)}
res = minimize(lambda x: -x[0] - x[1], [0.1, 0.1],
jac=lambda x: np.array([-1.0, -1.0]),
constraints=[ball], solver_selection="auto")
print(res.info.get("solver")) # 'socp' (None on the NLP fall-back path)
route_tol (default 1e-5) sets the relative tolerance for the held-out
validation; raise it if a genuinely-linear problem with noisy finite-difference
Jacobians is being conservatively rejected, lower it to be stricter. The
routing keys are consumed by minimize and never forwarded to the backend, so
the rest of options still reaches the NLP solver unchanged.
When you still need a typed entry point
Auto-routing handles LP, convex QP, and convex QCQP from the
minimize(fun, x0, …) shape. The remaining specialized solvers need structure
that a callable cannot carry — an explicit cone list (exp/power/PSD cones), a
symbolic objective to relax and bound — so each keeps its own pounce-native
entry point:
| Want | Entry point | You provide | Optimum |
|---|---|---|---|
| General nonlinear, fast local solve | minimize(fun, x0, …) | callables (fun/jac/hess) | local |
| LP / convex QP | minimize (auto) or solve_qp(P, c, A, b, G, h, lb, ub, …) | callables / matrices | global |
| Convex QCQP | minimize (auto / socp) or solve_socp(…, cones=…) | callables / matrices + cone list | global |
| SOCP / exp / power / PSD cones | solve_socp(P, c, A, b, G, h, *, cones, …) | matrices + cone list | global |
| Polynomial, certified global | sos_minimize(objective, *, inequalities, equalities, …) | a polynomial | global |
The solve_qp / solve_socp / sos_minimize functions are pounce-native (not
SciPy-shaped) by necessity — e.g. sos_minimize takes a polynomial as a
coefficient dict and returns a certificate, not callables and SciPy dicts. See
Choosing a Solver for the full map.
There is no
minimize_globalentry point — POUNCE has no spatial branch-and-bound solver. The only certified-global Python path issos_minimize, for polynomials.
Curve fitting
pounce.curve_fit is the data-fitting companion to minimize — a
scipy.optimize.curve_fit-style front end that adds parameter constraints,
robust losses, confidence intervals, and ∂params/∂data sensitivity, with the
covariance read from the solver’s reduced Hessian. See
Curve Fitting.
from pounce import curve_fit
res = curve_fit(model, xdata, ydata, p0=[1, 1, 0]) # model written with jax.numpy
print(res.summary())
Finding multiple minima
pounce.find_minima is the global-search companion to minimize: it drives
the same solver in a loop to discover many distinct minima (flooding,
deflation, tunneling, multistart, MLSL, basin-hopping). See
Finding Multiple Minima for the methods and references,
Choosing a Method for selection guidance
(including high-dimensional behavior), and notebooks
19,
20,
21
for the three families.
from pounce import find_minima
r = find_minima(fun, x0, method="deflation", jac=jac, hess=hess,
bounds=bounds, n_minima=6)
print(r.status, len(r), "minima; best f =", r.fun)
JAX integration
The pounce.jax subpackage provides five entry points:
| Surface | Use it for |
|---|---|
from_jax(f, g, …) | Build a one-shot pounce.Problem from JAX-traced f(x) and g(x). |
solve(p, …) | custom_vjp-wrapped differentiable solve over a parameter p. |
solve_with_warm(p, …, warm_start=) | solve + dual-triple (x, λ, z) warm-start hand-off across calls. |
vmap_solve(p_batch, …) / vmap_solve_parallel(…) | Batched solve over a leading axis of p; the _parallel variant uses a ThreadPoolExecutor and releases the GIL inside each solve. |
JaxProblem(f, g, n, m, p_example=, …) | Build-once / solve-many handle that caches JIT artefacts, the sparsity probe, and the underlying pounce.Problem across calls. |
One-shot build with from_jax
import jax.numpy as jnp
from pounce.jax import from_jax
def f(x): return jnp.sum((x - 1) ** 2)
def g(x): return jnp.stack([jnp.sum(x) - 5.0])
prob = from_jax(f, g, n=4, m=1, lb=jnp.zeros(4), ub=jnp.full(4, 10.0),
cl=jnp.zeros(1), cu=jnp.zeros(1))
x, info = prob.solve(x0=jnp.ones(4))
Sparse Jacobian/Hessian compression (sparse=)
By default the constraint Jacobian and the Lagrangian Hessian are
computed densely — jax.jacrev/jacfwd/hessian build the full
matrix, which is then sliced to the detected sparsity pattern. The
reported structure is sparse, but the AD work and memory are O(m·n)
(Jacobian) and O(n²) (Hessian) regardless of how sparse the true
matrices are. On a 10,000-variable banded system that means computing
~10⁸ entries per iteration to keep ~50,000.
Passing sparse=True switches both derivatives to CPR-style colored
AD (pounce#83): structurally-orthogonal columns are colored, one
JVP (Jacobian) / HVP (Hessian) is taken per color — k ≪ n colors —
and the compressed result is scattered back to the known nonzeros. The
per-iteration cost drops from O(n) to O(k) AD passes. This is the
same compression strategy the Rust .nl tape path already uses for its
Hessian.
prob = from_jax(f, g, n=4, m=1, lb=jnp.zeros(4), ub=jnp.full(4, 10.0),
cl=jnp.zeros(1), cu=jnp.zeros(1),
sparse=True) # colored JVP/HVP instead of dense slice
The flag is also accepted by JaxProblem,
where it applies to both the single-solve and the batched
block-diagonal paths. The reported structure, the values, and the
solution are identical to the dense path either way — only the cost of
producing the derivative values changes. The differentiable backward
(factor_reuse / implicit diff) is unaffected.
When to use it. sparse=True wins on problems whose Jacobian/Hessian
are genuinely sparse with bounded per-row fill (banded, block, finite
differences/elements, PDE-constrained, separable). On a dense problem
the coloring finds no orthogonality (k = n) and the flag is a small,
bounded overhead, so it is opt-in rather than the default. Measured
on a banded family (python/benchmarks/bench_sparse_ad_83.py):
| n | colors (Jac / Hess) | per-eval Jacobian | per-eval Hessian | full solve |
|---|---|---|---|---|
| 800 | 2 / 3 | 6.2× faster | 2.0× faster | 1.3× faster |
| 2000 | 2 / 3 | 18.4× faster | 5.4× faster | 7.6× faster |
| 5000 | 2 / 3 | 560× faster | 200× faster | — |
The color count stays constant in n while the dense path grows
linearly, so the gap widens without bound as the problem scales.
Pattern detection. Sparsity is found by probing the derivative at
random points and recording where entries are nonzero. Under
sparse=True a mis-probe is costlier — it corrupts the compression
seed, not just a reported nonzero — so detection unions 3 probes by
default (vs 1 for the dense path). Override with n_probes=.
The probe never materializes the full matrix. It sweeps a block of
rows (VJPs) or columns (JVPs/HVPs) at a time under a fixed byte budget
and reduces each block to index pairs before allocating the next, so
build memory is bounded by that budget plus the nonzeros found —
not O(n²) (pounce#464). The AD pass count is unchanged: it is still
O(n) passes, which is what jacfwd/jacrev would have cost anyway.
Supplying a known pattern. For a full-discretization method the
structure is known in closed form before any numbers exist — element i
couples only to element i-1, so the Jacobian is block-banded by
construction. Rediscovering that by probing is O(n) AD passes you
don’t need. Hand it over instead:
prob = from_jax(
f, g, n=n, m=m, cl=cl, cu=cu, sparse=True,
jac_pattern=(jac_rows, jac_cols), # (m, n), cyipopt convention
hess_pattern=(hess_rows, hess_cols), # lower triangle of the (n, n) Hessian
)
Detection is skipped entirely for whichever of the two you supply — the
other is still probed. JaxProblem, from_torch, and TorchProblem
take the same two arguments. Upper-triangle entries in hess_pattern
are folded onto their mirror, since H is symmetric.
The pattern must be a superset of the true structure. Extra entries
are harmless — they report a zero and may cost an extra color. A
missing entry is silently wrong: the dense path drops that derivative,
and sparse=True aliases it into a same-colored reported entry,
corrupting the others. Nothing validates this against the model, so the
contract is yours to keep. This is also the only reliable route for
truly value-dependent structure (branchy where/abs), which no random
probe can detect.
Differentiable solve
pounce.jax.solve(p, f=, g=, …) is a custom_vjp-wrapped solve that
differentiates x*(p) through the implicit function theorem on the
converged KKT system. Inequality rows that are not active at x*
are dropped from the KKT block before the implicit-diff back-solve, so
the gradient matches the analytic active-set sensitivity even on
slack-inequality problems (pounce#73).
import jax, jax.numpy as jnp
from pounce.jax import solve as psolve
def f(x, p): return jnp.sum((x - p) ** 2)
def g(x, p): return jnp.stack([x[0] + x[1] - 1.0]) # equality
def x_star(p):
return psolve(
p, f=f, g=g, x0=jnp.zeros(2), n=2, m=1,
lb=jnp.full(2, -10.0), ub=jnp.full(2, 10.0),
cl=jnp.zeros(1), cu=jnp.zeros(1),
options={"tol": 1e-10, "print_level": 0},
)
# Gradient of the L2 distance to the target as p moves:
loss = lambda p: jnp.sum(x_star(p) ** 2)
print(jax.grad(loss)(jnp.array([0.3, 0.7])))
Warm-start across a parameter trajectory
solve_with_warm returns the full primal-dual triple alongside x*,
and consumes one on the next call. The warm-state is opaque from the
JAX side (pytree of jnp arrays) but maps directly onto the
x0 / λ0 / z0 ports of the underlying solver — for a sequence of
nearby p values this often cuts solver iterations by an order of
magnitude (pounce#74).
from pounce.jax import solve_with_warm
trajectory = [jnp.array([0.3 + 0.01 * k, 0.7 - 0.01 * k]) for k in range(50)]
x, warm = solve_with_warm(
trajectory[0], f=f, g=g, x0=jnp.zeros(2), n=2, m=1,
lb=jnp.full(2, -10.0), ub=jnp.full(2, 10.0),
cl=jnp.zeros(1), cu=jnp.zeros(1),
warm_start=None, # first call → cold start
options={"tol": 1e-10, "print_level": 0},
)
xs = [x]
for p_k in trajectory[1:]:
x, warm = solve_with_warm(
p_k, f=f, g=g, x0=x, n=2, m=1,
lb=jnp.full(2, -10.0), ub=jnp.full(2, 10.0),
cl=jnp.zeros(1), cu=jnp.zeros(1),
warm_start=warm, # reuse λ, z
options={"tol": 1e-10, "print_level": 0},
)
xs.append(x)
Batched solve (vmap_solve / vmap_solve_parallel)
vmap_solve runs one solve per row of p_batch sequentially.
vmap_solve_parallel is the same surface but dispatches each row to a
ThreadPoolExecutor; the underlying Rust solve releases the GIL via
py.allow_threads, so workers actually run in parallel on multi-core
CPUs (pounce#74).
import numpy as np
from pounce.jax import vmap_solve_parallel
rng = np.random.default_rng(0)
batch = jnp.asarray(rng.standard_normal((32, 2)))
X = vmap_solve_parallel(
batch, f=f, g=g, x0=jnp.zeros(2), n=2, m=1,
lb=jnp.full(2, -10.0), ub=jnp.full(2, 10.0),
cl=jnp.zeros(1), cu=jnp.zeros(1),
workers=8, # ThreadPoolExecutor size
options={"tol": 1e-9, "print_level": 0},
)
assert X.shape == (32, 2)
Both batched surfaces are custom_vjp-wrapped, so a downstream
jax.grad/jax.jacobian over a batched loss works end-to-end.
Build once, solve many: JaxProblem
For iterative use — a parameter trajectory in a continuation loop, a
training step that calls the solver inside a batch, a notebook cell
that sweeps a knob — from_jax/solve rebuild the JIT artefacts, the
sparsity probe, and the underlying pounce.Problem on every call.
JaxProblem does that work once at construction and exposes the same
four method shapes against the cached state. On the
pounce#75 microbench shape (n=5, m=6, 20 sequential solves at
different p) this is roughly a 14× speedup, taking per-solve time
from ~96 ms down to ~7 ms (pounce#75).
from pounce.jax import JaxProblem
jp = JaxProblem(
f=f, g=g, n=2, m=1, p_example=jnp.zeros(2), # p_example fixes shape/dtype only
lb=jnp.full(2, -10.0), ub=jnp.full(2, 10.0),
cl=jnp.zeros(1), cu=jnp.zeros(1),
options={"tol": 1e-9, "print_level": 0},
# sparse=True, # colored AD on sparse problems (see above)
)
# Sequential, differentiable:
x = jp.solve(jnp.array([0.3, 0.7]), x0=jnp.zeros(2))
# Dual-warm-start trajectory (composes warm-state hand-off with reuse):
x, warm = jp.solve_with_warm(trajectory[0], x0=jnp.zeros(2), warm_start=None)
for p_k in trajectory[1:]:
x, warm = jp.solve_with_warm(p_k, x0=x, warm_start=warm)
# Batched parallel solve over a row-axis of p_batch:
X = jp.vmap_solve_parallel(batch, x0=jnp.zeros(2), workers=8)
Each worker thread in vmap_solve_parallel keeps its own cached
pounce.Problem via threading.local, so the per-thread build cost
is paid at most once per worker rather than once per batch row.
Factor-reuse backward (factor_reuse=)
JaxProblem.solve and solve_with_warm default to a k_aug-style
backward that reuses the IPM’s converged compound KKT factor
(pounce.Solver.kkt_solve) instead of assembling a dense
(n+m) × (n+m) block and running jnp.linalg.solve on it
(pounce#76). The held LDLᵀ factor turns the bwd back-solve from
O((n+m)³) into O(nnz(L)) and drops the explicit active-set masking
that the dense path does — the barrier rows on the bound multipliers
(z_l, z_u) already encode “active bounds force Δx_i = 0” exactly,
and the (v_l, v_u) rows do the same for slack inequalities. The
accuracy of the resulting gradient is O(μ) at the IPM barrier
parameter, which sits well below tol after convergence.
jp = JaxProblem(..., factor_reuse=True) # default; reuse the IPM factor
jp = JaxProblem(..., factor_reuse=False) # dense JAX backward
Pick factor_reuse=False when you want higher-order differentiation
(jax.grad(jax.grad(...)) through the solver) — the dense backward
stays JAX-traced and is itself differentiable, the factor-reuse one
crosses to the Rust host via pure_callback and is opaque to a
second-order trace.
When to pick which on batched_solve workloads (pounce#77)
factor_reuse=False is itself a form of factor reuse — it builds
the per-block (n+m) × (n+m) KKT at pounce’s converged
(x*, λ*, μ_l*, μ_u*) (saved in the custom_vjp residual) and
solves it under jax.vmap with a JIT-fused per-block
jnp.linalg.solve. So both modes reuse pounce’s converged solution;
they differ only in what they back-solve:
factor_reuse=True— back-solves pounce’s held LDLᵀ factor of the full stacked KKT (Rust-side, via FFI through a single-thread executor pin).factor_reuse=False— back-solves a freshly assembled per-block dense KKT in JAX, fused undervmap.
For batched_solve + jax.jacrev / jax.vmap minibatch projections
factor_reuse=False is faster at every scale we measured
(n = 3 through 48 per block, B = 64 stacked):
n=3 reuse bwd = 16.6 ms dense bwd = 20.6 ms reuse/dense = 0.80×
n=8 reuse bwd = 52.5 ms dense bwd = 38.5 ms reuse/dense = 1.36×
n=16 reuse bwd = 157.6 ms dense bwd = 57.2 ms reuse/dense = 2.76×
n=32 reuse bwd = 558.6 ms dense bwd = 103.6 ms reuse/dense = 5.39×
n=48 reuse bwd =1262.9 ms dense bwd = 137.4 ms reuse/dense = 9.19×
The dense path scales as B · (n+m)³; the factor-reuse path scales
as N · kkt_dim ≈ B² · n · (n+m) because jax.jacrev fans out
N = B·n cotangents and each triggers a back-solve of the full
stacked LDLᵀ even though only one block has nonzero signal.
Guidance:
- Single solve + many sensitivities —
jax.jacrev(jp.solve, argnums=0)(p, x0)and friends — keepfactor_reuse=True. One LDLᵀ back-solve per cotangent against the held factor beats JAX dense-solving a fresh(n+m) × (n+m)block. - Batched solve + jacrev / vmap —
jax.jacrev(lambda P: jp.batched_solve(P, x0))(pb)— setfactor_reuse=False. Treat the dense path as the default for minibatch projections.
Each fwd registers its converged factor in a bounded LRU on the
JaxProblem (default capacity 128). For very long-running training
loops with many distinct forward solves you can drop the cache
explicitly:
jp.clear_solver_cache()
Off-thread dispatch (training loops, jit(value_and_grad(...)))
pounce.Solver is a !Send PyO3 type (it holds an
Rc<RefCell<dyn TNLP>> interior), so any attempt to touch the held
factor from a thread other than the one that built it raises a PyO3
panic. JAX hits this whenever the bwd pure_callback lands on an XLA
worker thread — typical for jax.jit(jax.value_and_grad(...)) inside
a training step.
JaxProblem(factor_reuse=True) defends against this by routing every
pounce.Solver interaction (fwd register, warm-start solve, batched
solve, bwd kkt_solve) through a dedicated single-thread
ThreadPoolExecutor owned by the JaxProblem (pounce#77). All solver
touches are pinned to that one worker thread regardless of which
thread JAX dispatches from. vmap_solve_parallel bypasses the pin
(it doesn’t register with the factor cache), so its B-way thread
concurrency is preserved.
Pickle / distributed training
JaxProblem round-trips through pickle.dumps / pickle.loads, so
it works with the realistic distributed-training paths:
multiprocessing(start_method='spawn')— the default on macOS and whattorch.utils.data.DataLoader(num_workers>0)uses;- Ray and Dask actors via
cloudpickle; - Naive checkpointing for resume.
The per-process runtime state (JIT’d closures, threading.Lock,
threading.local, the factor-reuse executor, the held LDLᵀ factor
registry) is dropped from the pickle and rebuilt on the receiving
side. The sparsity-pattern arrays survive the round trip, so the
worker doesn’t redo the one-shot JAX probe. Held factors do not
survive — a fresh process has no history of fwd solves, so the
receiver’s registry starts empty and the bwd factor-reuse path picks
up from the next solve.
User-side requirement: f and g must themselves be picklable.
Module-level functions work with stdlib pickle; lambdas / inner
functions need cloudpickle (which is what Ray, Dask, and
torch.multiprocessing use by default anyway).
multiprocessing(start_method='fork') is not supported — JAX
itself warns that os.fork() is incompatible with its threading;
use spawn instead.
Stacked block-diagonal batched solve (batched_solve)
JaxProblem.batched_solve(p_batch, x0) runs one IPM solve over a
single NLP whose variables are [x^(1); ...; x^(B)], constraints are
concat(g(x^(k), p^(k))), and objective is Σ_k f(x^(k), p^(k)).
The Jacobian and Lagrangian Hessian are block-diagonal — each block-k
constraint touches only the block-k slice of X, and the objective
is a pure sum, so there’s no cross-block coupling. The IPM sees one
big sparse problem but does only B × (per-block factor cost) work
on the linear system.
p_batch = jnp.array([[0.3, 0.7], [0.5, 0.5], [-0.1, 0.4]])
x_batch = jp.batched_solve(p_batch, x0=jnp.zeros(2)) # (B, n)
custom_vjp-wrapped, so jax.grad/jax.jacobian through the
batched solve work end-to-end:
def loss(P):
return jnp.sum(jp.batched_solve(P, x0=jnp.zeros(2)) ** 2)
dloss_dP = jax.grad(loss)(p_batch) # (B, p_shape)
The backward path follows factor_reuse=:
factor_reuse=True(default) — oneSolver.kkt_solveagainst the stacked held LDLᵀ factor; the per-block∂²L/∂x∂p/∂g/∂parejax.vmap’d autodiff over the user’sf/g, then contracted with the per-blocku_x/u_gslices of the single back-solve. Composes (A) and (B) — one factor for both forward and per-batch sensitivities (pounce#76).factor_reuse=False—jax.vmapof the per-element dense(n+m) × (n+m)JAX KKT solve. Exact for the same reason: block- diagonal coupling means∂x^(k)*/∂p^(j) = 0fork ≠ j.
When to pick batched_solve vs the existing batched surfaces:
| Surface | Wins when |
|---|---|
vmap_solve | Long batches, want one solve per iterate sequentially. |
vmap_solve_parallel | Batch elements have very different convergence behaviour — slow blocks don’t drag fast ones (B independent IPMs in worker threads, GIL released per solve). |
batched_solve | Blocks have similar convergence behaviour (shared barrier homotopy and symbolic factorisation amortise) and B is large enough that the per-call Python overhead of B fwd dispatches becomes visible (one Rust crossing instead of B). |
Per-block lb/ub/cl/cu are tiled across the batch; the
parameter p is what varies, not the feasible region. Stacked
Problems are cached per (thread, B) in a tiny LRU (cap 4), so
calls in a loop with one or two batch sizes pay the build cost at
most once per worker.
Post-solve Jacobian and sensitivities (batched_solve_with_jacobian)
When you need the explicit per-block Jacobian J[k] = ∂x^(k)*/∂p^(k)
as a first-class result — for validation, linear-update layers, or
diagnostics — batched_solve_with_jacobian returns it directly from
the held KKT factor instead of wrapping batched_solve in
jax.jacrev:
x_star, (lam, zL, zU), J = jp.batched_solve_with_jacobian(p_batch, x0)
# x_star : (B, n) J : (B, n, p_dim) duals match batched_solve_with_warm
J’s row i is the reverse-mode VJP at cotangent e_i (the KKT
system is symmetric), so the whole Jacobian is one multi-RHS back-solve
against the held LDLᵀ factor — no NLP re-solve, no repeated public
jax.vjp calls. Pass wrt_cols (1-D p only) to keep just the
parameter columns you care about, e.g. wrt_cols=slice(0, ny) to drop
context columns; J then has trailing dim len(wrt_cols).
For the linear-update pattern — anchor once, then apply several nearby
sensitivity products — pin the factor with an AnchorState and reuse it:
with jp.anchor(p_batch, x0, wrt_cols=slice(0, ny)) as state:
dx = jp.batched_jvp_from_state(state, dp) # J @ dp (forward)
dp_bar = jp.batched_vjp_from_state(state, x_bar) # J^T @ x_bar (reverse)
batched_jvp_from_state is the cheap path for linear updates that only
need the directional sensitivity delta_x = J @ delta_p and never the
full J: it assembles the parameter-side RHS [∂²L/∂x∂p · dp; ∂g/∂p · dp]
and back-solves once against the held factor. When the state was anchored
with wrt_cols, pass the reduced dp (one entry per selected column);
otherwise pass a full (B,) + p_shape perturbation (zero out the columns
you don’t want to move).
anchor(...) (and batched_solve_with_jacobian(..., return_state=True))
return an AnchorState that holds the factor across calls. Prefer the
context-manager form; for handles that must outlive a single block
(e.g. stored on a projection layer), use explicit ownership:
state = jp.anchor(p_batch, x0)
... # later calls reuse `state`
state.reanchor(p_new, x0) # swap the solve in place (closes prior pin)
state.close() # release the held factor
Pinned factors are exempt from the backward LRU but capped
(_pinned_capacity, default 16) so a missed close() fails loudly
rather than leaking; a weakref finalizer reclaims the factor if a
handle is garbage-collected without close(). A worked example —
projection layer, full Jacobian, JVP/VJP-from-state, and the lifetime
patterns — is in
notebooks/13_post_solve_jacobian.ipynb.
Building on that held factor, PathFollower traces a whole solution
path \(x^*(\theta(s))\) while predicting most steps off the factor
instead of re-solving, and inverse_map_rhs runs the map backwards as an
ODE — see Path Following & Inverse Mapping.
PyTorch integration
The pounce.torch subpackage is a PyTorch frontend mirroring
pounce.jax, one-for-one. It is a thin adapter, not a second solver:
the numerical core (the Rust IPM) and the implicit-function-theorem
backward are framework-agnostic — only the array namespace differs. A
solve is a torch.autograd.Function you can drop inside a torch.nn
model and backprop through, with the same constraint-satisfaction
guarantee the JAX path gives. Install with pip install pounce[torch]
(torch.func requires torch ≥ 2.2).
Because PyTorch is eager, the adapter is smaller than the JAX one:
there is no pure_callback / ShapeDtypeStruct machinery (the forward
calls problem.solve(...) directly), no host-callback registry or
single-thread executor (the converged Solver is stashed on the
autograd ctx / AnchorState and read back in the backward on the same
thread), and no global jax_enable_x64 flag — float64 tensors are
requested explicitly (torch.set_default_dtype(torch.float64) or
.double() your inputs; the implicit-diff and KKT solves need double
precision and the layers validate it).
| JAX surface | PyTorch equivalent |
|---|---|
from_jax(f, g, …) | from_torch(f, g, …) |
solve(p, …) | solve(p, …) (torch.autograd.Function + KKT backward) |
solve_with_warm(p, …, warm_start=) | solve_with_warm(…) (dual triple + barrier-μ, pounce#86) |
vmap_solve / vmap_solve_parallel | vmap_solve / vmap_solve_parallel |
JaxProblem(…) | TorchProblem(…) (build-once, factor-reuse backward) |
solve_qp / solve_qp_batch / solve_socp / QpLayer | same names |
PathFollower / inverse_map_rhs | same names |
import torch
torch.set_default_dtype(torch.float64)
from pounce.torch import solve as psolve
def f(x, p): return torch.sum((x - p) ** 2)
def g(x, p): return torch.stack([x[0] + x[1] - 1.0]) # equality
p = torch.tensor([0.3, 0.7], requires_grad=True)
x_star = psolve(
p, f=f, g=g, x0=torch.zeros(2), n=2, m=1,
lb=torch.full((2,), -10.0), ub=torch.full((2,), 10.0),
cl=torch.zeros(1), cu=torch.zeros(1),
options={"tol": 1e-10, "print_level": 0},
)
(x_star ** 2).sum().backward() # dL/dp via the implicit function theorem
print(p.grad)
The differentiable conic layers are feasible-by-construction (the same “one roof” as cvxpylayers/theseus, off one core):
from pounce.torch import solve_qp
P = torch.eye(2); c = torch.tensor([-4.0, -4.0], requires_grad=True)
G = torch.tensor([[1.0, 1.0]]); h = torch.tensor([0.5])
x = solve_qp(P=P, c=c, G=G, h=h) # min ½xᵀPx+cᵀx s.t. Gx ≤ h
x.sum().backward() # OptNet implicit-diff gradients
Validation. Every layer is checked with torch.autograd.gradcheck
against finite differences, and a JAX↔Torch parity suite asserts both
frontends agree on x* and dL/dp to tolerance on shared fixtures
(python/tests/test_torch.py, test_qp_torch.py, test_socp_torch.py,
test_parity_jax_torch.py).
Thread-safety note.
torch.functransforms share a process-global layer stack and are not thread-safe;vmap_solve_paralleltherefore serializes the (already GIL-bound) Python derivative callbacks with a lock while the Rust IPM linear algebra still runs concurrently (GIL released). Double-backward is supported on the conic layers but not guaranteed on the NLP implicit-diff path (the parameter sensitivities are taken withtorch.func, outside the autograd graph) — setfactor_reuse=FalseonTorchProblemfor the in-framework dense backward if you need higher-order behaviour.
Notebooks
The notebooks under
python/notebooks/
work through getting started, JAX autodiff, implicit differentiation,
sensitivity analysis, the Pyomo integration,
NLP scaling
(set_problem_scaling + nlp_scaling_method=user-scaling), and
FBBT
(nonlinear bound tightening via presolve_fbbt=yes on Pyomo
models).
Rust API
POUNCE is written in Rust, so the Rust API is the solver itself rather than a
binding over it. Everything is reached through one crate — pounce-rs, a
facade that re-exports a curated public surface so your code does not depend
on POUNCE’s internal crate layout.
cargo add pounce-rs
[dependencies]
pounce-rs = "0.9"
The default build is the NLP path. The convex/conic, active-set QP, and sensitivity solvers are behind feature flags.
Why one crate. The solver is split across ~20 workspace crates (
pounce-nlp,pounce-algorithm,pounce-convex, …) whose boundaries move as the code evolves. Depending on them directly couples you to that layout.pounce-rsis the stability boundary; everything below is internal.
Two APIs
The builder — for the common case
Implement [Problem] — only objective is required — then configure and
solve. Anything you leave out is supplied: missing gradients and Jacobians are
approximated by finite differences, and the Hessian defaults to a
limited-memory (L-BFGS) approximation.
#![allow(unused)]
fn main() {
use pounce_rs::prelude::*;
// min (x0−1)² + (x1−2)² s.t. x0 + x1 == 3, 0 ≤ x ≤ 5
struct P;
impl Problem for P {
fn objective(&self, x: &[f64]) -> f64 {
(x[0] - 1.0).powi(2) + (x[1] - 2.0).powi(2)
}
fn n_constraints(&self) -> usize { 1 }
fn constraints(&self, x: &[f64], g: &mut [f64]) { g[0] = x[0] + x[1]; }
}
let sol = Nlp::new(P)
.var_bounds(&[0.0, 0.0], &[5.0, 5.0])
.constraint_bounds(&[3.0], &[3.0]) // equality: lower == upper
.x0(&[0.0, 0.0])
.option_num("tol", 1e-10)
.solve();
assert!(sol.success);
assert!((sol.x[0] - 1.0).abs() < 1e-5 && (sol.x[1] - 2.0).abs() < 1e-5);
}
n is inferred from var_bounds or x0 (they must agree). Options use the
same names as the CLI and upstream Ipopt — option_num, option_int,
option_str; see Solver Options.
The returned Solution carries success / status, x, objective,
multipliers, the constraint values g, the bound multipliers z_l / z_u,
and stats (wall time, iteration count, evaluation counts, final
infeasibilities). The vector fields are filled by finalize_solution, so they
stay empty if a solve aborts before finalizing — check success before
indexing.
To supply exact derivatives, implement gradient and jacobian and return
true; returning false (the default) selects finite differences for that
callback.
TNLP — for full control
For an exact Hessian, custom Jacobian/Hessian sparsity, or NLP scaling,
implement the [TNLP] trait directly and drive it with IpoptApplication.
This is the same trait the CLI and the C ABI sit on.
#![allow(unused)]
fn main() {
use pounce_rs::prelude::*;
use std::cell::RefCell;
use std::rc::Rc;
let mut app = IpoptApplication::new();
app.initialize()?;
let prob = Rc::new(RefCell::new(MyTnlp::default()));
let status = app.optimize_tnlp(Rc::clone(&prob) as Rc<RefCell<dyn TNLP>>);
assert_eq!(status, ApplicationReturnStatus::SolveSucceeded);
}
You provide get_nlp_info (sizes and nonzero counts), get_bounds_info,
get_starting_point, the evaluators (eval_f, eval_grad_f, eval_g,
eval_jac_g, eval_h), and finalize_solution to receive the answer.
eval_jac_g and eval_h are called in two modes — SparsityRequest::Structure
for the pattern, then SparsityRequest::Values — so the pattern is declared
once and reused across iterations.
The crate documentation on docs.rs has a complete HS071 walkthrough.
Iteration capture and logging
Opt into the per-iteration trajectory with .capture_iterations() on the
builder; the records land in sol.stats.iterations. Outside the builder,
with_iter_capture wraps any closure and returns the records alongside its
result:
#![allow(unused)]
fn main() {
use pounce_rs::prelude::*;
let (sol, iters) = with_iter_capture(|| {
Nlp::new(P)
.var_bounds(&[0.0, 0.0], &[5.0, 5.0])
.constraint_bounds(&[3.0], &[3.0])
.solve()
});
assert!(sol.success && !iters.is_empty());
}
On the IpoptApplication path, install collector_scope() for the duration of
the solve and read the history back from statistics(). init_subscriber()
turns on console logging without your crate taking a tracing dependency.
Feature flags
Everything beyond the NLP path is off by default and lands in its own module.
The two QP families both name their types QpProblem / QpSolution /
QpStatus, so they cannot share one flat namespace.
| feature | module | covers |
|---|---|---|
convex | pounce_rs::convex | LP, convex QP, SOCP / exponential / power / PSD cones, SOS; batched and warm-started solves; symbolic-factorization reuse; QP sensitivity |
qp | pounce_rs::qp, pounce_rs::sqp | sparse parametric active-set QP, and the SQP working-set warm-start contract |
sensitivity | pounce_rs::sensitivity | sIPOPT-style ∂x*/∂p predictors, parametric warm starts, reduced Hessian |
full | — | all three |
[dependencies]
pounce-rs = { version = "0.9", features = ["convex", "sensitivity"] }
Enabling a feature widens what the crate exports, not what it builds: the
default NLP path already compiles pounce-qp, pounce-linsol, and
pounce-feral transitively, so qp costs nothing at build time and only
convex and sensitivity add crates.
convex and qp also enable pounce_rs::linsol, which supplies the
sparse symmetric factorization those solvers take as an argument —
backend() for the default parallel FERAL factor, serial_backend() for the
inner-serial one used under an outer-parallel batch.
Convex: LP, QP, and conic
#![allow(unused)]
fn main() {
use pounce_rs::convex::{QpOptions, QpProblem, QpStatus, Triplet, solve_qp_ipm};
use pounce_rs::linsol::backend;
// min ‖x‖² − 0.5·x0 − 1.5·x1 s.t. x0 + x1 == 1, 0 ≤ x ≤ 5
let prob = QpProblem {
n: 2,
p_lower: vec![Triplet::new(0, 0, 2.0), Triplet::new(1, 1, 2.0)],
c: vec![-0.5, -1.5],
a: vec![Triplet::new(0, 0, 1.0), Triplet::new(0, 1, 1.0)],
b: vec![1.0],
g: vec![],
h: vec![],
lb: vec![0.0, 0.0],
ub: vec![5.0, 5.0],
};
let sol = solve_qp_ipm(&prob, &QpOptions::default(), backend);
assert_eq!(sol.status, QpStatus::Optimal);
}
P is the lower triangle of the Hessian in triplet form; an empty P is
an LP. Cone blocks beyond the nonnegative orthant are declared with ConeSpec
and solved by solve_socp_ipm. For many instances, solve_qp_batch_parallel
runs one per rayon worker, and QpFactorization reuses the AMD ordering and
symbolic analysis across instances that share a sparsity pattern. See
Convex Solver.
Active-set QP and SQP warm starts
pounce_rs::qp is the parametric active-set engine — a different solver
family from the convex IPM, for sequences of nearby QPs, and it accepts an
indefinite Hessian. pounce_rs::sqp is the NLP-level counterpart: carrying a
working set from one SQP solve into the next. See
Active-Set SQP & Warm Starts.
Sensitivity
#![allow(unused)]
fn main() {
use pounce_rs::prelude::*;
use pounce_rs::sensitivity::SensSolve;
let result = SensSolve::new(vec![2, 3]) // pinned constraint rows
.with_deltas(vec![-0.5, 0.0]) // Δp
.with_reduced_hessian()
.run(&mut app, tnlp);
let dx = result.dx.expect("populated when with_deltas was set");
}
A sensitivity-stage failure is reported through result.error, not
result.status — the underlying solve can converge while the post-solve step
fails. See Sensitivity Analysis and
Sessions.
Escape hatch
Each feature module also re-exports the crate behind it — pounce_rs::convex
re-exports pounce_convex, and so on — so anything outside the curated
surface stays reachable without adding a dependency. Reaching for it is a
signal the facade is missing something; those are worth
filing.
See also
- docs.rs/pounce-rs — the full API reference
- Choosing a Solver — which solver fits which problem
- Solver Options — the option names shared by every frontend
- Python API — the same solvers from Python
Path Following & Inverse Mapping
Tracing how a solution moves as a parameter changes is a re-solve loop by default: pick the next \(\theta\), solve the NLP, repeat. POUNCE replaces most of those solves with a back-solve on the KKT factor it already holds. Given the converged factor at one point, the sensitivity \(\partial x^*/\partial\theta\) is available for the cost of a triangular solve, so a step along the path is a prediction rather than an optimization.
PathFollower (in both pounce.jax and pounce.torch) wraps that idea
in a predictor–corrector loop:
- predict — extrapolate \(x\) and the multipliers along the
held-factor sensitivity (
jvp_from_state); - monitor — without solving, check the KKT residual at the predicted point and the active-set margin;
- correct — only when the monitor trips, take one warm-started,
barrier-\(\mu\) seeded re-solve that also re-anchors the factor
(
warm_anchor).
On a linear-response problem the predictor is exact and a whole path costs one solve. On a curved problem the monitor tolerance is the lever: loosen it to accept more predictor steps between re-solves.
The parametric problem
Everything on this page traces the solution of
\[ \min_x; f(x, \theta) \quad \text{s.t.} \quad g(x, \theta) = 0,; \mathrm{lb} \le x \le \mathrm{ub} \]
as \(\theta\) varies. Build it as a JaxProblem (or TorchProblem —
the API is identical, see Python API):
import jax.numpy as jnp
from pounce.jax import JaxProblem, PathFollower
def f(x, p):
return jnp.sum((x - p) ** 2)
def g(x, p):
return jnp.stack([x[0] + x[1] - 1.0])
jp = JaxProblem(
f=f, g=g, n=2, m=1, p_example=jnp.zeros(2),
lb=jnp.full(2, -5.0), ub=jnp.full(2, 5.0),
cl=jnp.zeros(1), cu=jnp.zeros(1),
options={"tol": 1e-9, "print_level": 0, "sb": "yes"},
)
Parameter continuation: follow
follow traces \(x^*(\theta(s))\) for a prescribed path
\(\theta(s)\), \(s \in [s_0, s_1]\). This is the operability-tracing
and uncertainty-mapping case: \(s\) is monotone by construction, so the
path cannot fold in \(s\).
def circle(s):
a = 2.0 * jnp.pi * s
return jnp.array([0.5 + 0.4 * jnp.cos(a), 0.4 * jnp.sin(a)])
pf = PathFollower(jp, monitor_tol=1e-6, ds0=0.05)
tr = pf.follow(circle, (0.0, 1.0), jnp.zeros(2))
print(tr.n_steps, tr.n_correctors, tr.n_accepts)
# 7 0 7 -> one anchor solve for the whole loop; naive would be 8
The objective here is quadratic, so \(\partial x^*/\partial\theta\) is constant, the predictor is exact, and the monitor never fires: zero correctors. Add curvature and the trade-off appears. With
def f_nl(x, p):
return jnp.sum((x - p) ** 2) + 0.02 * jnp.sum(x ** 4)
around the same loop (ds0=0.05, ds_max=0.1), sweeping monitor_tol
against the error versus a cold solve at every recorded \(\theta\):
monitor_tol | solves | accepts | max path error |
|---|---|---|---|
1e-6 | 10 | 2 | 9e-15 |
1e-3 | 9 | 3 | 2e-4 |
5e-3 | 5 | 7 | 9e-4 |
2e-2 | 3 | 9 | 2e-3 |
(12 solves if you re-solved at every step.) That is the whole predictor–corrector lever: you are paying accuracy for solves at a rate you set.
Result: PathTrace
Both entry points return a PathTrace dataclass:
| Field | Meaning |
|---|---|
s | path parameter at each recorded point (arclength in arclength mode) |
theta, x, lam | parameter, primal, and multipliers along the path |
n_steps | steps taken |
n_correctors | of those, how many needed a solve |
n_accepts | accepted on the predictor alone (no solve) |
active_set_changes | s values where the active set changed |
turning_points | \(\theta\) at detected folds (arclength mode) |
status | "ok", or a reason string on early stop |
n_correctors vs n_steps is the headline number: it is how many NLP
solves you avoided.
Step-size adaptation
The step grows by grow on an accepted predictor or an easy correction
(≤ 3 IPM iterations), shrinks by shrink on a hard one (≥ 10 iterations)
or a failed correction, and is clamped to [ds_min, ds_max]. When a
correction reveals the active set changed, the step resets to ds0
and the region is resolved finely — the s value is recorded in
active_set_changes. If a correction fails and the step would drop below
ds_min, the trace stops with status="corrector_failed" rather than
silently returning garbage.
The active_margin_tol knob is what keeps the predictor honest near a
critical-region boundary: a predicted point closer than this to an
active-set change forces a correction, so the predictor never
extrapolates across the discontinuity.
Tracing past folds: trace_arclength
Parameter continuation stalls at a turning point, where
\(\partial x^*/\partial\theta\) is singular and the path doubles back in
\(\theta\). trace_arclength parametrises the solution curve by
arclength instead, solving the stationarity/feasibility system
\[ R(x, \lambda, \theta) = \begin{bmatrix} \nabla_x f + J_g^{\mathsf T}\lambda \\ g \end{bmatrix} = 0 \]
along its curve in \((x, \lambda, \theta)\) space, with a tangent predictor and a Newton corrector on the augmented system \([R;\ \text{arclength}]\). Because arclength never reverses, the trace passes straight through the fold.
The classic test: the stationarity of \(f = x^4/4 - x^2/2 - \theta x\) is \(\theta = x^3 - x\), which folds at \(x = \pm 1/\sqrt3\) (\(\theta = \mp 0.385\)).
def f_cubic(x, p):
th = p[0]
return x[0] ** 4 / 4.0 - x[0] ** 2 / 2.0 - th * x[0]
jp_c = JaxProblem(f=f_cubic, g=None, n=1, m=0, p_example=jnp.zeros(1),
options={"tol": 1e-10, "print_level": 0, "sb": "yes"})
trc = PathFollower(jp_c).trace_arclength(
jnp.array([-1.3]), -0.4, ds=0.05, n_steps=120,
)
print(trc.turning_points) # [0.3843, -0.3805]
Both folds are found, in the order the trace reaches them, to the
accuracy of the ds=0.05 sampling — the exact values are
\(\pm 2/(3\sqrt3) = \pm 0.3849\). They are recorded where the
\(\theta\)-component of the tangent changes sign, so tighten ds if
you need the turning point located more precisely.
Colour is arclength, so the trace reads as one continuous walk: up the lower branch, through the first fold (star), back across the middle branch, through the second fold, and out along the upper branch. The right panel is the same run against arclength — \(\theta\) rises, reverses, and rises again. Those reversals are precisely where a method that treats \(\theta\) as the independent variable has nowhere to go.
(Regenerate with python3 scripts/make-docs-figures.py.)
direction sets the sign of the initial step in \(\theta\);
newton_tol / newton_max control the corrector.
Inverse / uncertainty mapping: inverse_map_rhs
A related problem runs the map backwards: given a prescribed path in output space, what input path produces it? For an output \(y = h(x^*(\theta), \theta)\) of the embedded optimizer, the Alves–Kitchin–Lima inverse map integrates
\[ \frac{d\theta}{ds} = \Big(\frac{\partial y}{\partial \theta}\Big)^{-1} \frac{dy}{ds}, \qquad \frac{\partial y}{\partial \theta} = \frac{\partial h}{\partial x} J + \frac{\partial h}{\partial \theta}, \]
with \(J = \partial x^*/\partial\theta\) off the held factor and the output Jacobians by autodiff. Note this is a linear solve against the sensitivity, not a Jacobian-vector product, so \(\partial y/\partial \theta\) must be square: the output dimension must equal the parameter dimension (with the default identity output, \(n = p\)).
inverse_map_rhs builds the right-hand side and hands the stepping to an
off-the-shelf integrator — no hand-rolled stepper, no NLP inversion:
Here the output is the solution itself (\(h = x^*\), the default), and \(f = (x - \theta)^2 + 0.05x^4\) makes the map explicit (\(\theta = y + 0.1y^3\)) so the trace can be checked analytically:
import diffrax
from pounce.jax import inverse_map_rhs
def f_inv(x, p):
return (x[0] - p[0]) ** 2 + 0.05 * x[0] ** 4
jp_inv = JaxProblem(f=f_inv, g=None, n=1, m=0, p_example=jnp.zeros(1),
options={"tol": 1e-11, "print_level": 0, "sb": "yes"})
# A closed loop in output space, and its velocity.
y_of_s = lambda s: jnp.array([0.5 + 0.3 * jnp.sin(2 * jnp.pi * s)])
dy_ds = lambda s: jnp.array([0.3 * 2 * jnp.pi * jnp.cos(2 * jnp.pi * s)])
rhs = inverse_map_rhs(jp_inv, dy_ds) # f(s, θ) -> dθ/ds
y0 = float(y_of_s(0.0)[0])
theta0 = jnp.array([y0 + 0.1 * y0 ** 3]) # θ0 with x*(θ0) = y(0)
term = diffrax.ODETerm(lambda s, theta, args: rhs(s, theta))
sol = diffrax.diffeqsolve(
term, diffrax.Dopri5(), t0=0.0, t1=1.0, dt0=0.01, y0=theta0,
stepsize_controller=diffrax.PIDController(rtol=1e-9, atol=1e-11),
max_steps=100_000,
)
A closed loop in output space must come back to a closed loop in input space; that round trip is the cheapest correctness check you have on an inverse map.
Under JAX the whole evaluation (solve, sensitivity, output Jacobians,
linear solve) rides one jax.pure_callback, so the RHS is traceable and
composes under jax.jit and diffrax. Under PyTorch it is a plain
callable — drop it into scipy.integrate or torchdiffeq.
warm=True warm-starts each inner solve from the previous evaluation’s
primal, duals, and barrier \(\mu\). The converged \(x^*(\theta)\) is
unique, so the result is unchanged up to solver tolerance; only the
iteration count drops, by a measured ~1.4–1.7× on smooth low-dimensional
maps. Interior-point methods warm-start weakly, so if the NLP is
expensive and the map is smooth, prefer PathFollower — its predictor
skips solves entirely rather than making each one cheaper.
When to use which
| Situation | Use |
|---|---|
| Active set may change along the path | PathFollower.follow — the robust default |
| The path folds (singular \(\partial x^*/\partial\theta\)) | PathFollower.trace_arclength |
| Smooth map, fixed active set, want adaptive stepping / dense output | inverse_map_rhs + diffrax / scipy |
All three run on the same held KKT factor; a predict step is one back-solve, never an NLP re-solve.
Scope and limitations
PathFollower supports equality constraints (cl == cu) and
variable bounds. Two-sided inequalities (cl != cu) are rejected with an
explicit error rather than silently mis-traced: the smooth-drift
monitor’s constraint residual (max|g|, valid only at \(g = 0\)) and
the arclength system \(R\) (which treats every row as \(g = 0\)) are
not valid for them. Reformulate inequalities with slack equalities.
trace_arclength additionally requires a scalar parameter and a
fixed active set along the traced branch; use follow for a
multi-dimensional path. Bifurcation and branch switching, Hopf detection,
general DAE continuation, and inequality-active folds are out of scope.
See also
notebooks/14_path_following.ipynb— runnable tour of all of the above, with the analytic checks.examples/inverse_map_diffrax.py— standalone 2-D coupled inverse map with a round-trip check.- Sensitivity Analysis — the underlying \(\partial x^*/\partial\theta\) and the active-set margin.
- Python API —
JaxProblem/TorchProblem, anchoring, and factor lifetime.
Curve Fitting
pounce.curve_fit fits a model f(x, *params) to data — the same call shape
as scipy.optimize.curve_fit —
but returns a much richer result and adds capabilities scipy’s fitter does not
have. It runs on pounce’s interior-point solver, so it inherits parameter
constraints, and because the solver keeps its converged factorization it can
hand back the parameter covariance (from the reduced Hessian) and the
data sensitivity ∂params/∂data essentially for free.
import numpy as np
import jax.numpy as jnp
import pounce
def model(x, a, b, c):
return a * jnp.exp(-b * x) + c # write the model with jax.numpy
x = np.linspace(0.2, 5, 40)
y = 3.0 * np.exp(-0.9 * x) + 0.5 + 0.05 * np.random.default_rng(0).normal(size=x.size)
res = pounce.curve_fit(model, x, y, p0=[1, 1, 0])
print(res.summary())
res.popt # fitted parameters
res.pcov # covariance matrix
res.perr # standard errors = sqrt(diag(pcov))
res.ci # (n, 2) confidence intervals at `alpha`
How it differs from scipy.optimize.curve_fit
| scipy.curve_fit | pounce.curve_fit | |
|---|---|---|
Least-squares fit + pcov | ✅ | ✅ |
Weighted (sigma, absolute_sigma) | ✅ | ✅ |
| Box bounds on parameters | ✅ | ✅ |
Relations between parameters (e.g. a + b ≤ 1) | ❌ | ✅ |
| Robust losses with covariance | partial | ✅ (sandwich) |
| Confidence intervals / goodness-of-fit in the result | ❌ | ✅ |
Data sensitivity ∂params/∂data | ❌ | ✅ |
| Exact derivatives via JAX | ❌ | ✅ |
The statistics follow the same conventions as scipy and
pycse.nlinfit: the covariance is
s² · (JᵀJ)⁻¹ with s² = SSE/(m − n) (the reduced χ²) unless
absolute_sigma=True, and confidence intervals use the Student-t quantile
popt ± t_{dof,1−α/2} · perr.
Already modelling in Pyomo?
pyomo_pounce.covariancecomputes the parameter covariance for an estimation model written directly in Pyomo — residuals as constraints, arbitrary surrounding structure — instead of af(x, *params)callable, using the same scale-and-invert-the-reduced-Hessian recipe read from the same held KKT factor. One caveat for nonlinear models:curve_fithere reports the Gauss-Newton covariance2·s²·(JᵀJ)⁻¹(the scipy/nlsconvention, always ≥ 0), whereascovariance()reports the observed-information covariance from the exact Hessian; they match for linear models and in the small-residual limit and differ by a few percent on a strongly-curved fit. See Parameter covariance and identifiability. Usecurve_fitwhen the fit is naturally a model-plus-data call (or when you want scipy-matching numbers); usecovariance()to interrogate a Pyomo model you already have.
Derivatives: prefer JAX
Accurate derivatives are what make the covariance and sensitivity sharp — and
they let the solver converge in a couple of iterations so the pounce-native
factor route is available. The Jacobian ∂f/∂p is resolved in this order:
- an analytic
jac=<callable>returning(len(x), n_params), - JAX autodiff (the default when the model is written with
jax.numpy), - a finite-difference fallback (used only if neither of the above applies; it emits a warning and the covariance falls back to the Jacobian form).
res = pounce.curve_fit(model, x, y, p0=[1, 1, 0]) # JAX (model uses jnp)
res = pounce.curve_fit(model, x, y, p0=[1, 1, 0], jac=myjac) # analytic
res = pounce.curve_fit(model_np, x, y, p0=[1, 1, 0]) # numpy model -> FD (warns)
Loss functions
Only smooth (C²) losses are supported, because the underlying solver is an interior-point method. Non-smooth L1/MAE is intentionally out of scope; use a robust loss instead.
loss | use |
|---|---|
"sse" (default), "chi2" | ordinary / weighted least squares |
"soft_l1" = "huber" | smooth pseudo-Huber, downweights outliers |
"cauchy" | strong outlier rejection |
"huber" and "soft_l1" are the same smooth (C²) pseudo-Huber loss: a
true piecewise Huber is only C¹ (its curvature jumps at the knee), which the
interior-point solver can’t use, so both names map to the C² form.
res = pounce.curve_fit(model, x, y, p0=[1, 1, 0], loss="huber", f_scale=0.1)
res.cov_source # "sandwich" (robust covariance estimator)
Parameter constraints
Box bounds express positivity / negativity / ranges; constraints=
expresses relations between parameters using the scipy-style dict format.
# positivity, ranges
pounce.curve_fit(model, x, y, p0=[1, 1, 0.2],
bounds=[(0, np.inf), (None, None), (0, 1)])
# a relation: require a + b <= 1 (ineq g(p) >= 0)
cons = [{"type": "ineq", "fun": lambda p: 1.0 - (p[0] + p[1])}]
pounce.curve_fit(model, x, y, p0=[0.4, 0.4, 0], constraints=cons)
When a bound or constraint is active at the optimum, the covariance is
projected onto the active-constraint nullspace (pounce’s reduced Hessian does
exactly this), and the affected parameter is flagged in res.active_mask with
an effectively degenerate confidence interval. res.cov_source reports
"reduced_hessian(projected)" in that case.
Data sensitivity: ∂params/∂data
Pass sensitivity=True to get res.dpopt_ddata, an (n_params, n_data)
matrix whose entry [j, i] is how fitted parameter j moves when data point
y_i is perturbed. This is the implicit-function-theorem influence
∂p*/∂y_i = 2 wᵢ² · H_S⁻¹ gᵢ, computed as a single batched back-solve against
the converged factor (Solver.kkt_solve_many).
res = pounce.curve_fit(model, x, y, p0=[1, 1, 0], sensitivity=True)
db = res.dpopt_ddata[1] # sensitivity of parameter b
i = int(np.abs(db).argmax()) # most influential point for b
print("most influential x:", x[i])
The result object
CurveFitResult carries everything in one place and supports dict-style access
(res["popt"]).
| field | meaning |
|---|---|
popt, pcov, perr, ci | parameters, covariance, std errors, confidence intervals |
correlation | normalized covariance |
residuals, sse, rmse, mae | fit residuals and error norms |
r_squared, adj_r_squared | coefficient(s) of determination |
chi_square, reduced_chi_square, dof | χ² statistics and degrees of freedom |
param_names | parameter names inferred from the model signature |
active_mask | which parameters sit on a bound |
cov_source | how the covariance was computed |
dpopt_ddata | data sensitivity (if requested) |
optimize_result | the raw solver info dict |
Methods: res.predict(xnew), res.confidence_band(...) (see below), and
res.summary() (a formatted report).
Confidence vs prediction bands
res.confidence_band(x, kind=..., sigma=...) returns (yhat, lower, upper),
but there are two different bands and they answer different questions.
-
Confidence band (
kind="confidence", the default) — uncertainty in the fitted curve itself, i.e. where the true meanE[y | x]lies. Its variance isgᵀ Σ g(delta method,g = ∂f/∂p,Σ = pcov). It is narrow, it shrinks toward zero as you collect more data, and most data points fall outside it — that is correct, not a miscalibration. -
Prediction band (
kind="prediction") — uncertainty in a new observationy = f(x) + ε. It adds the observation-noise variance:gᵀ Σ g + σ²(x). This is the band that contains about1 − alphaof the data; it does not shrink to zero, it floors at the noise level.
Both use the Student-t quantile t_{dof, 1−α/2} (not the normal z), so the
degrees of freedom are accounted for.
yhat, lo, hi = res.confidence_band(xx) # band on the curve
yhat, lo, hi = res.confidence_band(xx, kind="prediction") # band on new data
For the prediction band the noise level σ(x) is taken from the fit: the
sigma weights you supplied, scaled by the fitted variance s² (so a
heteroscedastic fit gives a heteroscedastic band — wider where the noise is
larger), or the homoscedastic level √s² if the fit was unweighted. Pass an
explicit sigma= (scalar or array over x) to override it, e.g. for new x
where you know the measurement noise.
Rule of thumb: use the confidence band to show how well the model is pinned down; use the prediction band to show where the next measurement will land. If “~95% of my points should be inside,” you want the prediction band.
Out-of-core data: curve_fit_streaming
When the dataset is too large to hold in memory, pounce.curve_fit_streaming
fits exactly the same model and objective as curve_fit, but reads the data
in mini-batches instead of as in-memory arrays. The solver’s objective,
gradient, and Gauss-Newton Hessian are all additive sums over data points, so
streaming and accumulating them produces the identical fit — only one batch
(plus an n_params × n_params matrix) is ever resident.
Instead of xdata, ydata you pass a data_source: a zero-argument callable
(a factory) that returns a fresh iterator of (x_batch, y_batch) — or
(x_batch, y_batch, sigma_batch) — tuples. It is called once per solver pass,
so it must yield the full dataset every time (re-open the file, re-slice the
mmap, …); a one-shot iterator is rejected.
import numpy as np
import pounce
# 50M points living on disk — re-read in 100k-row batches each pass
x_mm = np.load("x.npy", mmap_mode="r")
y_mm = np.load("y.npy", mmap_mode="r")
BATCH = 100_000
def data_source(): # fresh iterator every call
for i in range(0, x_mm.shape[0], BATCH):
yield x_mm[i : i + BATCH], y_mm[i : i + BATCH]
res = pounce.curve_fit_streaming(model, data_source, p0=[1, 1, 0])
print(res.summary())
res.popt, res.pcov, res.perr # identical to the in-memory fit
Notes and trade-offs:
- Re-readable, not one-shot. Each solver iteration (~10–50) makes one pass
over
data_source, so it must replay the whole dataset on every call. Uniform batch sizes avoid an extra JAX retrace on a smaller final batch. - Provide
p0. The data-driven seedcurve_fituses needs a full in-memory pass, so give a starting vector. With onlyn_paramsthe seed falls back to ones clipped intobounds. If the model signature doesn’t name the parameters and you omitp0, passn_params=. - What you get back is the same — all scalar diagnostics (SSE, χ², R², dof)
and the full covariance / standard errors / confidence intervals are computed
and are bit-for-bit the in-memory result. Everything else carries over too:
weighted fits (
sigmabatches), robustloss(the sandwich covariance is accumulated over batches),bounds, andconstraints(active sets project the covariance exactly as in the in-memory fit). - What is omitted — the two
O(n_data)outputs are not returned:res.residualsand the data sensitivityres.dpopt_ddataare bothNone(they are the size of the data and would defeat the purpose).confidence_bandstill works for newx, but uses a homoscedastic noise level since the per-pointsigmais not retained.
Multiple parameter sets: curve_fit_minima
Nonlinear least squares is generally non-convex, so the objective curve_fit
minimizes can have several local minima — distinct parameter sets that each
explain the data (peak-assignment ambiguity, frequency aliasing in sinusoids,
amplitude/decay trade-offs in sums of exponentials, sign/label symmetry, …).
pounce.curve_fit_minima drives find_minima over exactly
the same objective — same sigma weighting, robust loss, f_scale,
constraints, and resolved Jacobian — to enumerate those minima, then refines
each into a full CurveFitResult:
fits = pounce.curve_fit_minima(
model, x, y,
bounds=[(0, 3), (-10, 10), (0.1, 2.5)], # finite bounds = the search box
method="multistart", # or "deflation" | "flooding" | "mlsl" | ...
n_minima=5,
seed=0,
)
for r in fits: # ranked best (lowest SSE) first
print(r.popt, r.sse, r.r_squared)
fits[0].summary() # each is a full CurveFitResult
It reuses everything curve_fit does: the data-driven seed becomes the
search’s starting point, the model Jacobian is reused as the search gradient
and the Gauss-Newton matrix as the search Hessian — which sharpens the basin
escapes and lets find_minima certify each point as a true minimum (rejecting
saddles) before recording it. The returned list is ranked by SSE and may contain
fewer than n_minima entries when the landscape has fewer minima.
Finite
boundsare strongly recommended — they define the box the search samples / repels within. With the default unbounded box the search degrades to jittered restarts around the seed. Themethod,n_minima,max_solves,patience,dedup, andseedarguments pass straight through tofind_minima; see Finding Multiple Minima and Choosing a Method.
See python/examples/curve_fit_demo.py and the
22_curve_fit.ipynb
and
23_curve_fit_minima.ipynb
notebooks for complete, runnable walkthroughs.
Boundary Value Problems
pounce.bvp.solve_bvp solves two-point boundary value problems
dy/dx = f(x, y, p), a ≤ x ≤ b
bc(y(a), y(b), p) = 0
with a drop-in for scipy.integrate.solve_bvp. It
discretises the problem with the 4th-order Lobatto IIIA (Hermite–Simpson)
collocation scheme — the same one SciPy uses — and solves the resulting
square root-find as a pounce feasibility NLP (min 0 subject to the
collocation residual R(z) = 0).
The motivation is differentiability: because the discretised problem is
an NLP, the converged solution z*(θ) is differentiable with respect to
any parameter θ baked into f or bc, via the implicit-function theorem
on the collocation KKT system. The differentiable entry points live in the
autodiff frontends, pounce.jax.solve_bvp and pounce.torch.solve_bvp.
A runnable tour of every feature is in
python/notebooks/24_boundary_value_problems.ipynb, and a SciPy speed/accuracy comparison inpython/examples/bvp_scipy_compare.py(plus the GLC tritium-column case inpython/examples/glc_feral_vs_scipy.py). The GLC problem was suggested by Milan Rother and is adapted from pathsim-chem (MIT License).
Drop-in NumPy solve
import numpy as np
import pounce
# y'' = -|y|, y(0) = 0, y(4) = -2
def fun(x, y):
return np.vstack((y[1], -np.abs(y[0])))
def bc(ya, yb):
return np.array([ya[0], yb[0] + 2.0])
x = np.linspace(0, 4, 41)
y0 = np.zeros((2, x.size)); y0[0] = 1.0
res = pounce.solve_bvp(fun, bc, x, y0)
print(res.success, res.rms_residuals.max())
res.sol(np.linspace(0, 4, 9)) # cubic-Hermite interpolant, shape (n, 9)
The call signature and the returned bunch (sol, x, y, yp, p,
rms_residuals, niter, status, message, success) match SciPy, so
existing code consumes the result unchanged. Unknown parameters work the
same way — pass p=[...] and a fun(x, y, p) / bc(ya, yb, p):
# Eigenvalue: y'' + k² y = 0, y(0)=y(1)=0, y'(0)=k
def fun(x, y, p): return np.vstack((y[1], -p[0]**2 * y[0]))
def bc(ya, yb, p): return np.array([ya[0], yb[0], ya[1] - p[0]])
res = pounce.solve_bvp(fun, bc, x, y0, p=[3.0])
res.p # ≈ [π]
Differences from SciPy
- Mesh.
adaptive=True(default, like SciPy) refines the mesh to meettol.adaptive=Falsesolves the mesh you pass as-is — fast and predictable, and the mode the differentiable frontends use internally (a fixed mesh keepsθ ↦ ysmooth). verbosemirrors SciPy:1prints a one-line termination report,2also prints per-iteration mesh-refinement progress.- Solver (
method).method="newton"(default) runs a modified (frozen-Jacobian) Newton on the square collocation system, factorising theN×NJacobian with FERAL’s unsymmetric sparse LU (pounce._pounce.SparseLU) and reusing that factor across steps (refactoring only when progress stalls — the same trick SciPy’ssolve_newtonuses). Both scale linearly in the mesh; at equal mesh pounce is typically faster than SciPy (≈0.6–1.0×), including large nonlinear problems, because the factorisation dominates and it does far fewer of them. The Jacobian is the exact sparse collocation Jacobian (analytic per-node∂f/∂yblocks fromfun_jac/bc_jacif supplied, else a vectorised finite difference that perturbs each state across the whole mesh —O(n)funcalls, notO(n·m)).method="ipm"instead poses the system as a pounce feasibility NLP and solves with the interior-point method (factoring the2Nsaddle KKT each iteration — slower, but the basis for the constrained solver below). Accuracy is identical to SciPy either way. - Singular term
Sis not yet supported.
Differentiable solves (JAX / PyTorch)
The differentiable frontends take fun(x, y, p, theta) / bc(ya, yb, p, theta) (drop p when there are no unknown parameters), where theta is
the autodiff knob, and return a solution whose y / p participate in the
autodiff graph. Everything fun / bc close over is differentiable: a
physical coefficient, a boundary value, or the sensitivity of a solved-for
unknown parameter.
import jax, jax.numpy as jnp
import pounce.jax as pj
# Bratu: y'' + λ e^y = 0, y(0)=y(1)=0
def fun(x, y, lam): return jnp.vstack((y[1], -lam * jnp.exp(y[0])))
def bc(ya, yb, lam): return jnp.array([ya[0], yb[0]])
x = jnp.linspace(0, 1, 51)
y0 = jnp.zeros((2, x.size))
def y_mid(lam):
sol = pj.solve_bvp(fun, bc, x, y0, theta=lam)
return sol.y[0, sol.y.shape[1] // 2]
grad = jax.grad(y_mid)(1.0) # d y(0.5) / d λ
J = jax.jacobian(lambda l: pj.solve_bvp(fun, bc, x, y0, theta=l).y[0])(1.0)
The PyTorch frontend mirrors this exactly:
import torch
import pounce.torch as pt
torch.set_default_dtype(torch.float64)
lam = torch.tensor(1.0, dtype=torch.float64, requires_grad=True)
sol = pt.solve_bvp(fun, bc, x, y0, theta=lam) # fun/bc written with torch ops
sol.y[0, 25].backward()
lam.grad
What’s differentiable
| Target | How | Demo |
|---|---|---|
ODE/BC coefficient θ | jax.grad / .backward() through sol.y | examples/bvp_scipy_compare.py (a) |
| Boundary value | put it in bc and differentiate θ | (b) |
Solved-for unknown p* | differentiate sol.p | (c) |
Full solution dy/dθ | jax.jacobian over sol.y | (d) |
Vector θ | one reverse pass | (e) |
| Second derivative / Hessian | second_order=True | (f) |
All of these are validated against finite differences to ~1e-11 in
python/examples/bvp_scipy_compare.py, which also benchmarks accuracy and
speed against SciPy.
Differentiable solver backends (method)
The differentiable solve_bvp (both pounce.jax and pounce.torch) takes
the same method switch:
method="newton"(default) — the fast path. Forward is the FERAL sparse-LU Newton solve; the backward is the implicit-function-theorem VJPdz/dθ = −R_z⁻¹ R_θ, solvingR_zᵀu = vwith the same sparse LU (SparseLU.solve_transpose). Both directions stay on theNsystem — no2Nsaddle — so it is fast and differentiable. First-order only (the forward is an opaque callback).method="ipm"— routes the forward throughpounce.jax.solve/pounce.torch.solve(the interior-point feasibility NLP). Needed for second-order derivatives (below).
Second-order derivatives
With method="ipm", pass second_order=True to wrap the solve in a
custom_jvp whose tangent rule re-applies the implicit-function theorem to
the square collocation root-find,
dz/dθ = -(∂R/∂z)⁻¹ (∂R/∂θ),
and recovers z* through the same custom-ruled primitive, so JAX
recurses to arbitrary order:
def y_mid(lam):
sol = pj.solve_bvp(fun, bc, x, y0, theta=lam,
method="ipm", second_order=True)
return sol.y[0, sol.y.shape[1] // 2]
jax.grad(jax.grad(y_mid))(1.0) # d²y(0.5)/dλ² — works
The cost is one extra forward solve per differentiation level (the rule
re-solves to recover z*); the opaque forward is still only evaluated for
primal values. Leave it off for plain gradient-based training; turn it on
for Hessians / Newton-type outer loops.
Adaptive mesh refinement
Adaptive refinement is on by default (like SciPy), driven by tol /
max_nodes. Pass adaptive=False to solve the given mesh as-is:
res = pounce.solve_bvp(fun, bc, x, y0, tol=1e-6, max_nodes=2000) # adaptive
res = pounce.solve_bvp(fun, bc, x, y0, adaptive=False) # fixed mesh
Each round: solve on the current mesh (to round-off), estimate the relative
RMS residual of the continuous solution per interval with a 5-point Lobatto
quadrature at the superconvergent Gauss points x_mid ± ½h√(3/7), insert
nodes where it exceeds tol (one node, or two if it’s >100× over), and
re-solve warm-started off the previous solution. This is a faithful port of
SciPy’s estimator and refinement rule, so it reproduces SciPy’s mesh
sequence essentially node-for-node:
| problem | SciPy nodes | pounce nodes | solution agreement |
|---|---|---|---|
y''+y=0 | 6 → 31 | 6 → 31 | 1e-16 |
| Bratu | 5 → 29 | 5 → 29 | 6e-17 |
| `y’’=- | y | ` (kink) | 11 → 58 |
Adaptive is numpy-only — the differentiable pounce.jax /
pounce.torch paths are always fixed-mesh, because a parameter-dependent
mesh would make y(θ) nonsmooth and break the gradients. Pick a fixed mesh
fine enough for your θ range, or run an adaptive solve once to size it.
Constrained / optimal-control BVPs (pounce-unique)
pounce.solve_bvp_constrained solves a collocation BVP subject to bounds
on the states/parameters and inequality path constraints, optionally
minimising an objective:
dy/dx = f(x, y, p), bc(y(a), y(b), p) = 0
ylo <= y(x) <= yhi (state bounds, every node)
clo <= c(x, y, p) <= chi (path constraints, every node)
minimise J(Y, p) (optional)
This is a genuine NLP, so it goes through pounce’s interior-point method
(not the Newton path), and SciPy’s solve_bvp cannot express any of it.
A fully determined BVP (n + k boundary residuals) has a unique solution,
so constraints only bite when there is freedom — return fewer boundary
residuals and let the objective resolve the remainder (an optimal-control
collocation):
import numpy as np, pounce
# minimise ∫(y-1)² s.t. y''=0, y(0)=0 (slope free) — optimal control.
def fun(x, y): return np.vstack((y[1], np.zeros_like(y[0])))
def bc(ya, yb): return np.array([ya[0]]) # one boundary residual → 1 DOF
x = np.linspace(0, 1, 41); y0 = np.zeros((2, x.size)); y0[0] = x
obj = lambda Y, p: np.trapezoid((Y[0] - 1.0) ** 2, x)
r = pounce.solve_bvp_constrained(fun, bc, x, y0, objective=obj) # y(1) ≈ 1.5
rc = pounce.solve_bvp_constrained(fun, bc, x, y0, objective=obj,
y_bounds=([-np.inf, -np.inf], [1.2, np.inf]))
rc.y[0].max() # ≤ 1.2 — the bound is active and respected
path=path(x, Y, p) -> (q, m) with path_bounds=(clo, chi) adds inequality
path constraints at every node (assembled with a sparse block-diagonal
Jacobian). The objective’s gradient is finite-differenced; the Lagrangian
Hessian uses pounce’s limited-memory quasi-Newton (the path constraints
make it nonzero in general).
How it works
For a mesh x₀ < … < x_{m-1}, each interval contributes the Hermite–Simpson
collocation residual
y_mid = (y_i + y_{i+1})/2 - h/8 (f_{i+1} - f_i)
r_i = y_{i+1} - y_i - h/6 (f_i + 4 f(x_mid, y_mid) + f_{i+1}) = 0
Stacking the n·(m-1) collocation residuals with the n + k boundary
residuals gives a square system R(z) = 0 in the unknowns
z = [vec(Y); p] of size N = n·m + k. pounce solves it as min 0 s.t.
R(z) = 0. At the solution the interior-point method holds the KKT factor
of [[H, Jᵀ], [J, 0]] with J = ∂R/∂z; for this all-equality, no-bounds,
zero-objective problem the generic
implicit-diff backward collapses to the Newton
sensitivity
dz*/dθ = -(∂R/∂z)⁻¹ (∂R/∂θ),
which is exactly what jax.grad / autograd return — no BVP-specific
backward code. The collocation residual itself is shared verbatim across
the NumPy, JAX, and PyTorch paths (pounce/bvp/_core.py).
ODE / DAE Initial Value Problems
pounce.ode.solve_ivp integrates stiff initial value problems
M y' = f(t, y), y(t0) = y0
as a drop-in for scipy.integrate.solve_ivp with the
implicit Radau method. It implements the 3-stage Radau IIA collocation
scheme (order 5, L-stable) — the same method SciPy’s Radau uses, and the
classic RADAU5 of Hairer & Wanner. Each step’s coupled stage system is
solved by a simplified Newton iteration whose Jacobian is factored with
FERAL’s sparse LU.
Two things set it apart from SciPy:
- Mass matrix / DAEs. Pass
mass=Mto integrateM y' = f. WhenMis singular this is an index-1 differential-algebraic equation — somethingscipy.integrate.solve_ivpcannot do at all. - Differentiability.
pounce.jax.odeintandpounce.torch.odeintintegrate on a fixed mesh and return the trajectory differentiably with respect to the ODE parameters and the initial condition, via the implicit-function theorem on the collocation system (no per-step adjoint, no unrolled tape).
solve_ivp only implements method="Radau" — the implicit, stiff/DAE
capable method that is pounce’s niche. For non-stiff explicit integration,
SciPy or diffrax are the right tools, and solve_ivp raises for
those methods rather than silently substituting.
A SciPy speed/accuracy comparison, a DAE example, and a differentiability demo are in
python/examples/ode_scipy_compare.py.
Drop-in stiff solve
import numpy as np
import pounce.ode as po
# Van der Pol, mu = 1000 (very stiff)
mu = 1000.0
def f(t, y):
return [y[1], mu * (1 - y[0]**2) * y[1] - y[0]]
res = po.solve_ivp(f, (0.0, 3000.0), [2.0, 0.0],
method="Radau", rtol=1e-6, atol=1e-8, dense_output=True)
print(res.t.shape, res.y.shape) # (nsteps,) (2, nsteps)
ys = res.sol(np.linspace(0, 3000, 1000)) # continuous extension
The call signature and the returned object match SciPy: res.t, res.y
(n, n_points), res.sol (when dense_output=True), res.nfev /
res.njev / res.nlu, res.status / res.message / res.success. The
result is also dict-subscriptable like SciPy’s Bunch, so res["y"] and
"success" in res work too.
Provide an analytic Jacobian with jac=... (else it is estimated by finite
differences), and the usual t_eval, args, first_step, max_step,
rtol, atol controls.
Index-1 DAE via a mass matrix
A singular mass matrix turns the same solver into a DAE integrator. Robertson kinetics, written with the conservation law as an algebraic constraint:
import numpy as np
import pounce.ode as po
k1, k2, k3 = 0.04, 3e7, 1e4
def f(t, y):
return [-k1*y[0] + k3*y[1]*y[2],
k1*y[0] - k3*y[1]*y[2] - k2*y[1]**2,
y[0] + y[1] + y[2] - 1.0] # 0 = ... (algebraic)
M = np.diag([1.0, 1.0, 0.0]) # third equation is algebraic
res = po.solve_ivp(f, (0, 1e4), [1.0, 0.0, 0.0], mass=M,
rtol=1e-6, atol=1e-8)
The algebraic constraint is satisfied to round-off at every accepted step.
Inconsistent initial conditions
The algebraic components of y0 are determined by the differential ones,
so passing a rough guess for them is the normal case. When M is singular,
solve_ivp projects y0 onto the algebraic manifold 0 = f before
integrating — the same IDA_YA_YDP_INIT projection solve_dae uses — so
res.y[:, 0] is always a state the model admits:
# 0 = y1 - y0**2 => a consistent IC with y0 = 1 needs y1 = 1. Pass y1 = 5.
f = lambda t, y: [-y[0], y[1] - y[0] ** 2]
M = np.diag([1.0, 0.0])
r = po.solve_ivp(f, (0.0, 2.0), [1.0, 5.0], mass=M)
print(r.y[:, 0]) # [1. 1.] — projected onto the manifold
Pass consistent="assume" to opt out and use y0 verbatim (the old
behavior) when you know it is already consistent and rely on res.y[:, 0]
echoing your input. consistent is ignored for a non-singular mass (a plain
ODE has no manifold to project onto).
On-manifold output points
Radau IIA is stiffly accurate, so the constraint holds exactly at the solver’s
own accepted steps — but the dense-output polynomial only interpolates it
between them. For a linear conservation law (mass, atom, charge, or site
balance, sum(x) = 1) the interpolant satisfies the constraint exactly, so
there is nothing to fix. For a nonlinear algebraic constraint the
interpolated residual is small but nonzero. Pass project_output=True to
Newton-polish the algebraic components of every requested output point
(res.sol(t) and res.y at t_eval) back onto the manifold:
te = np.linspace(0, 2, 100)
r = po.solve_ivp(f, (0, 2), [1.0, 1.0], mass=M, t_eval=te,
project_output=True) # nonlinear 0 = y1 - y0**2
# max|y1 - y0**2| over te drops from ~5e-9 to ~1e-10
This is off by default, is skipped automatically for affine constraints
(where it buys nothing), and changes only what you read back — never the
trajectory, step sequence, or error control. See
python/examples/dae_manifold_gap.py to measure the interpolation gap on your
own DAE.
Differentiable integration (JAX / PyTorch)
For gradient-based work — fitting ODE parameters, neural ODEs, optimal
control — use the autodiff frontends. They integrate on a fixed mesh
t (make it fine enough to resolve the dynamics) and return the trajectory
differentiably w.r.t. the parameters theta and the initial condition
y0:
import jax, jax.numpy as jnp
import pounce.jax as pj
def f(t, y, theta): # dy/dt, JAX-traceable
k = theta[0]
return jnp.array([-k * y[0]])
t = jnp.linspace(0.0, 2.0, 81)
def y_final(k):
sol = pj.odeint(f, jnp.array([1.0]), t, jnp.array([k]))
return sol.y[0, -1]
val = y_final(0.7) # = exp(-0.7 * 2)
grad = jax.grad(y_final)(0.7) # exact d/dk via the implicit-function theorem
The PyTorch mirror is pounce.torch.odeint, with theta/y0 as tensors and
.backward() filling theta.grad / y0.grad. Both return a solution whose
y is (n, m) in SciPy layout and carries the autodiff graph; sol is a
(detached) cubic-Hermite interpolant for plotting.
Under the hood an IVP on a fixed mesh is just a boundary value problem with
bc(ya, yb) = ya - y0, so the differentiable path reuses pounce’s
Hermite–Simpson collocation and the same FERAL sparse-LU implicit-diff
back-solve as pounce.jax.solve_bvp. The result is the collocation
solution on the mesh you pass, and its gradients are exact for that
discretisation.
Performance
pounce.ode runs the same algorithm as scipy.integrate.solve_ivp(method= "Radau") (a faithful RADAU5), so it takes essentially the same number of steps
and reaches the same accuracy. The wall-clock difference is implementation
overhead: pounce’s stepper is pure Python, SciPy’s inner loop is compiled.
Practical guidance:
- Small / few-state stiff systems (state dimension up to roughly 10–20): pounce is at or below SciPy’s wall-clock. There is effectively no speed penalty for a single solve — and you get DAE support and differentiability on top.
- Large stiff systems (hundreds of states, e.g. a method-of-lines PDE): pounce is currently ~3–4× slower than SciPy in absolute terms, but still sub-second. That gap matters only when solving such a system many thousands of times in a loop — and if you need the differentiable path, SciPy is not an option at all.
Illustrative single-solve timings (best of 7; relative ratios are stable, the absolute milliseconds are machine-dependent):
| problem | states | pounce.ode | SciPy Radau |
|---|---|---|---|
| Van der Pol, μ=1000, t∈[0, 3000] | 2 | ~100 ms | ~105 ms |
| Brusselator (method-of-lines) | 100 | ~80 ms | ~24 ms |
| Brusselator (method-of-lines) | 300 | ~410 ms | ~94 ms |
These reflect three optimisations in the stepper, none of which change accuracy
or the public API: a RADAU5 stage predictor (warm-start each step’s Newton
from the previous step’s collocation polynomial), a wider step-size hold band
(reuse the cached factor across more steps), and reusing the LU pattern
across refactors (build FERAL’s symbolic analysis once per solve, refactor in
place). The last is what makes the large-n cost scale sensibly.
What it is and isn’t
- It is a faithful, L-stable Radau IIA(5) implementation that tracks SciPy’s
Radaustep-for-step on stiff problems and adds DAE and differentiability support SciPy lacks. - It is not a general non-stiff integrator: only
method="Radau"is implemented. - Event detection (
events=) is supported, matching SciPy: each event is a callableg(t, y)with optionalterminal(bool/ count) anddirectionattributes; crossings are root-found on the dense output and returned int_events/y_events(a terminal event stops withstatus=1). - The differentiable layer is fixed-mesh (the mesh keeps
theta → ysmooth); the adaptive solver is the non-differentiablesolve_ivp.
Fully-implicit DAEs
pounce.ode.solve_dae integrates a fully-implicit, index-1
differential-algebraic equation
\[ F(t, y, y’) = 0 \]
with the same Radau IIA(5) collocation as solve_ivp, written in
residual form. This is a pounce extension: scipy.integrate.solve_ivp has no
fully-implicit DAE solver (its closest relative, the mass-matrix form
M y' = f, is also available via solve_ivp(..., mass=M)).
import numpy as np
from pounce.ode import solve_dae
# Robertson kinetics as an index-1 DAE: two rate equations + a conservation law.
k1, k2, k3 = 0.04, 3.0e7, 1.0e4
def F(t, y, yp):
return np.array([
yp[0] - (-k1*y[0] + k3*y[1]*y[2]),
yp[1] - ( k1*y[0] - k3*y[1]*y[2] - k2*y[1]**2),
y[0] + y[1] + y[2] - 1.0, # algebraic constraint (no y')
])
res = solve_dae(F, (0.0, 1e4), y0=[1.0, 0.0, 0.0], rtol=1e-8, atol=1e-10)
print(res.y[:, -1], res.y[:, -1].sum()) # constraint held to round-off
Consistent initial conditions
A DAE solve needs (y0, y'0) with F(t0, y0, y'0) = 0. By default
(consistent="project") solve_dae computes them for you: it detects which
variables are algebraic (those that F does not depend on y' for — a
structurally-zero column of ∂F/∂y') and Newton-projects onto the constraint
manifold, holding the differential y and algebraic y' fixed and solving for
the differential y' and algebraic y (the IDA IDA_YA_YDP_INIT computation).
So a rough y0 (even one off the constraint) and yp0=None are fine:
# y0 violates the constraint (sum = 1.5) and no derivative guess is given —
# both are projected to a consistent state before integrating.
solve_dae(F, (0.0, 1e4), y0=[1.0, 0.0, 0.5], yp0=None)
Pass consistent="assume" with an explicit yp0 to skip the projection (you
guarantee F(t0, y0, yp0) == 0).
On-manifold output points
Radau IIA is stiffly accurate, so F_alg = 0 holds at every accepted step, but
the dense output only interpolates the constraint between steps. For an affine
constraint the cubic satisfies it exactly; for a nonlinear one the
interpolated residual is small but nonzero at intermediate res.sol(t) /
t_eval points. Pass project_output=True to Newton-polish the algebraic
components of each requested output point back onto 0 = F_alg, holding the
differential components fixed:
te = np.linspace(0.0, 2.0, 100)
res = solve_dae(F, (0.0, 2.0), y0=[1.0, 1.0], t_eval=te, project_output=True)
Off by default; skipped automatically when the algebraic rows are affine (it
buys nothing there — see solve_ivp’s DAE section and gh #216). It
changes only what you read from res.sol / res.y, not the trajectory or step
control.
Jacobians
jac(t, y, yp) -> (∂F/∂y, ∂F/∂y') is optional; both blocks are
finite-differenced (2n evaluations) when omitted. Supplying them avoids the
FD cost and improves robustness on stiff problems.
Scope
- Index-1 only. The stage matrix
I₃⊗∂F/∂y' + h(A⊗∂F/∂y)stays nonsingular for index-1 problems; higher index needs index reduction (not done here). - Same adaptive Radau engine as
solve_ivp— stiff-capable, sparse-LU stage solve, dense output (dense_output=True/t_eval=),args=. - Events are not supported.
Differentiable integration (JAX / PyTorch)
pounce.jax.daeint / pounce.torch.daeint integrate F(t, y, y', theta) = 0
on a fixed mesh and return the node trajectory differentiable w.r.t. the
parameters theta and the initial condition y0, via the
implicit-function theorem on the collocation system. As with
pounce.jax.odeint, the mesh is fixed (keeping the solution map smooth);
accuracy is controlled by the mesh. The default scheme is BDF2 (order=2,
L-stable, second-order); pass order=1 for backward Euler. F must be
framework-traceable.
import jax, jax.numpy as jnp
from pounce.jax import daeint
def F(t, y, yp, theta): # y0' + theta*y0 - y1 = 0 ; y0 + y1 = 1
return jnp.array([yp[0] + theta*y[0] - y[1], y[0] + y[1] - 1.0])
t = jnp.linspace(0.0, 2.0, 81)
y0 = jnp.array([0.5, 0.5])
loss = lambda th: daeint(F, y0, t, th)[0, -1] ** 2
g = jax.grad(loss)(1.3) # exact for the discretisation
The forward solve and the R_yᵀ back-solve run on the host (FERAL sparse LU);
the parameter VJP is taken by framework autodiff of the collocation residual at
the converged nodes. Gradients are validated against finite differences in the
test suite (python/tests/test_dae.py).
Glass Box / Black Box Optimization
pounce.trf_minimize solves problems where part of the model is an equation
and part is a program:
\[ \min_x f(x) \quad \text{s.t.} \quad h(x)=0,\; g(x)\le 0,\; y = d(w) \]
Here \(f, h, g\) are ordinary algebra with exact derivatives — the glass box — while \(d\) is a black box: a CFD solve, a converged unit model, a trained network. Something you can call but cannot hand to an NLP solver as equations. \(w\) and \(y\) are subvectors of \(x\).
This is common in process engineering, where a flowsheet is algebraic except for one unit that needs its own simulator.
Why not just fit a surrogate and optimize that?
Because it does not work, and it fails quietly. Consider
\[ \min\; x_1^2 + x_2^2 \quad \text{s.t.}\quad x_2 = x_1^3 + x_1^2 + 1 \]
whose solution is \((0, 1)\) with \(f = 1\). Replace the constraint with a linear surrogate \(x_2 = x_1 + b\), fit \(b\) so the surrogate matches the truth model at the current point, optimize, refit, repeat. Started exactly at the optimum, this iteration walks away and converges to \((-1, 1)\), where \(f = 2\) — a local maximum of the real problem (Biegler 2024, Fig. 2a).
The reason is that matching values gives feasibility but says nothing about optimality, which is a statement about gradients. The trust-region filter method fixes this by (a) correcting the surrogate to match the truth model in both value and slope, and (b) confining each step to a region where that correction is still valid.
Quick start
import numpy as np
import pounce
# Minimize (z-1)^2 + x^2 subject to z = sin(x), treating sin as a black box.
# The variable vector is v = [x, z]: v[0] is the black-box input w,
# v[1] is its output y.
res = pounce.trf_minimize(
fun=lambda v: (v[1] - 1.0) ** 2 + v[0] ** 2,
x0=[0.5, 0.0],
truth_model=lambda w: np.sin(w),
w_index=[0],
y_index=[1],
jac=lambda v: np.array([2 * v[0], 2 * (v[1] - 1.0)]),
truth_jac=lambda w: np.cos(w).reshape(1, 1),
)
print(res.x, res.fun, res.n_truth_evals)
You do not write the relationship y = d(w) into constraints yourself —
w_index and y_index tell the method which variables they are, and it
installs the surrogate constraint into each subproblem.
A worked, runnable version of everything below is in
python/notebooks/29_trust_region_filter.ipynb:
the failure mode plotted against the objective contours, the ZOC/FOC identities
demonstrated on a deliberately bad basis, a basis cost comparison, and the
Eason & Biegler benchmark with its convergence traces.
How it works
At each iteration the method:
-
Builds a corrected surrogate at the current point \(w_k\): \[ r_k(w) = \bar r(w) + \big(d(w_k) - \bar r(w_k)\big) + \big(J_d(w_k) - J_{\bar r}(w_k)\big)(w - w_k) \] The last two terms are the zero- and first-order corrections (ZOC/FOC). They guarantee \(r_k(w_k) = d(w_k)\) and \(J_{r_k}(w_k) = J_d(w_k)\) for any differentiable basis \(\bar r\).
-
Solves an ordinary NLP with \(d\) replaced by \(r_k\), and the trust region imposed as bounds on the decision variables: \(\max(l, u_k - \Delta) \le u \le \min(b, u_k + \Delta)\).
-
Evaluates the truth model at the trial point and measures the mismatch \(\theta = \lVert y - d(w) \rVert\).
-
Accepts or rejects via a filter on \((\theta, f)\) — a Pareto front of trade-offs rather than a penalty function with a parameter to tune.
Because ZOC/FOC makes the surrogate κ-fully linear automatically, the compatibility check and criticality phase of the original 2016 algorithm are unnecessary and \(\Delta\) need not shrink to zero.
Choosing a basis
The basis \(\bar r\) affects efficiency, not correctness — the corrections hold regardless of how good it is.
basis | Samples per iteration | Use when |
|---|---|---|
"zero" (default) | 0 (base point only) | Always start here. The surrogate is a plain linearization; this is also what pyomo.contrib.trustregion does by default. |
"quadratic" | \((n_w{+}1)(n_w{+}2)/2\) | Curvature matters and truth calls are cheap. Grows fast: 66 samples at \(n_w = 10\). |
You can also pass any object implementing the Basis protocol
(fit, predict, jacobian) — a low-fidelity physical model, a symbolic
regression fit, a Gaussian process. No adapter or registration is needed.
There is no "linear", and that is not an oversight
Only curvature survives the correction. Write the ZOC/FOC formula with an affine basis \(\bar r(w) = a + B(w - w_{\text{ref}})\). Its Jacobian is \(B\) everywhere, so the basis-dependent part is
\[ \bar r(w) - \bar r(w_k) - B(w - w_k) = B(w-w_k) - B(w-w_k) = 0 \]
leaving \(r_k(w) = d(w_k) + J_d(w_k)(w - w_k)\) — exactly the "zero" result.
An affine basis is therefore provably incapable of changing anything, while
costing \(n_w + 1\) truth-model calls per iteration to compute it. Before
adding any basis, check whether it is affine in \(w\); if it is, it cannot help.
Two findings from the literature worth internalizing before reaching for something fancy:
- Fit quality does not predict optimization performance. Pedrozo et al. (2025) benchmarked five surrogate families on a CO₂ pooling problem. Radial basis functions had the best R² of any model and needed 8 TRF iterations; Kriging needed 2; global polynomials were worst at both.
- Simple often wins outright. On Williams-Otto, Eason & Biegler (2016) found linear interpolation beat Kriging by 91 truth-model calls to 3141.
Fit the basis once and freeze it
By default a string basis is re-fitted from fresh truth-model samples every iteration. That is the wrong trade when the truth model is expensive, and it is not how the literature uses surrogates: Pedrozo et al. fit an ALAMO model once from a designed dataset and then let ZOC/FOC re-anchor it at each new point.
Pass a pre-fitted object and trf_minimize freezes it — fit is never called,
no per-iteration sampling happens, and each iteration costs one truth-model
evaluation plus a gradient. On the sin example:
| configuration | iterations | truth evals in the loop |
|---|---|---|
"zero" | 7 | 8 |
"quadratic", refit each iteration | 3 | 10 |
"quadratic", frozen | 3 | 4 (+3 upfront) |
Freezing keeps the quadratic’s three-iteration convergence but drops the in-loop cost from 10 calls to 4.
Freezing beats refitting the same basis; it does not automatically beat
"zero". At \(n_w = 3\) on a similar problem the numbers come out:
| configuration | iterations | in-loop | upfront | total |
|---|---|---|---|---|
"zero" | 9 | 10 | 0 | 10 |
"quadratic", refit | 4 | 41 | 0 | 41 |
"quadratic", frozen | 6 | 7 | 10 | 17 |
Freezing cuts the quadratic’s cost by more than half (41 → 17), but the free
"zero" basis still wins outright. The upfront design is
\((n_w{+}1)(n_w{+}2)/2\) calls — 10 at \(n_w=3\), 66 at \(n_w=10\) — paid
whether or not the curvature turns out to help. It amortizes when the run is
long or each truth call is genuinely expensive, and not otherwise.
Start with "zero". Reach for a frozen curved basis when you can see the
iteration count is the bottleneck and you have samples to spare.
from pounce.trf import QuadraticBasis, quadratic_design
design = quadratic_design(w0, 0.05)
basis = QuadraticBasis().fit(design, np.vstack([truth_model(w) for w in design]))
res = pounce.trf_minimize(..., basis=basis) # frozen automatically
The default is auto: string bases refit, user-supplied objects freeze — if
you fitted a model yourself it will not be clobbered. Override either way with
refit_basis=True|False.
This is also sound rather than merely cheap. Because ZOC/FOC forces \(r_k(w_k) = d(w_k)\) and \(J_{r_k}(w_k) = J_d(w_k)\) at every new base point, a basis fitted somewhere else entirely still converges to the truth model’s solution — it only contributes curvature.
Supply truth_jac if you possibly can
It is the single highest-value option here. It removes \(n_w\) truth-model calls per iteration and makes the surrogate exactly first-order accurate rather than accurate to the finite-difference step. Many simulators expose it — COMSOL’s sensitivity module, Aspen’s equation-oriented mode — and the ZOC/FOC variant is designed around the assumption that it is available.
Without it, the method finite-differences at the sampling radius \(\sigma_k\). That radius is deliberately not machine epsilon: the right perturbation is a property of the truth model. Eason & Biegler had to inflate it to \(\min(0.1, 0.8\Delta)\) for a boiler model “to compensate for greater numerical noise in the model outputs”, against \(10^{-5}\) for smooth steam tables.
Two radii, not one
trust_radius (\(\Delta\)) bounds the step. sampling_radius
(\(\sigma \le \Delta\)) bounds where the surrogate is fit. Eason &
Biegler (2018) introduced this separation and reported it more than doubled the
number of problems their test set could solve: with one radius doing both jobs,
the algorithm is forced into tiny steps near the solution purely to keep the
model accurate.
Convergence
trf_minimize reports success when both:
- \(\theta \le\)
feasibility_tol— the surrogate agrees with the truth model, so the point is feasible for the real problem; and - the step in \(w\) is below
criticality_tol— so the FOC gradient match is still valid at the solution.
The second is not optional. ZOC/FOC pins \(J_{r_k} = J_d\) only at the base point, so a large accepted step leaves the subproblem solving KKT conditions with a stale Jacobian. Testing feasibility alone will report convergence at points whose true gradient is of order one.
Limitations
Noise. The truth model must be deterministic and smooth. Eason & Biegler assume noise is negligible and list rigorous noise handling as open. For optimization against physical measurements, use a noise-aware method such as Bayesian optimization.
Local. Converges to a local KKT point. Pair it with
find_minima for a multistart sweep.
No restoration phase. The published algorithm calls a restoration procedure in two situations; this implementation approximates one and detects the other.
Incompatible subproblem. The glass-box constraints may have no solution inside
the current trust region — common on the first iteration, when the default
radius is simply too small for the constraints to be satisfiable. Contracting
would be exactly backwards, so trf_minimize expands the radius and retries,
logging the iteration as incompatible. On Eason’s example 1 this fires three
times before the first real step:
0 incompatible (Infeasible_Problem_Detected); Delta -> 2.000e-01
1 incompatible (Infeasible_Problem_Detected); Delta -> 4.000e-01
2 incompatible (Infeasible_Problem_Detected); Delta -> 8.000e-01
3 f= 0.2748077559 theta=4.513e-02 Delta=2.677e+00 f-step
If the radius reaches trust_radius_max and the subproblem is still
infeasible, the constraints are infeasible for reasons unrelated to the trust
region, and you get a clear error rather than a wrong answer.
Blocked filter. When the filter rejects every candidate, the iteration can
settle into an f-step / θ-step / rejected limit cycle in which θ creeps down but
the step length never shrinks. Real restoration would find a point acceptable to
the filter; without it, trf_minimize detects the stall and returns
success=False with a Stalled: message rather than silently burning its
iteration budget. If you hit it, try a larger trust_radius, a richer basis, or
a looser feasibility_tol.
References
- Eason, J.P. & Biegler, L.T. A trust region filter method for glass box/black box optimization. AIChE J. 62, 3124–3136 (2016).
- Eason, J.P. & Biegler, L.T. Advanced trust region optimization strategies for glass box/black box models. AIChE J. 64, 3934–3943 (2018).
- Yoshio, N. & Biegler, L.T. Demand-based optimization of a chlorobenzene process… AIChE J. 67, e17054 (2021).
- Biegler, L.T. The trust region filter strategy. Digital Chemical Engineering 13, 100197 (2024).
- Pedrozo, H.A. et al. Surrogate model optimization: a comparison case study with pooling problems of CO₂ point sources. Comput. Chem. Eng. 200, 109199 (2025).
WebAssembly: POUNCE in the Browser
POUNCE’s default build is pure Rust — no C, no Fortran, no BLAS to link —
so the entire solver compiles to WebAssembly and runs in a browser tab:
the AMPL .nl reader, the reverse-mode AD tape, the sparse LDL^T
factorization, and the interior-point algorithm. Nothing is sent to a
server.
Two pages ship with the docs, both published from main and both running
the solver locally in your tab:
- /demo — drop a
.nlfile on the page, see what is in the model, solve it, download the solution. - /demo/python — write a Pyomo model in Python and solve it, via Pyodide.
To run them from a checkout:
rustup target add wasm32-wasip1 # once
crates/pounce-wasm/build.sh --serve # the .nl page, :8000
crates/pounce-wasm/build.sh --serve-python # the Python page, :8000
Or make wasm to build the module without serving anything.
Hosting it
Each page is a static directory (crates/pounce-wasm/web/ and
crates/pounce-wasm/web-python/) — deploying either is a copy. Neither
needs a special server: no threads means no
SharedArrayBuffer, so none of the Cross-Origin-Opener-Policy /
Cross-Origin-Embedder-Policy headers that thread-enabled wasm requires,
and every URL the page fetches is relative, so it works under any base
path. If a host serves .wasm as something other than application/wasm,
the page falls back from streaming compilation to a buffered
WebAssembly.instantiate on its own.
GitHub Pages is what this repository uses: .github/workflows/docs.yml
builds the module and stages the two directories into the docs site at
/demo/ and /demo/python/, so both ship with every docs deployment from
main. They are version-independent — one live build each, not one per
archived release tag.
What you get
Dropping a model shows the problem summary POUNCE derives while building
its evaluator — sizes, degrees of freedom, how many rows are equalities,
how much of the model is nonlinear, Jacobian and Hessian sparsity, and how
the variable bounds break down. Solving streams the usual iteration table
into the page (that really is the solver’s stdout) and reports the exit
status, KKT residuals, evaluation counts, and the solution vector next to
the .col / .row names when you drop those alongside the .nl.
Solve options are ipopt.opt-format text — the same option names the CLI
and the Python API take.
Three downloads come off a finished solve:
| Download | What it is |
|---|---|
.sol | An AMPL solution file — byte-identical to what pounce model.nl writes, including the ipopt_zL_out / ipopt_zU_out reduced-cost suffixes. AMPL and Pyomo read it back. |
| CSV | One row per variable and per constraint: name, value, bounds, multiplier. |
| log | The solver output, as printed. |
The .sol and CSV are formatted inside wasm from the full solution, not
from the table on screen — the page truncates long vectors at 2,000 rows to
stay renderable, and a download that stopped there would be worse than none.
Dropping a new file resets everything: the page throws away its worker and starts a fresh wasm instance, so no parsed model, solver state, or grown heap carries from one file into the next.
The Python page
Pyodide supplies CPython compiled to WebAssembly; micropip installs Pyomo
(a py3-none-any wheel — nothing to build). You write an ordinary Pyomo
model, and:
from pyomo.environ import *
import pounce_browser
m = ConcreteModel()
m.x = Var([1, 2], initialize=0.5, bounds=(-10, 10))
m.circle = Constraint(expr=m.x[1]**2 + m.x[2]**2 == 1)
m.obj = Objective(expr=m.x[1])
m.dual = Suffix(direction=Suffix.IMPORT)
res = pounce_browser.solve(m, options="print_level 5")
print(res.status, value(m.x[1]), m.dual[m.circle])
solve() writes the model with Pyomo’s own NL writer, hands the .nl text
to the POUNCE wasm module, and loads the returned .sol back onto the
model, so x.value and model.dual[c] read exactly as after a local solve.
Variables and rows are matched by the writer’s own ordering
(NLWriterInfo.variables / .constraints), so the mapping cannot drift
from the file it just wrote — crates/pounce-wasm/tests/pyomo_roundtrip.py
pins that with a model whose optimum and multipliers are known in closed
form, and CI runs it on every PR with Node standing in for the browser.
The script box is a small editor — Python highlighting, line numbers,
Tab/Shift-Tab indent, indentation carried across Enter — built from a
highlighted <pre> behind a transparent <textarea> so the caret and undo
stay native. No editor library: a CDN dependency would be absent in exactly
the offline setup ?pyodide= exists for.
Two wasm runtimes are in play — Pyodide’s CPython and POUNCE — with separate
memories; all that crosses between them is .nl text one way and JSON plus
.sol text the other.
This is Pyomo’s modelling layer, not POUNCE’s own Python API: the model
reaches the solver as a file, so there are no Python callbacks mid-solve.
Running the real pounce-solver package in a browser would mean building
the compiled extension for Pyodide (emscripten), which this does not do.
The page needs the network for its first load — Pyodide from a CDN, Pyomo
from PyPI, about 15 MB, cached afterwards. Self-host both and pass
?pyodide=…&pyomo=… to avoid it entirely; see
crates/pounce-wasm/web-python/README.md. The solve itself is local either
way.
Numerical parity with the native build
The wasm build runs the same code, so it produces the same answers. Over
all 37 .nl fixtures in crates/pounce-cli/tests/fixtures, driven through
the same entry points on both sides:
- exit status: identical on 37 of 37
- iteration count: identical on 37 of 37
- objective: bit-identical on 34 of the 36 that return one; the two
exceptions (
scaled_feasible_a,feasible_x0_sentinel_bound) differ by one ulp, at objectives of 4.5e-10 and 7.1e-11
The 37th (presolve_overflow_feasible) returns InvalidNumberDetected
with no objective on either side — that is the fixture’s job.
Speed is what you would expect from wasm. Solver-internal wall time, same
build, same code path, native x86_64 vs wasm32-wasip1 under Node:
| model | n × m | native | wasm | ratio |
|---|---|---|---|---|
pooling_rt2stp | 46 × 72 | 9.9 ms | 40 ms | 4.0× |
jit1 | 25 × 32 | 8.1 ms | 43 ms | 5.3× |
airport | 84 × 42 | 20 ms | 80 ms | 4.1× |
autocorr_bern55-06 | 56 × 1 | 50 ms | 101 ms | 2.0× |
deb7 | 813 × 897 | 461 ms | 556 ms | 1.2× |
The larger the model, the closer wasm gets: small solves are dominated by
per-call overhead, while big ones spend their time in the sparse
factorization, where the gap narrows. Nothing here is tuned — no SIMD, no
wasm-opt.
How it is put together
| Piece | What it is |
|---|---|
crates/pounce-wasm | C-ABI entry points (pounce_load, pounce_solve, the exporters), bytes in / JSON out |
crates/pounce-wasm/web | the .nl page: index.html, app.js, worker.js, wasi.js |
crates/pounce-wasm/web-python | the Pyodide page, plus pounce_browser.py — the Pyomo ↔ POUNCE shim |
crates/pounce-wasm/build.sh | builds the module and stages it into both pages |
The target is wasm32-wasip1, not wasm32-unknown-unknown. WASI gives the
solver a clock (std::time::Instant::now() panics on
wasm32-unknown-unknown, and POUNCE times every solve) and a stdout to
write its iteration table to. Browsers do not implement WASI, so the page
carries a ~60-line shim, wasi.js, which answers clock_time_get from
performance.now() and turns each fd_write into a line in the log pane.
That shim is the entire cost of the approach: no wasm-bindgen, no npm, no
build step beyond cargo build.
A solve is one synchronous call into wasm that can run for seconds, so the module lives in a web worker and the page stays responsive.
Payloads cross the boundary as a little-endian u32 byte count followed by
that many UTF-8 bytes. Reading a length rather than scanning for a NUL
terminator keeps the reader’s correctness independent of what is in the
payload, and lets a bad pointer or length be reported as exactly that
instead of surfacing later as an unrelated parse error.
Limitations
- Single-threaded. No threads are spawned; rayon-parallel paths run serially. Results are unaffected.
- No AMPL imported functions. A model that calls compiled-C external
functions (
funcadd_ASL— IDAES property packages, for instance) needs a dynamic loader the browser sandbox does not provide. The summary flags such a model rather than failing mysteriously mid-solve. - No HSL. The optional
ma57backend links Fortran; the wasm build uses the default FERAL backend, like any stockcargo build. - 2.4 MB module, about 800 kB gzipped over the wire.
Embedding it in your own page
crates/pounce-wasm is a thin shim you can copy or fork. The ABI is four
exports — allocate, load, solve, free — and every payload is JSON:
const summary = fromWasm(wasm.pounce_load(nlPtr, nlLen, 0, 0, 0, 0));
const result = fromWasm(wasm.pounce_solve(optsPtr, optsLen));
const solFile = fromWasm(wasm.pounce_solution_sol()); // AMPL .sol text
const csv = fromWasm(wasm.pounce_solution_csv()); // every row
Both entry points catch panics and return {"error": …}, so a malformed
model cannot trap the instance. See crates/pounce-wasm/web/README.md for
the full walkthrough and crates/pounce-wasm/tests/smoke.mjs for a
headless (Node) driver of the same ABI.
Finding Multiple Minima
pounce.minimize finds a single local minimum from a starting point.
pounce.find_minima is its global-search companion: it drives the same
local solver in a loop to discover many distinct minima, or the global
one among them.
import pounce
result = pounce.find_minima(
fun, x0,
method="deflation", # see the method families below
jac=jac, hess=hess, # same as minimize; analytic derivatives recommended
bounds=bounds,
n_minima=6, # target number of distinct minima
max_solves=None, # budget; default 8 * n_minima
patience=8, # give up after this many solves with nothing new
dedup=1e-3, # minima closer than this are "the same"
seed=0,
)
result.minima # list of minima, sorted by objective (lowest first)
result.values # their objective values
result.x # the best (lowest) minimum
result.status # "target_reached" | "converged" | "budget_exhausted"
result.n_solves # solver calls used
result.trace # per-solve diagnostics
Every method reuses minimize, so bounds and constraints carry through
unchanged, and the acceptance test is shared: each candidate is polished
on the clean objective, checked against the bounds, and — when a Hessian is
supplied — certified as a true minimum (positive-semidefinite Hessian, so
saddles and maxima are rejected) before being de-duplicated and recorded.
The six methods fall into three families by how they escape a minimum they have already found.
Repulsion — transform the problem and re-solve
These modify the problem so the solver can no longer settle where it just did, then re-solve. They share the lineage of the filled-function method and metadynamics: make the found minimum unattractive.
flooding
Add a repulsive Gaussian bump to the objective at each found minimum
x*_k:
F(x) = f(x) + Σ_k A_k · exp(−‖x − x*_k‖² / 2σ_k²)
The bump does not move the stationary point (a Gaussian is flat on top); it
flips its curvature. The minimum turns into a saddle once the bump is
taller than the basin’s curvature — precisely when
A/σ² > λ_min(∇²f(x*)) — and the solver rolls off it into a new
basin. The bump is smooth with an analytic gradient and Hessian, so the
flooded problem is as solvable as the original.
- Knobs (
strategy_kw):sigma(width) andamplitude(height). Both are"auto"by default.sigmais per-dimension — a fraction (sigma_frac, default 0.1) of each variable’s bounds range — so variables on very different scales are handled automatically.amplitudeis set per minimum from the local curvature (amp_margin × μ_min, the well-tempered escape height, whereμ_minis the smallest generalized eigenvalue of the Hessian against the bump metric) and raised adaptively if the solver returns to a flooded basin — so no manual energy scale is needed (a steep PES whose wells are ~150 deep needs noamplitude=150). Override either with a scalar or a length-nvector (sigma). - Best for broad enumeration of all minima of a smooth objective.
- References. Ge, R. “A filled function method for finding a global minimizer of a function of several variables.” Mathematical Programming 46, 191–204 (1990). doi:10.1007/BF01585737. Laio, A. & Parrinello, M. “Escaping free-energy minima.” PNAS 99(20), 12562–12566 (2002). doi:10.1073/pnas.202427399. Grubmüller, H. “Predicting slow structural transitions in macromolecular systems: Conformational flooding.” Phys. Rev. E 52(3), 2893–2906 (1995). doi:10.1103/PhysRevE.52.2893. Adaptive bump heights: Barducci, A., Bussi, G. & Parrinello, M. “Well-tempered metadynamics.” Phys. Rev. Lett. 100, 020603 (2008). doi:10.1103/PhysRevLett.100.020603.
deflation
Instead of a finite local bump, add a singular pole penalty:
F(x) = f(x) + Σ_k η / (‖x − x*_k‖² + s)^(p/2)
Each found minimum becomes infinitely costly. The pole reaches further than
a Gaussian (it decays as 1/r^p rather than vanishing exponentially), so
it can clear a basin a narrow Gaussian would miss. This is the additive,
minimization-friendly realization of the deflation idea, whose original
form multiplies the residual of a nonlinear system by a deflation operator
to exclude known roots for a Newton iteration.
- Knobs:
eta(penalty strength),powerp,softs(softening that keeps the pole finite), andlength— the per-dimension pole scale, also"auto"from the bounds range by default (scalar or vector to override). - Best for enumeration on problems where the longer-reach repulsion helps; the most Newton/IPM-native of the repulsion methods.
- References. Brown, K.M. & Gearhart, W.B. “Deflation techniques for the calculation of further solutions of a nonlinear system.” Numerische Mathematik 16, 334–342 (1971). doi:10.1007/BF02165004. Farrell, P.E., Birkisson, Á. & Funke, S.W. “Deflation techniques for finding distinct solutions of nonlinear partial differential equations.” SIAM J. Sci. Comput. 37(4), A2026–A2045 (2015). doi:10.1137/140984798.
tunneling
Rather than climb out of a basin, tunneling crosses sideways at constant height to a point past the barrier, then descends. Between local solves it seeks a point at the height of the most-recently found minimum while being repelled from all known minima, and then re-minimizes there. The result is a monotonically non-increasing sequence of minima.
- Knobs:
eta,power,soft(the repelling poles). - Best for finding the global minimum and a descending trail to it, not exhaustive enumeration.
- Reference. Levy, A.V. & Montalvo, A. “The tunneling algorithm for the global minimization of functions.” SIAM J. Sci. Stat. Comput. 6(1), 15–29 (1985). doi:10.1137/0906002.
A worked example of all three is in
python/notebooks/19_find_minima_repulsion.ipynb.
Restart — choose the next start cleverly
These leave the objective untouched and only change where each local solve begins.
multistart
Random (or Sobol low-discrepancy) sampling of the bounds box, one local solve per start. Simple, a strong baseline, and embarrassingly parallel.
- Knobs:
sobol(low-discrepancy sampling, on by default),restart_jitter(used when no bounds box is given). - Best for a robust default, especially when local solves are cheap and can be parallelized.
mlsl
Multi-Level Single Linkage grows a pool of sample points and starts a local solve from a sample only when (a) no better sample lies within a shrinking “reduced distance,” and (b) it is not near an already-found minimum. The effect is that each basin is descended approximately once, instead of many times as plain multistart re-discovers knowns.
- Knobs:
samples_per_round,gamma(reduced-distance scale). - Best for expensive local solves on funneling landscapes, where avoiding redundant descents matters.
- Reference. Rinnooy Kan, A.H.G. & Timmer, G.T. “Stochastic global optimization methods part II: Multi level methods.” Mathematical Programming 39, 57–78 (1987). doi:10.1007/BF02592071.
See python/notebooks/20_find_minima_restart.ipynb,
which shows multistart spending ~15 solves (9 redundant) to find all six
camel minima where MLSL needs ~6 (0 redundant).
Hopping — a Markov chain over minima
basinhopping
From the current minimum, apply a random perturbation, locally minimize to a neighboring minimum, and accept or reject by a Metropolis rule on the objective. The chain is biased downhill, so it reliably reaches the global minimum while collecting the distinct minima it visits.
- Knobs:
step(perturbation size),temperature(acceptance). - Best for the global minimum on rugged, high-dimensional landscapes — the workhorse of cluster and protein optimization.
- References. Li, Z. & Scheraga, H.A. “Monte Carlo-minimization approach to the multiple-minima problem in protein folding.” PNAS 84(19), 6611–6615 (1987). doi:10.1073/pnas.84.19.6611. Wales, D.J. & Doye, J.P.K. “Global optimization by basin-hopping…” J. Phys. Chem. A 101(28), 5111–5116 (1997). doi:10.1021/jp970984n. Cousin with history feedback: Goedecker, S. “Minima hopping…” J. Chem. Phys. 120(21), 9911–9917 (2004). doi:10.1063/1.1724816.
See python/notebooks/21_find_minima_hopping.ipynb.
Beyond minima: saddle points and critical points
The same ideas extend to every stationary point of f — saddles
(transition states) and maxima included. A critical point has ∇f(x) = 0;
its Morse index (the number of negative Hessian eigenvalues) classifies
it: 0 = minimum, 1 = transition state, …, n = maximum. Two entry
points are provided.
find_critical_points — enumerate and classify
Stationary points are the roots of ∇f(x) = 0, which are exactly the minima
of the gradient-norm merit ½‖∇f(x)‖² (zero there). So find_critical_points
runs find_minima on that merit — using any enumeration method
("deflation", "multistart", …) — then keeps the points where ‖∇f‖ is
truly zero and labels each by its Morse index. This treats pounce as a
root-finder and reuses the whole find_minima machine.
r = pounce.find_critical_points(
fun, x0, grad=grad, hess=hess, bounds=bounds,
method="deflation", n_points=12, dedup=1e-2,
)
r.minima # index 0
r.saddles # 0 < index < n (transition states)
r.maxima # index n
for p in r.points:
print(p.kind, p.x, p.f, p.index)
find_saddles — eigenvector following
A saddle is a minimum in most directions and a maximum along a few. By
walking uphill along the index softest Hessian eigenvectors and Newton-
downhill in the rest, eigenvector following lands directly on an
index-index saddle; multistart enumerates several.
s = pounce.find_saddles(fun, x0, grad=grad, hess=hess, bounds=bounds,
index=1, n_saddles=4)
Together with the minima, the index-1 saddles between them form the transition-state network / disconnectivity graph — flooding fills the basins, and the saddles are the barriers crossed between filled basins.
reaction_network — states, barriers, and connectivity in one call
reaction_network packages the whole workflow: it finds the minima (stable
states), finds the index-1 transition states, and connects each
transition state to the two minima it joins — by descending its unstable
mode into each adjacent basin — returning the barrier table and the
minimum-energy paths.
net = pounce.reaction_network(
fun, x0, grad=grad, hess=hess, bounds=bounds,
n_states=3, n_transition_states=2,
minima_kw={"sigma": 0.4, "amplitude": 150.0}, # find_minima tuning
saddle_kw={"max_step": 0.05}, # find_saddles tuning
)
print(net.summary())
net.minima # stable states, sorted by energy (CriticalPoint)
net.transition_states # index-1 saddles
net.connections # each: .ts, .minima=(i,j), .barrier=(fwd,rev), .path (MEP)
net.barrier(i, j) # lowest single-step barrier from state i to state j
net.neighbors(i) # states reachable from i over one transition state
net.path_between(i, j) # the connecting minimum-energy path, oriented i -> j
This is the natural high-level entry point for reaction-barrier and
energy-landscape work: the connectivity it returns is the reaction network
(equivalently, a disconnectivity graph), and the barrier of an elementary
step i → j is E(transition state) − E(state i).
- References. Cerjan, C.J. & Miller, W.H. “On finding transition states.” J. Chem. Phys. 75, 2800 (1981). Henkelman, G. & Jónsson, H. “A dimer method for finding saddle points…” J. Chem. Phys. 111, 7010 (1999). doi:10.1063/1.480097. Henkelman, G., Uberuaga, B.P. & Jónsson, H. “A climbing image nudged elastic band method…” J. Chem. Phys. 113, 9901 (2000). doi:10.1063/1.1329672. E, W. & Zhou, X. “The gentlest ascent dynamics.” Nonlinearity 24, 1831 (2011). doi:10.1088/0951-7715/24/6/008.
Runnable demos: a landscape with 4 minima, 4 saddles, and 1 maximum in
python/examples/critical_points.py,
and a molecular reaction barrier on the Müller-Brown potential — one
reaction_network call locating the stable states and the transition states
between them, then reading off barrier heights and the minimum-energy path —
in python/examples/reaction_barrier.py.
Termination
The search stops on whichever fires first, reported in result.status:
| condition | meaning | status |
|---|---|---|
n_minima distinct minima found | got what you asked for | target_reached |
patience solves in a row find nothing new | landscape appears exhausted | converged |
max_solves reached | spent the budget | budget_exhausted |
patience is what makes the “fewer minima exist than requested” case
efficient: ask for 6, find 2, try a few more times, and stop with
converged rather than burning the whole budget. find_minima always
returns however many minima it actually found — falling short of n_minima
is not an error.
A solve is many function evaluations; max_solves counts solver calls. A
true per-evaluation ceiling belongs inside each solve via
options={"max_iter": ...}.
Choosing a method
See Choosing a Multiple-Minima Method, including how the families behave as the dimension grows.
Scope
find_minima covers methods that drive pounce’s local solver as their inner
loop. Rigorous deterministic global optimization (branch-and-bound, DIRECT),
population/stochastic globals (differential evolution, CMA-ES — already in
SciPy), and homotopy continuation (all stationary points of polynomial
systems) are different machinery and out of scope.
Choosing a Multiple-Minima Method
All six find_minima methods drive the same local solver; they differ in
how they leave a minimum once found. Use this page to pick one.
By goal
| Your goal | Prefer | Why |
|---|---|---|
| Enumerate all minima of a smooth, low-dimensional objective | flooding, deflation | repulsion clears each basin so the next solve finds a new one; analytic derivatives keep the inner solve fast |
| Just the global minimum | basinhopping, tunneling | both are biased downhill and do not try to cover the whole space |
| A robust, parallel baseline | multistart | independent starts, trivially parallel, no tuning |
| Expensive solves on a funneling landscape | mlsl | clustering avoids re-descending basins it has already mapped |
| Rugged, high-dimensional landscape (clusters, conformers) | basinhopping | a local random walk over minima; the standard tool at scale |
By problem structure
- Have an analytic Hessian? Repulsion methods (
flooding,deflation) exploit it directly and certify each result as a true minimum. Without a Hessian, saddle rejection is skipped and the restart/hopping methods are a safer default. - Constrained problem? All methods pass
bounds/constraintsthrough. Repulsion only touches the objective, so it is the cleanest with general constraints; restart and hopping sample/perturb inside the bounds box. - No bounds?
multistart/mlslfall back to jittering aroundx0(give aboundsbox for genuine global coverage).flooding/deflationandbasinhoppingwork without bounds. - Variables on very different scales? Handled automatically. The
repulsion bump widths (
sigma/length) are per-dimension and"auto"by default — sized to each variable’s bounds range — and the default dedup metric measures distance in that same scaled space, so a singlededuptolerance is scale-free. Giveboundsso the scales can be inferred; pass an explicit scalar or length-nvector to override. - Symmetric or periodic coordinates (e.g. a periodic box): pass a custom
distance=metric so that images of the same minimum de-duplicate correctly.
Tuning cheat-sheet
| method | key knobs | rule of thumb |
|---|---|---|
flooding | sigma, amplitude | both "auto" by default (sigma per-dimension from the bounds; amplitude per-minimum from local curvature) — leave them; override only to force a specific width/height |
deflation | eta, power, soft, length | length is per-dimension "auto" by default; raise eta if the solver returns to a known minimum |
tunneling | eta, power | increase patience; it descends in a chain |
multistart | sobol | leave Sobol on for coverage |
mlsl | samples_per_round, gamma | more samples/round on rugged landscapes |
basinhopping | step, temperature | step ≈ basin spacing; raise temperature to cross higher barriers |
If a run stops at converged with fewer minima than you wanted, raise
patience (search longer before giving up) and/or max_solves. If it stops
at budget_exhausted, raise max_solves.
Scaling to high dimensions
The honest headline: enumerating all minima is intractable in high
dimensions for every method here — and that is a property of the problem,
not of the solver. The number of local minima typically grows exponentially
with dimension (Rastrigin has on the order of k^n; molecular energy
landscapes grow exponentially with the number of atoms). No method can list
exponentially many minima. What changes with dimension is which goal
remains reachable and which methods stay efficient.
Two costs scale independently:
- Cost per local solve. This is just pounce’s interior-point solve and
scales well with sparse, large
n— provided the objective stays sparse. Here is the catch for repulsion methods: each Gaussian or pole term adds a densen×nHessian contribution, so withKfound minima the augmented Hessian is the sparse base plusKdense updates. On large sparse problems this destroys sparsity and the inner solve slows sharply. Restart and hopping never modify the objective, so they keep the original sparsity and the per-solve cost scales asminimizeitself does. - Number of solves needed. For coverage this grows exponentially for all methods. For the global minimum it grows much more slowly for the downhill-biased methods, which is why they remain usable at scale.
How each family behaves as n grows:
- Repulsion (
flooding,deflation). Two problems compound. A Gaussian of widthσcovers a vanishing volume fraction~ σⁿ, so filling space needs exponentially many bumps; and the bumps densify the Hessian (above). The standard high-dimensional fix — used by metadynamics in practice — is to flood in a low-dimensional collective-variable subspace rather than allncoordinates. In full coordinates these are best kept to roughlyn ≲ 10–20. - Restart (
multistart,mlsl). Each start is cheap and parallel, but the number of starts to cover (or to hit the global basin) grows exponentially. MLSL’s clustering relies on a reduced radius∝ (ln N / N)^(1/n); asngrows that exponent→ 0, distances concentrate, and MLSL degenerates toward plain multistart. So MLSL’s advantage is a low-to-moderate-dimension phenomenon; in high dimension prefer plain parallelmultistart(for the global basin) and spend the budget on more starts. - Hopping (
basinhopping). This is the family that scales best in practice, and it is exactly what the chemistry/physics community uses for hundreds to thousands of degrees of freedom. It performs a local random walk in minimum-space — it never tries to cover the domain — keeps the objective (and its sparsity) untouched, and the Metropolis bias funnels toward low minima. Pair it with multiple independent chains for parallelism.
Practical guidance:
n ≲ 10–20, want all minima:flooding,deflation, ormlsl.- High
n, want the global (or a few good) minima:basinhoppingfirst;multistartwith many parallel starts as a baseline;tunnelingfor a descending trail. - High
nand you still want flooding-style biasing: restrict the bumps to a handful of collective variables, as in metadynamics, rather than the full coordinate vector. - Always: each individual solve inherits pounce’s scalability; the bottleneck is the number of solves and, for repulsion, the loss of sparsity — not the local solver.
Finding Multiple Minima from the CLI
The pounce command line solves one problem from one starting point. The
--minima family turns that single solve into a global search: it drives
the same interior-point solver in a loop, escaping each minimum it finds, and
collects the distinct local minima into a deduplicated archive. It is the
pure-Rust counterpart of the Python find_minima API and
needs no Python — it works on built-in problems and on AMPL .nl files alike.
$ pounce model.nl --minima flooding --n-minima 10
Methods
--minima <method> selects one of six strategies. They share the same local
solver and acceptance test and differ only in how they leave a minimum once
found:
| method | how it escapes a found minimum | reference |
|---|---|---|
multistart | independent random / Sobol’ starts across the box | — |
mlsl | Multi-Level Single-Linkage clustering of sampled starts | Rinnooy Kan & Timmer (1987) |
basinhopping | Metropolis random walk over minima | Wales & Doye (1997) |
flooding | repulsive Gaussian bumps added at found minima (filled-function) | Ge (1990) |
deflation | softened 1/‖x−x*‖^p poles added at found minima | Farrell, Birkisson & Funke (2015) |
tunneling | equal-height tunnel term between descents | Levy & Montalvo (1985) |
--multistart is shorthand for --minima multistart. For help choosing,
see Choosing a Method — the guidance there applies
unchanged to the CLI.
Shared options
| flag | default | meaning |
|---|---|---|
--n-minima <N> | 10 | target number of distinct minima (a stop condition) |
--max-solves <N> | 8 × n-minima | hard cap on solver calls |
--patience <N> | 8 | stop after N solves in a row that find nothing new |
--dedup <d> | 1e-4 | minima within this per-dimension-scaled distance are the same |
--psd-tol <t> | 1e-6 | smallest Hessian eigenvalue tolerated by the saddle-rejection check |
--seed <S> | 0 | seed for sampling / Sobol’ scramble (runs are reproducible) |
--sobol / --no-sobol | on | use a scrambled Sobol’ sequence for box sampling |
A candidate is accepted when its solve converged, the point is finite and
inside the bounds, its objective Hessian is positive semidefinite within
--psd-tol (saddle rejection; skipped when no Hessian is available or the
problem is large), and it is not already within --dedup of an archived
minimum. The dedup distance is measured in a per-dimension-scaled space
(‖(a−b)/L‖, with L the box width per variable), so a single tolerance is
scale-free.
The search stops at the first of: target_reached (--n-minima found),
converged (--patience consecutive empty solves), or
budget_exhausted (--max-solves reached).
Strategy knobs
Each is optional and used only by the relevant method; omit them to take the
defaults (which mirror find_minima). "auto" widths are sized per dimension
from the bounds.
| flags | method |
|---|---|
--sigma, --sigma-frac, --amplitude, --amp-margin | flooding |
--eta, --power, --soft, --length, --length-frac | deflation, tunneling |
--gamma, --samples-per-round | mlsl |
--step, --temperature | basinhopping |
--restart-jitter | all (perturbation scale for restart fallbacks) |
The repulsion methods (
flooding,deflation,tunneling) run each escape solve underhessian_approximation = limited-memory— the analytic penalty term is added to the objective and its gradient, and the quasi-Newton update supplies curvature, so the dense augmented Hessian is never assembled. Each accepted point is then polished by re-solving the clean objective with the exact Hessian, so the reported minima sit on the true problem.
Output
The console prints a ranked table of the distinct minima (rank, objective, and scaled distance to the best), followed by the stop status and the number of solves:
find-minima: 6 distinct minima in 17 solves (target_reached)
rank objective dist-to-best
0 -1.03162845e0 0.000000e0
1 -1.03162845e0 4.772232e-1
...
Solution files. With .sol output enabled, the global best minimum is
written to the usual <stub>.sol (preserving the AMPL contract), and the
remaining minima, ranked by objective, to siblings <stub>.min001.sol,
<stub>.min002.sol, … (so min001 is the second-best point).
JSON report. --json-output writes the standard single-solve report for
the best minimum, plus a backward-compatible minima section:
"minima": {
"method": "multistart",
"status": "target_reached",
"n_solves": 17,
"n_minima": 6,
"minima": [{ "x": [...], "objective": -1.0316 }, ...],
"values": [-1.0316, -1.0316, -0.2155, ...]
}
Omitting --minima leaves the default single-solve output completely
unchanged.
Example
The six-hump camel function has six local minima (two global at
f ≈ −1.0316). Searching for all of them from an .nl model:
$ pounce sixhump.nl --minima multistart --n-minima 6 \
--max-solves 120 --patience 40 --dedup 1e-3 --seed 0
References
- Ge, R. “A filled function method for finding a global minimizer of a function of several variables.” Mathematical Programming 46, 191–204 (1990).
- Rinnooy Kan, A.H.G. & Timmer, G.T. “Stochastic global optimization methods part II: Multi level methods.” Mathematical Programming 39, 57–78 (1987).
- Levy, A.V. & Montalvo, A. “The tunneling algorithm for the global minimization of functions.” SIAM J. Sci. Stat. Comput. 6(1), 15–29 (1985). doi:10.1137/0906002.
- Wales, D.J. & Doye, J.P.K. “Global optimization by basin-hopping and the lowest energy structures of Lennard-Jones clusters containing up to 110 atoms.” J. Phys. Chem. A 101(28), 5111–5116 (1997).
- Farrell, P.E., Birkisson, Á. & Funke, S.W. “Deflation techniques for finding distinct solutions of nonlinear partial differential equations.” SIAM J. Sci. Comput. 37(4), A2026–A2045 (2015). doi:10.1137/140984798.
Algorithm & Workspace
Algorithm
POUNCE implements the interior-point filter line-search algorithm of Wächter & Biegler (2006) — the same algorithm upstream Ipopt uses. A solve proceeds as a sequence of barrier subproblems: for a decreasing sequence of barrier parameters μ, it takes primal-dual Newton steps on the perturbed KKT system, accepting each step through a filter line-search that balances objective descent against constraint infeasibility. When a regular step cannot be found, a restoration phase minimizes constraint violation to return the iterate to a filter-acceptable region.
See Acknowledgments for the papers behind each component.
Workspace layout
POUNCE is a Cargo workspace. Each crate maps onto a part of the upstream Ipopt source tree:
| Crate | Purpose |
|---|---|
pounce-common | Types, exceptions, journalist, options, tagged objects, cached results (Ipopt src/Common). |
pounce-linalg | BLAS-1, dense/compound vectors and matrices, triplet storage, CSC conversion (Ipopt src/LinAlg). |
pounce-linsol | Symmetric linear-solver trait layer — no FFI; backends plug in below. |
pounce-feral | Pure-Rust sparse symmetric LDLᵀ backend. The default. |
pounce-hsl | MA57 backend via libcoinhsl (optional, behind the ma57 feature). |
pounce-nlp | TNLP trait, TNLPAdapter, IpoptApplication entry point (Ipopt src/Interfaces). |
pounce-algorithm | IteratesVector, IpoptData, calculated quantities, KKT, line search, μ update, convergence check, main loop (Ipopt src/Algorithm). |
pounce-restoration | Restoration phase (Ipopt Algorithm/Resto*). |
pounce-presolve | Presolve / problem-reduction pass run before the IPM. |
pounce-l1penalty | ℓ₁-exact penalty-barrier wrapper for degenerate / MPCC NLPs. |
pounce-sensitivity | Parametric sensitivity (port of Ipopt contrib/sIPOPT). |
pounce-cinterface | C ABI shim — CreateIpoptProblem / IpoptSolve / FreeIpoptProblem. |
pounce-py | Python bindings (the pounce Python package). |
pounce-cli | The pounce command-line driver. |
The C ABI shim lets existing PyIpopt / cyipopt / JuMP / AMPL clients link against POUNCE in place of Ipopt.
Initialization and Warm Starts
POUNCE is a local NLP solver: every solve starts from a point, and that point often decides whether the solve takes 15 iterations or 150, or whether it converges at all. This page collects the initialization story in one place: where the starting point comes from on each frontend, what the solver does with it (the part that surprises people), how to warm-start each algorithm path, and how to diagnose a bad start. The per-algorithm details live in their own pages; this is the map.
Where the starting point comes from
| Frontend | Primal starting point |
|---|---|
Python Problem.solve(x0=...) | the x0 argument |
Python minimize(fun, x0, ...) | the x0 argument |
| CLI / AMPL | the .nl file’s initial-guess segment; zeros for variables without one |
| Pyomo | each Var’s .value, serialized into the .nl by Pyomo’s writer |
| GAMS | variable levels (x.L) via GMO |
| Rust | Nlp::new(problem).x0(&[...]), or TNLP::get_starting_point |
Two silent-zero traps hide in that table:
- Pyomo: a
Varwhose.valuewas never set is written as0in the.nlfile. A model initialized “nowhere” is actually initialized at the origin, which for many process models is outside every variable’s meaningful range (and a domain error forlog,/, and friends). - GAMS: levels default to
0unless assigned. Setx.Lbefore thesolvestatement.
Dual estimates can be seeded too: Problem.solve accepts
lagrange=, zl=, zu= keyword arguments, and the .nl format
carries constraint-dual guesses when the modeling layer writes them.
Dual seeds are ignored unless you opt into a warm start (below). The
scipy-style minimize facade does not expose dual seeding; use
pounce.Problem directly when you need it.
What the solver does with your point (cold start)
The default interior-point path ports Ipopt’s iterate initializer
(crates/pounce-algorithm/src/init/default.rs). The sequence:
- The primal point is pushed into the interior of the bounds.
Per component, with bounds
lo <= x <= hi:p_l = min(bound_push * max(|lo|, 1), bound_frac * (hi - lo)), likewisep_u, andxis clamped into[lo + p_l, hi - p_u]. One-sided bounds use thebound_pushterm alone; free variables are untouched. With the defaults (bound_push = bound_frac = 1e-2), a variable sitting exactly on its lower bound1.0starts at1.01instead. Your point is honored approximately, and the deliberately-at-a-bound part of it is not honored at all. This is the single most common reason a “perfect” starting point does not behave like one. - Slacks are set to
s = d(x)and pushed into the slack bounds the same way. - Duals get fixed defaults: constraint multipliers
y = 0(or a least-square estimate, seebound_mult_init_methodbelow) and bound multipliersz = v = bound_mult_init_val = 1.0. - The barrier parameter starts at
mu_init = 0.1(monotonemu_strategy, the default) regardless of how good your point is.
The knobs, all Ipopt-compatible:
| Option | Default | Meaning |
|---|---|---|
bound_push | 1e-2 | Absolute push off each bound (relative to `max( |
bound_frac | 1e-2 | Cap on the push as a fraction of the bound interval. |
slack_bound_push / slack_bound_frac | 1e-2 | Same, for inequality slacks. |
bound_mult_init_val | 1.0 | Initial bound-multiplier value. |
bound_mult_init_method | constant | constant / mu-based / least-square. |
constr_mult_init_max | 1e3 | Cap on the least-square constraint-multiplier estimate; 0 keeps y = 0. |
least_square_init_primal | no | Replace the starting x with the min-norm solution of the linearized constraints before the interior push. |
mu_init | 0.1 | Initial barrier parameter (monotone strategy). |
start_with_resto | no | Jump straight into feasibility restoration at iteration 1 (aborts if the start is already feasible). |
An infeasible starting point is fine: the IPM does not require
feasibility, and least_square_init_primal=yes can cheaply reduce
iteration-0 infeasibility on mostly-linear models (the
mehrotra_algorithm LP/QP cascade turns it on for you, along with
more aggressive bound_push / bound_frac / bound_mult_init_val).
A point where a function fails to evaluate is not fine; see
Diagnosing a bad start.
Warm-starting the interior-point path
From Python, the packaged form is one object:
x, info = prob.solve(x0=x0) # cold solve
ws = pounce.WarmStart.from_info(x, info) # captures x, duals, mu
x2, info2 = prob.solve(warm_start=ws) # warm re-solve
ws.save("state.npz") # reuse across processes
warm_start= is accepted by Problem.solve and pounce.minimize,
seeds the primal and dual iterates, applies the enabling options
below, and forwards the SQP working set when the state was captured
from that path. The rest of this section is what it does under the
hood (and the only route from the CLI or an options file).
Passing a previous solution as x0 is not a warm start by
itself. The IPM warm start is a package of three things, and skipping
any one of them silently degrades to (roughly) a cold solve:
- Opt in and seed the duals. Set
warm_start_init_point=yesand pass the previous multipliers. - Lower
mu_init. The default0.1makes the solver walk the barrier schedule down from scratch even when started at the optimum. Seed it near the converged complementarity (e.g.1e-7after atol=1e-8solve). - Tighten the warm-start pushes. The warm initializer applies
its own interior clamp with
warm_start_bound_push/_frac(default1e-3), which shoves an at-the-bound solution back off its bounds. Tighten them to keep the point.
x, info = make_problem().solve(x0=x0_cold) # cold solve
warm = make_problem()
warm.add_option("warm_start_init_point", "yes")
warm.add_option("mu_init", 1e-7)
for k in ("warm_start_bound_push", "warm_start_bound_frac",
"warm_start_slack_bound_push", "warm_start_slack_bound_frac",
"warm_start_mult_bound_push"):
warm.add_option(k, 1e-9)
x2, info2 = warm.solve(
x0=x,
lagrange=np.asarray(info["mult_g"]),
zl=np.asarray(info["mult_x_L"]),
zu=np.asarray(info["mult_x_U"]),
)
On HS071 this takes the re-solve from 11 iterations to 5, while
warm_start_init_point=yes alone saves nothing; the full runnable
comparison is python/examples/hs071_warm_start.py. On the CLI the
same options apply as KEY=VALUE pairs, with dual seeds coming from
the .nl file’s dual segment when present.
| Option | Default | Meaning |
|---|---|---|
warm_start_init_point | no | Master switch: honor supplied primal and dual seeds. |
warm_start_bound_push / warm_start_bound_frac | 1e-3 | Interior clamp used instead of bound_push / bound_frac. |
warm_start_slack_bound_push / warm_start_slack_bound_frac | 1e-3 | Same, for slacks. |
warm_start_mult_bound_push | 1e-3 | Floor on seeded bound multipliers (a carried-in z = 0 must not start on the barrier’s boundary). |
warm_start_mult_init_max | 1e6 | Cap on seeded equality multipliers. |
Even a well-executed IPM warm start has a structural limit: the barrier pushes iterates off the bounds, so the active-set information in a converged solution cannot be fully exploited. When you are solving a sequence of related NLPs (MPC steps, branch-and-bound nodes, homotopy paths), that limit is the reason the active-set SQP path exists.
Warm-starting the active-set SQP path
With algorithm=active-set-sqp, the warm-start payload is different:
alongside the primal/dual seeds it carries the working set (which
bounds and constraints are active), and an unchanged working set means
the next solve converges in a handful of QP iterations.
prob.add_option("algorithm", "active-set-sqp")
ws = None
for k in range(horizon_steps):
x, info = prob.solve(x0=x_prev, working_set=ws)
ws = info["working_set"]
x_prev = x
The two paths’ warm-start inputs are deliberately path-local: the
IPM-side options above (warm_start_init_point, mu_init,
bound_push, …) are silently ignored on the SQP path, and
working_set= is ignored on the IPM path. Details, the
classify_working_set helper for reconstructing a working set from
multipliers, and the GAMS sqp_state_file / marginal-based routes are
in Active-Set SQP & Warm Starts. Note the GAMS
warm-start features currently live in the native C link only, not the
pip link (see GAMS).
Sequences of solves: batch chaining and sessions
For MPC chains, parametric sweeps, and B&B node relaxations from
Python, solve_nlp_batch packages the whole IPM warm-start recipe
for you:
results = pounce.solve_nlp_batch(batch_t) # cold
results = pounce.solve_nlp_batch(batch_t1, warms=results) # warm
Each instance is seeded with the previous primal and duals, the
converged mu is threaded into mu_init, and
warm_start_init_point=yes is forced; see
Batched NLP solving.
For post-solve sensitivity queries against the converged KKT factor
(a different kind of reuse, no re-solve at all), see
Sessions. JAX users get warm-start hand-off along a
parameter trajectory via JaxProblem; see
the Python guide.
Diagnosing a bad start
The first stop is the preflight check, which evaluates the model once at its starting point (no solve) and reports everything this page has warned about: NaN/inf evaluations, bound violations, how far the interior clamp will move the point, initial constraint violation, and derivative scale spread.
pounce check-x0 model.nl # text report; --json for tools
pounce check-x0 model.nl --x0-file candidate.txt
report = pounce.preflight(problem_obj, x0, lb=lb, ub=ub, cl=cl, cu=cu)
print(report) # report.fatal, report.warnings, report.to_dict()
Exit code 0 means the model evaluates cleanly at x0 (warnings allowed); 21 means a solve from this point would abort. The other diagnostics:
Invalid_Number_Detectedmeans an evaluator returned NaN/inf, and the very first evaluation at the starting point is the usual culprit (log(0)or a division at an all-zeros default start). The interior clamp only repairs bound violations; it cannot fix domain errors on free variables. Move the start into the domain, or add bounds that keep the clamp inside it.derivative_test=first-orderruns the derivative checker at the starting point; wrong derivatives look exactly like a bad start (immediate restoration, tiny steps).- The interactive debugger (
--debug) breaks at iteration 0, so you can inspect the initial objective,inf_pr, andinf_dubefore a single step is taken, andresolvefrom an edited iterate. - Presolve (
presolve=yes) reports structural trouble that no starting point can fix, like rank-deficient equality blocks (LICQ check), and its bound tightening shrinks the box the interior clamp places you in. See Troubleshooting Recipes and FBBT. pounce-studio analyze-nlgives a structural pre-flight of a model file without solving.
No good starting point at all?
Three composable primitives cover the “generate or repair a point” workflows from Python:
# N diverse starts (the sampler behind find_minima): sobol / uniform /
# jitter / bounds midpoint. Feed them to solve_nlp_batch or race them.
starts = pounce.generate_starts(16, bounds=bounds, seed=0)
# Min-norm repair of a candidate onto the linearized constraints +
# bounds (the standalone form of least_square_init_primal).
x0 = pounce.project_to_feasible(problem_obj, x0, lb=lb, ub=ub, cl=cl, cu=cu)
# Cheap tournament: a few iterations from each start, ranked; continue
# the winner at full effort with a WarmStart.
best = pounce.race_starts(fun, starts, bounds=bounds, iters=10)[0]
res = pounce.minimize(fun, best.x,
warm_start=pounce.WarmStart.from_info(best.x, best.info))
When the model has many local minima and you want all of them (or a
managed search rather than a tournament), the
global search drivers (multistart, mlsl,
deflation, flooding, tunneling, basinhopping) manage
populations of starting points and warm-start bookkeeping for you,
from Python (pounce.find_minima) or the CLI (--minima).
Tutorial: active-set SQP and working-set warm starting
This is the user-facing walkthrough for pounce’s Phase 5b/5c
active-set SQP driver. It assumes you can already drive pounce’s
default IPM via the standard interface (Problem.solve in Python,
IpoptSolve in C, option nlp = pounce in GAMS).
The design rationale and algorithmic choices live in the design note — read that if you want to know why the solver works the way it does. This tutorial covers how to use the solver: switching to the SQP path, carrying a working set across solves, and stitching the parametric predictor + SQP corrector pattern together.
1. When to use the active-set SQP
Use it when the same NLP shape is solved many times under small perturbations — MPC closed-loop, parametric continuation, homotopy sweeps, sensitivity-driven design exploration. The IPM re-solves each instance from scratch (the central-path push at the beginning of a fresh solve typically costs 4–8 iterations even when the previous optimum is essentially correct); the SQP warm-started from the previous working set typically picks up where it left off in 0–3 outer iterations when the active set is stable, or grows by a few QP add/drop steps when one or two constraints flip.
Stick with the IPM (the default) for cold solves of a single problem. The IPM scales linearly in the active set; the active-set SQP’s per-QP cost grows with the number of active constraints, so a cold SQP solve does lose ground as the problem grows.
A warm one does not, which is the case worth being precise about. On the warm-start benchmark’s MPC sweep the warm-started SQP’s advantage over its own cold twin improves with problem size all the way to 1645 active constraints (0.17× → 0.02× wall time from N = 10 to N = 80, and 0.02–0.03× at n = 602–2402), because warm cost is set by how far the active set moved rather than by how large it is. An earlier version of this page warned against the SQP for “large-scale problems with thousands of active inequalities”; the measured behavior does not support that for warm-started sequences.
2. Switching to the SQP path
The switch is a single option flip — algorithm from its default
interior-point to active-set-sqp. Everything else (callbacks,
bounds, starting point, finalize_solution) is unchanged.
Python
import pounce
import numpy as np
prob = pounce.Problem(
n=2, m=1, problem_obj=MyNlp(),
lb=[0.0, 0.0], ub=[10.0, 10.0],
cl=[1.0], cu=[1.0],
)
prob.add_option("algorithm", "active-set-sqp")
prob.add_option("print_level", 0)
x, info = prob.solve(x0=np.array([0.5, 0.5]))
C
#include "pounce.h"
IpoptProblem prob = CreateIpoptProblem(/* ... */);
AddIpoptStrOption(prob, "algorithm", "active-set-sqp");
double x[2] = {0.5, 0.5};
double obj;
int status = IpoptSolve(prob, x, NULL, &obj, NULL, NULL, NULL, NULL);
Rust
The flip is one option, so it needs no cargo feature — the default
pounce-rs build reaches the SQP path. On the builder API:
#![allow(unused)]
fn main() {
use pounce_rs::prelude::*;
let sol = Nlp::new(MyNlp)
.var_bounds(&[0.0, 0.0], &[10.0, 10.0])
.constraint_bounds(&[1.0], &[1.0])
.x0(&[0.5, 0.5])
.option_str("algorithm", "active-set-sqp")
.solve();
}
or, driving IpoptApplication directly:
#![allow(unused)]
fn main() {
let mut app = IpoptApplication::new();
app.initialize()?;
app.initialize_with_options_str("algorithm active-set-sqp\n")?;
let status = app.optimize_tnlp(tnlp);
}
Carrying a working set across solves — §3 below — additionally needs
features = ["qp"], because the WorkingSet type belongs to the QP engine.
GAMS
* pounce.opt
algorithm active-set-sqp
Model mymodel / all /;
option nlp = pounce;
mymodel.optfile = 1;
Solve mymodel using nlp minimizing obj;
SQP-specific options
All SQP knobs live under the sqp_* namespace. The defaults
mirror SqpOptions::default().
| Option | Default | Meaning |
|---|---|---|
sqp_globalization | filter | filter or l1-elastic (Fletcher-Leyffer / Han-Powell) |
sqp_hessian | exact | exact, damped-bfgs, or lbfgs |
sqp_max_iter | 200 | outer iteration cap |
sqp_tol | 1e-8 | stationarity tolerance (max-norm) |
sqp_constr_viol_tol | 1e-6 | constraint-violation tolerance |
sqp_dual_inf_tol | 1e-4 | dual-infeasibility tolerance |
sqp_l1_penalty | 1.0 | initial ν (Han-Powell only) |
sqp_l1_penalty_safety | 0.1 | additive ν margin |
sqp_l1_penalty_max | 1e10 | ν upper clamp |
sqp_bt_reduction | 0.5 | backtracking factor |
sqp_bt_min_alpha | 1e-12 | minimum step before line-search failure |
sqp_print_level | 0 | 0=silent, 1=per-iter summary, 2+=trace |
sqp_lbfgs_max_history | 6 | L-BFGS history size |
Algorithm-path isolation guarantees
The two solver paths share the TNLP layer, the OrigIpoptNlp
adapter, the linear-solver backend, the options registry, and
finalize_solution. Beyond that they are deliberately
isolated, so toggling algorithm is always safe — no Phase 5
addition can change IPM behaviour, and no IPM warm-start setting
can change SQP behaviour. Concretely:
- The default (
algorithm = interior-point) is unchanged. No user who hasn’t typedactive-set-sqpever runs Phase 5 code. sqp_*options are silently ignored on the IPM path. Settingsqp_globalization,sqp_hessian,sqp_max_iter, … whilealgorithmisinterior-pointis a no-op. The option-list parser still validates them (out-of-range numeric values fail validation regardless ofalgorithm), but the IPM driver never reads the resolved values.- IPM warm-start options are silently ignored on the SQP path.
bound_push,bound_frac,slack_bound_push,mult_init_max,mu_init,mu_targetand the rest of the IPM-side initializer knobs sit on theAlgorithmBuilderbut are not consulted when the SQP outer loop runs. The one exception iswarm_start_init_point, which the C ABI also reads — see the next bullet. - Warm-start payloads are path-local.
IpoptApplication::set_sqp_warm_start(SqpIterates)/Problem.solve(working_set=…)/IpoptSetWarmStartWorkingSetfeed the SQP loop only — the IPM never readssqp_warm_start. The primal start is not path-local, though: every frontend warm-starts the SQP from the samex0it would have cold-started from (Problem.solve(x0=…), thexbuffer handed toIpoptSolve), so supplying a working set never displaces the caller’s iterate. Initial multipliers reach the SQP whenever the caller supplied both a working set and duals:zl=/zu=/lagrange=onProblem.solve, and — underwarm_start_init_point=yes, which is what makes them inputs at all in upstream Ipopt —mult_x_L/mult_x_U/mult_gonIpoptSolve. Without a working set they feed the IPM only. The SQP packs them signed aslambda_x = z_l − z_u. - You can flip between paths across solves on the same
Problemhandle. The application’s per-solve setup (restoration factory, options snapshot, statistics reset) is rebuilt for everysolve(), so a cold IPM solve followed by an SQP solve withalgorithmre-set in between is a supported pattern. This is exactly how the parametric corrector in §4 hands off from a cold IPM warm-up to the SQP corrector. - The C ABI is strictly additive. Existing cyipopt / JuMP /
AMPL clients link against the new
libpounce_cinterfaceunchanged; the four new entry points (IpoptGetWorkingSet,IpoptSetWarmStartWorkingSet,IpoptClearWarmStartWorkingSet,IpoptSolveWarmStart) are pure additions. info["working_set"]is always present, sometimesNone. Python callers that don’t touch the SQP path never have to read that key, but reading it is safe — it returnsNoneon the IPM path so a downstream loop won’t crash on a missing key.
This isolation is verified by the existing test suite: 868
workspace tests cover both paths, plus crosscutting tests like
application_sqp_warm_start_auto_clears_after_use (asserts the
SQP-side warm-start state doesn’t leak between solves) and
application_default_does_not_select_sqp (asserts the default
solver path is IPM).
Measuring whether the warm start paid
Do not judge a warm start by info["iter_count"]. That is the
outer SQP iteration count, and on a problem whose subproblem
is already a QP the outer loop terminates in one iteration
whether or not you warm started — the number reads the same cold
and warm while the work underneath differs by an order of
magnitude. The saved work is inside the QP subproblems, and two
further keys report it:
| key | meaning |
|---|---|
info["n_qp_solves"] | QP subproblems solved during this solve |
info["n_qp_ws_changes"] | active-set changes (adds + drops) across those QPs |
Both are 0 on the IPM path, which solves no QP subproblems.
n_qp_ws_changes is the measurement to watch across a sequence:
prob.add_option("algorithm", "active-set-sqp")
ws = None
for k in range(horizon_steps):
x, info = prob.solve(x0=x_prev, working_set=ws)
print(k, info["iter_count"], info["n_qp_ws_changes"])
ws, x_prev = info["working_set"], x
A warm start that is working drives the second column toward zero while the first stays flat. The warm-start benchmark is built on exactly this measurement, and reports what the effect is worth across eight problem families and all three solve paths.
3. The working-set warm-start contract
The §6 contract is the tuple (x, λ_g, λ_x, 𝒲) — primal, constraint
multipliers, packed bound multipliers, and the discrete working
set 𝒲 (which bounds and constraints are active at the optimum).
The first three are floating-point; only the last is the
parametric-warm-start payoff over the IPM, because IPM-side
multipliers are continuous interior-point estimates whereas the
active set is what tells the next QP which rows to keep in the
KKT block from iteration zero.
Python: carry across solves
prob.add_option("algorithm", "active-set-sqp")
ws = None
for k in range(horizon_steps):
# ... user code updates the parameter inside MyNlp ...
x, info = prob.solve(x0=x_prev, working_set=ws)
ws = info["working_set"] # (bounds_int8_array, constraints_int8_array)
x_prev = x
The status codes in the working_set tuple use these values
(int8 arrays):
0 = Inactive
1 = AtLower (active at lower bound)
2 = AtUpper (active at upper bound)
3 = Fixed (variable) or Equality (constraint)
C: carry across solves
IpoptBoundStatus *bounds = malloc(n * sizeof *bounds);
IpoptConsStatus *cons = malloc(m * sizeof *cons);
for (int k = 0; k < horizon_steps; k++) {
/* ... user code updates the parameter ... */
if (k == 0) {
IpoptSolve(prob, x, NULL, &obj, NULL, NULL, NULL, NULL);
} else {
IpoptSolveWarmStart(prob, x, NULL, &obj, NULL, NULL, NULL,
bounds, cons, /* in */
bounds, cons, /* out, may alias */
NULL);
}
/* read the WS out for next iteration */
IpoptGetWorkingSet(prob, bounds, cons);
}
Rust: carry across solves
[dependencies]
pounce-rs = { version = "0.9", features = ["qp"] }
The round trip is last_sqp_working_set out, SqpIterates in:
#![allow(unused)]
fn main() {
use pounce_rs::prelude::*;
use pounce_rs::sqp::SqpIterates;
let mut app = IpoptApplication::new();
app.initialize()?;
app.initialize_with_options_str("algorithm active-set-sqp\n")?;
let mut working = None;
let mut x = x0.clone();
for _ in 0..horizon_steps {
// ... user code updates the parameter inside the TNLP ...
if let Some(w) = working.take() {
app.set_sqp_warm_start(SqpIterates {
x: x.clone(),
lambda_g: lambda_g.clone(),
lambda_x: lambda_x.clone(), // packed: z_l − z_u
working: Some(w),
});
}
app.optimize_tnlp(Rc::clone(&tnlp));
working = app.last_sqp_working_set().cloned();
x = /* the x captured in finalize_solution */;
}
}
The warm start is consumed by the solve that follows it, so each pass
installs a fresh one; clear_sqp_warm_start() drops an unused one. Reading
individual rows uses the same status enums the Python int8 codes above
encode — BoundStatus::{Inactive, AtLower, AtUpper, Fixed} and
ConsStatus::{Inactive, AtLower, AtUpper, Equality}, both re-exported from
pounce_rs::sqp.
When the seed comes from a sensitivity predictor instead of a previous
solve (§4), there is no working set to carry —
pounce_rs::sqp::classify_working_set derives one from the predicted point
and its multipliers.
GAMS: working set persists automatically
The GAMS solver link reads variable and equation marginals (x.m,
con.m) at the top of each pouCallSolver invocation and
reconstructs the working set from them. No solve-statement
gymnastics required — every subsequent solve automatically
warm-starts from the previous solution’s marginals.
Use the §7.4(b) state file option for the precision-critical case where the marginal signs are ambiguous (degenerate active set):
* pounce.opt
algorithm active-set-sqp
sqp_state_file .mymodel.pou-ws
The link writes a small binary blob after each solve and reads
it at the start of the next, keyed by a checksum over
(n, m, x_l, x_u, g_l, g_u) so structural changes invalidate
the file cleanly.
4. Worked example: parametric continuation
The headline use case. You have an NLP
min f(x; p) s.t. g(x; p) = 0, x ≥ 0
and you want to trace x*(p) as p sweeps a path. The pounce
playbook is:
- Solve at
p₀with the IPM (better cold-start convergence than the SQP elastic phase). - Predictor: ask
pounce_rs::sensitivity::SensSolveforΔx ≈ ∂x*/∂p · Δpatp₀. - Classify the active set at the converged IPM iterate via
pounce.classify_working_set(...). - Update
pin your TNLP. Applyx* + Δxas the predictor. - Corrector: switch
algorithmtoactive-set-sqp, install the working set + predictor as warm start, solve. - The corrector lands on
x*(p₀ + Δp)in 0–3 outer iterations for small Δp.
Python (full code)
import numpy as np
import pounce
class ParamNlp:
"""min ½‖x − p‖² s.t. sum(x) = 1, x ≥ 0 with parameter p."""
def __init__(self):
self.p = np.zeros(3)
def set_p(self, p):
self.p = np.asarray(p, dtype=float)
def objective(self, x):
d = x - self.p
return 0.5 * float(d @ d)
def gradient(self, x):
return x - self.p
def constraints(self, x):
return np.array([float(x.sum())])
def jacobianstructure(self):
return (np.zeros(3, dtype=np.int64), np.arange(3, dtype=np.int64))
def jacobian(self, x):
return np.ones(3)
def hessianstructure(self):
idx = np.arange(3, dtype=np.int64)
return (idx, idx)
def hessian(self, x, lagrange, obj_factor):
return np.full(3, obj_factor)
nlp = ParamNlp()
def build_problem(algorithm):
p = pounce.Problem(
n=3, m=1, problem_obj=nlp,
lb=[0.0] * 3, ub=[1e20] * 3,
cl=[1.0], cu=[1.0],
)
p.add_option("algorithm", algorithm)
p.add_option("print_level", 0)
return p
# --- Step 1: cold IPM solve at p₀ ---
nlp.set_p([0.5, 0.4, -0.1])
x_ipm, info_ipm = build_problem("interior-point").solve(x0=np.full(3, 1.0 / 3))
print(f"IPM converged: x = {x_ipm}, f = {info_ipm['obj_val']:.4f}")
# --- Step 3: classify the active set at x_ipm ---
ws = pounce.classify_working_set(
x=x_ipm,
x_l=np.array([0.0, 0.0, 0.0]),
x_u=np.array([1e20, 1e20, 1e20]),
g=info_ipm["g"],
g_l=np.array([1.0]),
g_u=np.array([1.0]),
lambda_g=info_ipm["mult_g"],
z_l=info_ipm["mult_x_L"],
z_u=info_ipm["mult_x_U"],
m_eq=1,
)
bounds, cons = ws
print(f" working set: bounds = {bounds.tolist()}, cons = {cons.tolist()}")
# --- Step 4: perturb p and run the SQP corrector ---
nlp.set_p([0.52, 0.39, -0.05]) # Δp = (0.02, -0.01, 0.05)
x_sqp, info_sqp = build_problem("active-set-sqp").solve(
x0=x_ipm, working_set=ws,
)
print(f"SQP corrector: x = {x_sqp}, f = {info_sqp['obj_val']:.4f}")
print(f" info['working_set'] for the next step: {info_sqp['working_set']}")
Expected output (deterministic, ran live before this tutorial was checked in):
IPM converged: x = [5.5e-01 4.5e-01 1.2e-07], f = 0.0075
working set: bounds = [0, 0, 1], cons = [3]
SQP corrector: x = [0.565 0.435 0. ], f = 0.0033
info['working_set'] = (array([0, 0, 1], dtype=int8), array([3], dtype=int8))
The IPM lands x₃ essentially-zero (1.2e-7) — it’s an IPM
artifact of the central-path push; for classify_working_set’s
default primal_tol = 1e-6 that’s already inside the
“at the bound” band, so bounds[2] = 1 (AtLower). The SQP
corrector hits x₃ = 0 exactly because the working set tells
it x₃ is an active lower bound from iteration zero — no
central-path detour.
bounds = [0, 0, 1] means x[0], x[1] inactive (interior),
x[2] at its lower bound. cons = [3] means the sum constraint
is binding (equality).
Running it
Save as parametric_demo.py and run:
python parametric_demo.py
For an executable variant see
python/examples/sqp_warm_start_mpc.py (a 20-step parametric
sweep) and the Jupyter notebook in
python/notebooks/06_sqp_parametric_continuation.ipynb.
5. Choosing a globalization
sqp_globalization = filter (the default) follows Fletcher-Leyffer
2002 — a Pareto-frontier filter on (constraint violation, objective). Robust, no penalty parameter to tune, recommended
for general nonlinear NLPs.
sqp_globalization = l1-elastic is the SNOPT-style Han-Powell
merit φ(x; ν) = f(x) + ν · violation(x) with adaptive ν. The
new sqp_l1_penalty_safety (default 0.1) and sqp_l1_penalty_max
(default 1e10) options control the ν update:
ν ← clamp(max(ν, ‖λ_qp‖_∞ + sqp_l1_penalty_safety), 0, sqp_l1_penalty_max)
Use l1-elastic when you want behaviour close to SNOPT for
comparison studies, or when the filter is rejecting too many
trial steps on a problem where the merit decreases steadily.
6. Choosing a Hessian source
| Source | When to use |
|---|---|
exact | NLP provides eval_h; the QP’s inertia-control handles indefinite ∇²L |
damped-bfgs | Dense n×n Powell-damped BFGS; guaranteed PSD; n ≤ a few hundred |
lbfgs | Limited-memory BFGS with sqp_lbfgs_max_history pairs; large n |
The default exact is fastest when reliable. Switch to
damped-bfgs for ill-scaled nonconvex NLPs where the QP solver’s
inertia retries dominate the iteration cost. Use lbfgs only
when the dense n² BFGS storage is the bottleneck (n ≥ ~1000).
7. Pitfalls
- Calling
Problem.solve(working_set=…)with a stale working set whose dimensions changed. Validated and rejected withValueError. Pass the WS only when it came from a solve of the same problem shape. - Mixing IPM and SQP across solves without resetting state.
The IPM path ignores
set_sqp_warm_start, and the SQP path ignores the IPM warm-start options (warm_start_init_pointetc.). Each path’s warm-start input is path-local. - Degenerate active set after IPM convergence. The
multiplier-sign + primal-distance heuristic in
classify_working_setis lossy at degenerate optima — same trade-off CONOPT/IPOPT/KNITRO have under GAMS. The first QP step in the SQP corrector re-classifies any wrongly-tagged rows, so correctness is preserved; only the iteration count may be slightly higher than ideal. - L1Elastic with a hard cap. If your problem’s QP multipliers
spike (poorly scaled constraints), bump
sqp_l1_penalty_maxor rescale.
8. Where the code lives
| Concern | File |
|---|---|
| SQP outer loop | crates/pounce-algorithm/src/sqp/sqp_alg.rs |
| QP subproblem solver | crates/pounce-qp/src/solver.rs |
| Working set type | crates/pounce-qp/src/working_set.rs |
| Classifier | crates/pounce-algorithm/src/sqp/warm_start.rs |
| IpoptApplication hooks | crates/pounce-algorithm/src/application.rs |
| C ABI | crates/pounce-cinterface/src/lib.rs + include/pounce.h |
| Python binding | crates/pounce-py/src/{problem,warm_start}.rs |
| GAMS link | gams/gams_pounce.c |
| Design rationale | Design Note |
9. Reading list
- Hock, Schittkowski (1981) Test Examples for Nonlinear Programming Codes — the in-repo HS subset reference.
- Nocedal, Wright (2006) Numerical Optimization, ch. 18 — SQP fundamentals.
- Fletcher, Leyffer (2002) — filter line search.
- Han (1977) / Powell (1978) — l1-merit and damped-BFGS update.
- Wächter, Biegler (2006) — pounce’s IPM heritage.
- Gill, Murray, Saunders (2002) — SNOPT / l1-elastic phase 1.
- Forsgren, Gill, Wright (2002) — IPM vs SQP comparison.
- Kirches (2011) — parametric active-set SQP.
Design note — Active-set SQP for warm-started NLP sequences
Status: implemented. This note was originally the research →
plan half of the research → plan → implement workflow that
operationalized the C1 active-set SQP entry of the future-work
roadmap (dev-notes/research/future-work-roadmap.md, §3.2, §5 Phase 5).
The driver (Phase 5b/5c) has since landed and is wired through the
Rust API, C ABI, Python bindings, and the GAMS link; see the user
tutorial at Active-Set SQP & Warm Starts.
The note is retained as design rationale and pins each algorithmic
choice to the literature.
The target is a state-of-the-art sparse active-set SQP solver that (a) reuses pounce’s NLP / derivative / sparse-linalg foundation, (b) warm-starts on the working set across solves (not just primal-dual seeds), and (c) integrates symmetrically across the Rust API, C ABI, Python bindings, and GAMS link.
1. What this is
A sequential quadratic programming algorithm with a sparse parametric active-set QP subproblem — a second solver inside pounce sharing the model / derivative / linalg foundation but with its own iteration skeleton — designed for warm-started sequences of related NLPs:
- Model predictive control (MPC): re-solve a similar NLP every control step. The horizon shifts by one stage; the active set rarely changes.
- MINLP branch-and-bound: thousands of node relaxations differing by a few bound changes. Bounds-only active-set updates dominate.
- Parametric homotopy / continuation: trace the solution along a parameter path. Predictor (sensitivity) + corrector (SQP step from the predicted point) reuses the working set across path steps.
The motivation is the warm-start gap in interior-point methods: the barrier pushes iterates to the interior, so a near-optimal point from a previous solve sits near the bound boundary and cannot be exploited. Active- set methods, by contrast, carry the working set across solves; if the optimal active set is unchanged, the next solve converges in O(1) QP iterations. This is the documented reason qpOASES, SNOPT, and filterSQP dominate in MPC.
2. The architectural mismatch (read this first)
IpoptData / IpoptCalculatedQuantities are shaped around primal-
dual interior-point variables — slacks s, barrier μ, bound
multipliers z_l/z_u, complementarity quantities. Active-set SQP
has none of these: it carries (x, λ, 𝒲) where 𝒲 is the working
set — the indices of currently active inequalities and bounds — and
globalizes on a merit function or filter without a barrier at all.
This is therefore a new AlgorithmStrategy end to end — a Tier 3
addition in the roadmap’s tier ladder — and not an edit to the existing
loop. The existing IPM
(IpoptAlgorithm::optimize in
crates/pounce-algorithm/src/ipopt_alg.rs) is left untouched and
remains the default solver. Active-set SQP is opt-in via a new
top-level algorithm option (§7.1), parallel to the existing
linear_solver (Ma57/Feral) and mu_strategy (Monotone/Adaptive)
choices in alg_builder.rs:54-63.
The dual-skeleton commitment is the cost; the warm-start strength is the payoff.
3. What pounce already has that SQP can reuse
| Need | Existing component | Location |
|---|---|---|
NLP model trait (f, g, ∇f, J, ∇²ℒ) | IpoptNlp / TNLP | crates/pounce-algorithm/src/ipopt_nlp.rs, crates/pounce-nlp/ |
.nl and CUTEst frontends | pounce-cli, benchmarks/cutest | unchanged |
| Sparse storage (triplet + CSC) | SymTMatrix, triplet→CSR converter | crates/pounce-linalg/src/triplet.rs:374-405, triplet_convert.rs:40 |
| Sparse symmetric LDLᵀ with inertia | SparseSymLinearSolverInterface (FERAL, MA57) | crates/pounce-linsol/src/sparse_sym_iface.rs:42-84 |
| Multi-RHS solve sharing one factor | t_sym_solver.rs::multi_solve | crates/pounce-linsol/src/t_sym_solver.rs:174 |
| Inertia reporting (eigenvalue counts) | SparseSymLinearSolverInterface::provides_inertia | crates/pounce-linsol/src/sparse_sym_iface.rs:84 |
| Limited-memory BFGS / SR1 | hess/quasi_newton.rs | reused for SQP Hessian approximation |
| Filter acceptor | line_search/filter_ls_acceptor.rs | dominance test reusable for SQP filter |
| Convergence-check trait | conv_check::trait::ConvCheck | reused; KKT-error formula is identical |
| Option / journalist / iteration-output | pounce-common + output/ | reused; new fields for working-set events |
Warm-start primal/dual seeds from TNLP | init/warm_start.rs:60-100 | extended (§6) with working-set state |
| Parametric sensitivity (sIPOPT port) | pounce-sensitivity | provides predictor for parametric-homotopy use case |
The interfaces below pounce-nlp are stable enough that SQP inherits the full derivative and linalg layer unchanged. Everything new lives at the algorithm / solver level.
4. The algorithm — fully pinned
This section pins each algorithmic choice to literature. There is no remaining “decide during implementation” discretion at the level of algorithm class; only tuning constants are open.
4.1 Outer SQP loop — filter line search with Maratos correction
The outer loop is the filter SQP of Fletcher-Leyffer-Toint, with
the Wächter-Biegler second-order correction (Maratos effect) and
watchdog mechanism already implemented in line_search/. Filter
because:
- It avoids the penalty-parameter tuning of l1-merit (Han-Powell).
- It reuses pounce’s existing
FilterLsAcceptor(line_search/filter_ls_acceptor.rs) without modification — the dominance test on(‖c‖, f)is identical. - It is the globalization in filterSQP (Fletcher-Leyffer) and WORHP, the two open-source SQP solvers that compete with SNOPT on CUTEst, and the documented choice in Nocedal-Wright §18.10.
Alternative offered as opt-in: l1-elastic merit (the SNOPT
choice), via a sqp_globalization option. l1 is simpler to reason
about under MPCC-like degeneracies; filter is faster on smooth
nonconvex NLPs in published benchmarks (Fletcher-Leyffer-Toint 2002
§6; Wächter-Biegler 2006 Tab. 3-5).
References:
- Fletcher, Leyffer, “Nonlinear programming without a penalty function”, Math. Prog. 91 (2002), 239–269.
- Fletcher, Leyffer, Toint, “On the global convergence of a filter- SQP algorithm”, SIAM J. Optim. 13 (2002), 44–59.
- Wächter, Biegler, “Line search filter methods for nonlinear programming: Motivation and global convergence”, SIAM J. Optim. 16 (2005), 1–31.
- Wächter, Biegler, “On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming”, Math. Prog. 106 (2006) — pounce’s existing filter implementation.
4.2 QP subproblem — sparse Schur-complement parametric active-set
The QP subproblem solver is a sparse parametric active-set method with Schur-complement basis updates, the lineage of qpOASES extended to sparse Hessian and Jacobian. This is the SOTA choice for SQP subproblems in industrial MPC and for parametric / homotopy use: it is the only active-set QP family in the literature with proven cross-solve warm-start performance in the sparse regime.
Why this family (vs alternatives):
| Family | Sparse? | Indefinite H? | Parametric WS warm-start? | Reference |
|---|---|---|---|---|
| Goldfarb-Idnani (1983) | no (dense) | no (convex only) | partial | Goldfarb-Idnani 1983 |
| Range-space (SQOPT) | partial | yes | partial | Gill-Murray-Saunders 2008 |
| Null-space (Gould-Hribar-Nocedal) | partial | yes | partial | Gould-Hribar-Nocedal 2001 |
| qpOASES (online active set) | no (dense) | yes | yes (homotopy) | Ferreau et al. 2014 |
| Sparse Schur-complement parametric | yes | yes | yes | Kirches 2011, Janka 2017 |
| OSQP (ADMM) | yes | no (convex only) | seed only | Stellato et al. 2020 |
| PIQP / HPIPM (interior-point) | yes | yes | seed only | Schwan 2023, Frison-Diehl 2020 |
Only the sparse Schur-complement parametric method covers all three columns. It is what is needed.
Algorithm sketch. At any iterate the QP solver maintains a factorization of the base KKT matrix for some “base” working set 𝒲_base:
┌ H Aᵀ_𝒲 ┐
K_𝒲 = │ │, LDLᵀ via pounce-linsol (FERAL/MA57)
└ A_𝒲 0 ┘
When the working set changes (a constraint is added or dropped during
the homotopy), the new system is not refactorized. Instead, the
change is absorbed by a Schur-complement update: the modified
system has the form K_𝒲 + UVᵀ (low-rank correction), and solves
against the modified factor are obtained by the Schur-complement
formula
(K + UVᵀ)⁻¹ b = K⁻¹b − K⁻¹U (I + Vᵀ K⁻¹ U)⁻¹ Vᵀ K⁻¹ b
so each active-set change costs one rank-1 update of the dense Schur
complement S = I + Vᵀ K⁻¹ U plus one back-solve against the cached
sparse factor. This is the Bartels-Golub-Reid principle from
sparse simplex adapted to symmetric QP. When S grows too large
(default: 50 updates) or its condition number degrades, a fresh
sparse refactorization of K_𝒲 resets the cycle.
The homotopy itself follows qpOASES: between two QPs (H₀, g₀, A₀, b₀) and (H₁, g₁, A₁, b₁), the solver traces the parametric path
(H_t, g_t, A_t, b_t) = (1-t)·QP₀ + t·QP₁ for t ∈ [0, 1], jumping
the working set at each t where a multiplier hits zero or a
constraint hits its bound. If the active set is identical at the two
endpoints (the warm-start sweet spot), the homotopy completes with
zero working-set changes.
Why Schur-complement, not direct LDLᵀ update? Direct sparse LDLᵀ
factor updates (the symbolic+numeric reanalysis required when a
constraint row is added or dropped) are known to be unstable under
many updates because fill-in is not bounded (Davis 2006 §11). The
Schur-complement / Bartels-Golub-Reid approach bounds the
asymptotic update cost and is the technique that production sparse
simplex (CPLEX, Gurobi, HiGHS) and SOTA sparse parametric QP
(Kirches’s qpDUNES, the Janka parOSQP lineage) use.
References:
- Ferreau, Kirches, Potschka, Bock, Diehl, “qpOASES: a parametric active-set algorithm for quadratic programming”, Math. Prog. Comp. 6 (2014), 327–363 — the dense reference algorithm.
- Kirches, Fast Numerical Methods for Mixed-Integer Nonlinear Model-Predictive Control, Vieweg+Teubner (2011), Ch. 5–7 — the sparse Schur-complement extension; the canonical reference.
- Janka, Kirches, Sager, Schlöder, “An SR1/BFGS SQP algorithm for nonconvex nonlinear programs with block-diagonal Hessian matrix”, Math. Prog. Comp. 8 (2016), 435–459 — block-sparse extension.
- Kirches, Potschka, Bock, Sager, “A parametric active set method for quadratic programs with vanishing constraints”, Pacific J. Optim. 9 (2013) — MPCC structure handling, relevant to C4 reuse.
- Bartels, “A stabilization of the simplex method”, Numer. Math. 16 (1971); Reid, “A sparsity-exploiting variant of the Bartels-Golub decomposition”, Math. Prog. 24 (1982) — the Schur-complement basis-update lineage.
- Eldersveld, Saunders, “A block-LU update for large-scale linear programming”, SIAM J. Matrix Anal. Appl. 13 (1992).
- Gill, Murray, Saunders, “SNOPT: An SQP algorithm for large-scale constrained optimization”, SIAM Rev. 47 (2005) — the range-space active-set used inside SNOPT; competing family.
- Davis, Direct Methods for Sparse Linear Systems, SIAM (2006) — fill-in and refactor cost analysis.
4.3 Phase-1 / initial feasibility — l1 elastic mode
Active-set QP requires a feasible starting working set. The l1-elastic mode (Gill-Murray-Saunders, SQOPT) reformulates the infeasibility problem inside the same QP: each constraint gets a nonnegative elastic slack with a large linear cost γ, the working set starts empty, and elastic slacks are driven to zero as the homotopy proceeds. If the original QP is feasible the elastic slacks vanish at the solution; if infeasible the residual elastic slacks certify the minimal infeasibility.
This is preferred over the Big-M approach used in dense qpOASES
because it preserves sparsity (the cost vector grows by m entries,
the Jacobian by m columns; no large constants in H).
References:
- Gill, Murray, Saunders, User’s Guide for SQOPT 7.7, Stanford SOL Report (2008) — elastic-mode reference implementation.
- Friedlander, Saunders, “A globally convergent linearly constrained Lagrangian method for nonlinear optimization”, SIAM J. Optim. 15 (2005) — elastic mode as feasibility restoration.
4.4 Anti-cycling — EXPAND
Degeneracy in the working set (multiple constraints active with linearly dependent rows, or zero step lengths) can cause cycling in naive active-set methods. The SOTA anti-cycling rule is EXPAND (Gill-Murray-Saunders-Wright 1989): a small primal perturbation is introduced and grown over iterations so that the step length is always strictly positive, with periodic resets.
Bland’s rule (1977) and Wolfe’s rule (1963) are alternatives, but EXPAND is faster in practice and is the rule used by SNOPT, MINOS, LANCELOT, and qpOASES.
References:
- Gill, Murray, Saunders, Wright, “A practical anti-cycling procedure for linearly constrained optimization”, Math. Prog. 45 (1989), 437–474.
4.5 Indefinite reduced Hessian — inertia control + projected modified Cholesky
For nonconvex NLP subproblems the Hessian of the Lagrangian is indefinite. The QP must still be solved to a meaningful descent direction. Two-layer scheme, both standard:
- Detect via inertia of the LDLᵀ factor of
K_𝒲. pounce-linsol already exposes inertia viaprovides_inertia()/number_of_neg_evals(sparse_sym_iface.rs:84). The correct inertia for an SQP subproblem withmworking constraints is(n − m, m, 0); any deviation flags reduced-Hessian indefiniteness. - Correct via projected modified Cholesky on the reduced
Hessian: when wrong inertia is detected, shift
H ← H + δIwith δ chosen by the same inertia-correction logic pounce already uses inkkt/perturbation_handler.rs:141-356. This restores correct inertia at minimal modification.
References:
- Gould, “On modified factorizations for large-scale linearly constrained optimization”, SIAM J. Optim. 9 (1999), 1041–1063.
- Gould, Hribar, Nocedal, “On the solution of equality constrained quadratic programming problems arising in optimization”, SIAM J. Sci. Comput. 23 (2001), 1376–1395 — the inertia-correction prescription for SQP subproblems.
- Forsgren, “Inertia-controlling factorizations for optimization algorithms”, Appl. Num. Math. 43 (2002), 91–107.
4.6 Hessian approximation — exact, damped BFGS, L-BFGS
The SQP outer loop accepts three Hessian sources via the existing
HessianUpdater trait (hess/r#trait.rs):
- Exact
∇²ℒfrom the NLP (default when available). Indefinite on nonconvex problems; handled by §4.5. - Damped BFGS (Powell 1978): full dense BFGS with Powell’s
damping rule, guaranteed PSD. Default fallback when exact Hessian
is unavailable, for problems where
nis small. - Limited-memory BFGS / SR1 (Liu-Nocedal 1989, Byrd-Nocedal-Schnabel
1994): the existing pounce L-BFGS implementation. Default for
large
n. SR1 is the indefinite-Hessian variant preferred in Janka 2016 for nonconvex SQP block-sparse problems.
The QP subproblem absorbs whichever Hessian is supplied; only the indefinite-handling path (§4.5) differs.
References:
- Powell, “A fast algorithm for nonlinearly constrained optimization calculations”, in Numerical Analysis Dundee 1977 (1978) — damped BFGS for SQP.
- Liu, Nocedal, “On the limited memory BFGS method for large scale optimization”, Math. Prog. 45 (1989), 503–528.
- Byrd, Nocedal, Schnabel, “Representations of quasi-Newton matrices and their use in limited memory methods”, Math. Prog. 63 (1994), 129–156.
4.7 Iterative refinement
Single iteration of fixed-precision iterative refinement on every QP
solve, using the cached factorization. Standard practice; pounce-feral
and MA57 backends already implement it (t_sym_solver.rs::multi_solve
applies refinement when configured).
References:
- Wilkinson, The Algebraic Eigenvalue Problem, OUP (1965) — original.
- Higham, Accuracy and Stability of Numerical Algorithms (2nd ed., SIAM 2002), §12.
5. New crate pounce-qp — concrete types
Standalone crate. Depends on pounce-linalg and pounce-linsol;
depended on by pounce-algorithm (for SQP), pounce-sensitivity
(for the parametric corrector in Phase 5c+), optionally
pounce-presolve (for tighter feasibility checks in future work).
5.1 Types
All types are sparse from the start, using the existing
pounce-linalg storage conventions (SymTMatrix triplet → CSC for
the symmetric Hessian; GenTMatrix for the Jacobian).
#![allow(unused)]
fn main() {
// crates/pounce-qp/src/problem.rs
use pounce_linalg::triplet::{SymTMatrix, GenTMatrix};
/// A convex-or-nonconvex sparse QP:
/// min ½ xᵀ H x + gᵀ x
/// s.t. bl ≤ A x ≤ bu
/// xl ≤ x ≤ xu
/// Two-sided general bounds; H is symmetric (upper triangle stored)
/// and may be indefinite (caller sets `hessian_inertia`).
pub struct QpProblem<'a> {
pub n: usize,
pub m: usize,
pub h: &'a SymTMatrix, // symmetric, upper triangle, may be indefinite
pub g: &'a [f64],
pub a: &'a GenTMatrix, // m × n, sparse
pub bl: &'a [f64], pub bu: &'a [f64],
pub xl: &'a [f64], pub xu: &'a [f64],
pub hessian_inertia: HessianInertia, // PSD | Indefinite | Unknown
}
/// Discrete state per primal-and-constraint index. Carried across
/// solves to implement working-set warm start.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub enum BoundStatus { Inactive, AtLower, AtUpper, Fixed }
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub enum ConsStatus { Inactive, AtLower, AtUpper, Equality }
pub struct WorkingSet {
pub bounds: Vec<BoundStatus>, // length n
pub constraints: Vec<ConsStatus>, // length m
}
pub struct QpWarmStart {
pub x: Vec<f64>,
pub lambda_g: Vec<f64>, // length m
pub lambda_x: Vec<f64>, // length n (z_l − z_u, signed)
pub working: WorkingSet,
}
pub struct QpSolution {
pub x: Vec<f64>,
pub lambda_g: Vec<f64>,
pub lambda_x: Vec<f64>,
pub working: WorkingSet,
pub obj: f64,
pub status: QpStatus, // Optimal | Infeasible | Unbounded | MaxIter | …
pub stats: QpStats, // n_active_set_changes, n_refactor, time …
}
}
5.2 Trait surface
#![allow(unused)]
fn main() {
// crates/pounce-qp/src/solver.rs
use pounce_linsol::sparse_sym_iface::SparseSymLinearSolverInterface;
pub trait QpSolver {
/// Solve a single QP. `ws` is `None` for cold start.
fn solve(
&mut self,
qp: &QpProblem,
ws: Option<&QpWarmStart>,
opts: &QpOptions,
) -> Result<QpSolution, QpError>;
/// Parametric solve: trace the homotopy from a previous QP+solution
/// to a new QP. Falls back to `solve` if the previous solution is
/// `None`. This is the entry point SQP uses across outer iterations
/// to reuse the cached factorization across consecutive QPs.
fn solve_parametric(
&mut self,
qp_prev: &QpProblem,
sol_prev: &QpSolution,
qp_new: &QpProblem,
opts: &QpOptions,
) -> Result<QpSolution, QpError>;
}
pub struct QpOptions {
pub algorithm: QpAlgorithm, // ParametricActiveSet | …
pub linear_solver_factory: …, // injected from pounce-algorithm
pub max_iter: usize,
pub feas_tol: f64,
pub opt_tol: f64,
pub max_schur_updates_before_refactor: usize, // default 50, ref §4.2
pub anti_cycling: AntiCyclingChoice, // Expand (default), Bland, None
pub elastic_gamma: f64, // §4.3 penalty for elastic mode
pub print_level: i32,
}
}
The linear_solver_factory injection mirrors
alg_builder.rs::LinearBackendFactory (line 50) so pounce-qp
remains backend-agnostic: FERAL by default, MA57 when built with the
ma57 feature.
5.3 Internal structure
crates/pounce-qp/
├── Cargo.toml
└── src/
├── lib.rs
├── problem.rs — types from §5.1
├── working_set.rs — WorkingSet ops: add, drop, validate
├── kkt.rs — KKT assembly from QP + 𝒲
├── factor.rs — sparse LDLᵀ wrapper + Schur-complement state
├── schur.rs — block-LU update (Eldersveld-Saunders 1992)
├── homotopy.rs — parametric step engine (§4.2 t ∈ [0,1])
├── elastic.rs — phase-1 elastic mode (§4.3)
├── expand.rs — EXPAND anti-cycling (§4.4)
├── inertia.rs — indefinite handling (§4.5)
├── refine.rs — iterative refinement (§4.7)
├── solver.rs — QpSolver impl
└── options.rs — QpOptions, defaults
6. SQP iterate state and working-set warm-start contract
#![allow(unused)]
fn main() {
// crates/pounce-algorithm/src/sqp/iterates.rs
pub struct SqpIterates {
pub x: Rc<DenseVector>,
pub lambda_g: Rc<DenseVector>,
pub lambda_x: Rc<DenseVector>,
pub working: WorkingSet, // §5.1
pub h_approx: HessianStore, // exact | DampedBfgs | LBfgs (existing)
pub merit: Option<f64>, // l1-elastic mode or filter pair cache
}
}
The warm-start contract carried across calls to
SqpAlgorithm::optimize is the tuple (x, λ_g, λ_x, 𝒲, H):
(x, λ_g, λ_x)— already supported by the existinginit/warm_start.rsmachinery; reuse the seed-from-NLP path (warm_start.rs:60-100).𝒲(working set) — new. Encoded as(Vec<BoundStatus>, Vec<ConsStatus>). Transmitted via:- Rust: a new
SqpWarmStartIterateInitializerparallel to the IPM one, populated by an extendedTNLP::get_warm_start_working_sethook (Rust trait default: returnsNone⇒ cold-start the working set via §4.3 elastic mode). - C/Python/GAMS: §7.
- Rust: a new
H(Hessian) — already supported via the existing L-BFGS carry-forward path; reuse unchanged.
Cold-warm bootstrap (no prior 𝒲): elastic-mode QP §4.3 with
empty initial working set. The first QP infers 𝒲₀ from which
elastic slacks vanish at its solution.
Validation: before consuming a user-supplied 𝒲_prev, run a
linear feasibility check against the new bounds. If a previously
active bound is now infeasible, drop it (degrades to a cheaper warm
start, never to incorrectness). This is the same defensive check
qpOASES does on set_warm_start_x.
7. Integration with pounce — symmetric across interfaces
Each interface today is documented in the survey above. The integration plan below adds the same five-point contract (algorithm choice + suboptions + warm-start input + warm-start output + working-set typed-or-string surface) to each, without disturbing existing IPM users.
7.1 Rust / alg_builder.rs — the source of truth
New enum following the established LinearSolverChoice /
MuStrategyChoice pattern at alg_builder.rs:54-63:
#![allow(unused)]
fn main() {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AlgorithmChoice {
InteriorPoint, // default; existing IpoptAlgorithm
ActiveSetSqp, // new SqpAlgorithm
}
}
AlgorithmBuilder gains an algorithm: AlgorithmChoice field with
default InteriorPoint. build_inner branches on it, returning
either the existing AlgorithmBundle (IPM) or a new
SqpAlgorithmBundle. The two bundles share init, conv_check,
hess, iter_output; differ in main-loop driver.
New options registered in upstream_options.rs (the registry
pattern at lines 510-703 for the existing warm-start knobs):
| Option | Type | Default | Meaning |
|---|---|---|---|
algorithm | enum | interior-point | interior-point ‖ active-set-sqp |
sqp_qp_solver | enum | parametric-active-set | placeholder for future QP backends |
sqp_globalization | enum | filter | filter ‖ l1-elastic |
sqp_hessian | enum | exact | exact ‖ damped-bfgs ‖ lbfgs |
sqp_warm_start_working_set | bool | no | accept caller-supplied 𝒲 |
sqp_max_qp_iter | int | 200 | per-QP iteration cap |
sqp_qp_feas_tol | num | 1e-9 | QP feasibility tolerance |
sqp_elastic_gamma | num | 1e6 | elastic-mode penalty (§4.3) |
sqp_max_schur_updates | int | 50 | refactor frequency (§4.2) |
7.2 C API (crates/pounce-cinterface/)
Three additions, all backward-compatible (existing IPM users see no change).
(a) Option exposure. No new C entry point — AddIpoptStrOption
already accepts arbitrary option names. Setting algorithm via
AddIpoptStrOption(problem, "algorithm", "active-set-sqp") selects
the SQP path. This is identical to how linear_solver is selected
today.
(b) Working-set transfer. Three new C entry points in
include/pounce.h, ABI-stable (no change to existing structs):
/* Length-n status vectors. 0=Inactive, 1=AtLower, 2=AtUpper, 3=Fixed/Equality. */
typedef int IpoptBoundStatus;
typedef int IpoptConsStatus;
/* Retrieve the working set from the last solve. Returns 0 on success.
* Buffers must be sized n and m respectively. NULL buffer ⇒ skip that side. */
int IpoptGetWorkingSet(
IpoptProblem problem,
IpoptBoundStatus *bound_status_out, /* length n, or NULL */
IpoptConsStatus *cons_status_out /* length m, or NULL */
);
/* Supply a warm-start working set for the next solve. Buffers may
* be NULL ⇒ that side is cold-started. Caller-owned; copied. */
int IpoptSetWarmStartWorkingSet(
IpoptProblem problem,
const IpoptBoundStatus *bound_status_in, /* length n, or NULL */
const IpoptConsStatus *cons_status_in /* length m, or NULL */
);
/* One-shot solve with warm-start state. Equivalent to IpoptSolve
* preceded by IpoptSetWarmStartWorkingSet. Returns working set in
* the supplied output buffers if non-NULL. */
int IpoptSolveWarmStart(
IpoptProblem problem,
Number *x, Number *g, Number *obj_val,
Number *mult_g, Number *mult_x_L, Number *mult_x_U,
const IpoptBoundStatus *bound_status_in, /* in, or NULL */
const IpoptConsStatus *cons_status_in, /* in, or NULL */
IpoptBoundStatus *bound_status_out, /* out, or NULL */
IpoptConsStatus *cons_status_out, /* out, or NULL */
UserDataPtr user_data
);
IpoptProblem (lib.rs:67) gains an internal Option<WorkingSet>
slot; the existing IpoptSolve signature is unchanged. The C ABI
adds three symbols; existing cyipopt / JuMP / AMPL clients are
unaffected.
(c) Suboption strings. Already covered by §7.1’s option registry
via the existing AddIpoptStrOption / AddIpoptIntOption /
AddIpoptNumOption setters; no new C signatures required.
7.3 Python (crates/pounce-py/)
PyO3 bindings extend symmetrically. pounce.Problem.add_option
already accepts algorithm and the suboption strings from §7.1; no
binding change.
New methods on PyProblem (crates/pounce-py/src/problem.rs):
class Problem:
# existing ────────────────────────────────────────
def add_option(self, name: str, value): ...
def solve(self, x0,
lagrange=None, zl=None, zu=None,
# NEW kwargs, default None ⇒ cold:
working_set: Optional[WorkingSet] = None
) -> SolveResult: ...
# NEW ─────────────────────────────────────────────
def get_working_set(self) -> WorkingSet: ...
@dataclass
class WorkingSet:
bounds: np.ndarray # dtype=int8, length n
constraints: np.ndarray # dtype=int8, length m
@dataclass
class SolveResult:
x: np.ndarray
obj_val: float
mult_g: np.ndarray
mult_x_L: np.ndarray
mult_x_U: np.ndarray
working_set: Optional[WorkingSet] # populated when algorithm == "active-set-sqp"
info: dict
The MPC / parametric-continuation Python idiom becomes:
prob = pounce.Problem(...)
prob.add_option("algorithm", "active-set-sqp")
prob.add_option("sqp_warm_start_working_set", True)
ws = None
for step in range(horizon):
res = prob.solve(x0=x_prev, working_set=ws)
ws = res.working_set # carry across solves
x_prev = shift(res.x)
This is the same ergonomics as qpOASES’s Python binding, deliberately.
7.4 GAMS (gams/gams_pounce.c)
GAMS is the hardest case because the link is single-shot per solve
statement and there is no in-process persistence between solves. Two
mechanisms cover the use cases:
(a) Algorithm and suboption selection via the existing
pounce.opt option file (gams_pounce.c:220-273). No code change —
the option file already forwards unknown keys to the C API via
AddIpoptStrOption etc. Adding the keys from §7.1 to the documented
GAMS options list is the only deliverable here:
* pounce.opt
algorithm active-set-sqp
sqp_globalization filter
sqp_hessian exact
sqp_warm_start_working_set yes
(b) Working-set transfer across solves. GAMS has no native discrete-multiplier carry. Two mechanisms, both standard in GAMS solver links:
- Marginal-based reconstruction (the GAMS-native idiom). After
a solve, GAMS variable
.m(marginal) holds the bound multiplier and equation.mholds the constraint multiplier. The next solve’s link reads these and reconstructs an approximate working set by sign + tolerance test:bound_status[i] = AtLower if x.m[i] > tol else (AtUpper if x.m[i] < -tol else Inactive). This is lossy (degenerate cases ambiguous) but matches what CONOPT, IPOPT, and KNITRO already do under GAMS. Implemented ingams_pounce.c::pouCallSolver(:437) prior to building the problem. - Persistent state file (the precise idiom). The solver writes
a per-model state file (e.g.
.<modelname>.pou-ws) at the end of each solve and reads it at the start of the next. The state file holds(bound_status, cons_status)as a small binary blob, keyed by the model’s GMO checksum so a structural change invalidates it cleanly. The GAMS optionsqp_state_filecontrols the path; absence means cold-start.
Both mechanisms ship in Phase 5c; mechanism 1 is the default (no configuration required), mechanism 2 is opt-in for users who care about precision in degenerate cases. Documented limitation: full fidelity requires a GUSS-style scenario sweep within a single GAMS session.
7.5 Interface summary
| Layer | Algorithm switch | Working-set in | Working-set out | Bridge |
|---|---|---|---|---|
| Rust | AlgorithmChoice::ActiveSetSqp | SqpWarmStartIterateInitializer | SqpSolution.working | direct |
| C ABI | AddIpoptStrOption("algorithm", …) | IpoptSetWarmStartWorkingSet | IpoptGetWorkingSet | thin shim |
| Python | add_option("algorithm", …) | solve(…, working_set=ws) | res.working_set | PyO3 over C ABI |
| GAMS | pounce.opt | marginals ‖ state file | marginals ‖ state file | C code in pouCallSolver |
8. Test harness
The harness is layered: cheap analytical smoke tests on every commit, fixed reference problems on every PR, scaling sweeps weekly, full regression suite on phase-gate. Each layer below names specific problems, specific size parameters where applicable, and specific reference numbers from published literature so a regression is detectable rather than handwaved.
8.0 Analytical correctness ladder — CI smoke tests
Closed-form problems with hand-computable answers. Run on every
cargo test. Each catches a distinct class of bug in hour 1, not
week 3. These are the unit-level equivalent of pounce-feral’s
“factor a 3×3” smoke tests.
| # | Problem | Closed form | What it catches |
|---|---|---|---|
| 1 | Unconstrained QP, H=I, arbitrary g | x* = −g, one Newton step | KKT sign convention, gradient assembly |
| 2 | Equality-only QP: min ½xᵀHx + gᵀx s.t. Ax = b, with H, A full rank | [x*; λ*] = [H Aᵀ; A 0]⁻¹ [−g; b] (one linear solve) | KKT factor block layout, multiplier sign |
| 3 | Separable box-constrained QP: H = diag(h), xl ≤ x ≤ xu, no general constraints | x*_i = clip(−gᵢ/hᵢ, xlᵢ, xuᵢ) per coordinate | Bound-multiplier sign, working-set add/drop |
| 4 | Strictly convex QP with one redundant constraint | Same as without redundant; redundant row stays inactive | Degeneracy detection, EXPAND triggering |
| 5 | Infeasible QP (xl > xu on one coord) | Elastic mode returns minimal-infeas point | §4.3 phase-1 elastic detection |
| 6 | Indefinite Hessian, single equality, reduced Hessian PD | Solvable; reduced-Hessian inertia OK | §4.5 inertia-control trigger |
Implemented as #[test] functions in pounce-qp/src/tests/analytical.rs.
Total runtime budget: < 50 ms across all six. Exit: all six pass
to 1e-12 relative.
8.1 QP correctness — fixed reference set (Phase 5a, PR-level)
- Maros-Mészáros QP test set (Maros-Mészáros 1999): 138 problems,
sizes
n ∈ [2, 12955]. Format:.qps(QP-extended MPS); new reader inpounce-qp/src/maros.rs, sharing infrastructure with pounce-cli’s MPS handling. - Reference oracle:
- qpOASES (Ferreau 2014) for dense problems — exposes a C API we FFI.
- OSQP (Stellato 2020) for sparse convex problems — widely available, sparse, Python bindings.
- CPLEX/Gurobi via Python as tiebreaker on indefinite cases where qpOASES and OSQP disagree.
- Tolerance: 1e-6 relative objective, 1e-7 KKT residual.
- Exit: ≥ 95 % of Maros-Mészáros pass within tolerance. The remaining ≤ 5 % are documented as “known indefinite hard” with the per-problem reason. qpOASES itself reports ~97 % pass on its reference table (Ferreau 2014 Tab. 2), so 95 % is the floor.
8.2 QP scaling sweep — system size dependence (Phase 5a, weekly)
The deliverable here is a plot — iteration count and wall time vs
n — and a published reference curve to compare against. Three
families, each with a single size axis:
(a) LASSO QP (the canonical OSQP benchmark; Stellato 2020 Tab. 4).
Formulation min ½‖Ax − b‖² + λ‖x‖₁, reformulated as a sparse QP of
dimension 2n with 2n inequality constraints. Sweep
n ∈ {10², 10³, 10⁴, 10⁵} with fixed sparsity 5 %.
- Reference: OSQP paper Tab. 4 reports per-solve time ~0.01s / 0.1s / 1s / 12s respectively on a 2.6 GHz Xeon.
- Exit: within 3× of OSQP at every size; ideally within 2× by Phase 5a end. (Active-set is slower than ADMM on cold convex LASSO; the warm-start sequence in §8.4 reverses that.)
(b) MPC quadrotor scaling (Frison-Diehl HPIPM 2020 §5; the
acados reference). Linear-quadratic MPC with state dim 12, horizon
h ∈ {10, 20, 40, 80, 160}. Sweep yields n = 12·h, m = 12·h sparse
QPs with block-banded structure.
- Reference: HPIPM paper reports ~0.1 ms / 0.4 ms / 1.6 ms /
6.4 ms / 25.6 ms cold solve (linear in
h, as the block factor isO(h)). - Exit: linear scaling in
h(i.e., not super-linear) within 10× HPIPM at every horizon.
(c) Maros-Mészáros size buckets. Same problems as §8.1, sliced by
size. Bucket boundaries n ∈ [1, 10²) ∪ [10², 10³) ∪ [10³, 10⁴) ∪ [10⁴, ∞). Solve time per bucket reported as median + p95.
- Reference: qpOASES reports total time for the full set; per-
bucket numbers are computed once during Phase 5a and committed as
pounce-qp/benches/maros_baseline.json. Regression alert if median doubles or p95 quadruples between commits.
8.3 NLP correctness — fixed reference set (Phase 5b)
- Hock-Schittkowski test set (HS001-HS119; Hock-Schittkowski 1981).
The SQP-community gold standard. Tiny problems (
n ≤ 30mostly) with documented solutions. Every SQP paper reports HS results; filterSQP (Fletcher-Leyffer) and SNOPT (Gill-Murray-Saunders) both publish per-problem iteration tables.- Source: the CUTEst harness already contains all HS problems
(
benchmarks/cutest/problem_list.txtincludesHS001–HS119). - Exit: ≥ 117 of 119 converge; the two allowed failures are
HS013andHS099which most SQP solvers also fail (Wächter 2002 Tab. 6.1).
- Source: the CUTEst harness already contains all HS problems
(
- CUTEst small NLP subset (
n < 1000, the defaultproblem_list.txtminus large-scale entries — roughly 500 problems).- Reference numbers: Wächter-Biegler 2006 Tab. 5 and Fletcher- Leyffer 1999 Tab. 6 publish per-problem iteration counts for IPM and filter-SQP on CUTEst.
- Exit: total iteration count within 30 % of the median of {filterSQP, SNOPT, IPOPT} published numbers; success rate ≥ 90 %.
8.4 NLP scaling sweep — system size dependence (Phase 5b, monthly)
Two families giving a single size axis to test scaling claims:
(a) AC OPF — pglib-OPF (Babaeinejadsarookolaee 2019). Standard
power-grid benchmark, scales 14 → 30000 buses. Pounce’s CUTEst list
already has ACOPP14, ACOPP30, ACOPR14, ACOPR30; extending to
the full pglib-OPF set (14, 30, 57, 118, 200, 300, 1354, 2853, 9241,
13659, 30000 buses) gives a clean two-order-of-magnitude sweep with
real-world structure (sparse Jacobian, near-degenerate binding
limits — exactly what active-set should be measured on).
- Reference: the MATPOWER project publishes IPOPT solve times for pglib-OPF on each instance; PowerModels.jl benchmarks filterSQP and KNITRO on the same set.
- Exit: ≤ 2× IPOPT time at every bus count for cold solve. The warm-start advantage shows up in §8.5(a).
(b) Poisson boundary optimal control (Biegler 2010, Nonlinear
Programming, §11.3). PDE-constrained NLP: minimize tracking-cost
on u subject to −Δu = f + Bv on [0,1]² with boundary-control
v. Standard reference NLP scaling family. Mesh sweep
grid = 16 × 16, 64 × 64, 256 × 256, 1024 × 1024 gives
n ∈ {256, 4096, 65536, 10⁶} with smooth, well-characterized
continuous solution.
- Reference: Biegler 2010 Ch. 11 publishes IPOPT iter counts for exactly this family at each mesh.
- Exit: mesh-independent iteration count (≤ 30 outer iters at every mesh, as the continuous problem is well-posed).
8.5 Warm-start sweep — the actual deliverable (Phase 5c)
The headline result. A perturbation-magnitude axis × cold/warm comparison for each warm-start workload. Plotted as iteration count and active-set-change count vs perturbation size.
(a) MPC closed-loop with horizon shift. Quadrotor or autonomous- vehicle model (acados examples; Verschueren 2022). 200-step closed-loop simulation. At each step, the NLP is the horizon-shifted neighbor of the previous; the warm-start carries the working set shifted by one stage.
- Metrics:
- Mean SQP iterations per step (cold vs warm).
- Mean QP-subproblem active-set changes per SQP iteration (cold vs warm).
- 99th-percentile per-step wall time (worst case for real-time deployment).
- Reference: qpOASES paper (Ferreau 2014 Tab. 3-4) reports 5–50× iteration speedup on closed-loop MPC; acados paper (Verschueren 2022 Tab. 2) reports per-step times for HPIPM and qpOASES. Beat HPIPM-warm-start on worst-case latency — that’s the whole point.
- Exit: ≥ 5× iteration speedup, ≤ 3 active-set changes per step in the steady-state regime.
(b) Parametric continuation. Trace the solution of a parametric
NLP min f(x;t) s.t. g(x;t) ≤ 0 as t sweeps [0, 1] in 100
steps. Use the Beltistos parametric NLP benchmark (Pirnay-López-
Negrete-Wächter 2012) or a Wächter-Biegler 2006 §5 instance.
- Metrics: total iterations across the full path; size of largest discontinuous active-set jump.
- Reference: the
pounce-sensitivity(sIPOPT port) already has a baseline number for IPM-warm-start on the same path. Beat it. - Exit: ≥ 3× total-iteration speedup over IPM-warm-start.
(c) MINLP B&B trace. Record bound changes from a small MINLP
B&B run (one of the minlplib instances with documented bound-
tightening trace; Bussieck 2003). Replay the bound sequence,
warm-starting each child from its parent.
- Metrics: total iterations across the B&B trace.
- Reference: the
minlplibinstances have published Bonmin baselines; Bonmin uses IPOPT-warm-start internally. - Exit: ≥ 2× speedup vs Bonmin.
(d) Perturbation-size sweep. A synthetic perturbation axis on a fixed problem: start from a solved QP, perturb (i) one bound by ε ∈ {1e-6, 1e-3, 1e-1, 1}, (ii) ε of all bounds, (iii) drop one constraint, (iv) add one constraint. Plot iter count vs perturbation magnitude on log-x; the curve characterizes the “warm-start cliff” where active-set adaptation cost crosses cold-start cost.
- Reference: no published baseline; this curve becomes pounce-qp’s own published characterization. It’s what tells prospective users when warm-start helps.
- Exit: monotone in perturbation magnitude; sub-linear up to 10 % bound change.
8.6 Cross-phase comparison — the headline plot
One plot per benchmark family: iter count and wall time vs n, with
four curves on each panel:
- SQP cold
- SQP warm (with prior solve at the same
n) - IPM cold (pounce-default)
- IPM warm (pounce-default + warm_start_init_point=yes)
This is the deliverable that justifies the whole Phase-5 effort. The expected story: cold curves IPM ≤ SQP; warm curves SQP ≪ IPM at all sizes. Two-line summary in the eventual paper / README.
Committed in benchmarks/sqp_scaling/ alongside Phase 5c.
8.7 Unit tests — per-module
For each module in pounce-qp/src/:
factor.rs(sparse LDLᵀ wrapper): roundtrip factor-then-solve on the 6 analytical-ladder problems.schur.rs(Schur-complement updates): each rank-1 update validated against a full refactor of the equivalent KKT matrix, Frobenius-norm diff < 1e-10.expand.rs: anti-cycling verified on Beale’s cycling LP example (Beale 1955), Hoffman’s cycling LP (Hoffman 1953), and Maros 1996 §4.2 degenerate QP.elastic.rs: feasibility detection on the infeasible subset of Maros-Mészáros (problemsQPCBOEI2,QSCAGR25,QSCFXM1— documented infeasible).working_set.rs: random add/drop sequence (50 ops on a random working set), validated against ground-truth full KKT solves.homotopy.rs: parametric trace from QP₀ to QP₁ with identical optimal active set; verify zero working-set changes (the warm- start sweet spot).inertia.rs: indefinite-Hessian QP with reduced Hessian PD; verify §4.5 path produces a stationary point.
Total unit-test runtime budget: < 5 s; runs on every cargo test.
8.8 Phase-gate matrix
| Phase | Required passing |
|---|---|
| 5a (QP standalone) | §8.0 + §8.1 + §8.2 + §8.7 |
| 5b (cold SQP NLP) | All 5a + §8.3 + §8.4 |
| 5c (warm SQP) | All 5b + §8.5 + §8.6 |
| 5d (l1-elastic opt) | All 5c + side-by-side §8.5 comparison filter vs l1-elastic |
A phase is not declared shipped until every cell in its row passes the named exit criterion.
9. Per-workload notes
9.1 MPC
- Block-shift working-set carry:
𝒲_{k+1}[i] = 𝒲_k[i+1]with new terminal stage seeded cold. Modeling-layer convention; the solver only needs the warm-start API to be cheap. - The qpOASES paper (Ferreau 2014 Tab. 3) reports the homotopy completing in 1–3 working-set changes per shift in the well-warm- started regime. This is the headline benchmark for Phase 5c.
9.2 MINLP branch-and-bound
- Sibling/child relaxations differ in one bound. The previous solve’s 𝒲 is feasible for the child unless the bound change invalidates it; then one active-set update fixes it. Documented in Pirnay-Lopez- Negrellos-Wachter (2012) §4 for IPM warm start; the active-set numbers are categorically better.
9.3 Parametric homotopy / continuation
- Step in parameter
t:min f(x; t) s.t. g(x; t) ≤ 0. - Predictor:
pounce-sensitivitycomputesdx/dt,dλ/dtfrom the reduced Hessian at the previous solution. Reuse unchanged. - Corrector: one SQP solve from
(x + Δt·dx/dt, λ + Δt·dλ/dt, 𝒲_prev). If 𝒲_prev is still optimal, one QP iteration. - This is the workload where SQP outperforms a well-warm-started IPM most clearly. Cleanest demo target.
10. Implementation status
The driver shipped in four milestones, each with standalone value: 5a builds the standalone sparse QP solver, 5b the cold SQP NLP driver, 5c the working-set warm start and full-stack integration, and 5d the l1-elastic alternative. Everything below is implemented and self-tested; the only outstanding work is the external-oracle regression comparisons called out at the end.
Phase 5a — pounce-qp standalone sparse QP solver
- §4.2 active-set inner loop with cached-factor
resolveand an opt-in sparse Schur-complement update layer (QpOptions::use_schur_updates). TheSchurStateownsU, V, K₀⁻¹U, Sand applies Sherman-Morrison-Woodbury rank-2 updates per working-set change, cross-checked against a fresh factorization to 1e-9. - §4.3 l1-elastic mode (Gill-Murray-Saunders, SQOPT): an augmented QP
with two non-negative slacks per row and penalty γ, solved through the
standard active-set path; infeasibility is certified when residual
slacks exceed
feas_tol. - §4.4 anti-cycling: Bland’s rule plus the full GMSW EXPAND τ-growth with snap-reset, built on a Harris-style two-pass ratio test.
- §4.5 inertia control:
factorize_with_inertia_controlwraps every factor call site with a diagonal-shift retry onWrongInertia/Singular, matching thepounce-algorithmperturbation-handler defaults. - §4.7 iterative refinement inherited from
pounce-feral(on by default). - Test harness: the §8.0 analytical correctness ladder, a pure-Rust
Maros-Mészáros
.qpsreader (including RANGES), and per-module unit tests forkkt,elastic,refinement, andqps.
Phase 5b — cold SQP NLP driver
- Outer loop (
SqpAlgorithm::optimize) runs end-to-end on nonlinear NLPs, assembling each QP subproblem from the linearization (SqpQpData::build) and consuming any NLP the IPM consumes viaIpoptNlpAdapter(.nl, CUTEst, Python bindings). - Globalization: both an l1-merit line search (Han-Powell with ν
adaptation + Armijo backtracking) and §4.1 filter globalization
(Fletcher-Leyffer 2002), selectable via
sqp_globalization. - Hessian sources: exact, §4.6 damped BFGS (Powell 1978, guaranteeing
PD iterates so the QP solver needs no inertia control), and L-BFGS (a
circular curvature-pair history seeded by the Nocedal-Wright γI
scaling), selectable via
sqp_hessian. - Dispatch:
add_option("algorithm", "active-set-sqp")routes throughoptimize_sqp_tnlp, which builds the NLP chain (TNLPAdapter → OrigIpoptNlp → IpoptNlpAdapter) and mapsSqpStatusback toApplicationReturnStatus; the IPM path is unchanged when the defaultinterior-pointis selected. - Options: eleven registered
sqp_*suboptions (globalization,hessian,max_iter,tol,constr_viol_tol,dual_inf_tol,l1_penalty,bt_reduction,bt_min_alpha,print_level,lbfgs_max_history), all defaulting toSqpOptions::default()and applied throughapply_sqp_options.
Phase 5c — working-set warm start and integration
- Rust:
SqpAlgorithm::optimize_with_warm_startconsumes the §6 tuple(x, λ_g, λ_x, 𝒲)and feeds the working set into pounce-qp’ssolve_with_working_set;IpoptApplicationexposesset_sqp_warm_start,clear_sqp_warm_start, andlast_sqp_working_set(input iterate consumed once and auto-cleared, output working set valid until the next solve overwrites it). - C ABI (§7.2):
IpoptGetWorkingSet,IpoptSetWarmStartWorkingSet,IpoptClearWarmStartWorkingSet, andIpoptSolveWarmStart, withPOUNCE_WS_*status codes. Existing cyipopt / JuMP / AMPL clients are unaffected — no existing signature changes. - Python (§7.3): the
working_set=(bounds, cons)kwarg onProblem.solve, theset/clear/get_working_setmethods, theinfo["working_set"]return key, and a module-levelpounce.classify_working_set(...)so parametric-continuation users can wire IPM-converged multipliers in without dropping into Rust. - GAMS (§7.4): the marginal-based reconstruction path (§7.4(a))
classifies the working set from
gmoGetVarM/gmoGetEquM, with the opt-in persistent state file (§7.4(b),sqp_state_file) as the lossless alternative — a binary format with an FNV-1a checksum keyed by(n, m, x_l, x_u, g_l, g_u)so structural changes invalidate cleanly and fall back to §7.4(a). - Sensitivity corrector:
classify_working_setbuilds aWorkingSetfrom any (primal, multipliers, bounds) snapshot, completing the parametric “predictor (sensitivity) + corrector (SQP)” pattern. The worked end-to-end pipeline ships aspython/examples/sqp_warm_start_mpc.py,gams/examples/parametric_sqp_warm_start.gms, and thetests/parametric_sqp_corrector.rsintegration test, which validates a cold IPM solve → active-set classification → predictor step → SQP corrector to the exact perturbed optimum at 1e-8.
Phase 5d — l1-elastic alternative
Shipped and self-tested: sqp_l1_penalty_safety and sqp_l1_penalty_max
clamp the Han-Powell ν update, and comparison tests certify that the
Filter and L1Elastic globalizations converge to the same optimum on the
shared Hock-Schittkowski fixtures (HS28, HS35).
Deferred — external-oracle regressions
These are benchmarking comparisons that each need a third-party solver or problem distribution wired in; none gate algorithmic completeness:
- Maros-Mészáros 138-problem regression vs qpOASES / OSQP. The in-repo
framework parses each
.qpsand asserts against a supplied optimum; the distribution and reference-optima table are what remain. - Hock-Schittkowski 119-problem regression vs CUTEst.
- AC OPF (pglib-OPF) and Poisson boundary-control scaling sweeps vs MATPOWER / PowerModels.jl.
- A measured ≥5× iteration-count drop on the MPC and parametric suites vs HPIPM / qpOASES / acados, and a small-NLP iteration-count comparison vs filterSQP / SNOPT.
Phases 5a and 5b each have standalone value (the sparse QP solver and the cold SQP NLP driver); 5c is where the warm-start payoff lands; 5d is the comparison work.
11. Risk
- Maintenance. Two solver paths is a permanent maintenance
liability. Mitigation: SQP shares the
IpoptNlp, derivative, scaling, options, journalist, conv-check, and Hessian layers unchanged; only the iteration skeleton + QP subproblem are net new. - Indefinite-Hessian failure modes. Reduced-Hessian indefiniteness
with bad scaling can defeat §4.5 inertia control. Mitigation: SR1
fallback (§4.6) and the same
kappa_ddamping pounce already applies inmu/adaptive.rs. - Schur-complement growth. If the working set changes O(n) times
before a refactor, the dense Schur block becomes a cost concern.
Mitigation: refactor cap
sqp_max_schur_updates(default 50, §7.1); Davis 2006 §11 and Eldersveld-Saunders 1992 give empirical guidance. - GAMS state-transfer ambiguity. Mechanism §7.4(a) is lossy on degenerate active sets. Mitigation: §7.4(b) state file as opt-in; documentation calling out the limitation.
- Benchmark target completeness. No MPC, MINLP, or parametric
workload sits in
benchmarks/today. Phase 5c ships with at least one each (§8.2) committed alongside.
12. Design decisions
The scope-and-policy questions raised during design were resolved as follows:
- Hessian default for cold SQP. Exact Hessian, with a damped-BFGS auto-fallback when the QP repeatedly fails — fastest when reliable, robust on hard nonconvex problems.
- GAMS state-file format. Binary, with a checksum keyed by the problem structure so a changed shape invalidates the file cleanly.
- C API entry-point granularity. Both the three-call primitive
(
IpoptSet… / IpoptSolve / IpoptGet…) and the one-shotIpoptSolveWarmStartconvenience wrapper ship; the sequence is the primitive, the one-shot is convenience. pounce-sensitivityintegration. Landed in Phase 5c, so the parametric workload is a real end-to-end test rather than a unit-test stub.- Crate placement.
crates/pounce-qp/, matching the existing workspace convention.
13. References
Algorithm — outer SQP
- Fletcher, Leyffer (2002), Math. Prog. 91, 239–269 — filter SQP.
- Fletcher, Leyffer, Toint (2002), SIAM J. Optim. 13, 44–59 — convergence of filter SQP.
- Wächter, Biegler (2005), SIAM J. Optim. 16, 1–31 — filter line search.
- Wächter, Biegler (2006), Math. Prog. 106 — IPOPT reference.
- Nocedal, Wright, Numerical Optimization (2nd ed., Springer 2006), Ch. 16 (QP), Ch. 18 (SQP).
Algorithm — QP subproblem
- Ferreau, Kirches, Potschka, Bock, Diehl (2014), Math. Prog. Comp. 6, 327–363 — qpOASES, dense parametric active set.
- Kirches (2011), Fast Numerical Methods for Mixed-Integer Nonlinear Model-Predictive Control, Vieweg+Teubner — sparse Schur-complement extension; the canonical reference for §4.2.
- Janka, Kirches, Sager, Schlöder (2016), Math. Prog. Comp. 8, 435–459 — block-sparse SR1/BFGS SQP.
- Goldfarb, Idnani (1983), Math. Prog. 27 — dual active-set for convex QP (competing family).
- Gill, Murray, Saunders (2005), SIAM Rev. 47, 99–131 — SNOPT.
- Gould, Hribar, Nocedal (2001), SIAM J. Sci. Comput. 23, 1376–1395 — null-space, indefinite Hessian (§4.5).
- Stellato, Banjac, Goulart, Bemporad, Boyd (2020), Math. Prog. Comp. 12 — OSQP (operator-splitting alternative).
Algorithm — sparse linear algebra and updates
- Bartels (1971), Numer. Math. 16 — basis-update lineage.
- Reid (1982), Math. Prog. 24 — Bartels-Golub-Reid sparse variant.
- Eldersveld, Saunders (1992), SIAM J. Matrix Anal. Appl. 13 — block-LU update used for the Schur complement.
- Davis, Direct Methods for Sparse Linear Systems (SIAM 2006) — fill-in analysis.
Algorithm — anti-cycling, elastic mode, inertia
- Gill, Murray, Saunders, Wright (1989), Math. Prog. 45, 437–474 — EXPAND.
- Gill, Murray, Saunders (2008), User’s Guide for SQOPT 7.7 — l1-elastic mode.
- Friedlander, Saunders (2005), SIAM J. Optim. 15 — elastic globalization.
- Gould (1999), SIAM J. Optim. 9, 1041–1063 — modified factorizations.
- Forsgren (2002), Appl. Num. Math. 43 — inertia control.
Algorithm — Hessian approximation
- Powell (1978), in Numerical Analysis Dundee 1977 — damped BFGS for SQP.
- Liu, Nocedal (1989), Math. Prog. 45, 503–528 — L-BFGS.
- Byrd, Nocedal, Schnabel (1994), Math. Prog. 63 — compact representations.
Test harness and benchmarks
- Hock, Schittkowski, Test Examples for Nonlinear Programming Codes, Lecture Notes in Economics and Mathematical Systems 187 (Springer 1981) — the HS001–HS119 reference set used in §8.3.
- Maros, Mészáros (1999), Optim. Methods Softw. 11/12 — Maros-Mészáros QP test set.
- Maros (1996), Computational Techniques of the Simplex Method, Springer — degenerate-QP cycling examples used in §8.7.
- Beale (1955), “Cycling in the dual simplex algorithm”, Naval Res. Logistics Quart. 2 — cycling LP smoke-test instance.
- Hoffman (1953), “Cycling in the simplex algorithm”, National Bureau of Standards Report 2974 — second cycling smoke-test.
- Stellato, Banjac, Goulart, Bemporad, Boyd (2020), Math. Prog. Comp. 12 — LASSO scaling reference numbers in §8.2(a).
- Frison, Diehl (2020), “HPIPM: a high-performance quadratic programming framework for model predictive control”, IFAC- PapersOnLine 53 — MPC scaling reference numbers in §8.2(b).
- Babaeinejadsarookolaee et al. (2019), “The power grid library for benchmarking AC optimal power flow algorithms”, arXiv:1908.02788 — pglib-OPF used in §8.4(a).
- Biegler, Nonlinear Programming: Concepts, Algorithms, and Applications to Chemical Processes, SIAM (2010), §11.3 — Poisson optimal-control scaling family used in §8.4(b).
- Verschueren et al. (2022), Math. Prog. Comp. 14 —
acadosMPC benchmark suite used in §8.5(a). - Pirnay, López-Negrete, Wächter (2012), Math. Prog. Comp. 4 — Beltistos parametric NLP and IPM warm-start comparison baseline used in §8.5(b).
- Bussieck, Drud, Meeraus (2003), INFORMS J. Comp. 15 — MINLPLib instances used in §8.5(c).
- Wächter (2002), An Interior Point Algorithm for Large-Scale Nonlinear Optimization with Inexact Step Computations, PhD thesis, CMU — HS failure documentation referenced in §8.3.
Roadmap context
- The future-work roadmap’s C1 entry — the active-set SQP track this note operationalizes.
- Sister design notes cover C3 (the composite-step Byrd-Omojokun trust-region globalization) and C5 (the matrix-free interior-CG / Krylov-KKT track).
NLP and Linear-System Scaling
Optimization problems whose objective, constraints, or KKT system span many orders of magnitude often converge poorly — or not at all — without some form of rescaling. pounce inherits two independent scaling layers from Ipopt and adds a third option at the linear-system level (see issue #61).
The two layers are conceptually separate:
| Layer | Option | What it touches |
|---|---|---|
| NLP scaling | nlp_scaling_method | The objective f and each constraint row c_i, before the IPM sees them. Changes algorithmic behavior (filter, tol, μ). |
| Linear-system scaling | linear_system_scaling | Symmetric scaling of the KKT augmented system D K D for the factorization. Purely numerical — the IPM sees the same iterates. |
You can configure them independently. Defaults match upstream Ipopt:
nlp_scaling_method = gradient-based, linear_system_scaling = none.
NLP-level scaling
| Option | Default | Effect |
|---|---|---|
nlp_scaling_method | gradient-based | none / gradient-based / user-scaling. |
nlp_scaling_max_gradient | 100.0 | Cutoff above which gradient-based scaling applies. Per-row scale = min(1, max_gradient / ‖∇c_i‖_∞). |
nlp_scaling_min_value | 1e-8 | Floor on computed scale factors — prevents inverting near-zero gradients. |
nlp_scaling_obj_target_gradient | 0.0 | When > 0, pins the scaled objective gradient ∞-norm to this value. Overrides the max_gradient cutoff. |
nlp_scaling_constr_target_gradient | 0.0 | Same as above, per constraint row. |
obj_scaling_factor | 1.0 | Constant multiplier on the objective, applied after the automatic factor. |
gradient-based (default)
Evaluates ∇f and ∇c_i once at the starting point x_0 and
chooses per-row scales that pull each gradient ∞-norm into a
reasonable band. Single-shot is mandatory — recomputing per iteration
would invalidate the filter’s history (Wächter, 2013).
The clamp at 1.0 means scaling never amplifies a small row; it only damps large ones.
user-scaling
The TNLP is asked for obj_scaling, a per-variable x_scaling, and a
per-constraint g_scaling via the get_scaling_parameters callback.
Use this when you know the natural units of your problem (e.g. mass in
kg vs. distance in mm) and can supply better scales than the
gradient-based heuristic.
If the TNLP’s get_scaling_parameters returns false (the default),
pounce falls back to no automatic scaling.
Per-variable factors are a change of variables.
OrigIpoptNlpmodelsobj_scalingand per-constraintg_scalingonly (the design in issue #61), sox_scalingis applied one level below the algorithm instead: a wrapper substitutesx̃ = d ⊙ x, the IPM works in the scaled coordinates, and everything reported back — the solution, the duals, the bound multipliers, and every sensitivity accessor — is in your own units (issue #486). No clone of the model is made and nopropagate_solutionstep is needed, which is what distinguishes this from Pyomo’score.scale_model.Factors must be finite and strictly positive. Zero and negative are refused rather than applied: a negative factor reverses a variable’s direction and swaps its bounds. A factor that would push a finite bound past
nlp_lower_bound_inf/nlp_upper_bound_inf— turning a bounded variable into a free one — is refused too, naming the threshold it crossed. Absent bounds stay absent: the±1e19sentinel is an ordinary finite number, so it is passed through unscaled rather than multiplied into range.One user-visible consequence:
tolkeeps comparing scaled quantities, matching upstream Ipopt, so the sametolstops at a different point than it would on the unscaled model.
Setting user scaling
-
From an
.nlfile (AMPL, Pyomo, any NL-writing frontend) — attach ascaling_factorsuffix to the objective, to constraints, or to both, and passnlp_scaling_method=user-scaling. This is the same channel Ipopt reads through ASL. In Pyomo:m.scaling_factor = Suffix(direction=Suffix.EXPORT) m.scaling_factor[m.obj] = 1e-3 m.scaling_factor[m.mass_balance] = 1e2 SolverFactory('pounce').solve(m, options={'nlp_scaling_method': 'user-scaling'})Components the suffix does not list are unscaled, as are components listed with a factor of
0(AMPL’s suffix default). With noscaling_factorsuffix at all the option falls back to no scaling. See Pyomo for the pyomo-pounce specifics. -
From C — call
SetIpoptProblemScaling(problem, obj, x_scaling, g_scaling)thenAddIpoptStrOption("nlp_scaling_method", "user-scaling"). Seecrates/pounce-cinterface/include/pounce.h. -
From Rust — implement
TNLP::get_scaling_parameterson your problem type. -
From Python —
pounce.Problem.set_problem_scaling(obj_scaling, x_scaling=..., g_scaling=...), followed byadd_option("nlp_scaling_method", "user-scaling"). Walked through end-to-end inpython/notebooks/07_scaling.ipynb.
Specialized solvers. A model that classifies as an LP, convex QP, or SOCP normally routes to
pounce-convex, which equilibrates internally and never reads the TNLP scaling callback. Whennlp_scaling_method=user-scalingis set and the.nlcarriesscaling_factorsuffixes,solver_selection=autodeclines that fast path and uses the general NLP interior-point solver so the scaling is honored; an explicitsolver_selectionis respected and warns.
Target-gradient overrides
nlp_scaling_obj_target_gradient and
nlp_scaling_constr_target_gradient are subtle. When set to a
positive value, they override the max_gradient cutoff and the 1.0
clamp: the scaling is computed unconditionally as
target / max_gradient_norm, so the scaled gradient ∞-norm becomes
exactly the target. Useful when you have a specific numeric range you
want the IPM to see.
The default 0.0 means “use the cutoff path” — i.e. only scale rows
that are above nlp_scaling_max_gradient.
Linear-system-level scaling
| Option | Default | Effect |
|---|---|---|
linear_system_scaling | none | none / ruiz. mc19 and slack-based are accepted by the option registry but not yet implemented — both fall back to none. |
linear_scaling_on_demand | yes | Defer scaling computation until a linear solve is poor; reduces overhead for well-conditioned KKT systems. |
The KKT augmented system is symmetric; all linear-system scalers in
pounce use the symmetric form D K D (single diagonal) to preserve
that structure for the downstream factorization (MA57, MUMPS,
FERAL/SSIDS).
none— first-class choice. The inner linear solver (MA57, MUMPS, FERAL) often does its own scaling under some configurations; stacking pounce-level scaling on top can hurt. Default. Usema57_automatic_scaling=yesto get MA57’s internal scaling instead.ruiz— iterative symmetric ∞-norm equilibration (Ruiz, CERFACS TR/PA/01/14). Pure Rust, no Fortran dependency. Converges geometrically; capped at 10 iterations. The only implemented scaler today; recommended starting point when MA57’s internal scaling is off.mc19(not yet implemented) — intended HSL MC19 row/column scaling (Curtis-Reid 1972; minimizes Σ log²|a_ij|). Accepted by the registry but currently logs a warning and falls back tonone.slack-based(not yet implemented) — intended slack-aware scaling. Accepted by the registry but falls back tonone.
Worked example — nql180
nql180 is one of the Mittelmann NLP benchmarks where both default
pounce and default Ipopt fail to clear the strict tol gate (see
issue #25). Forcing
Ruiz symmetric equilibration on the augmented KKT system is enough to
push pounce all the way to “Optimal Solution Found”:
pounce nql180.nl presolve=yes linear_system_scaling=ruiz \
linear_scaling_on_demand=no
| default | + Ruiz (forced) | |
|---|---|---|
| Exit status | Solved To Acceptable Level | Optimal Solution Found |
| Iterations | 41 | 50 |
| Primal infeasibility | 4.0e-11 | 1.2e-15 |
| Dual infeasibility | 1.0e-5 | 3.1e-4 |
| Complementarity | 1.2e-9 | 9.9e-10 |
| Overall NLP error | 2.4e-7 | 9.9e-10 |
The four-orders-of-magnitude primal-feasibility improvement and ~3
orders on the overall NLP error are the textbook Ruiz benefit:
symmetric ∞-norm equilibration lowers the condition number of the KKT
matrix enough that the back-solve residuals drop the extra fractional
digits needed to clear tol. The extra nine iterations are well spent
— the 50-iter Ruiz solution is mathematically of strictly higher
quality than the 41-iter unscaled “acceptable” solution.
linear_scaling_on_demand=no forces always-on Ruiz; the default
(yes) defers scaling computation until the linear solver flags an
iterate as poorly scaled, which is the right behavior for problems
that don’t need it (most of the Mittelmann set, where the iter count
is unchanged with or without Ruiz).
Reporting
All scaling effects are undone before the solve report (final objective, multipliers, dual residuals, KKT termination metric) is handed back to the user. You always see quantities in the natural units of your TNLP.
Internally, the IPM operates in scaled space: stopping criteria
(tol, acceptable_tol) compare scaled values, the barrier parameter
μ is in scaled units, and the filter’s history is built from scaled
function values.
When to override the defaults
Reach for non-default scaling when:
- The constraint Jacobian has entries spanning many orders of magnitude
(chemistry, power-flow, mixed-unit mechanics). Try
mc19orruizat the linear-system level, after disabling MA57’s internal scaling. - The IPM stalls with small step sizes but no clear infeasibility.
Worth turning
nlp_scaling_method=noneto see whether the default gradient scaling is doing the wrong thing; then re-enable with problem-specific target gradients. - You know the natural units of your problem better than the solver
can infer from gradients at
x_0. Wireuser-scaling.
Otherwise the upstream-Ipopt-style defaults (gradient-based at the
NLP level, none at the linear-system level with MA57’s internal
scaling on) are a reasonable starting point.
References
- Wächter, A. On the effects of scaling on the performance of Ipopt. arXiv:1301.7283 (2013). https://arxiv.org/abs/1301.7283
- Ruiz, D. A scaling algorithm to equilibrate both rows and columns norms in matrices. CERFACS TR/PA/01/14. https://cerfacs.fr/wp-content/uploads/2017/06/14_DanielRuiz.pdf
- Curtis, A. R. and Reid, J. K. On the Automatic Scaling of Matrices for Gaussian Elimination. (1972). HSL MC19 reference.
- pounce issue #61.
Feasibility-Based Bound Tightening (FBBT)
pounce supports feasibility-based bound tightening on nonlinear
constraints: interval-arithmetic propagation through the constraint
expression DAG to discover variable bounds the user did not write
down (e.g. x² + y² ≤ 1 ⇒ x ∈ [-1, 1], exp(x) ≤ 10 ⇒
x ≤ ln 10). It pairs with the linear bound-tightening already in
the presolve pipeline (which only handles linear constraints).
Tracks issue #62. References: Belotti, Cafieri, Lee, Liberti (2010).
When it helps
- The Jacobian / objective row magnitudes are wildly different from what the user-declared bounds suggest.
- A nonlinear equality or one-sided inequality is much tighter than
the user’s
[lo, hi]box. - Loose bounds were inherited from a modeling tool that doesn’t propagate constraints back to variable boxes (most modeling tools don’t).
FBBT cannot help when:
- The TNLP has no structural-expression representation. Today only
.nl-loaded problems (NlTnlp) expose one. Python (PyTnlp), C-callback (CCallbackTnlp), and Rust closure-based problems silently opt out. - The expression uses operators FBBT doesn’t reason about
(
Funcallto AMPL imported functions, variable-exponent powers,sin/cosreverse pass). Those subtrees become opaque and block tightening through them, but the rest of the constraint still propagates normally.
Options
| Option | Default | Effect |
|---|---|---|
presolve_fbbt | no | Master switch. Requires presolve=yes and an ExpressionProvider. |
fbbt_tol | 1e-6 | Minimum per-variable bound improvement to keep iterating. |
fbbt_max_iter | 10 | Outer-sweep cap. |
fbbt_max_constraints | 0 | Per-sweep cap on constraints inspected (0 = unlimited). |
FBBT runs after the linear bound-tightening (Phase 1) and before the redundant-constraint pass (Phase 2), so any FBBT-derived tightening feeds forward into row drops, the LICQ check, and the bound-multiplier warm starts.
Reading the presolve banner
With presolve_fbbt=yes, the per-solve presolve banner prints two
lines instead of one:
Presolve: tightened 170 bounds (82 newly-finite), dropped 46 redundant rows, LICQ=Full
Presolve FBBT: 10 sweeps, 1362 variable tightenings (Σ|Δ|=7.5e20)
Fields:
sweeps— number of outer iterations actually executed (≤fbbt_max_iter). Hitting the cap is informational, not an error.variable tightenings— total count of per-variable(x_lo[j], x_hi[j])updates that strictly improved the box.Σ|Δ|— sum of absolute bound improvements across all updates. Provided as a coarse “how much did we move” signal — not part of the FBBT algorithm.
If FBBT detects infeasibility (the constraint bound is disjoint
from the interval enclosure at the current variable box), it stops
and emits pounce: FBBT detected infeasibility (witness constraint N). The solve continues with the partially-updated bounds — the
IPM will then report infeasibility through its own channels.
Should I turn it on?
The issue’s design says: default off until benchmark evidence justifies a flip. Today’s evidence:
- On small problems (e.g.
tutorial_flow_density.nl): FBBT moves iteration count slightly, sometimes up, sometimes down. - On larger problems (e.g.
gaslib11_steady.nl): FBBT enables additional redundant-row drops and can promote the LICQ verdict fromStructuralRanktoFull, but the iteration count change is mixed.
So: try it on your problem. If you see fewer iterations or a
cleaner LICQ verdict, keep it on; if it costs iterations, turn it
off again. The cost of FBBT itself is small (one pass over the
expression DAGs per sweep, capped at fbbt_max_iter).
Soundness guarantees
FBBT uses outward-rounded interval arithmetic. Every operation
widens its result by one ULP outward so accumulated floating-point
error always increases the interval, never shrinks it. The
consequence: FBBT may produce a looser tightening than ideal, but
it cannot drop a feasible point. The pointwise soundness fuzz tests
in crates/pounce-presolve/src/fbbt/{forward,reverse,orchestrator}.rs
verify this property on random sample grids.
Operator support
Forward + reverse rules cover the operators that account for ~all nonlinear constraints in practice:
| Operator | Forward | Reverse |
|---|---|---|
+ - * / neg | ✓ | ✓ |
pow (integer constant) | ✓ | ✓ (branch-selecting for even powers) |
pow (variable / non-integer) | opaque | opaque |
sqrt exp ln abs | ✓ | ✓ (with domain clipping) |
sin cos | ✓ (loose) | declines to tighten |
log10 | rewritten as ln / ln(10) | follows the rewrite |
AMPL imported Funcall | opaque | opaque |
n-ary Sum | folded into binary Add | follows the fold |
Opaque slots evaluate to [-∞, +∞] on the forward pass and block
reverse propagation through them — they don’t pollute the rest of
the constraint.
Extending support to new TNLP sources
FBBT consumes the pounce_nlp::expression_provider::ExpressionProvider
trait. Any TNLP can opt in by implementing:
#![allow(unused)]
fn main() {
impl ExpressionProvider for MyTnlp {
fn constraint_expression(&self, i: usize) -> Option<pounce_nlp::FbbtTape> {
// Build a tape from your problem's symbolic structure.
// Return None to decline (FBBT becomes a no-op on that
// constraint).
}
}
}
FbbtTape is a flat tape of FbbtOp nodes; the existing
NlTnlp implementation in crates/pounce-cli/src/nl_fbbt_translate.rs
is the canonical template (it walks an AMPL Expr tree, preserving
CSE sharing via Rc::as_ptr keying). Building a similar tape from
a Pyomo, JAX, or sympy expression is a finite-effort project.
References
- Belotti, Cafieri, Lee, Liberti. On feasibility based bounds tightening. (2010). https://enac.hal.science/hal-00935464v1/document
- Liberti et al. Feasibility-based bounds tightening via fixed points. COCOA 2010. https://www.lix.polytechnique.fr/~liberti/fbbt-cocoa10.pdf
- Puranik, Sahinidis. Domain reduction techniques for global NLP and MINLP optimization. Constraints 22 (2017). https://arxiv.org/pdf/1706.08601
- pounce issue #62.
Auxiliary-Equality Preprocessing
POUNCE’s auxiliary-equality preprocessing pass identifies small, self-contained equality sub-systems in an NLP and solves them before the IPM starts. Variables determined by those sub-systems are pinned to their values; the equality rows are dropped from the problem the IPM sees. The IPM then handles the reduced problem, which is smaller, often better-conditioned, and sometimes solvable in zero iterations.
The pass is a port of ripopt PR #32 by
David Bernal Neira to pounce’s TNLP wrapper. It lives
entirely inside pounce-presolve and is enabled by setting two
options:
pounce problem.nl presolve=yes presolve_auxiliary=yes
What it does, in words
For each call to the inner TNLP, the wrapper:
- Builds a bipartite graph between equality constraint rows and variables, using the Jacobian sparsity.
- Finds a maximum matching (Hopcroft-Karp).
- Runs a Dulmage-Mendelsohn partition, slicing the graph into three pieces: overdetermined, underdetermined, and square (the piece where rows and variables pair up one-to-one).
- Decomposes the square piece into independent connected components, and each component into an ordered sequence of blocks via Tarjan SCC.
- Classifies each block by how it’s coupled to the rest of the problem: pure equality, objective-coupled, inequality-coupled, or both.
- Solves each pure-equality (or, with
aggressivecoupling, objective-coupled) block via a small dense-LU Newton step and verifies the full-space residual is within tolerance. - Applies accepted blocks by clamping the fixed variables’
bounds (
x_l = x_u = value) and dropping the dropped rows. - After the IPM finishes, recovers the Lagrange multipliers for the dropped rows via a small dense-LU stationarity solve, and hands the user back a complete full-space KKT solution.
If the model has no eliminable structure, the pass is a tested no-op and the IPM runs as usual.
When it helps
The pass is most valuable when an NLP contains:
- Algebraic auxiliary variables that appear in one or two linear constraints with no other coupling (common in process-engineering and energy-system models).
- Internal chains where one variable is defined as a function of
another (e.g.
T_out = T_in + delta_TwithT_inalready known). - Mass-balance equalities that form a small square block on a subset of stream variables.
ripopt reports gaslib11_steady going from 204 / 200 vars / cons to
140 / 136 vars / cons under this pass, and tutorial_flow_density
going from 6–7 IPM iterations to 0.
Coupling classes
Every candidate block is classified by what it touches:
| Class | Touches inequality? | Touches objective grad? | Eliminated under safe? | Eliminated under aggressive? |
|---|---|---|---|---|
PureEquality | no | no | yes | yes |
ObjectiveCoupled | no | yes | no | yes (postsolve candidate) |
InequalityCoupled | yes | no | no | no |
ObjectiveAndInequalityCoupled | yes | yes | no | no |
safe is the default. Inequality-coupled blocks are never
eliminated in v1 — fixing such a variable could violate the
inequality.
Options
See Solver Options → NLP Presolve for the full list. The two switches you most often touch are:
| Option | Default | Effect |
|---|---|---|
presolve_auxiliary | no | Master switch. Off → pass is a no-op. |
presolve_auxiliary_coupling | safe | none / safe / aggressive policy. |
Diagnostics
The pass populates an
AuxiliaryPreprocessingDiagnostics
struct on every call. From Rust:
#![allow(unused)]
fn main() {
use pounce_presolve::{wrap_with_presolve, PresolveOptions};
let opts = PresolveOptions { enabled: true, auxiliary: true, ..PresolveOptions::defaults() };
let wrapped = wrap_with_presolve(inner, opts)?;
// ... run a solve ...
// Access via the typed handle returned by PresolveTnlp::new:
// let diag = typed.auxiliary_diagnostics();
// println!("{diag}");
}
The Display impl produces output like:
auxiliary-preprocessing: 1 of 1 candidate block(s) eliminated, fixing 2 variable(s) and dropping 2 row(s) in 0 ms
max block dim: 2, max residual: 0.000e0
coupling: pure=1, obj=0, ineq=0, both=0
Per-stage timings (stage_time_ms.incidence_ms /
matching_ms / dm_ms / components_ms / btf_ms /
block_solve_ms / residual_check_ms) and per-class accept counts
are also available.
From the command line, set presolve_auxiliary_diagnostics=yes to
have the same summary emitted to stderr automatically after every
Phase-0 pass:
pounce problem.nl presolve=yes presolve_auxiliary=yes \
presolve_auxiliary_diagnostics=yes
Limitations (v1)
Both linear and nonlinear blocks are eliminated. The linear path reuses the pre-fetched Jacobian; the nonlinear path drives Newton through TNLP callbacks.
Fixed variables are assumed to be interior to their original bounds at the optimum; postsolve sets their bound multipliers to zero implicitly. Lifting this assumption — handling the case where a fixed variable is at an original bound — is a known follow-up.
The pass currently runs once, at the start of the solve. Iterative re-elimination (running the pass again on the reduced problem) is not supported in v1.
Interaction with the rest of presolve
The auxiliary pass runs before the existing bound-tightening
phase (presolve_bound_tightening=yes). The two phases interact at
the bounds: aux clamps x_l[i] = x_u[i] = value for variables it
fixes; bound tightening then propagates the remaining constraints.
The orchestrator filters out aux-dropped rows before tightening
runs, so they can’t propagate contradictions back over the clamps.
If tighten_bounds still flags infeasibility — for example because
an aux-fixed value disagrees with a kept-row’s bound — the
orchestrator rolls back the aux pass for that solve and re-runs
tightening on the unfiltered rows. A one-line warning lands on
stderr when this happens.
Interaction with sensitivity / reduced-Hessian post-processing
When the input .nl file carries sensitivity suffixes
(sens_init_constr / sens_state_*) or the CLI is invoked with
--compute-reduced-hessian, the entire presolve layer — including
auxiliary preprocessing — is silently disabled. The user sees a
single warning on stderr (pounce: disabling presolve — ...) and
the solve proceeds without any presolve transformation. This is
because the existing sensitivity / reduced-Hessian code paths
assume the IPM’s variable and row indices match the user’s
original .nl. Lifting this restriction is tracked separately
(pounce#19).
Caveat: nonconvex problems can land at a different local optimum
When the auxiliary pass eliminates a block, it pins the block’s
variables to a specific feasible point of the equality system —
the one Newton converges to from the probe point. On convex
problems this is the unique local optimum and the IPM would reach
the same values anyway. On nonconvex problems with multiple
feasible solutions to the equality system, the auxiliary pass may
fix variables to a feasible point in a different basin of
attraction than where the un-presolved IPM would eventually settle.
The full-space objective then differs between
presolve_auxiliary=yes and presolve_auxiliary=no, both
solutions remain feasible and locally optimal.
The vendored gaslib11_steady.nl benchmark in
crates/pounce-cli/tests/fixtures/aux_presolve/ exhibits exactly
this — presolve_auxiliary=yes converges to objective ≈ 1.825e-02
while the un-presolved path settles at ≈ 3.286e-02. Both points
satisfy the model’s KKT conditions; aux just lands in a different
basin. The regression test for gaslib11_steady deliberately does
NOT assert objective parity for this reason; the test name and
comments document the constraint.
If matching the un-presolved path’s local optimum is important
for your workflow, leave presolve_auxiliary=no until iterative
re-elimination or a multiple-basin-aware policy lands (tracked on
pounce#53).
Worked example
Run any of these to see the pipeline in action:
cargo run -p pounce-presolve --example pipeline_demo
cargo run -p pounce-presolve --example phase0_via_tnlp
The first runs the algorithmic pipeline directly on a hand-crafted
problem and prints each stage’s output. The second wraps a real
TNLP with presolve_auxiliary=yes and exercises the end-to-end
elimination + multiplier recovery.
References
- Issue tracking the port: pounce#53.
- Upstream: ripopt PR #32 by David Bernal Neira
(@bernalde). The
tutorial_flow_density{,_perturbed}.nlandgaslib11_steady.nlfixtures vendored intocrates/pounce-cli/tests/fixtures/aux_presolve/originate from that ripopt PR. - Design notes:
dev-notes/auxiliary-equality-preprocessing.mdin the pounce repo.
Troubleshooting Recipes
When a pounce solve fails, stalls, or settles for “acceptable” instead of “optimal”, the default options aren’t always the best fit. This page collects concrete, reproducible recipes that turn failures into successes (or improve already-successful solves) on real problems.
Each entry follows the same shape:
- When to try it — symptoms in the iter table or the final report that point to this knob.
- The knob — exact option(s) and CLI invocation.
- Worked example — before/after table on a named problem so you can verify the recipe reproduces on your machine.
A recipe earns a place on this page when there’s a named problem where it demonstrably helps. “Should help in theory” entries belong in the reference pages (Scaling, FBBT, Options), not here. If you find a new win, the contribution guide (CONTRIBUTING.md) walks through adding it.
Quick lookup by symptom
| Symptom | Recipe |
|---|---|
| Exit “Solved To Acceptable Level” but you need strict optimality | Ruiz linear-system scaling |
| Hundreds of small steps, slow convergence on a problem with loose bounds | FBBT on nonlinear constraints |
Search Direction is becoming Too Small early in the iter table | Ruiz linear-system scaling, then μ-strategy switch |
| Restoration phase fires repeatedly | ℓ₁ exact-penalty wrapper |
| Iterates wander on an LP-like / linearly constrained problem | mehrotra_algorithm=yes |
| Hundreds of iterations, monotone μ stair-steps slowly toward optimal | mu_strategy=adaptive |
| Iter count looks fine but seconds-per-iter is dominated by the linear solve on a hard QCQP / banded problem | feral_ordering=auto_race |
alpha_pr halves toward 1/128 while ||d|| grows and the dual residual stalls | feral_singular_pivot_floor |
Presolve: bound-tightening and row drops
presolve=yes (start here)
The pounce presolve pipeline drops fixed variables, propagates bounds from linear rows, detects empty / redundant constraints, and warm-starts bound multipliers. It is off by default to match upstream Ipopt’s no-surprises behavior; turn it on for any non-trivial NLP.
pounce problem.nl presolve=yes
Cheap, almost always helpful, and a prerequisite for FBBT.
FBBT (feasibility-based bound tightening)
Interval propagation through the nonlinear constraint DAG to discover
variable bounds the user did not write down (x² + y² ≤ 1 ⇒
x ∈ [-1, 1], exp(x) ≤ 10 ⇒ x ≤ ln 10, etc.). Full reference
in Feasibility-Based Bound Tightening.
When to try it. Hundreds of small steps in the iter table, the
primal infeasibility stuck against a bound, or a problem that’s
clearly under-constrained from the modeler’s side. Requires a
structural-expression representation, which today means an .nl
input.
The knob.
pounce problem.nl presolve=yes presolve_fbbt=yes
Worked example — clnlbeam (Mittelmann):
presolve=yes | + presolve_fbbt=yes | |
|---|---|---|
| Exit status | Optimal Solution Found | Optimal Solution Found |
| Iterations | 552 | 65 |
| Wall time | 41.4 s | 8.2 s |
FBBT discovers tight nonlinear bounds the linear sweep missed; the IPM then has a much smaller feasibility gap to close and converges in roughly one-eighth the iterations.
Not every problem benefits. On corkscrw and arki0003 FBBT
produces no measurable change or a slight regression — the
infrastructure is cheap (one pass per constraint per outer sweep,
capped at fbbt_max_iter=10), so the worst case is a few percent of
extra presolve time.
Scaling
Full reference in Scaling. The two layers are independent.
Ruiz scaling on the augmented KKT system
When to try it. Exit status is “Solved To Acceptable Level” with
small step sizes near the end, or dual_inf plateaus several orders
above tol while primal feasibility is already at machine epsilon.
That pattern signals a poorly-conditioned KKT augmented matrix — the
back-solve loses the last few fractional digits the convergence check
needs.
The knob.
pounce problem.nl presolve=yes linear_system_scaling=ruiz \
linear_scaling_on_demand=no
linear_scaling_on_demand=no forces always-on Ruiz; the default
(yes) defers scaling until the linear solver flags an iterate as
poorly scaled. For diagnostic runs, force it on.
Worked example — nql180 (Mittelmann):
| default | + linear_system_scaling=ruiz | |
|---|---|---|
| Exit status | Solved To Acceptable Level | Optimal Solution Found |
| Iterations | 41 | 50 |
| Primal infeasibility | 4.0e-11 | 1.2e-15 |
| Dual infeasibility | 1.0e-5 | 3.1e-4 |
| Complementarity | 1.2e-9 | 9.9e-10 |
| Overall NLP error | 2.4e-7 | 9.9e-10 |
Symmetric ∞-norm equilibration improves primal feasibility by four
orders of magnitude and overall NLP error by ~3 orders, letting the
solver clear the strict tol gate. The extra nine iterations are
well spent. Resolves issue #25.
Worked example — WM_CFy (Mittelmann ampl-nlp, n=8709, m=12850):
| default | + linear_system_scaling=ruiz | |
|---|---|---|
| Exit status | Optimal Solution Found | Optimal Solution Found |
| Iterations | 605 | 241 |
| Wall time | ~2300 s | ~543 s |
| Overall NLP error | 3.4e-9 | 2.6e-9 |
A 4× wall-time speedup on a problem that previously sat in the “hard
W-B” bucket: every Ipopt + linear-solver combination tried in
issue #29 had failed
to converge within a 600 s budget. Ruiz wasn’t just an iteration-count
win — at 605 iters / 2300 s default-pounce was the only configuration
that even finished; Ruiz cuts that to under ten minutes. Same
underlying mechanism as nql180: the augmented KKT system is
ill-conditioned enough that the back-solve burns iterations chasing
residuals symmetric ∞-norm equilibration fixes in one preconditioning
pass.
Pairing mu_strategy=adaptive with Ruiz on this problem solves to a
~50× tighter NLP error (5e-11) but takes twice as long (491 iters,
1100 s). For a tighter solution at any cost, use both; for a fast
solve, Ruiz alone wins.
NLP-level scaling: when the default hurts
The gradient-based default at the NLP level is computed once at
x_0 and is sometimes the wrong fingerprint of the problem — for
instance when the starting point lives near a flat region of the
objective. If the IPM stalls with no clear infeasibility and the
unscaled gradients in the report look reasonable, try turning NLP
scaling off:
pounce problem.nl nlp_scaling_method=none
Or, if you know the natural units of your problem better than the
solver does, supply user-scaling (see Scaling for the
end-to-end recipe).
μ-strategy
Monotone vs. adaptive
Monotone (the default) decreases the barrier parameter μ in geometric steps; adaptive uses a quality-function oracle to pick each new μ based on the current iterate’s complementarity. Adaptive is more aggressive in well-conditioned regions and more conservative near degeneracy.
When to try it. Convex or nearly-convex problems where the monotone schedule wastes iterations stair-stepping toward a μ that the iterate clearly accepts; alternately, ill-conditioned problems where monotone overshoots and triggers restoration.
The knob.
pounce problem.nl mu_strategy=adaptive
Pair with mu_oracle=quality-function (the default) or
mu_oracle=probing for the Mehrotra-style affine probe.
Worked example — arki0009 (Mittelmann):
mu_strategy=monotone (default) | mu_strategy=adaptive | |
|---|---|---|
| Exit status | Optimal Solution Found | Optimal Solution Found |
| Iterations | 358 | 108 |
A 70 % iteration-count reduction with no quality regression. The quality-function oracle picks larger μ-decrements when the complementarity gap is well-balanced, skipping the slow stair-step that monotone is forced into on this instance.
nql180 is also rescued by mu_strategy=adaptive alone
(Acceptable → Optimal in 61 iters) — so for that problem you have a
choice between the Ruiz recipe (above) and the adaptive-μ recipe.
Ruiz gives a numerically cleaner solution (primal infeasibility
1.2e-15 vs ~5e-12); adaptive μ is one knob instead of two and has no
linear-system overhead.
Mehrotra predictor-corrector
For problems that are LP-like (linear or mildly nonlinear constraints, quadratic objective), the Mehrotra predictor-corrector mode short-circuits the filter line search and accepts every trial step:
pounce problem.nl mehrotra_algorithm=yes
This sets a Mehrotra-canonical configuration (adaptive_mu_globalization=never-monotone-mode,
accept_every_trial_step=yes, alpha_for_y=bound_mult, larger
bound_push and bound_mult_init_val). On well-conditioned LP-like
problems it routinely cuts iteration counts in half. On nonconvex
NLPs it can destabilize — see
issue #58 for the
trade-off discussion.
Restoration & ℓ₁ exact-penalty wrapper
When restoration fires repeatedly, the standard IPM is stuck on an infeasible subproblem the filter cannot accept. The ℓ₁ exact-penalty wrapper rephrases the constraints as an additive penalty term and solves a sequence of bound-constrained subproblems instead:
pounce problem.nl l1_exact_penalty_barrier=yes
Or, only invoke the wrapper as a fallback when standard restoration fails:
pounce problem.nl l1_fallback_on_restoration_failure=yes
This is the recipe for problems with rank-deficient constraints, ill-defined bounds at the starting point, or pathological LICQ violations — anywhere the filter’s history rules out feasibility restoration paths the wrapper can still find.
Worked example: certifying genuine infeasibility
The built-in infeasible-eq problem is the smallest fixture that
exercises the fallback end-to-end:
min x0^2 + x1^2
s.t. x0 + x1 = 1 (g0)
x0 + x1 = 2 (g1)
The two equalities are mutually contradictory, so no x exists with
||g(x)||_∞ = 0. The standard solve diagnoses this without the
wrapper:
$ pounce --problem infeasible-eq
...
EXIT: Converged to a point of local infeasibility. Problem may be infeasible.
That message is the filter giving up: it found an iterate where the constraint gradients are linearly dependent and no admissible step reduces infeasibility further. The output does not tell you whether the problem is genuinely infeasible or whether the filter rejected a feasible neighborhood that another method could reach. Re-run with the wrapper to find out:
$ pounce --problem infeasible-eq l1_fallback_on_restoration_failure=yes
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) ...
0 0.0000000e+00 2.00e+00 0.00e+00 -1.0 0.00e+00 - ...
1 1.1250000e+00 5.00e-01 4.22e-09 -1.0 7.50e-01 - ...
2r 1.1250000e+00 5.00e-01 9.99e+02 -0.3 0.00e+00 - ... ← restoration
...
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) ... ← second inner solve
0 3.0202000e+00 9.90e-03 0.00e+00 -1.0 0.00e+00 - ...
...
6 1.5000000e+00 2.22e-16 2.53e-14 -8.6 1.88e-06 - ... ← wrapper converges
in the slacked
problem
EXIT: Converged to a point of local infeasibility. Problem may be infeasible.
Read this trace carefully. The wrapper’s inner solve converges to
KKT tolerance on the slacked problem — inf_pr falls to 1e-16 in
six iterations because the added slack variables s+, s- absorb the
inconsistency g0 ≠ g1. But pounce reports the overall verdict on
the original constraints, so the final Constraint violation = 0.5
is unchanged: that’s the irreducible gap (g1 − g0)/2. Two
independent solvers (filter IPM and ℓ₁-penalty barrier) landing on
the same least-infeasible iterate, from different starting strategies,
is what makes this an infeasibility certificate rather than a
diagnosis of solver fragility.
The recipe in plain English:
- Standard solve says “local infeasibility” → may or may not be a real obstruction; could be filter history, LICQ degeneracy, or a bad starting point.
- Wrapper agrees on the same least-infeasible iterate → trust the certificate; reformulate the model.
- Wrapper promotes to
Solve_Succeeded→ the standard filter was rejecting a feasible neighborhood it could not reach; the model itself is fine.
Implementation note — running this case used to panic with
restoration factory invoked more than oncebecause the CLI wired a one-shot restoration factory into the application. The fix (pounce#24) routes through a multi-pass provider so the wrapper can mint a fresh restoration phase per inner solve. The regression test that guards it (crates/pounce-cli/tests/l1_fallback_no_panic.rs) uses this sameinfeasible-eqbuiltin.
The second-opinion ladder (what those extra solves in your log are)
Before shipping a local-infeasibility verdict the CLI re-solves the problem along up to two different trajectories and only keeps the verdict if they agree. You will see this in the log:
EXIT: Converged to a point of local infeasibility. Problem may be infeasible.
pounce: local infeasibility — re-solving along 2 different trajectories before
believing it (second-opinion ladder: feral_scaling=mc64,
mu_strategy=adaptive).
pounce: second opinion — re-solving with feral_scaling=mc64…
pounce: feral_scaling=mc64 re-solve did not recover (InfeasibleProblemDetected).
pounce: second opinion — re-solving with mu_strategy=adaptive…
pounce: mu_strategy=adaptive re-solve recovered the problem — promoting (SolveSucceeded).
Status: Solve_Succeeded
Note the trailing Status: line. Each rung prints its own EXIT: banner,
so a laddered run has several and only the last one is the verdict that
shipped — if you are parsing pounce’s output, read Status: and ignore the
banners. It carries the upstream IPOPT enumerator spelling
(Infeasible_Problem_Detected, Maximum_Iterations_Exceeded, …).
The two rungs probe different things, and the distinction matters when you are reading a log:
| rung | option | varies |
|---|---|---|
feral_scaling=mc64 | feral_infeasibility_scaling_retry | the linear algebra |
mu_strategy=adaptive | infeasibility_mu_strategy_retry | the barrier trajectory |
The first rung is evidence only when the trajectory is
hypersensitive — two equally backward-stable scalings staying
bit-identical for many iterations, then diverging by ~1 ULP into
different basins (discs.nl is the canonical case). When it is not,
MC64 retraces the same iterates and agrees for the same reason the
first solve was wrong, so the scaling rung agreeing is not by itself
a reason to believe the verdict. That is why the barrier rung exists
(pounce#524: CUTE
cresc4 is feasible, Ipopt solves it in 71 iterations, and the MC64
re-solve reproduced the failing trajectory bit-identically).
Things worth knowing:
- A rung is promoted only if it returns
Solve_Succeeded/Solved_To_Acceptable_Level, so an overturned verdict always comes with a point that passed the ordinary convergence check. - Rungs are applied to your baseline options, not stacked on each
other, and a rung that would change nothing (you already set
mu_strategy=adaptive) is skipped. - The extra solves are spent only on runs that would otherwise report failure. Nothing changes on a successful solve.
- Both rungs are on by default; set them to
nofor upstream IPOPT’s behaviour of shipping the first verdict. - If a rung recovers the problem, that is a signal about your model as well as about the solver: the verdict was trajectory-dependent, so the starting point or the scaling of the formulation is worth a look.
When the residual is small but the verdict still says infeasible
Some models cannot reach a small absolute residual no matter how well
they are solved. An ill-conditioned change of variables — a moving-boundary
PDE on a Landau coordinate, say — can leave a row carrying a coefficient
of 1e9, so a residual of 1e-3 is eleven relative digits: the equation
is satisfied about as well as double precision allows, and no absolute
tolerance will ever be met. That is exactly the regime the acceptable-level
fallback exists for, and the exit you want is
Solved_To_Acceptable_Level.
Set acceptable_tol to a level you can actually reach, and read the
result there:
$ pounce model.nl -AMPL tol=1e-6 acceptable_tol=1e-3
Three things are worth knowing about how that interacts with the infeasibility detector:
acceptable_constr_viol_tol(default1e-2) is the feasibility band the acceptable-level exit uses, and it is separate fromconstr_viol_tol. Widening the latter does not widen the former.- Tightening
constr_viol_toldoes not make POUNCE readier to call a model infeasible. The rapid-infeasibility detector’s violation floor is clamped so it never convicts a point whose violation sits inside the band the defaults call acceptable (pounce#519). If you are still seeingInfeasible_Problem_Detected, the point is outside the band you declared: compare the reportedOverall NLP erroragainst youracceptable_tol, and theConstraint violationagainstacceptable_constr_viol_tol. - If the solve did pass through an acceptable iterate before giving up, that point is returned rather than discarded, whichever internal route reached the verdict (pounce#505).
If the residual is large relative to its own row — not just in absolute terms — the verdict is the honest one, and the ℓ₁ wrapper above is the way to corroborate it.
Linear solver choice
linear_solver=ma57 (when built with HSL):
pounce problem.nl linear_solver=ma57
For problems that go many hundreds of iterations, the round-off chain of the inner sparse factorization matters — MUMPS, FERAL/SSIDS, and MA57 do not produce bitwise-identical iterates, and on the worst-case instances the difference can be the difference between convergence and a μ-reset spiral (issue #58, issue #64).
Pair with ma57_automatic_scaling=yes (default in HSL builds) and
leave linear_system_scaling=none — MA57’s internal scaling and a
pounce-level Ruiz pass should not be stacked.
FERAL ordering: when the adaptive dispatcher guesses wrong
When linear_solver=feral (the default) and per-iter wall time is
dominated by the linear solve — typical on dense / quadratically-
coupled KKT systems where iteration counts look reasonable but
seconds-per-iter are high — the fill-reducing ordering choice often
matters more than any other knob. By default, feral_ordering=auto
picks AMD / AMF / METIS from cheap pattern features. This is right
in the common case but can miss badly on a single hard problem.
The safe recipe is to measure the right ordering rather than guess:
pounce problem.nl feral_ordering=auto_race
This runs symbolic factorization on AMD, METIS, SCOTCH and KaHIP and
keeps the one with the smallest factor_nnz. Costs ~4× a single
symbolic pass — paid once per problem because symbolic factorization
is cached across numeric refactorizations with the same pattern, so
the overhead is invisible to the per-iter cost on anything but a
one-iter problem.
feral_ordering=amd (concrete pin) is the right escalation when the
race itself is showing AMD winning consistently — pinning skips the
race entirely on subsequent runs. See the full
feral_ordering table for the
other variants.
feral_singular_pivot_floor: a reduced Hessian that collapses to singular
When to try it
alpha_pr walks down 1/2, 1/4, … 1/128 with a matching ls count,
||d|| grows instead of shrinking, and the run exits with dual_inf
parked a couple of orders of magnitude above tol — or reaches tol
only after a long tail of tiny steps. Feasibility is usually already at
machine precision, and the objective is right to many digits; only the
dual residual will not come down. The lg(rg) column in that tail is
typically churning — small values re-escalating iteration after
iteration — rather than settling.
That combination means the reduced Hessian Zᵀ W Z has become
numerically singular, so the Newton step runs off along a direction
whose curvature is at the noise floor and the line search has no choice
but to cut the step to nothing. It shows up on problems whose solution
set is a manifold rather than a point — degenerate eigenvalue models
are the classic case — and it is not something the exit criteria can
fix, because the iterate handed to them is the problem.
Since #544 pounce already
handles the sharpest form of this automatically: when the KKT is
singular to working precision its inertia count is meaningless, and
feral_inertia_pivot_floor (default 1e-12) routes that case to δ_c
rather than answering an unmeasurable test with δ_w. The recipe below
is for what remains — it attacks the same degeneracy higher up, capping
the null-direction step outright, and on some models that is still
markedly faster.
To confirm before reaching for the knob, dump the KKT systems and look at the smallest pivot:
pounce problem.nl --dump kkt:all --dump-dir /tmp/dump-problem
The knob
pounce problem.nl feral_singular_pivot_floor=1e-8
FERAL force-accepts a pivot at the working-precision floor and still
reports a clean factorization with the right inertia. This option is
pounce’s analog of MA57’s CNTL(2): after a successful factor the
smallest accepted D-block pivot is compared against the floor, and a
factor below it is reported singular so the perturbation handler
escalates δ_w. The default 1e-20 almost never fires — deliberately,
because on a bounded problem a tiny pivot usually comes from the
barrier blocks (Σ_x = z/x as a bound activates) and is both expected
and harmless. Raising it is a per-problem call, not a global default:
airport, jit1 and pooling_rt2stp all converge to Optimal with
smallest pivots between 1e-12 and 1e-21, and a 1e-8 floor would
flag every one of them.
Start at 1e-8 and back off toward 1e-10/1e-12 if the extra
factorizations cost more than they save.
Worked example: eigenb2 (Vanderbei)
110 variables, 55 equality constraints, no bounds at all. Zᵀ W Z’s
smallest eigenvalue falls from 1.4e+02 at iteration 2 to 1.4e-11 by
iteration 36, against ‖W‖ ≈ 1.3e+02. The KKT is singular to working
precision down that tail, so its negative-eigenvalue count stops being
measurable — FERAL reports anywhere from 43 to 64 against an expected
55.
Since #544 the default solve certifies Optimal (before it, this
exited Solved To Acceptable Level in 67 iterations). The knob is now
a speedup rather than a rescue:
| options | iterations | dual inf | exit |
|---|---|---|---|
| (defaults) | 68 | 4.70e-09 | Optimal Solution Found |
feral_singular_pivot_floor=1e-8 | 39 | 1.25e-09 | Optimal Solution Found |
feral_singular_pivot_floor=1e-8 mu_strategy=adaptive | 30 | 4.98e-09 | Optimal Solution Found |
The fixture is committed, so this reproduces without a benchmark corpus:
pounce crates/pounce-cli/tests/fixtures/eigenb2.nl \
feral_singular_pivot_floor=1e-8
Full diagnosis in
dev-notes/issue-541-eigenb2-degenerate-reduced-hessian.md
(issue #541).
Diagnosing before you reach for a knob
Before trying recipes, dump the per-iter diagnostic categories that pounce supports:
pounce problem.nl --dump kkt --dump iterate \
--dump-dir /tmp/dump-problem
The dumps land as JSONL under /tmp/dump-problem/. Two categories
have wired dump sites today:
--dump kkt— KKT residuals and condition-number proxy; large values motivate Ruiz scaling.--dump iterate— primal/dual values; needed to spot whether a small step is bound-snapping or infeasibility-driven.
The
--dump muand--dump restocategories are accepted by the CLI but not yet wired to a dump site, so they currently emit no data. For the μ trajectory and restoration entries/exits, use the Studio queries below (which read the iteration stream from the solve report).
The Studio MCP (pounce-studio) wraps these dumps in higher-level
diagnostic queries (diagnose, find_stalls, restoration_windows),
which is the recommended workflow when iterating on options.
Logs, colors, and machine-readable output
POUNCE routes diagnostics through tracing.
The knobs are environment variables (see
Options › Logging and colored output),
not solver options.
When to try it
- You want more detail than the iteration table shows (which phase fired, why restoration triggered, linear-solver fallbacks).
- A downstream tool (Studio, CI) needs to parse per-iteration data.
- Color is garbling a log file, or you want color forced through a pipe.
The knobs
| Goal | Invocation |
|---|---|
| Verbose, everything | RUST_LOG=debug pounce problem.nl |
| Just the restoration phase | RUST_LOG=pounce::restoration=debug pounce problem.nl |
| Separate logs from results | pounce problem.nl > result.txt 2> solve.log |
| Plain text (no color) | NO_COLOR=1 pounce problem.nl |
| Force color through a pipe | `CLICOLOR_FORCE=1 pounce problem.nl |
| Line-delimited JSON iterations | POUNCE_LOG_FORMAT=json pounce problem.nl 2> iters.jsonl |
Logs go to stderr; the iteration table, final summary, and --dump
output are program output on stdout. The colored table uses a
tiger/rust theme — restoration lines get a kind-dependent background and
the row text reddens as the step length alpha shrinks, so a stalling or
restoration-heavy solve is visible at a glance. When stdout is not a
terminal (or NO_COLOR is set) the table is emitted as plain text with
the same column layout.
Subsystem debug gates
For output finer than RUST_LOG=<target>=debug gives on its own, several
subsystems have a POUNCE_DBG_* gate that switches on extra per-iteration
diagnostics (adaptive-μ oracle decisions, the quality-function σ sweep,
inertia-perturbation choices, restoration internals, KKT-matrix dumps, …).
Most emit at debug level, so pair the gate with the matching RUST_LOG
target. The full table — including which gate takes a value and which
prints straight to stderr — is in
Options › Environment overrides.
Contributing a new recipe
A recipe earns a place here when:
- There is a named, reproducible problem where the recipe
demonstrably helps. Mittelmann benchmark (
benchmarks/mittelmann/nl/) is preferred but any committed.nlworks. - The before/after numbers are captured at
print_level=3or higher and pasted into the worked-example table. - The recipe is not a special case of an existing one. (If your problem needs three knobs together, write one entry; if your problem benefits from a knob already documented here, file a PR to add a second worked example under that entry.)
Open a PR adding to this file with the table populated. The
maintainer-side review checks that the numbers reproduce against the
current main and that the recipe really is a recipe — not a
problem-specific accident.
Benchmarks
The benchmarks/ directory contains comparison harnesses that run
POUNCE against upstream Ipopt across several test suites: the Vanderbei
CUTE-in-AMPL collection, Mittelmann ampl-nlp, CHO parameter estimation,
GasLib pipelines, water-network design, electrolyte thermodynamics,
AC optimal power flow, and large-scale synthetic NLPs. Every suite is
.nl-driven — a directory of AMPL .nl files solved by both pounce
and ipopt.
Common targets:
make benchmark # full sweep: every suite + composite report
make benchmark-report # regenerate benchmarks/BENCHMARK_REPORT.md
make benchmark-cho # one suite at a time
make benchmark-gas
make benchmark-water
make benchmark-mittelmann
make benchmark-vanderbei # Vanderbei CUTE-in-AMPL collection (733 problems)
One suite is deliberately not .nl-driven:
the warm-start benchmark measures the cost of
solving a sequence of related problems, cold versus warm, across all
three of POUNCE’s solve paths. Carrying a working set between solves
needs an in-process handle, so it runs through the Python API instead of
the CLI, and it reports on its own rather than into the composite
report.
The benchmark inputs themselves — the .nl problem files — and the
per-run logs and JSON results are regenerated locally and not tracked in
the repository. See
benchmarks/README.md
for the full list and per-suite details.
The Warm-Start Benchmark
Every other suite in benchmarks/ answers “how fast does POUNCE solve
this problem?” This one answers a different question: when you solve a
sequence of related problems, how much does starting from the previous
answer actually save — and which of POUNCE’s three solvers should you
use?
That question has no meaning for a single isolated solve, which is why it needs its own suite. The unit of work here is a parametric family plus a path: one problem shape, one scripted sweep through its parameter space, solved end to end. MPC horizons, continuation and homotopy, sensitivity sweeps, and design exploration all have this shape.
There is no standard public benchmark for this. The nearest things —
the qpbenchmark test sets,
WARP, the AC-OPF learning datasets —
are either QP-only, interior-point-only, or ship their instances
stripped of the sequence structure that makes warm starting meaningful.
benchmarks/warmstart/README.md has the full survey.
The three solvers under test
POUNCE has three solve paths that can take a sequence, and they warm start in genuinely different ways:
| solver | algorithm / entry point | what it carries between solves |
|---|---|---|
| general NLP filter-IPM | interior-point (the default) | the previous primal-dual point and the converged barrier parameter μ |
| active-set SQP | algorithm = active-set-sqp | the previous working set — which bounds and constraints were active — plus the point |
| convex QP interior point | pounce.solve_qp (solver_selection=qp-ipm) | the previous primal-dual point |
Each runs cold and warm, giving six arms:
| arm | solver | seeded with | runs on |
|---|---|---|---|
cold-ipm | NLP filter-IPM | nothing | every family |
warm-ipm | NLP filter-IPM | previous point + μ | every family |
cold-sqp | active-set SQP | nothing | every family |
warm-sqp | active-set SQP | previous working set + point | every family |
cold-sqp-hom | active-set SQP, homotopy inner QP | nothing | every family |
warm-sqp-hom | active-set SQP, homotopy inner QP | previous working set + point | every family |
cold-qp-ipm | convex QP IPM | nothing | QP families only |
warm-qp-ipm | convex QP IPM | previous primal-dual point | QP families only |
The -hom pair differs from cold-sqp / warm-sqp in exactly one
option, sqp_qp_use_homotopy: the inner QP’s cold solve traces the
§4.2 parametric homotopy — start from the box-only relaxation, tighten
the row bounds along t ∈ [0,1], jump the working set at each event —
instead of the conventional phase-1/phase-2 scheme. It is the algorithm
pounce-qp is named for.
Each warm arm is scored against its own cold counterpart. That
pairing is the whole point: warm-sqp beating cold-ipm would confound
“warm started” with “switched algorithms”, and only the paired
comparison isolates the warm start.
The problems
Fourteen families in the default sweep, each run at three step sizes
(tiny ×0.1, small ×1, large ×4 of its natural per-step parameter
increment), for 42 rows and 855 solves per arm, plus three more in an
opt-in large tier. Warm-start payoff is a function of how far the
problem moved, so a single step size would measure one point on a curve
and call it the answer.
| family | n | m | active-set regime | perturbation enters | curvature |
|---|---|---|---|---|---|
simplex_proj | 20 | 1 | flipping | objective | convex |
moving_bound_qp | 40 | 3 | flipping | variable bounds | convex |
degenerate_corner | 6 | 3 | dual degenerate (a multiplier passes through zero) | objective | convex |
redundant_rows | 6 | 5 | rank-deficient (LICQ fails; duplicated rows) | objective | convex |
degenerate_vertex | 4 | 12 | primal degenerate (12 rows tight in 4 variables) | objective | convex |
hanging_chain | 30 | 15 | flipping contacts | mixed | convex |
rosenbrock_ring | 10 | 1 | one clean activation switch | constraint RHS | nonconvex |
rosenbrock_ring_cycle | 10 | 1 | switch crossed in both directions | constraint RHS | nonconvex |
double_well_chain | 12 | 0 | none — empty active set throughout | objective | nonconvex |
nmpc_vanderpol | 47 | 32 | closed-loop MPC | constraint RHS | nonconvex |
mpc_horizon_10 | 32 | 22 | control saturation | constraint RHS | convex |
mpc_horizon_20 | 62 | 42 | control saturation | constraint RHS | convex |
mpc_horizon_40 | 122 | 82 | control saturation | constraint RHS | convex |
mpc_horizon_80 | 242 | 162 | control saturation | constraint RHS | convex |
plus an opt-in large tier (--tier large), the same MPC carried out
to a scale where the sparse factorization is what the cost is made of:
| family | n | m | nnz(J) |
|---|---|---|---|
mpc_horizon_200 | 602 | 402 | 1402 |
mpc_horizon_400 | 1202 | 802 | 2802 |
mpc_horizon_800 | 2402 | 1602 | 5602 |
The seven mpc_horizon_* families are the same linear-quadratic MPC
problem at seven horizons — only N differs, so reading down them
isolates problem size from every other property. The parameter walks the
initial state around a circle, which keeps every step about as hard as
the last while rotating the set of saturated controls. Nothing dense is
ever built for them: they declare their block-banded Jacobian and
diagonal Hessian structurally, and the convex-QP arm receives sparse
matrices, because at N = 800 a dense Hessian alone would be 46 MB
rebuilt every iteration and passing dense data to the QP solver is
60–80× slower by its own diagnostic — which would have made the QP arm
look bad for a reason that has nothing to do with the QP arm.
The three degeneracy families cover the three distinct ways an
active-set QP meets degeneracy, which are not interchangeable:
degenerate_corner fails strict complementarity (a zero multiplier),
redundant_rows fails LICQ (duplicated equality rows throughout, and a
duplicated inequality pair that activates together partway along the
path), and degenerate_vertex is primally degenerate (12 constraints
tight at a 4-variable vertex, so the ratio test is a mass of ties —
the case Harris’s two-pass test and GMSW EXPAND exist for). The
benchmark reports that pounce prunes that vertex’s active set to its
maximal independent subset: |A| never exceeds 4 of the 12 tight rows.
Apart from the horizon sweep, the families are deliberately small and analytic: this is a measurement of warm-start behavior, and small problems measure it cleanly.
How a result is produced
Three rules make the arms comparable:
- Every arm sees the identical parameter sequence. For
nmpc_vanderpol, whose path depends on its own solutions, the sequence is recorded once from the reference arm and replayed for the others. - Step 0 of a warm arm is a cold solve — there is nothing to warm from — and is excluded from the speedup ratios while still counting in the totals.
- Every step is checked. A step must return success, actually achieve a small KKT residual and be feasible (verified by the harness, not taken from the solver’s status), and not land on a worse optimum than the reference. A warm start that converges quickly to the wrong answer is a failure, not a win.
In the run reported below, every step of every arm passed — 42 rows, 6228 solves (855 steps for each of the six callback arms, 549 for the two QP-only ones), with zero correctness failures.
Results
Run on POUNCE 0.9.0, tol = 1e-8, one machine, all 42 rows, on a build
that includes the fix for
#428 — which this suite
found and which moved most of the SQP numbers below.
Does warm starting pay?
Totals across all 855 steps of all 42 rows:
| arm | Σ outer iterations | Σ solve time | incorrect steps |
|---|---|---|---|
cold-ipm | 10288 | 7.71 s | 0 |
warm-ipm | 3628 | 3.55 s | 0 |
cold-sqp | 4238 | 30.31 s | 0 |
warm-sqp | 1501 | 3.46 s | 0 |
Both solvers cut outer iterations by roughly 3×. But for the active-set SQP that number badly understates the effect — its wall time falls by 8.8× on the same iteration count — for a reason worth understanding before reading any further.
The metric trap: outer iterations hide the SQP’s warm start
On a problem whose subproblem is already a QP, the SQP outer loop
terminates in one iteration whether or not it was warm started. The
work a working-set warm start actually saves is inside the QP
subproblems, and it is reported separately as
info["n_qp_ws_changes"] — active-set changes (adds + drops) summed
over the step QPs.
The two extremes make the point:
| family | SQP outer iterations, cold→warm | QP active-set changes, cold→warm |
|---|---|---|
simplex_proj @ tiny (a QP) | 1.00× — flat | 16.0× (285 → 0) |
double_well_chain @ tiny (unconstrained) | 8.33× | 1.00× (0 → 0) |
They are mirror images. On a QP, everything happens inside; with no
constraints there is no working set to carry, so the entire effect is in
the outer loop and comes from the primal point alone. Neither column
alone summarizes this benchmark. double_well_chain exists precisely
to be that zero mark.
Warm-start effect per family
SQP is the ratio of inner QP active-set changes (raw totals in
parentheses); IPM is the ratio of outer iterations. Higher is better;
worse counts steps where warm cost more than cold.
| family | scale | SQP cold→warm | worse | IPM cold→warm | worse |
|---|---|---|---|---|---|
simplex_proj | tiny | 16.00× (285→0) | 0 | 5.05× | 0 |
simplex_proj | small | 17.46× (313→0) | 0 | 4.42× | 0 |
simplex_proj | large | 18.62× (335→0) | 0 | 4.17× | 0 |
moving_bound_qp | tiny | 6.45× (104→0) | 0 | 5.01× | 0 |
moving_bound_qp | small | 11.53× (207→0) | 0 | 2.00× | 0 |
moving_bound_qp | large | 13.74× (467→19) | 0 | 1.93× | 0 |
degenerate_corner | tiny | 1.87× (19→1) | 0 | 4.67× | 0 |
degenerate_corner | small | 1.87× (19→1) | 0 | 3.56× | 0 |
degenerate_corner | large | 1.98× (26→3) | 0 | 3.08× | 0 |
redundant_rows | tiny | 2.27× (42→0) | 0 | 5.25× | 0 |
redundant_rows | small | 3.16× (73→2) | 1 | 4.08× | 0 |
redundant_rows | large | 5.50× (114→2) | 1 | 3.54× | 0 |
degenerate_vertex | tiny | 2.16× (46→4) | 1 | 4.05× | 0 |
degenerate_vertex | small | 2.23× (50→4) | 1 | 3.17× | 0 |
degenerate_vertex | large | 2.16× (46→4) | 1 | 2.97× | 0 |
hanging_chain | tiny | 4.00× (57→0) | 0 | 1.25× | 0 |
hanging_chain | small | 4.44× (67→0) | 0 | 1.54× | 0 |
hanging_chain | large | 6.84× (124→1) | 0 | 0.85× | 17 |
rosenbrock_ring | tiny | 2.37× (30→1) | 0 | 11.37× | 0 |
rosenbrock_ring | small | 2.26× (28→1) | 0 | 8.75× | 0 |
rosenbrock_ring | large | 1.72× (18→1) | 0 | 6.73× | 0 |
rosenbrock_ring_cycle | tiny | 2.32× (29→1) | 0 | 9.16× | 0 |
rosenbrock_ring_cycle | small | 2.25× (28→1) | 0 | 8.62× | 0 |
rosenbrock_ring_cycle | large | 1.52× (17→4) | 0 | 6.14× | 0 |
double_well_chain | tiny | 1.00× (0→0) | 0 | 3.00× | 0 |
double_well_chain | small | 1.00× (0→0) | 0 | 2.29× | 0 |
double_well_chain | large | 1.00× (0→0) | 0 | 2.14× | 0 |
nmpc_vanderpol | tiny | 18.80× (366→2) | 0 | 3.63× | 0 |
nmpc_vanderpol | small | 12.55× (348→14) | 0 | 1.96× | 0 |
nmpc_vanderpol | large | 7.33× (425→72) | 0 | 1.05× | 8 |
mpc_horizon_80 | tiny | 54.75× (1105→2) | 0 | 5.17× | 0 |
mpc_horizon_80 | small | 42.98× (1552→21) | 0 | 2.23× | 0 |
mpc_horizon_80 | large | 8.21× (1176→123) | 0 | 1.12× | 3 |
Payoff tracks active-set churn, not problem size
Read down any family and the pattern is the same: the further the
problem moves per step, the less a warm start buys. churn is the
mean number of working-set entries that change between consecutive
steps.
| family | churn/step at tiny → large | SQP payoff at tiny → large |
|---|---|---|
nmpc_vanderpol | 0.21 → 2.95 | 18.8× → 7.3× |
mpc_horizon_80 | 0.21 → 5.58 | 54.8× → 8.2× |
moving_bound_qp | 0.05 → 1.63 | 6.5× → 13.7× |
hanging_chain | 0.00 → 0.47 | 4.0× → 6.8× |
simplex_proj | 0.00 → 0.21 | 16.0× → 18.6× |
The two MPC families are the clearest cases: a 14× and 27× increase in
churn costs a 2.6× and 6.7× reduction in payoff. This is the practical
rule — warm starting pays in proportion to how stable your active set
is, and problem size has little to do with it. (The families at the
bottom, whose churn stays below one entry per step even at large,
show the opposite sign: there the warm start stays essentially exact
while the cold solve gets harder, so the ratio rises.)
Warm starting can make things worse
Two rows show it, both at the largest step size:
hanging_chain @ large,warm-ipm: 0.85× — the warm-started IPM needed more iterations than a cold solve on 17 of 19 steps. The previous solution sits exactly on the constraint boundary, which is the worst possible starting point for a barrier method when the active set has since moved.nmpc_vanderpol @ large,warm-ipm: 8 of 19 steps worse, where a 4× control interval makes the plant state jump far enough that the previous point is a poor guess. The SQP arm no longer regresses on this row (it did before #428 was fixed), but its payoff still falls from 18.8× to 7.3× across the same span.
This is why the benchmark reports regressions per step rather than only a mean. A single averaged speedup would hide both.
How it scales: the MPC horizon sweep
The same linear MPC at four horizons, warm/cold wall-time ratio — below 1.00 means warm starting won:
| N | n | mean |A| | tiny SQP / IPM | small SQP / IPM | large SQP / IPM |
|---|---|---|---|---|---|
| 10 | 32 | 31.0 | 0.17 / 0.37 | 0.17 / 0.38 | 0.24 / 0.69 |
| 20 | 62 | 61.2 | 0.08 / 0.37 | 0.09 / 0.70 | 0.12 / 0.74 |
| 40 | 122 | 118.3 | 0.04 / 0.32 | 0.04 / 0.59 | 0.11 / 0.85 |
| 80 | 242 | 204.1 | 0.02 / 0.26 | 0.03 / 0.49 | 0.10 / 0.86 |
Read down the SQP columns: the warm start does not merely survive the
horizon, it improves with it — 0.17 → 0.02 at tiny, and even at
the largest perturbation 0.24 → 0.10. At N = 80 a warm-started solve is
50× faster than a cold one at small steps and still 10× faster at large
ones. The reason is that cold cost grows with the problem while warm
cost is set by how far the problem moved, which is a property of the
path, not of n.
Reading across, the familiar pattern holds: bigger steps cost more (0.02 → 0.10 at N = 80), because more of the active set has to change.
The mechanism is in the working sets. The fraction of the active set
that changes per step is essentially horizon-independent — about 3% at
large for every N, by construction, since the same angular
perturbation moves proportionally the same constraints:
| N | mean |A| | churn/step at large | as a fraction | SQP inner work, cold → warm |
|---|---|---|---|---|
| 10 | 31.5 | 1.05 | 3.3% | 242 → 14 |
| 20 | 61.2 | 2.26 | 3.7% | 486 → 41 |
| 40 | 118.8 | 4.21 | 3.5% | 893 → 76 |
| 80 | 203.1 | 5.58 | 2.7% | 1176 → 123 |
Absolute churn does grow with the problem (1.05 → 5.58 changes per step), and the warm arm’s inner work grows with it — but the cold arm’s grows faster, which is why the ratio improves. The rule stands as first stated: payoff tracks how much the active set moves, and problem size has little to do with it.
An earlier revision of this page reported the opposite — a crossover where warm-started SQP turned harmful above N = 20, reaching 2.57× at N = 80. That was #428, found by the large tier below and now fixed; the numbers above are the same measurement on the fixed solver.
At large scale: where the benchmark found a defect
Carrying the same MPC out to n = 2402 is what exposed #428, and the before/after is the clearest single result in the suite.
At default settings the warm-started SQP did not produce an answer
on the large tier: warm-sqp and warm-sqp-hom returned
Maximum_Iterations_Exceeded with zero outer iterations on 7 of 8 steps
at every one of N = 200/400/800, leaving x at the warm-start point,
while every other arm solved all 8 cleanly.
Inner working-set changes for one step, before and after the fix:
| N | n | m | cold | warm, before | warm, after |
|---|---|---|---|---|---|
| 10 | 32 | 22 | 11 | 0 | 0 |
| 20 | 62 | 42 | 25 | 43 | 1 |
| 40 | 122 | 82 | 48 | 1 | 1 |
| 80 | 242 | 162 | 66 | 164 | 3 |
| 200 | 602 | 402 | 66 | 403 | 3 |
| 400 | 1202 | 802 | 66 | 795 | 3 |
| 800 | 2402 | 1602 | 66 | 1589 | 3 |
The warm arm was Θ(m) — 1589 pivots at N = 800, 24× the cost of not warm starting at all. It is now flat at 3 across a 75× range of m, at the same optimum to 1e-11.
The cause was not gradual erosion but a step function in how far the
problem moved. Before, at N = 200, zero changed entries of the true
active set cost 0 pivots and one cost 400. solve_with_working_set
pins the hinted rows to their new boundaries; once the active set has
moved, that pinned point violates some other row by roughly the distance
the parameter moved, and a feasibility pre-check in solve routed the
whole thing to elastic phase-1 — whose recovery re-solve starts from a
cold working set. The hint was discarded rather than repaired. The fix
repairs it: the violated rows are known, so they are pinned too and the
KKT re-factored, keeping the |A| − 1 entries the hint got right. Now the
cost tracks the movement, as it should:
| Δφ | entries of the true active set that changed | warm pivots, before | after |
|---|---|---|---|
| 0.002 | 0 | 0 | 0 |
| 0.005 | 0 | 0 | 0 |
| 0.01 | 1 | 400 | 0 |
| 0.02 | 2 | 401 | 1 |
| 0.05 | 4 | 403 | 3 |
On the large tier at default settings, the whole picture inverts. Every arm is now correct on every step, and the SQP goes from unusable to the fastest thing on the board:
| N | n | warm-sqp wall vs its cold twin | warm-ipm | warm-qp-ipm |
|---|---|---|---|---|
| 200 | 602 | 0.03 | 0.58 | 0.48 |
| 400 | 1202 | 0.03 | 0.57 | 0.54 |
| 800 | 2402 | 0.02 | 0.41 | 0.50 |
Inner active-set work drops 514 → 11 per path (46.7×) identically at all three horizons. At n = 2402 a warm-started SQP sweep takes 1.34 s against 12.12 s cold.
This also revises the caveat in Active-Set SQP & Warm Starts about preferring the IPM for “large-scale problems with thousands of active inequalities”. With #428 fixed, this problem shows no such crossover up to 1645 active constraints — the active-set path wins by 30–50× there.
The parametric homotopy: a sharply mixed trade
The -hom arms differ from their twins in one option, so the delta is
the homotopy alone. Comparing inner QP active-set work on the cold
arms, where the homotopy actually engages (warm inner QPs mostly skip
the cold path):
| family | conventional → homotopy, cold inner work | ratio across the three scales |
|---|---|---|
simplex_proj | 978 → 1400 | 0.63–0.74× |
moving_bound_qp | 793 → 587 | 1.02–3.33× |
degenerate_corner | 69 → 30 | 1.91–2.73× |
redundant_rows | 247 → 30 | 3.91–11.27× |
degenerate_vertex | 154 → 132 | 1.09–1.25× |
hanging_chain | 257 → 257 | 1.00× |
rosenbrock_ring | 79 → 79 | 1.00× |
rosenbrock_ring_cycle | 77 → 77 | 1.00× |
double_well_chain | 0 → 0 | — (no inner QP work at all) |
nmpc_vanderpol | 1205 → 3575 | 0.33–0.36× |
mpc_horizon_10/20/40/80 | 9179 → 29839 | 0.25–0.37× |
| all 42 rows | 13038 → 36006 | 0.36× |
Above 1.00× the homotopy did less work. The split is not random — it tracks exactly what the homotopy was built for:
- It wins on degenerate geometry.
redundant_rows, whose active set is linearly dependent, is its best case by a wide margin, and it improves with perturbation size (4.2× → 12.3× fromtinytolarge) because the conventional cold solve degrades there while the homotopy does not.degenerate_corneranddegenerate_vertexfollow the same pattern. This is the netlib-like geometry #412 reported it gaining 20 problems on. - It loses badly on well-conditioned MPC-shaped QPs. Every
mpc_horizon_*family andnmpc_vanderpolcost about 3× the inner work with the homotopy on, consistently across scales, andsimplex_projcosts ~1.4×. - It is inert on four families — exactly 1.00×, because their inner QPs never take the cold path far enough for it to matter.
Net over all 42 rows it does 2.8× more inner work (0.36×), because the losers are also the largest problems. That is an argument for keeping it off by default on the SQP path and reaching for it on degenerate models, which is what the option now allows.
Three-way: which solver for a sequence of QPs?
Five families are literally convex QPs, so all three solvers can take them. Interior-point iterations and active-set pivots are not the same unit of work, so the like-for-like column is each solver against itself:
| family | scale | convex QP IPM cold→warm | NLP IPM cold→warm | SQP cold→warm (inner) | fastest warm arm |
|---|---|---|---|---|---|
simplex_proj | tiny | 160→46 | 182→28 | 300→15 | warm-qp-ipm |
simplex_proj | small | 162→75 | 190→38 | 328→15 | warm-qp-ipm |
simplex_proj | large | 173→96 | 200→45 | 350→15 | warm-sqp |
moving_bound_qp | tiny | 202→94 | 228→43 | 109→5 | warm-sqp |
moving_bound_qp | small | 195→121 | 224→116 | 212→5 | warm-sqp |
moving_bound_qp | large | 229→125 | 240→126 | 472→24 | warm-sqp |
degenerate_corner | tiny | 196→74 | 223→41 | 20→2 | warm-qp-ipm |
degenerate_corner | small | 174→77 | 170→40 | 20→2 | warm-qp-ipm |
degenerate_corner | large | 177→98 | 177→53 | 29→6 | warm-sqp |
redundant_rows | tiny | 189→75 | 249→41 | 42→0 | warm-qp-ipm |
redundant_rows | small | 173→80 | 207→44 | 82→11 | warm-qp-ipm |
redundant_rows | large | 171→83 | 176→43 | 123→11 | warm-qp-ipm |
degenerate_vertex | tiny | 215→73 | 192→39 | 50→8 | warm-qp-ipm |
degenerate_vertex | small | 199→87 | 149→38 | 54→8 | warm-sqp |
degenerate_vertex | large | 195→92 | 141→38 | 50→8 | warm-qp-ipm |
Geometric-mean wall time over those fifteen rows:
| cold-ipm | cold-sqp | cold-qp-ipm | warm-ipm | warm-sqp | warm-qp-ipm |
|---|---|---|---|---|---|
| 99.1 ms | 62.5 ms | 61.6 ms | 50.1 ms | 30.9 ms | 29.5 ms |
The dedicated convex solver is fastest on 9 of the 15 rows and the
active-set SQP on the other 6, with the SQP taking the rows where the
active set churns hardest. The two are within 5% of each other on the
aggregate — on a problem that really is a QP, either warm-started path
is a reasonable default. Note that this ranking is recent: before
#417 was fixed the
convex solver’s warm start was capped at 1.2–1.5× and warm-sqp led 8
of 9 rows.
What to take from this
- For a sequence of convex QPs —
solve_qpwarm-started with the previous result. It leads on most rows and needs no callbacks. - For a general NLP whose active set is stable between solves —
algorithm = active-set-sqpcarrying the working set. This is where the largest effects live (up to 55× less inner active-set work, and a 50× wall-time win on the largest default horizon), and the whole reason the active-set path exists. - Scale is not the thing to worry about; movement is. On the horizon sweep the SQP’s warm/cold ratio improves with N (0.17 → 0.02 at small steps), because cold cost grows with the problem while warm cost is set by how far the active set moved. At n = 2402 a warm-started sweep runs 30–50× faster than cold. What costs you is a large step, not a large problem.
- For a problem with no active set to speak of — unconstrained, or
with constraints that never bind — the warm start still helps, but
only through the primal point. Either solver is fine; the working set
buys nothing (
double_well_chain: 0 → 0). - When each step moves the problem a long way — check whether warm starting is helping at all. It can cost more than a cold solve, and the IPM path is more exposed to this than the SQP path.
- On degenerate models — dependent rows, vertices where many
constraints meet — try
sqp_qp_use_homotopy. It cuts inner active-set work by 2–12× on the degeneracy families and is the algorithm the active-set engine was designed around. Leave it off for MPC-shaped problems, where it roughly doubles the work. - Always verify. A fast wrong answer is the failure mode that matters, which is why the harness re-checks KKT residuals and objectives itself rather than trusting a status code.
See Active-Set SQP & Warm Starts for how to drive the warm-start APIs, and Initialization and Warm Starts for the interior-point side.
Defects this benchmark found
All three are fixed. They are listed because they show what the suite is for — two of them lived in the same configuration (nonconvex, indefinite Hessian, nothing active) that no other suite exercised:
| issue | what it was |
|---|---|
| #416 | Exact-Hessian SQP spent its entire inner-QP iteration budget making zero working-set changes; a budget of 20 gave bit-identical answers ~9× faster. Fixed in #419. |
| #423 | The #416 fix regressed unconstrained problems: with nothing able to block a negative-curvature direction, the solve died at iteration 1. Caught by double_well_chain on its first run against the new build. Fixed in #424. |
| #417 | The convex QP warm start left ~40% of its iterations unclaimed — not from the seeding but from a fraction-to-boundary parameter pinned at 0.95. Fixed in #422. |
| #428 | The SQP’s working-set hint was discarded — not repaired — the moment the active set moved by one entry, costing one inner pivot per constraint row (1589 at n = 2402, against 3 now). Invisible below N ≈ 80; at n ≥ 602 it stopped the warm-started solve returning an answer at all. Found by the large tier on its first run, fixed in #429. |
sqp_qp_use_homotopy was a no-op | Found while adding the -hom arms: the option was registered but apply_qp_subproblem_options never read it, so setting it on the SQP path did nothing while its documentation described what it would do. The inverse of #360 (read-but-unregistered), and invisible to that issue’s guard, which only checked one direction. Fixed here, with a bidirectional guard. |
Running it
The harness drives POUNCE in-process through the Python API, so it needs the extension built:
cd python && maturin develop --release
Then:
make -C benchmarks warmstart-selftest # finite-difference checks, no solver needed
make -C benchmarks warmstart-run # full sweep -> results.json + results.md
make -C benchmarks warmstart-quick # 3 families, one scale
or, for a narrower run:
python -m warmstart.run --families simplex_proj,nmpc_vanderpol --scales large -v
python -m warmstart.run --arms cold-sqp,warm-sqp --tol 1e-10
python -m warmstart.run --tier large --scales small # n = 602 → 2402
--tier large is opt-in because a single active-set solve there takes
seconds; --tier all runs both.
Results land in benchmarks/warmstart/results.json (every step of every
arm) and results.md. Both are regenerated per run and gitignored.
Adding a problem family or a new solver is documented in
benchmarks/warmstart/README.md;
nothing outside adapters/ imports a solver, so the families and the
protocol are reusable against any solver with a warm-start API.
Limits of these numbers
- Mostly small problems (n ≤ 47 outside the horizon sweep, which
reaches n = 242 by default and n = 2402 with
--tier large). The sweep gives one scaling curve on one problem shape; it is not a substitute for a large-scale study across problem classes, and the scaling it reports is specific to this MPC. - The large tier is one problem class. Linear-quadratic MPC has a particular structure — banded, mostly equalities, a large active set that barely moves — and #428 was found there. Whether a large problem with a different sparsity pattern behaves the same way is untested, and is the obvious next family to add.
- A published conclusion here has already been wrong once. The horizon sweep’s crossover held for one revision of this page before the large tier showed it was a solver defect. The measurements were right and the mechanism inferred from them was not; treat the explanations here as the current best reading of the numbers rather than as established behavior.
- Wall time carries Python callback overhead for the four callback-driven arms. Iteration and active-set-change counts are the primary measurements; times are a cross-check, and vary 10–30% between runs on the same machine.
- The QP arms are handed matrix data once per step, where the other arms re-evaluate the model every iteration. That is a real advantage of the QP path on a QP, not an artifact, but it does mean the wall times are not measuring identical work.
- One machine, one run. Iteration counts are deterministic and reproducible; timings are not.
Color Theme
POUNCE’s terminal output uses one tiger / rust / warm palette across
every colored surface — the iteration table, the branded wordmark, and
the interactive debugger. This page is the single reference for what the
colors mean; the palette itself lives in
pounce-common::style
(a pure, unit-tested module — no I/O, no globals).
For the environment variables that turn color on/off (NO_COLOR,
CLICOLOR_FORCE, RUST_LOG, POUNCE_LOG_FORMAT) see
Solver Options → Logging and colored output.
The palette
| Name | Hex | Role |
|---|---|---|
ALPHA_COOL | #000000 | iteration-row text at α = 1 (full Newton step) |
ALPHA_HOT | #cc2200 | iteration-row text at α → 0 (stalling); molten-claw base |
TAN | #8a6d3b | restoration soft-stay row background (s) |
AMBER | #b56a12 | restoration soft-exit row background (S) |
RUST_DEEP | #6e260e | restoration hard row background (R / resto-phase rows) |
CREAM | #f5e6c8 | restoration-row text at α = 1 |
BRIGHT_YEL | #ffe03a | restoration-row text at α → 0; molten-claw top |
TIGER_ORANGE | #e87a1e | WARN logs, banner accents, molten-claw mid |
Two further surfaces reuse these or a small extension:
| Name | Hex | Role |
|---|---|---|
| steel-hi → steel-lo | #d2d6dc → #5c6068 | wordmark letter sheen, top row → bottom row |
| gold | #ffb000 | debugger banner highlight (interior-point debugger, help) |
| dim | #7a7e88 | debugger banner gloss text |
Where the colors appear
The iteration table
Two orthogonal channels encode solver state on each row:
- Background = restoration kind, keyed off the row’s
alpha_primal_chartag:ssoft-stay → tan,Ssoft-exit → amber,Rhard (and the dedicated restoration phase’sr-suffixed rows) → deep rust.- Normal (non-restoration) rows have no background.
- Tiny-step tags (
t/T) deliberately get no background — that stall is shown by the foreground instead.
- Foreground = a smooth gradient on the primal step length α ∈ [0, 1]
(a visual stalling cue):
- Normal rows: black (α = 1, full step) → hot red (α → 0).
- Restoration rows: cream (α = 1) → bright yellow (α → 0), so the text stays legible on the dark background.
- α is clamped to
[0, 1]; a non-finite α is treated as a full step (no false stalling alarm).
So at a glance: a dark row means restoration (its shade tells you which kind), and redder / yellower text means a shorter step (the solver is struggling to move).
The branded wordmark (pounce logo)
Printed atop a normal solve and at the top of the debugger REPL. The
POUNCE block letters carry a top-lit steel sheen (light silver
#d2d6dc at the top row fading to dark steel #5c6068 at the bottom),
and three diagonal molten claw slashes rake across them, glowing
bright yellow → tiger-orange → deep red top-to-bottom — the project
logo’s forged-metal-with-lava look.
The interactive debugger
The REPL open banner (--debug) reuses the same wordmark, then a command
cheat-sheet whose shortcut keys are tiger-orange, the
interior-point debugger line and the help hint are gold, and the
descriptive gloss is dim grey. Pause banners and command output are
otherwise uncolored. (--debug-json emits no color — its stdout is a
pure JSON channel.)
viz kkt / viz L open in the external Plotly viewer
(pounce-dbg-viz),
which is a separate visual language: the sparse-matrix heatmaps use a
diverging red–blue scale keyed on entry value (sign + magnitude),
not the terminal palette.
Logs
WARN-level log lines (on stderr) take the tiger-orange accent;
other levels use the subscriber’s defaults.
Terminal support & downgrade
- Truecolor (24-bit) is used when the terminal advertises it via
COLORTERM— every color above is emitted as exact RGB. - 256-color terminals get a graceful fallback: each RGB color snaps
to the nearest xterm 6×6×6 cube color (
downgrade/nearest_ansi256). The theme still reads correctly, just quantized.
When color is emitted
Color is opt-out and TTY-aware:
- The iteration table is colored only when stdout is a terminal
(via
anstream::AutoStream, which strips escapes from redirected output while keeping identical column alignment). - The debugger banner is colored only when stderr is a terminal.
NO_COLOR(any value) disables color everywhere;CLICOLOR_FORCEforces it even into a non-terminal sink. See Solver Options.
Because the policy is consistent, redirected logs/output are always plain text — safe to diff, grep, and ingest.
For contributors
Add or change colors in pounce-common::style, never with inline ANSI:
the constants, the α-gradient (alpha_gradient_rgb), the restoration
mapping (resto_background_rgb), the composed iteration_row_style, and
the truecolor downgrade all live there and are unit-tested without a
TTY. Print sites style through anstyle + anstream (or, for the
debugger banner on stderr, gate on stderr().is_terminal() and
NO_COLOR). Keep the two iteration-table channels — background =
restoration kind, foreground = step length — orthogonal.
Acknowledgments
POUNCE’s nonlinear-programming core is a Rust port of Ipopt, the interior-point nonlinear programming solver by Andreas Wächter, Lorenz T. Biegler, and the COIN-OR community. Its algorithm, console output, and option semantics are modeled directly on that codebase, which is released under the EPL-2.0.
It is a sibling of ripopt, an earlier memory-safe interior-point NLP optimizer in Rust by the same author (DOI 10.5281/zenodo.19542664).
Convex solver inspiration
The specialized convex conic solver (pounce-convex; see
Convex Solver) is a pure-Rust port of ideas — not a
wrapper — from two reference projects, gratefully acknowledged:
- Clarabel by Paul Goulart and Yuwen Chen (University of Oxford). POUNCE’s homogeneous-free conic interior-point design — a quadratic objective handled directly over a product of symmetric cones, with Nesterov–Todd scaling for the second-order cone and a diagonal-plus-rank-1 sparse KKT representation — follows Clarabel’s approach. Clarabel is itself a pure-Rust solver; POUNCE shares the spirit but is an independent implementation.
- PaPILO, the presolving library of SCIP (the Zuse Institute Berlin optimization suite). POUNCE’s transaction-stack presolve with full primal and dual postsolve — forcing constraints, dominated columns, bound tightening with global dual recovery, parallel/duplicate rows, iterated to a fixpoint — is modeled on PaPILO’s catalog and postsolve discipline.
Contributors
- David Bernal Neira (@bernalde)
designed and prototyped the auxiliary-equality preprocessing pass
in ripopt PR #32.
POUNCE’s
pounce-presolve::auxiliaryPhase-0 orchestrator (issue #53) is a port of that work — Hopcroft-Karp matching, Dulmage-Mendelsohn partition, Tarjan SCC, block-triangular reduction, damped-Newton block solver, reduction frame with multiplier recovery — and ships with thetutorial_flow_density{,_perturbed}.nlandgaslib11_steady.nltest fixtures David vendored. - Milan Rother (@milanofthe)
suggested the boundary value problem solver and the tritium
gas-liquid-contactor (GLC) test problem behind
docs/src/bvp.mdandpython/examples/glc_feral_vs_scipy.py. The GLC model is adapted from pathsim-chem (src/pathsim_chem/tritium/glc.py, MIT License, Copyright (c) 2025 PathSim).
Key references
- Wächter, A., Biegler, L.T. “On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming.” Mathematical Programming 106(1), 25–57 (2006). DOI 10.1007/s10107-004-0559-y — the algorithm POUNCE implements.
- Wächter, A., Biegler, L.T. “Line search filter methods for nonlinear programming: Motivation and global convergence.” SIAM Journal on Optimization 16(1), 1–31 (2005). DOI 10.1137/S1052623403426556
- Wächter, A., Biegler, L.T. “Line search filter methods for nonlinear programming: Local convergence.” SIAM Journal on Optimization 16(1), 32–48 (2005). DOI 10.1137/S1052623403426544
- Fletcher, R., Leyffer, S. “Nonlinear programming without a penalty function.” Mathematical Programming 91(2), 239–269 (2002). DOI 10.1007/s101070100244 — the filter concept underlying the line search.
- Pirnay, H., López-Negrete, R., Biegler, L.T. “Optimal sensitivity
based on IPOPT.” Mathematical Programming Computation 4(4),
307–331 (2012). DOI
10.1007/s12532-012-0043-2
— the sIPOPT method behind
pounce-sensitivity. - Duff, I.S. “MA57—a code for the solution of sparse symmetric
definite and indefinite systems.” ACM Transactions on Mathematical
Software 30(2), 118–144 (2004). DOI
10.1145/992200.992202 — the
optional
ma57linear-solver backend. - Goulart, P.J., Chen, Y. “Clarabel: An interior-point solver for
conic programs with quadratic objectives.” (2024).
arXiv:2405.12762 /
Clarabel.rs — the
conic interior-point design behind
pounce-convex. - Gleixner, A., Gottwald, L., Hoen, A. “PaPILO: A Parallel Presolving
Library for Integer and Linear Optimization with Multiprecision
Support.” INFORMS Journal on Computing 35(6), 1329–1341 (2023). DOI
10.1287/ijoc.2022.0171 —
the presolve catalog and dual-postsolve model behind
pounce-convex::presolve. - Domahidi, A., Chu, E., Boyd, S. “ECOS: An SOCP solver for embedded systems.” European Control Conference (2013), 3071–3076. DOI 10.23919/ECC.2013.6669541 — the sparse second-order-cone KKT representation.
- Amos, B., Kolter, J.Z. “OptNet: Differentiable Optimization as a
Layer in Neural Networks.” ICML (2017), 136–145.
arXiv:1703.00443 — the implicit
differentiation behind the
pounce.jaxconvex layers. - Wilkinson, M.D. et al. “The FAIR Guiding Principles for scientific data management and stewardship.” Scientific Data 3, 160018 (2016). DOI 10.1038/sdata.2016.18 — the provenance model behind the JSON solve report.