The pivot: from ML potentials to differentiable optimization
A decade of ML surrogates for DFT taught me differentiable programming (JAX, PyTorch)
Argued in a perspective that AD is the next modeling paradigm in ChemE
This year's goal: an end-to-end differentiable optimization stack
differentiable programs
differentiable solvers
differentiable surrogates with uncertainty
I learn by doing, and agents made it possible to do this in a year
Some of what I have built in 2025-26 (repo created)
Dec 2025
difflow: flowsheets in JAX
Feb 2026
ripopt (abandoned), discopt, jaxsr
Apr 2026
feral: sparse LDLᵀ
May 2026
pounce: interior point
Oct 2026
jax-go: global opt of JAX code
J. R. Kitchin, V. Alves, C. D. Laird, Nat. Chem. Eng. 2, 11 (2025), doi:10.1038/s44286-024-00170-x
The differentiable optimization landscape · hover an icon for details and links
Programs: difflow, optimization of ordinary Python
Write a flowsheet as regular Python + JAX; derivatives come from AD
Sequential modular solve with JAX solvers (optimistix, lineax, diffrax)
Same model in residual form → a simultaneous solve with pounce
Unit models can be implicit blocks inside an equation-oriented model
Plugins: bio, rare-earth extraction, carbon capture, gas networks, refinery, power
In one cascade study, matched IDAES within 0.3%
Why not Pyomo? I wanted to understand it by building it, do new things that are not practical in Pyomo, and programs cover models with no closed-form equations
github.com/jkitchin/differentiable-flowsheets
How a chemical engineer ends up writing solvers
I want sensitivities of a steady-state model (e.g., degree of rate control)
Sensitivities at an optimum come from the KKT system
That needs a robust interior point NLP solver
That needs a sparse symmetric indefinite factorization with inertia
... so in February I challenged myself: build Ipopt in Rust, with a new linear solver
I expected it to fail and show the limits of agentic coding. It kept working.
feral → pounce: a pure-Rust interior point family
feral: sparse LDLᵀ, certified inertia
pounce: NLP / QP / conic
NLP core began as a faithful port of Ipopt: same filter line search, logs, options
Grew into a family: active-set SQP, LP/QP, SOCP, exp/power cones, small SDP, SOS for polynomials
sIPOPT-style sensitivity and reduced Hessian, end to end
pip install pounce-solver, Pyomo plugin, C ABI, WebAssembly (solve in the browser)
Default build is pure Rust: no Fortran, no HSL; MA57 optional
From GitHub, Oct 28 2026. pounce: ~19 MB of Rust source.
The workflow that worked
Literature first: curated LLM wikis (crucible) on the optimization literature
Benchmark-driven: ~1,200 NLPs, ~180k KKT matrices; the benchmarks are the truth
Phases with testable goals; CI on every push
A notebook of results and failed attempts
Adversary agents to break the code
Claude Code (Opus 4.5 → 4.8), Max plan, $100-200/month
This is v2. ripopt (Feb) became too hard to improve; lessons from it shaped feral + pounce.
Differentiable solvers: sensitivities from the implicit function theorem
Steady state F(θ;g)=0, with gi=lnki:
dgidθ=−(∂θ∂F)−1∂gi∂F
∂F/∂θ is the Jacobian pounce already factored
Other partials: AD of the same rate law, no new model
No finite differences, no re-solve per parameter
Example: degree of rate control for a 17-step propylene oxidation mechanism, in seconds in Python (previously a differentiable stiff integrator in Julia)
Y. Yang, S. K. Achar, J. R. Kitchin, AIChE J. 68, e17653 (2022), doi:10.1002/aic.17653
Equations: discopt, an algebraic modeling language and MINLP solver
Python modeling API, Rust core; AD tape shared with pounce
Spatial branch and bound over convex relaxations (McCormick, RLT, univariate envelopes), in-house simplex for node LPs; HiGHS for pure MILP
What is the best condition? Active learning: surrogate + EI/UCB, next batch
Does this factor matter? Factorial, mixture, Latin square designs, ANOVA
How well can I estimate parameters? D/A/E-optimal design from the Fisher information (JAX autodiff), identifiability checks
discopt doe new response-surface-2d -o c.xlsx --input T:300:500 --n 8
discopt doe fit c.xlsx # after the lab fills in results
discopt doe extend c.xlsx --n 4
The /doe skill lets the agent run the campaign; the spreadsheet is the hand-off to the lab
5. Lessons from working with Claude
What worked
Engineering practice, applied to agents
Tests and benchmarks are the specification. The agent iterates against them
Small phases with testable goals, then commit
Write down failures, so the next session does not repeat them
Adversary agents: one writes, another tries to break it
Give the agent literature, not just prompts (crucible)
Port first, then extend: a faithful Ipopt port gave a reference to diff against
What surprised me
Unfamiliar languages (Rust, TypeScript) stopped being a barrier
Breadth is cheap; depth and robustness are still expensive
Solvers designed for agents: JSON solve reports, an LLM-native debugger, skills shipped in the repo
Agents are good at the tedious parts of numerical software: option plumbing, readers, test harnesses
I am the loop: steering, judging and saying "no" is the job
Hazards? Of course
Fooling yourself
Easy to generate faster than you can read or verify
Code is only as good as the trustworthy benchmarks that test it
Confident wrong answers, e.g. a certified-but-wrong global optimum
Maintenance and debt
Can these codes be maintained without LLMs? Probably, but much slower
Attribution
Agents do not know where ideas came from, and can invent references
Licensing and provenance: pounce is EPL-2.0, like Ipopt
Cost and access: $100-200/month; outages; models change under you
We are not thinking big enough, and sustainability is open
A 10-100x acceleration in generation needs a 10-100x larger user and contributor base
Pyomo and IDAES took two decades of federal investment; how do we sustain this?
Rust is getting native AD; a move from Python/Julia to Rust could take a year, not a decade
There is not one tool to rule optimization: we need a landscape of pathways
Thanks to contributors: Devin Griffith (sensitivity in pounce), Germán Heim (pounce in oximo), Milan Rother (RSLAB, pounce in the browser), Victor Alves (first pounce user), David Bernal (global optimization in discopt)
Concluding thoughts
Agents let a domain scientist build real numerical software in months
They need knowledge (skills), memory (crucible) and a human interface (scimax)
The payoff is a differentiable stack: programs, solvers, and surrogates with UQ that reuse one model for many tasks
Correctness still comes from benchmarks, mathematics and experiments
I would love your critique, your benchmarks, and your contributions
Thank the host. The invitation asked for three things: crucible, pounce and discopt, and my
experience working with Claude. This talk is all three, organized around the
differentiable optimization landscape we have been building this year.
Caveat up front: I am a chemical engineer who learned optimization by building it. Many of
you know these algorithms far better than I do; I am here to show what is now possible
for someone like me, and to hear where I got it wrong.
~1 min. Each dot is a paper; size ~ citations.
2002-08 DFT surface science; 2009-15 CO2 capture, oxides; 2016- ML for simulation;
2021- optimization, automation, AI for science. The constant thread is software that
makes the science faster. The new part this year: solvers.
people
scimax: interface
agent
knowhow: knowledge
crucible: memory
lab and compute
The picture: people stay in charge and talk to the agent through scimax, where we
already write notes, code and papers. The agent gets our know-how from skills
(knowhow), remembers across sessions through crucible, and acts on the lab and
compute through tools like discopt-doe and the Opentrons skill. The next slides
take each piece in turn.
Inspired by Karpathy's LLM knowledge-base pattern. The wiki is the agent's domain;
I rarely edit it. Each conversation adds up instead of evaporating. Every project
(including MFI) has its own crucible; a CLAUDE.md directive tells the agent to check
it before answering domain questions. Useful for proposals, student meetings, lab notebooks.
knowhow is the tool and a collection. Any git repo with SKILL.md files works,
so a group can publish its own lab's skills and students install them in one line.
Compatible with npx skills. Safety: shows hook and MCP scripts before installing.
Offer: happy to help anyone write skills for their own instruments or courses.
I never knew TypeScript or VS Code internals; Claude Code made it possible to port
scimax in a few weeks, extended to Markdown and LaTeX. I use it nearly daily.
The broader point: agents let domain experts build real software tools.
Install from the VSIX on GitHub releases.
The landscape post (kitchingroup.cheme.cmu.edu/posts/differentiable-optimization-landscape)
is the written version of this section.
Three columns. Programs: write ordinary Python + JAX (difflow), sequential modular.
Equations: an algebraic modeling language (discopt) with modules and plugins, simultaneous.
Solvers in the middle are shared: feral -> pounce -> discopt, plus jax-go which certifies
global bounds directly on JAX programs. Surrogates with UQ along the bottom feed both
sides: as JAX callables in programs, as embedded constraints in equation models.
Hover to show the details live.
Started right after Opus 4.5 (Nov 2025); proof of concept in days. Learned about
sequential modular vs simultaneous the practical way. The IDAES comparison is from the
NNL rare-earth cascade work.
GRC Catalysis, Feb 2026: I said as a community we were not thinking big enough. I did not
know Rust or interior point methods well. Many people did not think it would work.
The first attempt (ripopt) had to be abandoned; the second is feral + pounce.
scimax-hidden
## Interior point methods in one slide (for me, not for you)
Log barrier, $\mu \to 0$:
$\displaystyle \min_x \; f(x) - \mu \sum_i \ln s_i \quad \text{s.t.} \quad c(x) = 0$
Each Newton step solves the KKT system
$$
\begin{bmatrix} W + \Sigma + \delta_w I & J^\top \\ J & -\delta_c I \end{bmatrix}
\begin{bmatrix} \Delta x \\ \Delta \lambda \end{bmatrix}
= - \begin{bmatrix} \nabla_x \mathcal{L} \\ c(x) \end{bmatrix}
$$
<div class="cols small">
<div>
- Sparse, symmetric **indefinite**, ill-conditioned as $\mu \to 0$
- Needs $LDL^\top$ with pivoting and the **inertia** $(n, m, 0)$ to choose $\delta_w$
- Most of the time, and most of the difficulty, is in this solve
</div>
<div>
- **Bonus:** at the solution, the factored KKT matrix gives $\partial x^*/\partial p$ with one back-solve per parameter (sIPOPT)
- That is what makes the **solver differentiable**
</div>
</div>
scimax-hidden
Be careful with this audience: the profile is on our own harness of ~1,200 problems, against
Ipopt with MA57. It is evidence the port is solid, not a claim of beating mature commercial
solvers. Feral's job is the KKT solve plus a trustworthy inertia count.
Try it: jkitchin.github.io/pounce/demo (drop an .nl file, solved client-side).
X_RC,i = d ln r / d ln k_i at fixed K_i (Campbell). PO desorption limited (step 14, +0.99),
O2 poisoned (step 17, -0.91). The point for this audience: the solver is a differentiable
layer, so a model built once serves estimation, sensitivity and design.
This is the part of the landscape where this audience (MINOTAUR, filterSQP) knows far more
than I do. I would value your critique. discopt is broad because agents make breadth cheap;
depth and robustness still come from benchmarks (MINLPLib, QPLIB, CUTEst).
David Bernal contributed major global optimization pieces.
Hover a model for what it is and how it gets its uncertainty. Four regions: symbolic
grove (jaxsr, iBART, SISSO), decision forest (linear/hyperplane trees, soft trees),
neural meadow (ICNN, monotonic, periodic, KAN, LLPR, DPOSE), and distributions (flows).
Written version: kitchingroup.cheme.cmu.edu/posts/differentiable-surrogate-zoo
scimax-hidden
## Uncertainty drives the next experiment
<div class="cols">
<div class="side">
$$
q(P) = \frac{q_{\max} K P}{1 + K P}
$$
- Low-pressure data only determine the lumped $q_{\max} K$
- Every curve in the 95% region fits the data equally well
- The **Fisher information** from the solver says where to measure next

</div>
<div class="center">


**Before: very uncertain. After: very confident.**
</div>
</div>
<!--
Same machinery as the DRC slide: sensitivities of a solved model. This is the bridge to
the lab: design of experiments is an optimization problem over a differentiable model.
-\->
scimax-hidden
This is the experiment-planning engine in the MFI loop. The agent reads the workbook,
fits, proposes the next runs, and the human approves. Browser app (Pyodide, nothing
leaves your machine): jkitchin.github.io/discopt-doe/app/ -- good for teaching too.