The differentiable optimization landscape
The big pivot into optimization
Over the past year I have been making a big pivot in research into optimization. I have always learned by doing, and this post documents some of that journey. This transition has been happening for a while, really starting about a decade ago when I started using machine learning to build surrogate models for DFT. That work led to this perspective on differentiable programming in scientific programming. That has led to this year’s work in differentiable optimization. My goal is to build an end-to-end differentiable optimization stack that integrates differentiable programming, differentiable solvers, and differentiable surrogates.
Automatic differentiation and program-based optimization
Coming from the machine learned potential field, which uses pytorch and jax with regular Python to solve their optimization problems, that is the approach I started first in difflow. The idea was pretty simple, you write regular Python code, and use jax solvers to solve the problem. I found this appealing because I have decades of programming experience in Python. Of course I was familiar with Pyomo, and although it is written in Python, it is very unPythonic with its operator overloading and heavy object-oriented programming. A year ago I did not fully understand how the solves that Pyomo uses actually have automatic differentiation and I thought jax would enable something they can’t do in cases where you don’t have equations. Agentic coding played a huge role in getting difflow off the ground, I started it shortly after Opus 4.5 was released in Nov 2025 and quickly got a proof of concept working. I learned about sequential modular solves, and that I could reformulate a residual approach that also allows a simultaneous solve. difflow is still being actively developed and fills a part of the landscape that is not easily filled by other tools.
Interior point solvers and differentiable programming
In February 2026 I set out a challenge at a GRC conference that as a community we were not thinking big enough about agentic coding. I suggested building a new version of Ipopt in Rust, with a new linear algebra library. I actually thought at the time it would probably not work since I was not familiar with Rust, or interior point methods, and that it would establish the boundaries of what is possible with agentic coding. Many people did not think this would work either, but I tried it anyway, and week after week it kept working better. I did have to abandon the first attempt (ripopt) and start over, but pounce emerged from this as one of the best open source interior point solvers, and built a small community of contributors. pounce is where I learned about the way automatic differentiation works in equation-oriented solvers, and that we can use sensitivity analysis to get differentiable solvers in Python.
Algebraic modeling languages and mixed integer programming
At that same GRC, I build jaxsr as a symbolic regression modeling tool. This got me interested in mixed integer programming and global optimization, and I started discopt. This forced me to get comfortable with the need for an algebraic modeling language and the kinds of analysis that are possible. My original goal in discopt was to also build a full Rust solver for mixed integer programming that leveraged pounce for the NLP parts. I tried that for a while, but currently outsource the mixed integer programming to the open source solver HiGHS. Several people have asked why didn’t I just use Pyomo? The simple answer is I learn by doing, and this is how I did it. I also wanted to try to address some limitations I have heard about Pyomo, primarily performance. For large problems it can take longer to build your model than to solve it. discopt addresses some of those issues, and let me develop part of the ecosystem in the landscape above.
Differentiable surrogate models and uncertainty quantification
In between these efforts we had a need for differentiable surrogate models. I learned in this paper that some surrogate models are MIP representable which has desirable properties for optimization. In parallel efforts in molecular simulation we were developing uncertainty quantification and this started making its way into our efforts in differentiable surrogates with uncertainty quantification. This means we can build surrogates that have properties we want, e.g. monotonicity, convexity, and uncertainty quantification, MIP representable, etc. and use them in optimization. This is a new paradigm that is not easy with other tools.
Why do we need new solvers at all?
There are a plethora of options, very good commercial ones, and open-source ones. I work in higher education, and I think it is important that free, open-source solvers exist for students and researchers. I got into computational catalysis as a student because there was a free code I could use, read, modify and learn from. It is often not possible to do this with closed codes. Even if my codes aren’t as good as other codes, if they open the door for others to learn then a part of the mission is accomplished.
New starts mean new things are possible. pounce is more comprehensive than Ipopt in interior point methods, with integrated sensitivity and other great features that are 1) pip-installable, 2) capable of running in your browser, and with interfaces to use in other interfaces. discopt led to new integrations of ideas, and eventually to the development of jax-go which bridges the gap between program and equation-based optimization. difflow is a new way to think about optimization, and the differentiable surrogates are a new way to think about optimization with uncertainty quantification.
Agentic optimization is a new thing with these solvers. The tight integration of agentic coding with differentiable programming and differentiable solvers is a new paradigm that is more possible with these new solvers because they were designed with this in mind. We have integrated agentic debugging, and agentic optimization.
What is next?
I have laid out an ambitious vision for the end-to-end differentiable optimization stack, and I think we are making good progress on it. It is a lot, and we don’t use all of them every day. Some of these tools are stabilizing. Feral is pretty stable now. I probably work on pounce once a week or so. discopt and difflow are still changing fast. It would probably be a lie to say there won’t be new packages coming. I still have many private projects that haven’t seen the light of day yet, but will eventually.
We (the Kitchin group) are beginning to enter a new era of applications. Inspired by the algebraic modeling language we made discopt-mkm for microkinetic modeling, and discopt-calphad for alloy phase behavior. discopt-doe is a new direction in design of experiments. We have projects in process modeling, alloy design, rare earth separation, and more.
It is always dangerous to predict the future, but I think 5 years from now we will have a much more integrated landscape of differentiable optimization tools. Right now programmatic automatic differentiation is mostly only available as addons like jax, pytorch in Python, and something like that in Julia. AD is coming to Rust as a native feature in a few years, and predict this will be a game changer for optimization. The pace of development I have demonstrated this year means it will probably only take a year or so to transition from a Python or Julia focused solution space to Rust if we decide to. These changes come with 20/20 hindsight and lessons learned from everything we know now, so they happen very quickly. I think the next 5 years will be very exciting for differentiable optimization.
I still think we are not thinking big enough as a community. I am grateful to a few people who made substantial contributions to these codes: Devin Griffith for major contributions to sensitivity in pounce, Germán Heim for integrating pounce into his modeling language oximo. Milan Rother forked feral into RSLAB and was the inspiration for pounce in the browser. Victor Alves for using pounce before anyone else. David Bernal for major contributions in global optimization in discopt, and ideas linking it to pounce. All this work was done by a very small number of people, and I think we can do better as a community.
It is an open challenge to figure out how to sustain these efforts. Pyomo and IDAES were built over two decades, with generous federal funding. This is a challenge today as that funding becomes more scarce. We need new models to support this kind of work that balances keeping open-access to the code, and generating revenue to support the people who build it.
Along this journey I have come to appreciate that 1) I still have a lot to learn in this space, and 2) there is not one tool to rule in optimization; there are many ways to solve problems, and some ways are better than others for certain problems. That means you have to have a landscape of pathways to choose, and historically I have chosen the roads less traveled. I have learned a lot along the way, and I hope this post helps others to learn from my journey.