Systems and Toolchains for AI Engineers
"The box labeled 'ML Code' is actually tiny in proportion to the rest of the system."
Sculley et al., NeurIPS 2015
Read the paper, 9 pages
Redrawn after Sculley et al. (2015). Our own artwork, not the paper's figure.
You will see this cited as "less than 10% is ML code."
That number is not in the paper.
data → storage → pipelines → features → training → evaluation → deployment → monitoring
Not in the model class. In the joints:
Integration failure: a fault in the joints between components, not inside any one of them. Cross-validation cannot see it.
Pipeline: commercial labs → PHE → contact tracing
.xls
.xls caps a sheet at 65,536 rows
No error. No rejection. Just gone.
BBC: how it happened
15,841 positive cases, 25 Sep to 2 Oct 2020
75% of them in the final three days
PHE statement
Every model downstream computed correctly on the data it received.
The missing control was the least glamorous thing in the pipeline:
Did the number of records I sent equal the number that arrived?
Glue code: connective tissue that only moves data between components. It dominates the codebase.
Undeclared consumer: someone depends on your output and you do not know they exist.
Pipeline jungles: glue accreting without redesign.
Hold onto the undeclared consumer.
The premise was good: flu surveillance ran on a 1 to 2 week lag, search queries are available immediately. See the epidemic sooner.
The validation was good too:
Stricter than most deployed ML gets today.
Ginsberg et al., Nature 2009
Wrong, in the same direction, for two years.
Lazer et al., Science 2014
Search 50M terms against ~1,000 points → you will find winter.
They were already deleting terms like high school basketball by hand.
"part flu detector, part winter detector"
Missed the non-seasonal 2009 H1N1 entirely.
Google shipped 86 search changes in Jun-Jul 2012
Every one an improvement to search. None of those engineers knew a flu model consumed their output.
The undeclared consumer, in-house.
100 wrong weeks out of 108, same direction.
The failure was not the model. It was the absence of a baseline comparison in production.
Raw sensor voltage ≠ calibrated concentration
Surfaces when someone recalibrates.
UCI Air Quality: metal-oxide sensors, Italian roadside, hourly for ~13 months, 9,357 records
Fit a calibration on March to May 2004 only. Apply it for the rest of the deployment.
Dataset
The model learned spring, so it is worst exactly when conditions are least like spring.
Same failure as Flu Trends, on a gas sensor.
Same data. Same model. Two ways of scoring it.
Random split → Temporal split →
The gap is small enough to pass unnoticed.
If you only ever saw 0.78, nothing would look wrong. This is how the error survives review.
Provenance: the record of which data, which code, and which parameters produced a given number.
If the answer is "a notebook run out of order on a laptop that has been reimaged," there is no answer.
Control function commanding nose-down stabilizer trim.
One bad sensor was sufficient.
Left AoA sensor biased by ~21°
Traced to a replacement sensor mis-calibrated during an earlier repair
A measurement channel nobody had validated, driving a flight control surface.
Boeing's hazard assessment evaluated erroneous data from both air data channels → "beyond extremely improbable"
But the MCAS path was exposed to a single failure.
House T&I Committee report, p. 106
Incorrect assumptions about flight crew response, plus incomplete review of flight deck effects, are what made single-sensor reliance appear acceptable.
JT610, Oct 2018: 189 killed. ET302, Mar 2019: 157 killed.
If (3) is "an operator will compensate," that needs evidence, not assumption.
A CSV has no schema, no types, no constraints, and no way to read one column without reading all of them. A spreadsheet datastore is the far end of that road.
A fourth, the vector store, indexes by similarity.
Not "which database is best" but "what is my access pattern?"
DuckDB, Parquet format
Batch runs on a schedule over bounded data. Most engineering work needs only this. Streaming processes records as they arrive: buys latency, costs correctness.
A pipeline that silently passes bad data is worse than one that crashes.
Declare what you expect: column types, physical ranges, null rates, row counts
Fail loudly when reality disagrees.
pandera
It is an optimization loop.
Automatic differentiation: frameworks record the operations you perform, then replay them backwards for exact gradients.
You never hand-derive anything. That is the whole trick behind PyTorch and JAX.
torch.autograd, gently
Where there is structure to exploit:
But: a boosted tree on good features beats a neural net on bad ones, most of the time.
Beat the baseline first.
Both failures today were evaluation failures.
After a hundred runs, "which config produced this?" is unanswerable from memory. One run = one reproducible fact.
MLflow quickstart
Making it run somewhere else, repeatedly, for someone who is not you.
Container: an image bundling code, dependencies and system libraries into one artifact that runs anywhere the runtime exists.
A VM virtualizes hardware and boots an OS; a container shares the host kernel and isolates the process. Milliseconds, not tens of seconds.
What is a container?
FastAPI: your model as an HTTP endpoint, with typed request and response schemas.
That schema is also a data contract with your callers.
Then the questions turn operational: latency budget, throughput, cost per prediction, behavior under load
FastAPI
Most courses stop at deployment.
Models degrade because the world moves, exactly as the calibration figure showed.
Watch: input distributions, prediction distributions, the gap against ground truth once labels arrive
MLOps automates that loop: CI that tests data and models and not only code, reproducible retraining, staged rollout so a bad model does not reach everyone, and the ability to roll back.
Flu Trends is what absence looks like.
A next-token predictor trained on a very large corpus.
What matters operationally:
Two integration patterns:
For engineering: agents over your database, your simulation, your instrument.
NIST AI RMF
uv
LLM and agent frameworks are deliberately not standardized: that ecosystem changes much faster than a semester, so we teach interfaces and evaluation techniques rather than vendor's (OpenAI, Anthropic) specifics.
l01-reproducibility.ipynb
UCI Air Quality → plot → naive split → fresh checkout
Diagnose each break before I do.
Compare failures with your neighbor on 3. None of the three announces itself while you are making it.
Install before next class git, Python 3.11+, uv Assignment 1 released next session
git
Practice module ~10 min, ends in a PDF you upload: https://kitchingroup.cheme.cmu.edu/f26-06763/game/#/l01
Full notes, with all sources on the course website.
110 minutes. Budget: 10 / 30 / 20 / 35 / 10, leaving ~5 slack. The tour is the part students actually need today. If you are running long, cut MCAS detail, not the tour.
Worth 60 seconds. Sets the tone for citation discipline all semester.