What today is about

Features for time-series data: which columns belong in the table when the rows came from a process that changes over time.

All of them share one property: what you measure now depends on what the system was doing a moment ago.

What today is about, what is being stored

Every one of those has something stored in it, which is why its past belongs in the table.

How would you know your feature table is right?

You build a table, fit a model, and it reports R-squared 0.94.

That number is just as high for a table that:

• quietly used a value from the future
• grouped on the wrong column
• leaned on a sensor that was only repeating its last reading

The score cannot see any of those. It goes up either way.

So we pick a problem whose answer is checkable

One loop all class: the A and C feed of the Tennessee Eastman plant, from Lecture 5.
xmv_4 is the valve, xmeas_4 is the flow it sets.

The process behind it is first order, so the two fitted coefficients convert into:

• a time constant, in minutes
• a gain, in flow per percent of valve

A plant engineer who knows that line can tell you whether ten minutes is credible. Nobody can tell you whether an R-squared is credible.

The plant, and where these channels are

P&ID from Lyu, Botcha, Kulkarni, Pagaria, Alves, Sunshine and Kitchin (2026)

Textbook time-series vs real data

The textbook way to get a time constant is a step test: move the valve, wait for it to settle, read the response off the chart.

Now go and schedule one.

The logged data has a problem?

At 57.6 % the flow is 8.578 going up and 8.997 coming down. If you had waited: 8.790.

Simulated at the time constant and gain measured from the plant's own data. No operator will ramp a live feed valve so you can draw this.

The logged data has a problem?

None of the three is a measurement error.

The loop is dynamic: it takes time to respond, so it is almost never at steady state.

Flow depends on the valve and on where the flow was three minutes ago. A table with one column for each has thrown the second one away.

Structuring data for time series modeling

The valve alone cannot tell you which branch of the loop you are on. One extra column can: the previous flow.

Roadmap

1. The loop, and the data we have of it
2. From differential equations to discrete columns
3. Leakage: could you have known this at the time?
4. Four more things you do to time series tables
5. Live demo
6. Fitting, and converting to physics
7. More than one past value
8. ARX systems
9. Two things the record cannot tell you

The loop, the column naming

The logged data uses two column families:

• xmv_* are the manipulated variables (MVs): what the operator or the controller moves
• xmeas_* are the measurements: what the plant reports back

Here: xmv_4 is the valve, xmeas_4 is the A and C feed flow. Samples every 3 minutes.

Rieth et al. 2017, Harvard Dataverse, CC0 / Downs and Vogel, 1993

The loop, what the recorded data looks like

The valve is under automatic control, so it rattles instead of ramping. No loop, just a cloud.

Continuous and discrete, two kinds of model

Most models you have written are probably in continuous time. Mole balances, energy balances, rate laws, forces acting on a structure: all differential equations, all describing an instant.

Continuous vs discrete?

Your data is discrete. One row every 3 minutes, nothing in between. A dataframe has rows, not a continuum.

So a model you can fit to a table has to be in discrete time.

And the discrete model is what tells you which columns the table needs.

Continuous vs discrete? the continuous model

A vessel or line with one place to store something (heat, mass, momentum):

Continuous vs discrete?, discretizing it

We need sample from sample . That needs an assumption about the input in between, and the plant gives it to us free: the valve holds its position between samples.

That is zero-order hold (ZOH). Integrate across one interval with u held constant:

Continuous and discrete, the model gives the feature list

Read that equation as a table specification, not as physics.

To predict row you need exactly two numbers from row :

That is where the features come from. Not a guess, not everything in the file. The discretized model names them.

Continuous and discrete, the lag plot

A straight line whose slope is a, simulated with the valve held steady. Read the coefficient off the chart before you fit anything.

Continuous and discrete, getting the physics back

Leakage, the definition

Leakage, the credit card example

Predict fraudulent transactions. The file has: customer, amount, location, fraud found, chargeback received.

A chargeback is the customer disputing the charge. It happens after somebody decided it was fraud.

• the model learns "chargeback means fraud". Validation looks excellent
• it goes live. There is no chargeback column yet, because you are trying to catch the fraud first
• accuracy collapses

Nothing about the data was wrong. That column just did not exist yet.

IBM, data leakage in machine learning

Leakage, the same thing in our plant data

And the subtle one: xmeas_23 to xmeas_41, the slow analysers.

Leakage, two ways it gets in

1. Target leakage. A column holds information from after the row's timestamp. The chargeback, faultNumber, the analysers. A property of the table. Ours today.

2. Train-test contamination. Training rows come from after the test rows. Shuffle a time series at random and every test row has its neighbours in training.

The second needs a held-out score to show, so it belongs with the sessions that fit models. Carry the fix: split on time, not at random.

Data preparation, build the clock and check it

No timestamps in this file. Just sample, an integer counter. Build the clock, then check it:

ts = start + sample * 3 minutes
df.select(pl.col("ts").diff().value_counts())

Why bother: shift(1) reaches back one row, not three minutes. If one row is missing, that lag quietly spans six minutes instead, and the fitted time constant is wrong.

Data preparation, putting rows back on the grid

Rows are missing. Put them back, so every row is 3 minutes apart:

df.upsample("ts", every="3m")

That creates empty rows. Do you fill them?

Data preparation, rolling window statistics

pl.col("xmeas_4").rolling_std(window_size=10).over(GROUP)

• rolling standard deviation screens for a sticking valve or a chattering transmitter
• rolling min and max find a channel sitting at its range limit
• longer than your lag depth adds something new.

Data preparation, channel screening

run.select(pl.col("^xmeas.*$").diff().eq(0).mean())

19 of 41 channels, at exactly 0.500 and 0.800.

Data preparation, what the plateaus are

Composition analysers running on their own cycle, with the historian holding the last result in between. Zero-order hold, again.

Cycles from Downs and Vogel 1993, table 5.

What to watch

1. The matrix, printed. Read one row across: these were the flow and the valve at this moment, and that is what the flow did next.

2. It loses rows. 480 in; one lag leaves 479, three lags leave 477. A row needs all of its past and its answer.

3. The coefficient splits. With one lag, y[t] gets 0.753. With three, it gets 0.417 and the rest lands on y[t-1] and y[t-2].

4. The gain goes to zero when the valve is frozen.

Fitting, deviation variables

y = y - y.mean()      # xmeas_4 sits at 8.79
u = u - u.mean()      # xmv_4 sits at 57.6

Both columns are almost entirely offset. A model with no constant term spends both coefficients explaining that offset and gets a badly wrong.

In process control this is working in deviation variables.

Fitting, what came back

Two coefficients in, minutes and flow-per-percent out.

Minutes, and flow per percent of valve. A plant engineer understands this.

Multiple lags, the general form

One lag is where you start, not a rule. There are situations you may need more lag/regressors.

df.with_columns([
    pl.col("xmeas_4").shift(k).over(GROUP).alias(f"xmeas_4_lag{k}")
    for k in (1, 2, 3)
])

Nothing about the method changes.

Multiple lags, the regression vector

Collect the right-hand side into one row, the regression vector :

Stack one row per sample and you have the design matrix . The whole model is

lstsq solves that for at any width of .

Model structures, the family

Model structures, the same arithmetic

Ours:

An RNN cell:

Some of where you were, plus some of the input. An LSTM adds gates deciding how much of to keep, which is why its name says memory.

What changes across the family is how much state you carry and how nonlinear the update is.

Model structures, why the table came first

Every model in that list uses the same feature table you just built.

If the past is missing from the table, or a column leaked, no amount of capacity further down the list repairs it.

Limitations, the valve that never moved

Same code, same clean table, one different day: the operator left the valve alone all day.

rank of the design matrix: 2  ->  1
b = 0.0000      K = 0.000      (the truth is K = 0.130)

The data really does contain no evidence about the valve. It never moved, so the data cannot say what it would have done.

Limitations, persistent excitation

How much is enough? A single step is plenty. What matters is the rank: a constant input gives rank 1 and no gain, a single step gives rank 2 and a usable fit.

The fix is an experiment, not an algorithm. Move the setpoint, or add a small deliberate wiggle (a PRBS).

Limitations, closed-loop identification

   open loop     valve  ->  measurement

   closed loop   valve  ->  measurement
                   ^              |
                   +-- controller +

The controller moves the valve because the measurement moved. In the record, the effect arrives first and the cause follows.

This is closed-loop identification, and it is the normal condition of historian data.

Limitations, what to do

• log the setpoint as well as the valve, so the thing that moved on its own is in the table
• fit on days where the setpoint stepped, not where the plant held still
• ask for a deliberate wiggle: minutes of a small perturbation buys a number no logged data contains

Recap

• A dynamic loop draws a loop, so its past has to become a column
• Leakage: every value in a row was knowable at that row's timestamp
• The group key travels with the shift, and here it is two columns, not one
• Two coefficients convert to a time constant and a gain: 10.7 min on our record, 10.6 on the plant's
• A valve that never moved, and a controller in the loop, both return confident wrong numbers

Recap, the names you can now look up

You built an ARX(1,1) model by zero-order hold discretization, widened it to a regression vector, and met two reasons the data refuses to answer.

None of this is new. Process control has been identifying models from plant data for over fifty years, and the ARX structure you just built is where that field starts. What changed is the size and "nature" of the models, not the table underneath them.

• Nelles, Nonlinear System Identification: From Classical Approaches to Neural Networks, Fuzzy Models, and Gaussian Processes, 2nd ed., Springer 2020. Goes from simple ARX to neural networks.

Next

Practice module for this session, for participation credit

Assignment 4 is released at the next session, on 2026-09-21