Homework 4: Feature Engineering

Open In Colab

Homework 4: Feature Engineering#

Practice creating and transforming features for machine learning.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler, PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

Problem 1: Scaling and Normalization#

Process data has features on very different scales.

# Load process data from URL
url = "https://raw.githubusercontent.com/jkitchin/s26-06642/main/dsmles/data/hw04_process_data.csv"
process_data = pd.read_csv(url)
process_data.describe()
temperature pressure flow_rate catalyst_loading conversion
count 100.000000 100.000000 100.000000 100.000000 100.000000
mean 394.036149 5.480486 56.584120 1.033183 0.927325
std 59.497882 2.638001 26.408362 0.557559 0.125485
min 301.104423 1.062569 10.455543 0.127348 0.487552
25% 338.640152 3.178041 34.919188 0.574268 0.883697
50% 392.828491 5.550624 60.629944 1.068465 1.000000
75% 446.040624 7.895652 77.713025 1.497977 1.000000
max 497.377387 9.870854 99.104847 1.981960 1.000000

1a. Apply StandardScaler to the features. Show the mean and std of scaled features.

# Your code here

1b. Compare linear regression R² with and without scaling. Does scaling change the predictions?

# Your code here

1c. Compare the raw coefficients vs scaled coefficients. Which feature is most important according to the scaled model?

# Your code here

Problem 2: Polynomial Features#

Sometimes relationships are nonlinear.

# Load kinetics data from URL
url = "https://raw.githubusercontent.com/jkitchin/s26-06642/main/dsmles/data/hw04_kinetics_data.csv"
rate_data = pd.read_csv(url)
rate_data.head()
temperature rate_constant
0 374.908024 45.886528
1 490.142861 4667.733233
2 446.398788 886.598550
3 419.731697 331.722042
4 331.203728 2.916731

2a. Fit a linear model using temperature to predict rate_constant. Report R².

# Your code here

2b. Create polynomial features of degree 2 (include interaction terms). How many features do you have now?

# Your code here

2c. Fit a linear model with polynomial features. How much does R² improve?

# Your code here

2d. Create a physically-motivated feature: 1/T (for Arrhenius). Does this improve the model?

# Your code here

Problem 3: Categorical Encoding#

Handle non-numeric features.

# Load catalyst data from URL
url = "https://raw.githubusercontent.com/jkitchin/s26-06642/main/dsmles/data/hw04_catalyst_data.csv"
catalyst_data = pd.read_csv(url)
catalyst_data.head()
catalyst_type support preparation loading_wt_pct activity
0 Ni Al2O3 coprecipitation 2.45 94.0
1 Cu SiO2 sol-gel 2.08 55.1
2 Pt TiO2 coprecipitation 3.40 58.5
3 Ni Al2O3 sol-gel 3.51 71.6
4 Ni SiO2 impregnation 4.39 65.4

3a. One-hot encode the catalyst and support columns using pd.get_dummies().

# Your code here

3b. Why should you use drop_first=True when one-hot encoding? Demonstrate with an example.

# Your code here

3c. Fit a linear regression predicting activity. Which catalyst has the highest predicted activity?

# Your code here
Contents