Homework 9: Nonlinear Methods

Apply nonlinear regression methods to chemical engineering data.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsRegressor
from sklearn.tree import DecisionTreeRegressor
from sklearn.svm import SVR
from sklearn.metrics import mean_squared_error, r2_score

Problem 1: K-Nearest Neighbors Regression

Predict reaction rate from operating conditions.

# Load reaction kinetics data from URL
url = "https://raw.githubusercontent.com/jkitchin/s26-06642/main/dsmles/data/hw09_reaction_kinetics.csv"
rate_data = pd.read_csv(url)
rate_data.head()

	temperature	concentration	rate
0	374.908024	1.825705	26.763705
1	490.142861	0.555168	181.816505
2	446.398788	0.375300	40.160332
3	419.731697	1.029960	66.704053
4	331.203728	1.972736	2.943014

1a. Scale features and fit KNN regressors with k=1, 5, 15, 30. Report test R² for each.

# Your code here

1b. What happens to the bias-variance tradeoff as k increases? Which k performs best?

Your answer here:

1c. Create a predicted vs actual plot for your best KNN model.

# Your code here

Problem 2: Decision Trees

Use a decision tree for regression.

# Load process data from URL
url = "https://raw.githubusercontent.com/jkitchin/s26-06642/main/dsmles/data/hw09_process_data.csv"
process_data = pd.read_csv(url)

X = process_data[['temperature', 'pressure', 'catalyst_loading']]
y = process_data['conversion']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

2a. Fit decision trees with max_depth = 2, 5, 10, and None (unlimited). Compare train and test R².

# Your code here

2b. Which max_depth gives the best test performance? Is there evidence of overfitting?

Your answer here:

2c. Plot feature importances for your best tree. Which feature is most important?

# Your code here

2d. What are the advantages and disadvantages of decision trees compared to linear regression?

Your answer here:

Problem 3: Model Comparison

3a. Compare Linear Regression, KNN (k=5), and Decision Tree (max_depth=5) using 5-fold cross-validation on the process data. Which performs best?

# Your code here

3b. For the rate data in Problem 1, would you expect linear regression to perform well? Why or why not?

Your answer here:

3c. When might you prefer a simpler model (like linear regression) even if a complex model (like KNN) gives slightly better accuracy?

Your answer here: