Homework 10: Ensemble Methods

Open In Colab

Homework 10: Ensemble Methods#

Apply Random Forests and Gradient Boosting to a chemical engineering problem.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
from sklearn.metrics import mean_squared_error, r2_score
import xgboost as xgb

Dataset: Polymer Properties#

Predict polymer tensile strength from synthesis conditions.

# Load polymer properties data from URL
url = "https://raw.githubusercontent.com/jkitchin/s26-06642/main/dsmles/data/hw10_polymer_properties.csv"
data = pd.read_csv(url)

feature_names = ['temperature', 'pressure', 'molecular_weight', 'crystallinity',
                 'additive_concentration', 'cooling_rate']
X = data[feature_names].values
y = data['tensile_strength'].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

print(f"Training samples: {len(X_train)}")
print(f"Test samples: {len(X_test)}")

Training samples: 240
Test samples: 60

Problem 1: Random Forest#

1a. Train a Random Forest with 100 trees and max_depth=10. Report train and test R².

# Your code here

1b. Plot feature importances. Which features matter most?

# Your code here

1c. How does performance change with the number of trees? Plot test R² vs n_estimators for [10, 25, 50, 100, 200].

# Your code here

Problem 2: Gradient Boosting#

2a. Train an XGBoost model with default parameters. Compare to Random Forest.

# Your code here

2b. Use GridSearchCV to tune max_depth (3, 5, 7) and learning_rate (0.01, 0.1, 0.2). What are the best parameters?

# Your code here

2c. Plot the learning curve showing how test error decreases with number of boosting rounds.

# Your code here

Problem 3: Model Comparison#

3a. Compare Random Forest and XGBoost using 5-fold cross-validation. Which performs better?

# Your code here

3b. Create a predicted vs actual plot for your best model.

# Your code here

3c. A colleague suggests decreasing cooling_rate to improve tensile strength. Based on your model’s feature importances, is this a good strategy? What would you recommend instead?

Your answer here:

Contents