Optional: Introduction to Deep Learning

Optional: Introduction to Deep Learning#

Neural networks for complex pattern recognition.

Learning Objectives#

  1. Understand neural network basics

  2. Build simple networks with PyTorch

  3. Train and evaluate models

  4. Know when deep learning is appropriate

import numpy as np
import matplotlib.pyplot as plt

# Note: PyTorch must be installed separately
# pip install torch
try:
    import torch
    import torch.nn as nn
    import torch.optim as optim
    TORCH_AVAILABLE = True
except ImportError:
    print("PyTorch not installed. Run: pip install torch")
    TORCH_AVAILABLE = False

PyTorch not installed. Run: pip install torch

Neural Network Basics#

A neural network is a function approximator:

  • Input layer: Features

  • Hidden layers: Learned representations

  • Output layer: Predictions

Each layer applies: \(\text{output} = \text{activation}(W \cdot \text{input} + b)\)

if TORCH_AVAILABLE:
    # Create synthetic data
    np.random.seed(42)
    torch.manual_seed(42)
    
    n_samples = 1000
    X = np.random.randn(n_samples, 5).astype(np.float32)
    y = (np.sin(X[:, 0]) + 0.5*X[:, 1]**2 - X[:, 2] + 
         np.random.normal(0, 0.1, n_samples)).astype(np.float32)
    
    # Convert to tensors
    X_tensor = torch.from_numpy(X)
    y_tensor = torch.from_numpy(y).unsqueeze(1)
    
    print(f"X shape: {X_tensor.shape}")
    print(f"y shape: {y_tensor.shape}")

if TORCH_AVAILABLE:
    # Define a simple neural network
    class SimpleNN(nn.Module):
        def __init__(self, input_dim, hidden_dim=32):
            super().__init__()
            self.network = nn.Sequential(
                nn.Linear(input_dim, hidden_dim),
                nn.ReLU(),
                nn.Linear(hidden_dim, hidden_dim),
                nn.ReLU(),
                nn.Linear(hidden_dim, 1)
            )
        
        def forward(self, x):
            return self.network(x)
    
    model = SimpleNN(input_dim=5)
    print(model)

if TORCH_AVAILABLE:
    # Training loop
    criterion = nn.MSELoss()
    optimizer = optim.Adam(model.parameters(), lr=0.01)
    
    losses = []
    for epoch in range(200):
        optimizer.zero_grad()
        predictions = model(X_tensor)
        loss = criterion(predictions, y_tensor)
        loss.backward()
        optimizer.step()
        losses.append(loss.item())
        
        if epoch % 50 == 0:
            print(f"Epoch {epoch}, Loss: {loss.item():.4f}")
    
    # Plot training curve
    plt.figure(figsize=(10, 5))
    plt.plot(losses)
    plt.xlabel('Epoch')
    plt.ylabel('Loss')
    plt.title('Training Loss')
    plt.grid(True, alpha=0.3)
    plt.show()

if TORCH_AVAILABLE:
    # Evaluate
    model.eval()
    with torch.no_grad():
        y_pred = model(X_tensor).numpy().flatten()
    
    from sklearn.metrics import r2_score
    print(f"R² Score: {r2_score(y, y_pred):.4f}")
    
    # Predicted vs actual
    plt.figure(figsize=(8, 8))
    plt.scatter(y, y_pred, alpha=0.5)
    plt.plot([y.min(), y.max()], [y.min(), y.max()], 'r--')
    plt.xlabel('Actual')
    plt.ylabel('Predicted')
    plt.title('Neural Network Predictions')
    plt.grid(True, alpha=0.3)
    plt.show()

%pip install -q jupyterquiz
from jupyterquiz import display_quiz

display_quiz("https://raw.githubusercontent.com/jkitchin/s26-06642/main/dsmles/optional/quizzes/deep-learning-quiz.json")

Note: you may need to restart the kernel to use updated packages.

When to Use Deep Learning#

Use Deep Learning

Use Traditional ML

Large datasets (>10K)

Small datasets

Images, text, sequences

Tabular data

Complex patterns

Simple relationships

Prediction focus

Interpretability focus

For most chemical engineering tabular data, tree-based methods (XGBoost) often work better!

Contents