Regression

Regression — predicting a continuous output from input features — is one of the most widely used tools in chemical engineering data analysis. Predicting product yield from process variables, estimating reaction rates from spectroscopic data, and building surrogate models for expensive simulations all reduce to regression.

This module takes a machine-learning perspective: we focus on how to choose models with appropriate complexity, how to validate them honestly, and how to handle the high-dimensional datasets that arise in modern process data. The Dow Chemical distillation column dataset appears throughout as a running industrial example.

Topics

• Topic 2.1 — Non-parametric Models: Kernel regression, k-nearest neighbor regression, and spline interpolation. Introduces the bias–variance tradeoff without parametric assumptions.

• Topic 2.2 — Model Validation: Train/test splits, k-fold cross-validation, bootstrap resampling, and outlier sensitivity. Emphasizes honest estimation of generalization error.

• Topic 2.3 — Complexity Optimization: Regularization (ridge, LASSO, elastic net), hyperparameter selection via BIC/AIC and cross-validation, and the danger of data leakage.

• Topic 2.4 — High-dimensional Data: Feature extraction, correlation analysis, and scaling strategies for datasets with many input variables.

• Topic 2.5 — High-dimensional Regression: Multiple linear regression, principal component regression (PCR), and partial least squares (PLS) for datasets where the number of features rivals or exceeds the number of samples.