Skip to content

ML 04 Regression

Sam

Fit Linear Regression via either:

  1. a closed-form solution (Normal Equation): a mathematical equation that gives the result directly

  2. iterative optimization (GD: batch, stochastic, mini-batch): initialize model parameters randomly ---> tweak to min cost function

Iterative optimization (gradient descent)

  • Batch: full dataset per step; converges on convex MSE “bowl.” Image

  • Stochastic: computes the gradients 1 instance at a time

  • Mini-batch: computes the gradients on small random sets of instances

Model complexity / Regularization

See Regularization.

Evaluation

See Lift.

Sam

  • MAE = Mean absolute error

  • MAPE = Mean absolute pct error

  • RMSE = Root mean squared error

  • SMAPE = Symmetric mean absolute percentage error.

    • Goes from 0 to 200%. Apply when comparing average error of different models. Does not apply when we are looking at each observation.