COMP 5013 at Lakehead, built from the course textbook (An Introduction to Statistical Learning with Python): regression, classification, resampling, regularization, splines and GAMs, trees, SVMs, neural networks, PCA and clustering, plus the Python used in the labs and the mixture and ICA material from the course outline.
You’re reading a free preview. Sign up to see every topic and question — and to play this deck as games.
ISLP section 2.2: measuring fit with MSE and error rate, training versus test error, overfitting, the bias-variance decomposition, the Bayes classifier and the K-nearest neighbors classifier.
43 total questions
Card #1
Formula for the mean squared error (MSE) of a regression fit on n observations
Card #2
How does ISLP define overfitting precisely, in terms of test MSE?
Card #3
Degrees of freedom (of a fitted curve)
1. Three models are fitted to the same data. Training MSEs: A 4.1, B 2.3, C 0.9. Test MSEs on a large held-out set: A 5.0, B 3.1, C 6.2. Which model should you choose?
2. The true f is very close to linear. As you increase the flexibility of a smoothing spline, what does ISLP expect to happen to the test MSE?
3. At a test point x0, a method has Var(f(x0)) = 0.5 and Bias(f(x0)) = 0.6, and Var(ε) = 1.0. What is the expected test MSE at x0?
1. In ISLP Figure 2.9, linear regression sits at the most restrictive end of the flexibility axis, with ___ degrees of freedom.
Hint: Intercept and slope
2. As the flexibility of a method increases, the training MSE declines ___, while the test MSE traces a U-shape.
Hint: It never goes back up
3. Whether or not overfitting has happened, we almost always expect the training MSE to be ___ than the test MSE, because most methods try to minimise the training MSE.
Hint: Comparative of small
