1 Introduction

2 Overview of Supervised Learning

2.1 Introduction

2.2 Variable Types and Terminology

2.3 Two Simple Approaches to Prediction

2.3.1 Linear Models and Least Squares

2.3.2 Nearest-Neighbor Methods

2.3.3 From Least Squares to Nearest Neighbors

2.4 Statistical Decision Theory

2.5 Local Methods in High Dimensions

2.6 Statistical Models, Supervised Learning and Function Approximation

2.6.1 A Statistical Models for the Joint Distribution Pr(X,Y)

2.6.2 Supervised Learning

2.6.3 Function Approximation

2.7 Structured Regression Models

2.7.1 Difficulty of the Problem

2.8 Classes of Restricted Estimators

2.8.1 Roughness Penalty and Beyesian Methods

2.8.2 Kernel Methods and Local Regression

2.8.3 Basis Functions and Dictionary Methods

2.9 Model Selection and Bias-Variance Tradeoff

3 Linear Methods of Regression

3.1 Introduction

3.2 Linear Regression Models and Least Squares

3.3 Subset Selection

3.4 Shrinkage Methods

3.5 Methods Using Derived Input Directions

3.6 Discussion: A Comparison of the Selection

3.7 Multiple Outcome Shrinkage and Selection

3.8 More on the Lasso and Related Path Algorithms

3.9 Computational Considerations

4 Linear Methods for Classiflcation

4.1 Introduction

4.2 Linear Regression of an Indicator Matrix

4.3 Linear Discriminant Analysis

4.4 Logistic Regression

4.5 Separating Hyperplanes