## Regularization - Intro to Machine Learning

• 0:01 - 0:05
One very powerful place that you can use regularization, is in regression.
• 0:05 - 0:07
Regularization is a method for
• 0:07 - 0:11
automatically penalizing the extra features that you use in your model.
• 0:11 - 0:13
So, let me make this a little bit more concrete.
• 0:13 - 0:17
There's a type of regularized regression called Lasso Regression.
• 0:17 - 0:20
And, here's the rough formula for the Lasso Regression.
• 0:21 - 0:22
A regular linear regression would say,
• 0:22 - 0:28
I just want to minimize the sum of the squared errors in my fit.
• 0:28 - 0:31
I want to minimize the distance between my fit, and
• 0:31 - 0:35
any given data point, or the square of that distance.
• 0:35 - 0:40
What Lasso Regression says is yeah, we want a small sum of squared error.
• 0:40 - 0:43
But, in addition to minimizing the sum of the squared errors,
• 0:43 - 0:46
I also want to minimize the number of features that I'm using.
• 0:46 - 0:49
And, so I'm going to add in a second term here, in which I
• 0:49 - 0:54
have a penalty parameter, and I have the coefficients of my regression.
• 0:54 - 0:58
So, this is basically the term that describes how many features I'm using.
• 0:58 - 1:00
So, here's the result of this formulation.
• 1:00 - 1:05
When I'm performing my fit, I'm considering both the errors that come from that
• 1:05 - 1:09
fit, and also the number of features that are being used.
• 1:09 - 1:11
And, so let's say I'm comparing two different fits,
• 1:11 - 1:13
that have different number of features in them.
• 1:14 - 1:16
The one that has more features included,
• 1:16 - 1:19
will almost certainly have a smaller sum of the squared error.
• 1:19 - 1:23
because, it can fit more precisely to the points.
• 1:23 - 1:25
But, I pay a penalty for using that extra feature.
• 1:25 - 1:29
And, that comes in the second term with the, with the penalty term, and
• 1:29 - 1:31
the coefficients of regression that I'm going to get for
• 1:31 - 1:34
that additional feature that I'm using.
• 1:34 - 1:37
And, so what this is saying is that the gain that I get,
• 1:37 - 1:39
in terms of the, the precision,
• 1:39 - 1:44
the goodness of fit of my regression, has to be a bigger gain than the, the loss
• 1:44 - 1:48
that I take as a result of having that additional feature in my regression.
• 1:49 - 1:53
So, this precisely formulates, in a mathematical way, the trade off between
• 1:53 - 1:59
having small errors and having a simpler fit that's using fewer features.
• 1:59 - 2:01
And, so what Lasso Regression does,
• 2:01 - 2:05
is it automatically takes into account this penalty parameter.
• 2:05 - 2:09
And, in so doing, it helps you actually figure out which features that
• 2:09 - 2:12
are the ones that have the most important effect on your regression.
• 2:12 - 2:15
And, once it's found those features, it can actually eliminate or
• 2:15 - 2:20
set to zero, the coefficients for the features that basically don't help
Title:
Regularization - Intro to Machine Learning
Description:

more » « less
Video Language:
English
Team:
Udacity
Project:
ud120 - Intro to Machine Learning
Duration:
02:21
 Udacity Robot edited English subtitles for 11-19 Regularization Udacity Robot edited English subtitles for 11-19 Regularization Cogi-Admin edited English subtitles for 11-19 Regularization

# English subtitles

## Revisions Compare revisions

• API
Udacity Robot
• API
Udacity Robot
• API