English subtitles

← Information Gain Calculation Part 10 - Intro to Machine Learning

Get Embed Code
5 Languages

Showing Revision 3 created 05/25/2016 by Udacity Robot.

  1. As it happens when we split based on speed limit we get
  2. perfect purity of the branches that we make as a result.
  3. So our information gain is going to be equal to 1.
  4. We started out with an entropy of 1.
  5. At the end we had an entropy of 0.
  6. So the information gain is going to be 1.
  7. So this is the best information gain that we can have,
  8. definitely this is where we want to make a split.
  9. And just to sketch out the decision tree it would look something like this.
  10. Where when we look at samples where the speed limit is in effect,
  11. so these first two rows where the, the answer for
  12. speed limit is yes, then we get all of our slow examples over there.
  13. On the other side when there's no speed limit, everything is going to be fast.
  14. So this has just been a very simple calculation.
  15. It still took us a while to get through, but
  16. I hope you have a little bit of a better sense for
  17. what information gain is in decision trees and why it's so important.
  18. So, it's calculations like this that the decision tree is figuring out when it
  19. does the training.
  20. It looks at all the training examples, all of the different features that
  21. are available to it, and it uses this information gain
  22. criterion in deciding which variables to split on, and how to make the splits.