This max df argument will actually shrink down the size of my vocabulary.

And, it will use it based on the number of documents that

a particular word occurs in.

So, if there's a word that occurs in more than 50% of the documents,

this argument says, don't use it in the tfidf,

because it probably doesn't contain a lot of information in it.

because it's so common.

So, this is an example of another place where you could do

some feature reduction, some dimensionality reduction, as, as we also call it.

But of course,

you also always have your old standby of doing something like, SelectPercentile.

So, I hope what you found in that coding exercise underscores this point that

we're talking about right now, that features are not the same as information.

You just got rid of 90% of your text features, but

your classifier accuracy basically didn't suffer at all.

And in fact, in some ways the performance improved because it's able to run so

much more quickly on the smaller number of features.

So, this, obviously,

is going to be something that you want to be keeping in mind.

Especially, when you're working with very high dimensionality data.

Data that has lots and lots of features.

You want to be skeptical of all of those features and think,

which are the ones that are really going to get me the most bang for my buck?