https:/.../fd7d2c53-752b-4844-9919-ad9000efd7f2-7c7c7abc-0256-4d5b-a428-ad9101277c2a.mp4?invocationId=891d3622-3009-ec11-a9e9-0a1a827ad0ec

This video I'm going to talk about, descriptive statistics are learning outcomes are first no, what a statistic is.

We talk about statistics. What is a statistic? Identify whether the mean or the median is more appropriate for summarizing some data.

And also to be able to strive both the central tendency and the spread of a data series.

So a statistic is a value that collect computed from a collection of data.

And it often summarizes a variable or particularly summarizes observations of a variable.

You've probably seen means medians, et cetera, before. Those are all examples of statistics.

In other contexts. We can get an additional statistic. But it's this one.

Excuse me. We can get additional statistics.

But if there's one value that summarizes the observations of a variable and it becomes useful for a variety of things.

So when we have a we have a variable. We have if we have a set of observations, there's a few different questions we want to ask of it.

And the readings talk about some conceptual questions, but these are getting at some just direct.

How are is the actual data values themselves laid out?

So one is where's the variables centered? This is called a measure of central tendency, a measure.

If it's a numeric variable, it measures how large is of value tend to be.

We also want to ask how spread out it is around that value.

So ways to do this. The mean of a data series is the sum divided by the number.

And so we have some data points here.

These are the scores of each of the players on the Chicago Bulls in the nineteen ninety eight game six of the NBA finals.

So we add up all of the values we're going to get. Eighty seven. There's ten of them.

And we we have a mean of eight point seven. This is often informally called an average.

When someone says average, they're usually talking about the mean.

But average itself is not a very specific term. It just means one of these measures of central tendency.

And so we want to be precise about. We often use it in informal discussion when we want to be precise.

Average is not a good enough term. We need something like mean, but the mean measures.

If every instance had the same value, what would it be?

If if the total is some resource or quantity, how much if it was evenly distributed among all the instances, how much would it be?

So all points per player kind of value. And if we go back, we think back to the question.

That I I told you to ask in the first week when we have a statistic. We have a metric.

How do I change this? Or if I have a definition that's defined as better, how do I improve it?

But how do you move this? Well, the way you move this is you increase the total score, more points.

But crucially, it does not matter where the total is increased among your data points.

One one value can get all of the increase in total to produce an increase in mean.

So we see a strong suit like there's an outlier here, Michael Jordan scored forty five points.

He could score 10 more points and that would have the same effect on the mean as every player scoring.

One more point. So. We want to measure how spread out the values are, one measure is the standard deviation.

So this is sample standard media. So the sample standard deviation and what it is, we take the mean.

X Bar is the main river from the previous slide. We subtract the mean from each value and then we square it and squaring does two things.

One. It makes it makes everything positive. We want to make all the values positive.

There's usually two ways to do it. Take the absolute value or take the square. But also squaring emphasizes larger values.

More one of the reasons it's really useful.

A third reason squaring is useful that does is that in a variety of contexts you'll see in future classes, particularly around machine learning.

It's really useful to have differentiable statistics and you can take the derivative of the square of the square.

But if we didn't have this square root, we would get the sample variance as squared.

But this is a measure of what this measures is the mean squared difference from distance from the mean.

If we just wanted to measure the mean distance, the total distance from the mean is.

Zero. Because we're subtracting the mean from every value.

And if you push through the algebra of that, I'll leave that as an exercise for you in order to better understand the algebra of these statistics.

But the sum of a bunch of values, minus the mean is zero.

So we needed to be positive. And what this measures is if the mean is the center or the expectation of our values.

How far away do values tend to be are if if it if the variance or the standard deviation is small?

That means the value is tightly clustered around the mean.

And if it's large, that means they're spread out quite a bit around the mean,

the state taking the square root of the variance means that the result is back in the original units rather than the square of the units.

And we can see here the standard DeeVee. So I've plotted here we have the mean.

And then we've got the standard deviation. I've plotted these points are one standard deviation away from the mean and each direction we

see that's pretty spread out in this particular data set because we have quite a bit of spread.

There's a number of values that are clustered over here. But then we've got this value over here.

So to compute these, we use some of the methods that we talked about in the group and aggregate video mean SDD is the standard deviation vare.

One note is that the number high versions of standard deviation and variance compute a slightly different statistic.

The population standard deviation and variance instead of the sample standard deviation and variance.

You can change this by passing a d o f option to them when you call them.

The difference is that they divide by N instead of N minus one.

And we'll but when we're computing the standard deviation or the variance of a set of data that we've collected,

that's a set of observations of the things we care about. We generally are going to want the sample standard deviation or variance.

So outliers are particularly large or small values and they draw the mean towards them and they also affect the standard deviation,

one of the reasons why the standard deviation was so large.

So we've got Michael Jordan's score over here and that that pulls the outlier or that pulls the mean quite a bit.

So the red the red line, again, is our mean, the blue line.

That's what the mean we would compute if we didn't have Michael Jordan's score of forty five.

That's so much larger than everybody else's. So we can see that one value is pulling the mean quite a bit to the right.

This is one of the downsides of using the mean.

So when we have a heavily skewed distribution like this, the skew is what we call it, when there's I mean,

there's a lot of stuff in one place and there's a few values that are way off in one direction.

We can have outliers without skew. We can have some very large and very small outliers.

If we if they're relatively balanced, they actually don't affect the mean too much because one pulls it way high, the other polls it way low.

It's when we were outliers tend to skew off in one direction that the mean starts to become a problem.

So the median is another way of measuring how where values tend to be and the way we computed that we sort the values and pick the middle one.

If there's an even number of values, we take the mean of the middle, too. So we have.

So we have five, ten values. So right in here is going to be the dividing line between them.

It's two and seven. The mean of two and seven is four point five.

So the median X tildy of our of our series is 4.5.

Now if we ask how do we move the median?

So remember, we can move the mean by just scoring more points, even if Michael Jordan is the one who scores all of the more points.

But the only way to increase the median is to increase the small values.

And similarly, you decrease the median by decreasing the large values.

And so no matter how many score points, more points, Michael Jordan scores.

We don't move the median. We can only move the median by having the players who scored the fewest points, scored more points and the.

We could have that that seven scored another point than we would. We would move the median just a little bit.

But primarily, we need to have the smallest values,

need to increase in order to increase our median and or at least the values in the smaller half of the distribution.

One can't really common use for medians as it is when we're talking about income and wealth.

Statistics mean income for a region is almost never reported.

What you report, you usually report the median income because income is usually a skewed distribution.

A few people have a very large income. A lot more people have a significantly smaller income.

And the mean would would be pretty high if the mean would be pretty high,

because you have these large incomes, the median wage ends up being reflecting the typical experience.

Of of people in the population when we have skewed values, so like standard deviation is a mean based measure of spread.

A. Measure of spread more connected to the median is the range which we often want to compute in general.

What's the maximum minus the minimum? And then the inter cortical range and the inter cortical range.

So the median is the point is the fiftieth percentile or the point five court quanti all.

The inter cortical range is the distance between the first and third quartiles.

The point, seven point two five point seventy five positions. If you split the data in half at the median, it's the medians of the two halves.

And so in our data set, we've got it split into two halves are lower, quartile is zero, value is zero.

Our upper one is eight. So the intercourse range is eight. And this gives you it gives you the width of the middle 50 percent of the data.

And so it gives you a measure of how spread out the data is. That is similarly robust outliers like the median is.

So if we want to, we can get a quick summary of a data series with the described method.

The described method prints its results, it does not return,

it makes for a few subtle differences in what you're doing in the room or what you're going to see in the room when you run it in a notebook.

But it gives us the count. It gives us the mean and standard deviation.

We have our main based metrics of central tendency and spread,

and then it gives us the minimum maximum and gives us the men the max and then the the points of off of the quartiles so that we can see the median.

This is gonna be our median. And then we can also see the inter cortical range, two to thirty six.

This particular data set, this is a heavily,

heavily skewed data set because one way we can see that it's skewed is that the median is significantly less than the mean.

That's evident. That's indication of scale. That mean is pulled way up.

Also, if you look at the seventy fifth percentile is thirty six.

But the mean is four hundred and twenty three. So.

If you pick a movie at random, it's very unlikely to have the mean or larger number of ratings.

I don't know exactly where. What what quantity of 423 is going to be at.

But probably somewhere over 80 percent, possibly even over 90.

So. 80, 90 percent of the movies.

So this is the number of ratings per movie. 80 to 90 percent of the movies have less than the mean number of ratings.

It's really emphasizes this difference between the mean and the median.

So how do you pick the mean works well for centered values. There's no excessively large or small values, especially skewed in one direction.

The mean is going to be approximately equal to the median.

A lot of other computations, as well as our ability to try to predict future values, really depend on the mean.

And also the mean is the central tendency so that if we take the total deviation from it, we get zero.

Now, the median is significantly more robust to outliers.

And so we're just trying to describe data and we have a strong skew.

We have outliers on one side of the data. Then we're it's of gives us a statistic that is not as strongly affected by them.

And if we think that as a as I indicated,

as we think back to the question of how do I change this value if we're using the statistic as our evaluation criteria, as our target or our goal?

That becomes very important because if our goal is to raise the mean, say, if our goal is to raise the main number of ratings,

we could do that by just getting a bunch of more ratings for the most popular movies.

But if our goal is to raise the median number of ratings, we can only do that by getting more people watching.

And rating less popular movies becomes a huge difference. It divides high into low the median value.

Is such that if you picked a random random observation, it's equally likely to be greater or less than the median.

That's not true for the mean. But it doesn't tell you, like, how far away the values are on its own.

So it's limited in its ability to generate predictions.

So when we when we think about what's when we want to do it really comes down to the question we want to answer.

And then also the other things are going to use it for. So the mean is answering.

If we distributed the points equally, how many would each player have? And the median gets to the distribution of players about the around the value.

We want to find one that players are equally likely to have more or less.

Another one, quick, is the mode, it's the most common value. It doesn't work for continuous, doesn't really work for continuous variables.

It's really, really useful for categorical variables. If you got a categorical variable that has like three codes.

The mode want to know which one. The common is super, super valuable thing.

It's also useful for integers in any other discrete variable. So wrap up the mean and the median, describe where a value tends to be stick.

Standard deviation, variance range and enter cortical range. Measure how spread out it is.

The mean is very computationally useful. We're going to need it a lot.

But it's very sensitive to outliers, median based. The median and the median based statistics like the ICU are are more robust outliers.

One of the things we're also going to see later on is we we can do data transformations

to get data to be less skewed and then we compute the mean and the transform

space and we can wind up with methods that are going to be give us the computational

benefits of the mean while also not having the outliers causing as many problems.