STATS 250 Week 02(a): Chapter 2 Turning Data into Info

A lecture from Statistics 250 - Introduction to Statistics and Data Analysis.

Instructor: Brenda Gunderson

View the course materials:
https://open.umich.edu/education/lsa/statistics250/fall2011

Creative Commons Attribution-Non Commercial-Share Alike 3.0 License
http://creativecommons.org/licenses/by-nc-sa/3.0/

... especially with a large class.

Alright!

We haven't seen you since last Thursday. I don't see you quite as often as my Monday, Wednesday, Friday class.

If you're still having questions about switching a lab because you had something else change in your schedule,

or still waiting to get into the class, Angie at that statsstudents services email is the best place to contact someone

or you go right over there. She's in West Hall in the Stat department, and she can actually sometimes help you

out much better if you go over to see her personally. So [that] is where you can find Angie,

who is manning my wait-list every morning, issuing permissions for the drops that went through at night,

so if you still have a question regarding that see Angie.

The prelabs are working, it wasn't just our site, it wasn't all the 1570 students bringing site-maker down,

it was some other issue in site-maker certainly, but the prelabs are now available and working fine

as did their ... fix yesterday and re-load,

so I hope you can also see them adequately and ... go back to them as needed

when you start to do your first real homework set and have to make that histogram or boxplot.

Watch the 2 minute video on how to do that rather than have to go through a whole bunch of other stuff to remember how that works.

So the videos are there, we still will allow you to finish up your prelab, you know, by tomorrow afternoon, by tomorrow evening,

the last lab ends at 7, so if you didn't get it done for your lab yesterday because of the difficulties with site-maker

we will extend that, just email your GSI, work out getting that prelab 1, we just want you to do it, so that

you are ready to do some of the other things we do in labs and homework, so you get your point if you do it,

you get no points if you don't ...so complete that please.

Alright, urh, the other item that's coming up on Thursday is when you go to your lecture book homework tool

you will start to see a homework there for you on Thursday morning.

This is my testing student homework side,

but I have the ability to see anything that's hidden and I have created that first homework, and so, once you click on

the little pencil thing that say "I'm gonna start writing that homeowork" there's 3 questions ... and you'll be able to click

through those 3 questions and put in your answers, some of them are multiple choice, some of them are typing up

a little explanation, and one of them is an upload of a graph to get that practice too.

The practice homework going up Thursday is practice. I'm putting it under the "required hand in" just so that

there are points on it, and then it will be submitted by the Wednesday after at 11pm automatically for you.

The GSIs will get it and they will grade it to go through that process, because we have a few new GSIs this term who haven't done that yet.

And then you will see what the graded homework looks like back to you, maybe with some feedback too if

you didn't get something right. It is again just practice so if you're not in the class and haven't subscribed yet that's fine,

if you don't do it that's fine, it's not gonna affect your grade, except that it does give you a full practice,

so that when that 1st homework really does come up and you're working on it maybe just the little issues or questions

you have will be gone ... by then.

So that will be available, I'll probably give you another highlight on that again on Thursday.

So, office hours have also started but they're pretty quiet right now, but if you do have an issue,

if you do want to see how virtual sites works with someone that could help you a little more on that

come on in, 274 West Hall, Monday through Wednesday, we're there from 9 to 9.

Thursday 9 o'clock in the morning 'til 5, and Friday just in the morning time, 9 to 12.

You're welcome to go to any of the office hours even though they may not be your GSI's office hours,

and instructor office hours are also posted on c-tools, you can go to myself or Dr. ?.

The first real homework will not be open until January 20th, but the practice one will start this Thursday

and go from Thursday morning and you got a week basically to do it.

The following Wednesday night at 11pm, it will be automatically submitted for you.

We're working through chapter 2. We actually should finish much of that up today, maybe a couple more slides on Thursday.

For Thursday, I am asking you to read the chapters 3 and 4.

They're short chapters, they're reading chapters, (um)

there are not really any formulas in those chapters, maybe just 1 small one.

I've posted partially complete notes for those 3 and 4.

You've got the notes in your binder and you can look at a few of the answers.

I am going to go through the ones that are missing as a recap of those chapters at the beginning on Thursday,

and then I'll put up a clicker practice quiz, 5 points, I'll actually keep those scores just to show you the histogram

of them later; but it'll be as if it were a little intermediate practice for you to see if you got those couple ideas

from those 2 reading chapters, more concepts on different types of bias, an experiment versus an observational study and things like that.

So try reading through the chapters or at least just look at the notes that are online, they're on c-tools under lecture info,

just read through them before Thursday and then come ready to recap ideas and try out a little practice quiz.

So those are my announcements, questions or comments on anything before we recap our histograms that we ended up with,

question or comment on anything?

Alright,

then we're going to recall the 2 types of variables. We're in chapter 2, the 2 types of variables were what?

categorical or ... quantitative.

And what was the very first graph we looked at that described a quantitative variable's distribution,

that gave us the values or categories that your variable can take on and how often they occurred? and how many?

that first graph we looked at had bars and it was called a

bar chart or bar graph

and then we saw the pie graph along with that,

and the basic thing for a numerical summary is just summarizing the counts or percentages that fall into

each category, we did that for the sleep deprived status variable,

and then we went on to our quantitative variable where we had students say how much hours of sleep they typically get per night

and the quantitative variable to show its distribution, and by distribution we just mean,

give me an idea of are that that variable can take on

and how often they tend to occur,

and the nice picture for the distribution of a quantitative variable was also one with bars but its wasn't

a bar chart, it was called a ... histogram.

And the histogram is what we looked at, we gonna recap that on page 8 of your notes, and then

move on to start summarizing the data that's quantitative with numbers.

Numbers such as the mean and the median, and we'll get through at least to standard deviation by the end of today.

So, page 8, we had our histogram of the Amount of Sleep for College Students

we did a clicker question at the very end where we talked about the shape of that distribution of amount of sleep

we talked about it being approximately symmetric and another word that would work would be unimodal

one main mode or peak, and then some of you did want to select the skewness aspect

it was, it's not perfectly symmetric, a slight skewness to the left, but it's quite slight,

I would leave that as a secondary and not the primary thing I would be looking for.

The summary here approximately symmetric, unimodal, centered around 7 hours,

most of the values between 4 and 10, and I can kind of picture that histogram back with that description.

So this type of description is what you're asked to write out, even by hand, with your histogram that you make for your prelab.

Alright ... so lets move to looking at nummerical summaries that would be appropriate.

Oh, we have one more histogram here for "What if".

So, you have a response, you measured for a study, it's quantitative, so you make a histogram.

You look at it graphically first, and this is the picture that you get.

So this is not unimodal, it's called what? [softly: bimodal]

Bimodal, more than one mode or peak. A bimodal distribution, what would it tell you?

I certainly see 2 groups, kinna 2 subgroups.

I see a groups of observations, people that had low scores or low values on the response, and another group that had high scores here.

So I see a bimodal distribution.

I see that there seems to be 2 subgroups in my data set.

And that's one thing I would comment on and want to find, try to figure out why that occurred.

So there appears to be ... 2 subgroups.

I would not just note that and then go on and start calculating means and standard deviations because

I want to first figure out why? what made these 2 groups?

Investigate why.

Maybe it turns out ... that the lower observations were for the males in your data set,

the higher observations for females.

Maybe it's the old versus the young, or some other aspect that you measured.

You want to try to investigate why. It might be a gender issue, it might be an age issue ... it could be ... a region issue,

or something else that affects that response and gives you these 2 subgroups, which is why in a data set

we don't just record only the outcome of interest, we record a lot of other variables too,

they may not be the main response variable but they might help to explain features that we see in our response,

we might have to control for or account for age or gender in our analyses if we see such a picture like this.

So I would want to try to figure out why, so we look at our data set with other variables that have been measured,

may be more demographic but they might help us to understand what's going on and

then of course, I wouldn't just leave the data set group together after this 'cause if you calculate a mean

the mean's going to be sort of a balancing point and that's not really reflecting of the group very well.

wouldn't make sense to summarize this data together.

I'd rather probably lead toward analyzing the data separately by my subgroups from there on out.

So we might end up analyzing data separately by my subgroups,

from there on out. Or at least include that variable that just gives us that distinction as a variable to help control for that factor.

It's important to look at your data first.

If your histogram turned out to be like this and I asked you "Should you calculate a mean?"

Well you can, you can always have the calculator do that for you. SPSS gives you the mean by default with the histogram

so it doesn't even, you know, make you (you know) check first, but it would not make sense to report it.

It would not be meaningful in this case.

Alright, couple of histogram comments are laid out there for you.

The bar chart, we usually have a gap between the bars, 'cause it separates the different categories, whereas

histograms there's often one right next to the other, unless of course there's a gap and there's no observations there

and the you have an outlier maybe sticking out at the end.

How many classes depends ... on a little bit of judgement but the computers and calculators will do a default kinna algorithm to work out

a reasonable number of categories. You can always go in and change that slightly,

you don't want too many, because if you had too many categories then you're going to have just a few values

in each, it's going to be kind of a "flat pancake" kind of histogram.

If you have too few, then you have everybody in just a couple of classes, and that's not

looking like a very good picture of a distribution either.

We like to put "relative frequency" just another name for "proportion" or "percent" on the Y axis whenever you're comparing observations.

If I wanted to prepare the histogram for male versus female college students, in terms of Amount of Sleep,

I could do that as long as my axes were matching up.

Lots of defaults, lots of options, and you get to see that in SPSS a little bit too.

Alright, urh, last comment here, is just that one of the sections talks about "dot plots" and "stem and leaves plots" in your textbook

and we're not going to have you do those particular types of graphs but some of the examples in there do comment

on the shape of the resulting distribution. So it's still good to just kinna glance through,

but you won't be asked to make those particular types of graphs.

The histogram's a good choice overall.

The example that you have on page 10, you can try out on your own ... that was a, one I think on an exam in a spring term.

It is another histogram of a quantitative variable. See if you can go through and answer those couple questions.

The solutions are already posted on c-tools. I'll be posting lecture notes filled in up until the end of drop/add

and so, anything I missed though or also skip in class I will post no matter what.

So under lecture noted you can find part 1 of chapter 2 notes already filled in.

Let's turn to the numerical summaries for the rest of today.

Numerical summaries only appropriate for quantitative data, even if you had categorical data that you coded

it may not make sense to do any kind of averaging or finding a median there.

But it certainly makes sense for a quantitative variable where we saw a reasonably unimodal, homogenous set of observations.

So you've all calculated a mean at some point, and a median perhaps.

Measures of center.

We're gonna have a couple formulas along the way, they're going to be based on data looking like those Xs

X1 is the first observation in your data set, X2 is the second one. That is representing your set of data.

So n represents the number of items in your data set, its sometimes called the sample size.

And how do you calculate a mean of a set of data?

Add them up, divide by the total number, right?

The formula for that then would be what?

X1 add X2 add all those observations up,

up to the last observation and divide by the total number that's represented here by ... n.

The symbol for that type of mean, when it's the mean of your sample, which is usually what kind of number or data you have.

Usually you don't have the entire population of values but rather just a sample.

It's called the sample mean, it's represented by X - bar.

It's probably a function on one of your calculators where you can put some data in and press that to

get the mean, and it's the mean of your sample.

We have a different notation if it is the mean of the population. We've a greek letter "mu" µ that represents a population mean

but usually what we're looking at when we're summarizing data it's from a sample, part of the larger population,

not the entire population.

Anything calculated on a sample, again, is called a statistic ... so that sample mean is easy to find,

um, a shorthand notation would be to use that summation notation, the summation there, that big "sigma" Σ

just means add 'em up, add up whatever's after it, and so that's adding up the Xs and dividing by the total.

The median, how do we find that?

The median is the bullet value that's in the middle, but you have to first order your data from smallest to largest.

Let's say you had an odd number of observations, 5 of them,

then there is, if you order them, a middle value. And that would be your median.

What if you have only an even number of observations, say 4 ... so the median's gonna be right here.

Any value that's between those 2 numbers could be the median, it would divide your data set into 50% above and 50% below,

but in this case we define it to be the average of those 2 middle numbers so we all get the same answer.

So the median ... when n is an odd number of observations,

the median or 'm' is going to be the middle observation, 'cause there is a unique one.

Maybe the same values, some on the either side, but it will be THE middle value,

whereas if your number of observations is even ... then you're going to define the median to be the average of the 2 middle observations.

Alright, let's try it out.

We've got our small set of data from that study where students were looking at

whether you're getting the same amount of french fries in your small orders, we have the data provided for you there.

Again the note right here is whenever you have quantitative data, the first step should be,

not calculate the mean, some standard deviation or something like that but to graph it.

Proper graph or histogram to show the distribution, to show the values that are possible is a (uh) histogram for quantitative variable.

I say we have sort of a unimodal, bell-shaped picture again.

Roughly symmetric and a range from the 60s up to the lower 80s, and it's centered around the mid to lower 70s.

So there's our histograph. It makes sense to calculate a mean to summarize this data, we have some variability there but it is somewhat homogenous.

So let's calculate that mean first. How'd we do it? and what would it be represented by?

The symbol would be

X - bar.

We'd have to plug into our calculator or ... computer, this data set, add up all 12 observations, and divide by 12.

Now if you have a calculator. I would, I rarely ask you to calculate a mean on an exam, I know you can do it.

You can do it with a computer or calculator. I usually give you some basic summary measures.

If we were to calculate this, would a value like 82 even make sense?

Right? ... 82 would be way over here.

Is that the balancing point of this?

Does that look like the mean?

So you just want to make sure that any calculation you do do can take it back to the picture you got that it makes somewhat sense with that picture.

82 would not make sense; 69 may not start to make sense either. But how about more in the middle? A 73.6,

and the units here are always good to include when you're reporting some numerical summaries and

the units here are always good to include when you're reporting some numerical summaries,

and the weight in grams. 73.6 is visually about that balancing point, it makes sense with my histogram.

We have had pictures of a histogram, where we ask you to pick what you think might be the mean, and maybe which one might be the median

and you should be able to kind of visualize that but not have to calculate, just from the histogram, and know how would it relates back.

Alright, how 'bout the median?

Median you need the data in order, and I provided that for you. We have 12 observations,

so the median's going to fall in the middle between ... that 72 and that 74. So the median here will be what?

73, it would be the average of the 2 middle observations... 73 grams.

Now the mean was 73.6 and the median was 73. The fact that those are kind of similar, does that make sense here too?

If you had a roughly symmetric distribution, then the mean and the median would be very close to one other, and that supports it also.

Alright, well we have a couple of "what ifs" there on the bottom.

What if we had put the data in our calculator quickly and instead of putting the 63 in we actually just put in a 3.

So our smallest observation was a 63, what if we had accidentally just typed in the 3 only.

How would that affect my measures of center here that we just calculated? How about the median?

Would the median change at all? No.

The median doesn't use all the values, it uses them in terms of its place, in terms of there size or order, but it doesn't use every value in a data set.

So the median's not going to be affected at all. The median would stay the same.

How about the mean?

It will change; the mean uses every value that 3 would enter in instead of a 63 is part of the total on the top.

The mean is going to do what do you think? It's going to go up? go down? It's gonna go down.

One smaller value, being smaller yet, is going to drag that mean down.

That the mean would decrease

The mean would decrease

to 68.6

There would only be a couple observations below the mean, 10 of them above it.

Much different that what we had before. So the mean would be smaller.

So we talk about the mean IS affected by extreme observations, the median is not so.

It is more of a resistant measure.

Top of the next page, let's fill that idea in.

The mean IS sensitive

to extreme observations whereas the median is our more resistant measure of center.

So which measure of center might be the better one to report if in looking at your histogram it was strongly skewed

maybe with a couple of outliers, either on the high end or low end?

The median.

The median would paint a picture of what really is more of the middle observation, where more of the typical values

tend to be, rather than the mean. The mean can be affected by those extreme observations.

Very good!

Alright, couple pictures to show the relationship between the mean and the median.

What is the descriptor of this first one again?

We call this ... bell-shaped, symmetric, unimodal, all of those words would work.

And, the symmetry is the main idea here. How would the mean and median compare? and where will they be?

Both right in the middle ... and approximately equal to each other.

The smoothed out histogram being shown on the right,

upper right, that is also

symmetric. What's the other word to describe this one?

UNIFORM, not unimodal but uniform. It actually doesn't have really any mode, or they're all modes if you will,

'cause they're all equally likely. Symmetric, uniform, it's still symmetric.

So it would be quite easy here to also find ... the mean and the median, you just need to find the midpoint.

So if I gave you the end points, you would be able to find the mean and the median for that one even without

having more detail of how many observations and all the individual values.

Alright, you got two skewed distributions, This first one here is again what? skewed to the?

Skewed to the right, look at where the tail ends up being pulled out. Skewed to the right.

That is more for income data, sales of home. That type. And skewed to the right means we've got a lot of values that are small

and then we're throwing in a few really large values into that data set, and averaging.

So the mean and median are not going to be the same.

Which one will tend to be higher?

Got a bunch of ones, twos and threes, and you throw in a couple of really large numbers, it's going to pull the mean

towards those large values.

Alright...Skewed to the right, relative to each other, the median would be the smaller one and be less than the mean.

Exactly how much smaller, and exactly where they're placed.

Median's going to be the value on the axis so that if you looked at the area to right and left it should be both about 50%

'cause the median divides things in half that way. The mean should be more visually the balancing point

of that histogram or that smooth curve. But relative to each other that's how they should compare.

The mean gets pulled in direction of the tail.

Our other distribution here is skewed to the left.

Very typical of your exam scores in this class.

The mean and median again will not be equal to each other but it would be the mean that gets pulled down

compared to the median ...in this distribution.

Which is why I often report the 5 number summary or the median ... as far as my measure of center

when I report scores on your exams, rather than the mean, the mean can get pulled down by

that one or two or few low scores that end up happening.

And what's the descriptor for our last graph there?

Bimodal

Two main modes or peak, it's still a roughly symmetric

and what would the mean and median be there?

Kind of in the middle again.

Does either one of those measures of center really represent what we think of as the center as being that sort of typical value?

Not very well! ... So neither does a very good job as a summary measure here.

One of the old exams had a picture that ended up being somewhat bnimodal, showing distinct clusters

and the question was "should you report the mean here for this quantitative variable? is that an appropriate measure?"

If yes go ahead and report it, and it was right in the output summary, if no, explain why not.

And for that type of picture, I would say no.

And the reason is that you seem to have two subgroups and you shouldn't be aggregating them, combining them together.

Alright, there's measures of center. Mean and median are the typical ones, we know when it's appropriate to use either one

and when it's appropriate to perhaps use one over the other.

Median preferred when its skewed, or strong outliers in your data set.

Measures of center are done. Measures of spread come next.

So what if you had the distribution of scores on an exams that were somewhat

unimodal, bell-shaped, symmetric, and the mean was reported to be 76

and you scored an 88, how do you feel?

Good?

You're above the mean at least, right? um?. How good should you feel?

Let's suppose 76 is our mean there, and 88 is right about here ... so there's one model.

Would you feel better with the distribution of scores looking like that number one, or number two?

Number two, right? Number two relative to the peers and the scores, you're looking much better

compared to the first distribution. They both have the same shape, they both have the same center, but

the differ in their spread, or variation. And it's important to know that aspect of your model too.

And a score from one distribution being 88 from 76 may look better relative to that idea of how spread out the values are.

So a couple measures of spread to go along with measures of center.

The two easiest ones of course, or the easiest one that is, is the range.

Just look at the overall range of your data set. The range is defined to be the max. minus the min.

So if I have you calculate the range I want you to take the largest value and subtract the smallest one off and get that spread of that 100% of your data.

If I ask you to just comment on spread, one comment you can make is "it goes from 10 to 22"

and that's kind of giving me the range, but it's not computing it.

Another way of giving you some idea of the breakdown is to report some percentiles.

In fact alot of your standardized tests report your score, and what percentile you got.

In the two pictures we just looked at you would have a higher percentile when there is less spread in you model.

So percentiles tell you a value so that you know what percent are below it and what percent therefore are above it.

Common percentiles, we've already done one of 'em.

The median is actually a percentile of your distribution. It is the 50th percentile 'cause it cuts you data set in half, half below, half above.

If you were to take the lower half of your data set, below the median, and pretend that's your data set and find the median again

that would be what is called the first quartile or the 25th percentile, and the first quartile is denoted typically with a Q1.

If were to take your data set, find the median and take all the values above the median, find the median of that set again

you'd have the upper quartile or Q3 or third quartile ... also known as the 75th percentile

So it's just dividing each half of the data set in half again,

and those two quartiles then give you another little bit of positional idea in your data set.

If you take those numbers, the median, the quartiles, along with the min. and the max. and put them in a table,

this is the way your textbook lays out this summary, it's called the 5 Number summary.

It gives you all in one kind of picture here. A couple measures of spread.

One measure of spread would be the max. minus the min. so you get the range by looking at that distance.

You get the median as your measure of center, and then another measure of spread that sometimes used

instead of the overall range is called the interquartile range or I Q R.

Interquartile range.

It's the measure of the spread for the middle 50% of your data, 'cause the range depends on your two most extreme values.

What if one of those values is an outlier, so your range looks distorted to be quite large

when most of your data is really in this range instead, a smaller range.

IQR, interquartile range.

So nice set of summaries for when you have skewed data or outliers would be a 5 number summary.

It gives your center and spread in a couple of ways.

Let's try it out. Our French Fries data set again

Why don't you go ahead and try to work out the 5 number summary, we found the median already

Min. and max. are easy to put in.

And then let's find those quartiles.

So 12 observations,

middle,

(silence)

So I'm finding Q1 and Q3. Not too bad here right? You just take your data that's below and above that median position

What is going to be Q3? ... Any of these lower observations, there's 6 of them down there,

so the middle falls right here now, between the 69 and the 70.

So Q1, the lower quartile would be what? 69.5 and on the upper side there's another 6 observations in that upper half and

finding the middle there would be Q3 78.5.

Alright, so the range is easily found, the actual computation of the range would be the maximum value minus the min.

The overall range of the data covers ... 20 grams.

For a spread.

The interquartile range defined again to be Q3 minus Q1

Looking at the spread of the middle 50% of your data would be that 78.5 minus the 69 ... 69.5

A difference there of only 9 grams instead

So the middle 50% cover a range of 9 grams. You can compare from one data set to another with common measures such as IQR

to see which one looks like it's more spread out in terms of that middle 50%. ...Now quick "What if" here

What if that 83 were not there? So instead of having 12 observations, I have 11,

so now what will be my median of this data set?

The 72

and looking at finding Q1 and Q3, what would I do?

So now, I'm looking at just the 5 observations below the 72, I'm not going to include the 72 in that lower half.

So the definition of Q1 when you have a median being one of the values in your data set is everything strictly below and strictly above

that works for both an odd and an even.

So them my Q1 would have been a 69, and looking at only the 5 observations above, my Q3 would have been the 78

just so you know how the calculations the computers are doing, that's typically the method of using it

sometimes they specifically take .25 times the sample size to find the position of that 25th observation

or 25th percentile, and even interpolate, but this is method that most calculators and computers use.

Alright, the test score example is another one that you can try out on your own,

it's really not too difficult to work through, just report values from that 5 number summary.

It's very similar to one of the examples in chapter one, which I asked you to read initially,

and it went through a 5 number summary there and pulling out a few values.

So try that one out, that would be posted up on c-tools ... very soon.

We're gonna look at another graph today ... called boxplots.

You probably saw that in your prelab if you looked at your prelab video or maybe even ... saw one in lab.

It is a picture of your 5 number summary ... in graphical form.

So you take your 5 number summary ... you take your quartiles Q1 and Q3

and you use those to form your box. The length of the box is your IQR, visually,

so Q1 and Q3, so this length here is really your interquartile range, it's visually showing you that spread of the middle 50% of your data.

Inside the box, wherever it occurs you put your median with a line drawn in the middle of the box.

Now here it's shown more in the middle, it could be that your Q1 and your median are the same, if you had a lot of repeats in your data set it's possible

so there might not even be a line in the middle of the box.

And then you do a little check. You check to see if there are any outliers, 'cause if there's outliers,

I want to still show them visually in the graph that we're making here.

There's a rule for checking for outliers, it's called the 1.5 times IQR rule

'cause you calculate 1.5 times IQR, and that quantity's your step, your amount that you go out from the quartiles

you take Q1 and you go down one step,

you take Q3 and you go up ... one step

and those values are your fences or your boundaries.

Turns out with a little bit of theory you can show that any values that are OUTSIDE those fences are unusual,

they would be deemed an outlier using this rule.

So you take 1.5 times the IQR, you go out a step from you quartiles, put like a little boundaries there, fences

and say do I have any observations that are outside those fences?

If I do, I want to plot them separately and draw attention to them 'cause they're sticking out from the rest,

if I don't ...then just extend the lines out to the actual min. and max. 'cause there were no outliers in that data set

so you plot them individually if you have any, if not just extend them out to the smallest and largest observation

if there are outliers you say apart from those outliers what are the min. and max. and draw your boxplot accordingly.

So this boxplot right here is for your 12 orders of french fries, the weights.

I do not see any points plotted outside the length of those boxes, those whiskers going out individually,

I didn't have any outliers, in this example. If you remember the histogram that we looked at a little bit ago

it didn't have any outliers either. With this set of 12 observations.

So I see no outliers on either end here, they would be plotted separately,

I don't see any outliers, so what we're going to do is confirm that there aren't any outliers

trying out this rule once ... and then pretend that we do have one larger value and see how that affects the boxplot overall.

Boxplots are gonna be made for you typically, and I just want you to know how they're constructed

so let's do one little check on how that rule works.

What we're doing basically is verifying that there are no outliers in our data set.

We didn't see it visually in the histogram, the boxplot we just saw there didn't show it , so let's see

what that rule is that shows us there aren't any outliers.

The interquartile range is the first thing to compute, and that was a diference of 9 grams, we just did that together ... a minute ago.

Calculate that thing called a step

1.5 times the IQR.

So this is my step amount ...13.5.

13.5 is not put on my actual boxplot anywhere, it's just the amount I'm going to go out from the quartiles

to figure out where these fences can be drawn or thought through.

So the lower boundary or fence is taking Q1 and subtracting one step. So go from 69.5 which is Q1

subtract off one step amount and come up with your lower boundary which is a 56.

Now that's not necessarily gonna be a value in your data set, but you're going to ask yourself,

"in my data set did I have any values that were even smaller than that lower boundary?"

Any observation that fell below ... 56. What was our lowest value?

63. So we have no outliers on the low side.

Any observation that fall below this lower boundary? no, so there are no low outliers.

If my lowest value was a 55 or a 54 that would start to be deemed an outlier, and I would be plotting that separately

but there is no outlier on the low end. The 56 didn't appear on my boxplot that you had at the top of that page

but that is the boundary that I'm using to determine if there would be a point I would plot separately.

What about on the upper end? The upper boundary or fence is 78.5 going up one step

This is a boundary of 92.

Again, you may or may not even have that value in your data set, but you're gonna be asking

do I have any values that go beyond that number? Any values on the high end, above 92.

Our largest value was? ... 83

So the answer here is also no. So there are no ... high outliers or large outliers.

So we've just confirmed the graph that we just made was the graph using that rule, just that it didn't have any outliers

so we just drew the lines out to the min. and max.

What would happen if there were an outlier? How does the boxplot change?

So we're going to pretend on the next page that that largest value of 83 is really a 93 so we will have an outlier

and see how the graph changes.

So if we change from 83 to 93, just makes the one large value larger a bit larger, does the median change?

is the median affected by those extreme kind of values? no, still 73.

In fact the quartiles are going to be the same too. We're just shifting that one large value out there further

so it's still going to be the same Q1 and Q3. Everything's the same except for the largest values which is a 93 now.

The boundaries that we computed on the previous slide. That 56 lower boundary and that 92 still stand as the boundaries to determine if we have outliers

but now on the high side, 92's out there and I have a 93.

It goes beyond ... so now I do have one high outlier, the value of my 93.

That's the point that's plotted separately here.

A different symbol's used depending on the package, it might be dots or asterisks, but any point that's plotted separately is an outlier by this rule.

There actually could have been, without knowing the data set, at least one outlier here, 'cause there could be two values at that same high value

But we plot that separately and then ... the lower box whisker goes out to still the 63 'cause we didn't have any low outliers

That upper whisker or line goes up to an 80. Notice that this doesn't go up to the 92, which was our boundary, 'cause we didn't have any values at 92,

that was just used to determine if we have an outlier or not.

The upper end here should go up to the largest value in our data set that's NOT an outlier which we would have plotted separately

and our largest value that is not an outlier is?

80

Why does the line extend out to 80?

It's the largest value

that is not an outlier.

The 50, the lower boundary of 56 and the upper boundary of 92 don't appear on the graph.

They are sort of your invisible fences, one kind of fence right there, and one way down over here just to

determine which values, if there are any, would be plotted separately.

So now I can see truly where most of the values are. I see the outlier plotted separately, I see the gap

which I would have seen in the histogram too.

You have the histogram and just that one larger outcome was moved way out larger you would have

had a gap of values and seen that outlier through the histogram, but also drawing to your attention the boxplot.

Alright ... so the boxplots are going to be made for you, I don't like to have you work our this IQR rule by hand

and try to draw them out, but know what these values are that apply separately and how they came to be plotted that way.

Couple notes then to fill in.

Boxplots are very helpful when you want to make comparisons.

You can make side by side boxplots, one boxplot for males, one for females, to compare the two distributions quite easily.

So side by side boxplots are good for ... comparing ... two or more ... sets of observations.

It automatically puts them on the same axes, the same scaling so that you don't have to go in and click the axes and make the two histograms have the same

X and Y axis scaling, so you can compare them, puts 'em right next to each other.

We do that quite often for a visual check when we look at multiple comparisons for different groups,

watching out for those points that are plotted separately.

They're called what again? ... They're outliers.

And my point here is that they are still part of the data set.

I don't want you to ignore them, in fact they were plotted there separately so you would be drawn to see them.

Maybe that outlier is the most interesting observation in your data set.

Here's all the data, that observation's really good. How did you get to be THAT good for that combination of factors that I put together?

I might wanna focus on that one rather than the rest.

Alright, so they're still part of the data set, don't ignore them, they're there to show you that they did stand out

from the general rest of the data set.

One thing that boxplots do not do very well, that we can't see from a boxplot alone very well is shape.

You can't confirm shape from a boxplot only.

What graph does a much better job of showing us the shape of the distribution of our quantitative variable? ... A histogram.

Your boxplot can look ... beautifully symmetric but your data set underlying it may not be.

Just because your median is right in the middle to Q1 and Q3, you don't know what happens between Q1 and the median and Q3.

How are the data distributed that way?! They could be all clumped to one end, and then clumped to the middle on the other side,

so it may not ... have that same pattern that you're thinking that the boxplot tends to show.

So it doesn't confirm shape, you can start to see skewness. A boxplot showing some skewness would be

a long tail going out and a bunch of outliers on one end, so you can start to see or visualize that,

BUT it still doesn't confirm the shape, we would wanna use a histogram instead.

And then one of the graph you're gonna look at in labs #2

um, no labs next week, remember?! cause there's no Monday classes, so we don't have labs at all for next week.

But the 2nd lab we're gonna look at Q-Q plots... That'll show you whether you have a bell-shaped model in a better way.

We're gonna look at time plots 'cause a lot of data we gather over time. So a couple more graphs that we'll look at.

And then my last comment really helps you on an exam ... or a quiz if we were to give one

and that is that "Show me what you're doing when you're reading values off the graph!"

If you're looking all over and reading that value that was the outlier a 92 instead of 93

well, I might give you credit for something that's reasonable, based on where the axes are and how fine they are for you to read values from.

Show me what you're doing so I can see your approach ... and still give you credit for the right process.

Alright, so on the next ... Oh! I got my comic of the day

Recycling, go green (sings softly: aluminum cans, bottles and) (soft laughter) Recycle those boxploh ... uh yeah, Ok!

What I have next for your is to try out a question that was on an exam, in the past. We have here scores

on a standardized test, for children, and some of those children ate breakfast, some of them did not.

SO we have their scores compared side by side. Side by side boxplots. So I'm gonna ask you to review

the notes on that page 16 that we just went through, or keep them in mind anyway, work with a neighbor.

Try out these 3 questions, and then we'll click them in. I'm gonna give you some choices then for the actual answers

but try them out , and we'll see how you do.

This is a chance for you to talk with your neighbor and work out a problem.

(silence)

(inaudible talking in background)

So this question that you have, was on an exam before but not with multiple choice on the exam.

If your answer's not exactly there ... it's probably close to one of those answers perhaps ... pick the closest one.

Do you notice in the questions there are sometimes words that are bold? ... or italicized?

That usually means they're important ...to help you guide to the right boxplot.

We're asking here for the approximate lowest score or grade... for a child who... "does" have breakfast.

So I want "Do you have breakfast? YES!"

That's the boxplot I wanna work with.

Looking pretty good! Here's your distribution, most of you indeed are picking the right answer. Which is ... B) ... 4.5

Plus 5 points minus 10 over here

Alright!

The... lowest value 4.5 is this value right here. What is that called? That's an ... outlier. (Outlier)

But is it still part of the data set? (Yes) (It is)

It's the lowest value. There could've even been two children that had 4.5. But it is the minimum.

If you ignored the outliers, yes, then the next smallest observation is 6.

But 6 is not the minimum value here, for that data set.

And of course the other, um, answer of 4 was for the other group in case you went to the wrong boxplot.

Highlighting that.

Now if someone wrote, and I saw someone in my other classes wrote 4.6, I would have given credit for that

'cause that's close to what I have there.

You know it's pretty much right in the middle, but if you read it off as 4.4 or 4.6 you'd still credit,

especially if you were to circle it or bring it over and kinda tell me what you think that number is, on the axis.

Alright, a little more curious about this ... next question ...and go ahead!

Among children who did NOT eat breakfast ... 25% has a score of at least how many points?

(inaudible taking in background)

We're a little more split on this one. (Uh oh)

For our question 2.

Alright, let's take a look.

Wanna change your answer? Um?

Question 2, I see a 25%, so I'm sort of thinking a quartile rather than the median. So the choice over these three.

Let's take a look. I gonna tell you right up front that the right answer is ... A)

Let's see WHY?

Alright we're looking at which boxplot? The one with the children that did ... not eat breakfast. So the "NO" group.

I do see what? what is the median for this one? About 6...and a half.

That would be my median... and my quartiles are about this 5.5 ... and then about 7.5

And I see a 25%, but 25% does not necessarily mean it's gonna be a Q1.

When you take Q1, you got 25% percent below and what percent above Q1? ... 75

When you take Q3, you got another 75, 25 split. There's 25% above it and 75% below it.

So depending on which quartile you use depends on which direction you were looking to go for that 25%.

So often in these kind of questions it's almost easier to just say "I know it's a Q1 or Q3", 'cause it could be kinda tricky that way.

Put each one in and see if it makes the sentence correct.

So let's try the one that was the most common answer. Most of you took 25% and said "oh it's Q1, 5.5"

So what if we put 5.5 in our sentence here? It says ... you're looking at the percent of students that had a score of ...at least ... 5.5 points.

So at least 5.5. points means 5.5 points or ... more.

And what percent of the students had a score of 5.5 points or more? or at least 5 and a half points?

75% ... NOT 25%.

There are 75% ... that had a score of at least 5 and a half points.

Let's put in the other Q3 of 7.5. We put in 7.5 and say what percent of the students had a score of at least 7 and a half points.

At least means that many points or? ... more. And what percent are there?... 25%

The correct answer A).

If it were "at most" instead of "at least" ... if I had at most there then you would be calculating that many points or?... less,

and then you do want 25%, so that's Q1.

So is it "at least" or "at most"? is it more than or less than? and you want the 25% in that spot , or the 75% tells you which one to use.

You just have to think it through a little bit, or kinda try out both numbers and see which one ends up

making it correct... noting the direction you're going and what percent you want there... in that direction.

Alright, so that one I put there specifically 'cause that was one on a quiz, I think for spring term and I did have a number get it wrong.

I'm doing it now so you won't get it wrong if it were asked somewhere on your exam.

Our last one is a true/false. Let's see what you choose there.

We have a very nice symmetric boxplot for our "not eating breakfast" group.

The statement is that this implies the distribution for scores, is set nice bell-shaped symmetric distribution.

No outliers in that one.

And ... most of you paid attention to the notes on the previous page.

Notes on the previous page would lead you to say "This is false".

It is false. ... It does not imply the distribution is bell-shaped, just because the distribution is symmetric.

A symmetric boxplot ... does not necessarily guarantee that your data when you plot it would be symmetric.

It tells you that you median and your Qs, your Q1 and Q3, are nice and symmetric from each other,

and even if you went up to a max. and min. it would be the same in distance.

But what happens in between, to that 25% chunks you've got?

They could be distributed in different ways that are not mirrored or symmetric.

And even if it were symmetric, does it mean it's gonna be this type of symmetry? bell-shaped?

Isn't there other symmetry models? Bimodal will give you a symmetric boxplot. Just the same as a unimodal bell-shaped curve will give you a symmetric boxplot.

So you can't see the clusters.

You can't see bimodality from your boxplot. It hides that aspect of shape.

Boxplots hide clusters and gaps, you don't see them as readily. So shape is not best to imply from any boxplot.

Look at the histogram along with it. So false!.

Alright ... now we did have one outlier, at least one outlier in our data set, there could actually be two children or more there, at that low value.

What do you do with outliers?

A small section in your text. Section 2.6 talks about how to handle them.

Gives you a couple good examples for you to look at.

The primary idea is that you just can't throw them out, and say "oh they don't match, let's just not use them".

You have to take a look at your data, it might be a legitimate value, it might represent something that's been gathered under different conditions.

Joel did the measuring of these parts, knew how to operate this machine. Jim came in while he went out, to the bathroom, and did a measurement and it was way wrong,

and that would indicate that you shouldn't use it 'cause it wasn't calibrated correctly or whatever.

It could just be natural variability, and you occasionally get a value that stands out,

but more than likely it could be that someone else made the measurement. The measurement was entered incorrectly,

you can go back and check your records, just a switching around of the values or something,

but you can't throw it out unless there's a legitimate reason to say it doesn't fit with the rest of the data for this situation,

it's measured by different person or on a different machine ... or under a different condition.

It could be that it's an interesting value to look at, and that might be your focus.

It could be that that particular person or item belongs to a different group.

That was the only male in your data set and everybody else that you asked the question of were all females.

So that could give you a different rating or response.

Alright, so, some good examples there. We have techniques that are more resistant to outliers than others,

we have a median that's better to report than a mean ...if there were an outlier.

Question in the back? ... or just stretching?

Alright, one last measure of spread ....Standard Deviation... and we end with our pictures of the day.

So how many of you have heard of standard deviation too as another nummerical summary?

How it's calculated, how you interpret it, would be our last focus.

So if you have the median as your measure of center, then you're most likely gonna report the range or the IQR, interquartile range, to go along with it.

If you had the mean, as a reasonable measure of center, then often you see next to that is the standard deviation as a measure of spread.

'Cause what does the standard deviation do? It measures the spread of your observations from, the mean.

The standard deviations takes every value of a new data set, and looks at how far away every observation is from the mean.

We're going to interpret the standard deviation in the following way.

We want to interpret it as being roughly the average distance, that your values are ... from the mean.

It's that typical distance, that average distance, of the values from the mean.

Now it's not exactly the average.

If you were going to calculate the average, you'd have a little different formula than what we're gonna write up here.

But it does take distances from the mean, and we want to interpret it or view it as being roughly that average distance,

so it can be kind of a yardstick for us ... to see how far away we are from the mean.

So here's how it's computed. Let's work that out.

We're gonna take every observation and look at how far away it is from the mean.

So the first observation would be X1

and what was the mean of a sample again? We represent that by X - bar, that represents the sample mean. The mean of your set of data.

So there's the first distance, and I'm gonna calculate the next distance, my next observation from the mean.

And I'm gonna do that for all of the observations.

Every single value's used in calculating the standard deviation. Just as it is for calculating the mean.

And if I were to average these, I would be starting to add them up then, right?

Average these distances. And if you did that calculation right there, that would always come out to be zero.

'Cause some of those distances are positive, some values were above the mean and some were below the mean.

And so every negative and positive ends up, when you work them all out as a total, to add up to zero. That's a property of the mean.

The total of the distances of every value from the mean is zero.

So you can't really just average that 'cause you always get zero.

So what's one way of getting rid of the negatives versus positives is to take ... absolute value, and that would work

that would be called "the mean absolute deviation", which is a measure of spread,

which really is more of that average distance idea, but it doesn't work very well mathematically.

Trying to integrate or show properties of this kind of measurement here, this statistic when there's absolute values to work with ... not so fun.

So another way to get rid of it is to ... square everything.

Then a value that's 2 below the mean contributes 4, just as a value that sits 2 above the mean contributes the same distance of 4.

But it's in squared units now.

So if I were to use squares, I would average those squares by doing what?

Add all those squared distances up and divide by ... 'n', we're gonna divide by n minus 1.

n minus one, sometimes gonna be called, later for us, the degrees of freedom.

Those quantities that were in the top numerator there,

and before we put the squares there, we said they added up to zero.

So if you has 10 numbers and you know that the distances add up to zero, 9 of those those numbers could be anything

giving any of the distance at all, that last one though, has to be fixed, because it has to make that total add up to zero.

So there are 10 observations but only 9 that are free to vary, and be whatever they want to be.

That last one's fixed by this constraint, because we're looking at that distance from the mean.

It also turns out, if you do a "n-1" on the bottom ... you're gonna get a better estimate out from your data

for what the true standard deviation would be for the population, than if you didn't correct for that, losing of one degree of freedom.

So later it's gonna be called

degrees of freedom

when we use some inference using this as our measure of spread.

Right now, we have, looking at roughly the average of the squared deviation of the values from the mean,

and I don't want to have a measurement in squared units when it comes out, so we take the square root of that whole thing.

And you got the standard deviation ... represented by that little letter 's' for our sample standard deviation.

's' goes along with X - bar... those are the symbols for a sample standard deviation.

The shorthand notation that's on your yellow card, that has your second formula there, I think the mean is listed there too.

would be that you take the sum of the X minus the X - bars, and square them, dividing that total or sum by that 'n-1'.

So that's just that more compact notation.

You've already likely have on your calculator an 's' button too.

Or when you have it do some basic summaries it will give you that standard deviation, which is nice.

What we calculated before we took a square root ... is called s² (s squared), that's your variance.

Variance is not in original units. Variance is in squared units so we wanna bring it back to the original units to be my yardstick.

Here's the average and my give or take, my standard deviation each way.

So that's how it's computed. Let's try it out once.

There's an example in the bottom that you can look at, on your own.

We're gonna calculate it once but mostly focus on the interpretations that we have in the middle of that next page. Page 19.

This is probably the only time you're gonna calculate the standard deviation by hand, working it out.

The rest of time I want you to use the calculator or I provide it on the exam for you to interpret.

These are our observations, the average was 73.6, that was our X - bar.

The standard deviation or 's' for our sample.

We're gonna need a very large square root.

We're gonna have to start calculating distances of each of the values from their mean, so the first one's a 77 ... minus the 73.6 ... but I do have to square that.

We're kind of cheating a little bit here, and putting in the first and the last observation...

to show would it would be that you would have to work out and do.

With only a very basic calculator this would take a bit of time, especially with a mean of 73.6, in decimal form like that,

but I would do all of those squared terms and add them up.

So, an observation that's further away from the mean contributes more.

Leading to a higher spread or more variation than if you had a lot of observations that are close to the mean.

What will we divide by here? ...12 is our observation number, so 12 minus ...1

Under the square root you would get a thirty...five point four

35.4 has what name again?

It's called the variance. It's call the variance.

So sometimes you'll see reports that show the variance and the standard deviation.

Standard deviation's preferred because we bring it back to original units, and that would be about 5.9

and so we can say the standard deviation's 5.9 grams versus a variance of 35.4 grams squared.

Hard to think of squared units.

So there's out standard deviation. If you looked at the histogram, the values did vary from the mean

some of them are more than 5.9 away, some of them are closer ... than 5.9 away.

But the average distance can be thought of as being ABOUT 5.9 grams.

Probably the more important thing is to know what this number tells you in terms of an interpretation or viewing it,

and then also when you have certain shapes of distributions it's very useful for a yardstick as we will see ...not today, on Thursday.

So interpretation's coming first. This is gonna be an important page.

We're definitely gonna have on real homework number 1, somewhere interpreting a standard deviation.

I think in one of your modules, it might even be in module one, there's some good examples to look at.

If you didn't get to them in lab that were wrong interpretations and right ones, in picking out the right ones.

The weights of our small orders of french fries. They weren't all the same ... they did vary.

They vary, they are about how far way from the mean? Roughly how far on average?

So these weights of small order of french fries are roughly about ... 5.9 grams away from their mean, which happened to be this 73.6.

And then we've got that important clarifier there ... "on average".

I'm not saying that every order of french fries have the weight that is exactly 5.9 grams away from the mean.

There were some that were further away, and some that were closer; and if you look at the average of those distances, it would be ABOUT 5.9.

So come, some keys parts here is that we are clarifying it 'cause we didn't calculate the absolute values,

we did square things and then took the square root.

We are talking about an average distance... and I give you the frame of reference from which you're looking at

the values from what? from their mean.

Another way to write it correctly is to start out with the "On average" so you ...

"On average, these weights did vary ... by ABOUT how much... from their mean?"

by ABOUT 5.9 grams .. from their mean, which happens to be known here as 73.6.

So there's a couple examples of where you have all the right parts ... for an interpretation.

A standard deviation's thought of as being roughly an average distance. The average distance that the values vary from ...

what frame of reference from? The mean.

A data set that had a standard deviation of 5.9 compared to a data set that had a standard deviation of 59.

Much more spread from the mean on average, compared to the other one.

Alright, standard deviation. Couple of notes

Well, how would we get a standard deviation of 0?

What would would it mean?

There's no variation, no spread. Has to occur when all the values are ... the same.

So 's' could be zero, that's the smallest it could be. It represents that there is no spread at all, no variation,

and it would occur of course when all the observations ...are the same.

Otherwise, what kind of values do you get for an 's', a standard deviation? It'd have to be positive.

You can't get an 's' of -2.8. It would be marked really wrong.

The larger the value, the more spread. The closer to zero, the more consistent the values are, and close to the mean on average.

Now, the mean was our measure of spread that we, or measure of center that we talked about was somewhat sensitive to those extreme observation,

an extreme observation would pull the mean quite down or up; and the standard deviation looks at every value from that mean

so the standard deviation is also ... SENSITIVE to extreme observations.

So when do we prefer to use the mean and the standard deviation, to go along with a graph, to show or summarize a quantitative variable?

We do that when our distribution looks to be reasonably symmetric, and bell-shaped.

If our distribution overall looks reasonably symmetric, unimodal, bell-shaped ... it's even more ideal.

Then the standard deviation and a mean work very well, in fact we're gonna see in the beginning of next class

this empirical rules that allows us to really see how it works with a bell-shaped model.

5 number summary ... uses a median, uses the interquartile range to measure spread,

which are more resistant and better to use when you have what kind of distributions?

Strongly skewed distributions, not just slightly skewed, slightly is okay, to use a mean and standard deviation, but

strongly skewed distributions, OR if you have outliers.

Strongly skewed distributions or if outliers, 'cause they will affect the mean and the standard deviation.

The last bullet ... and then a picture of the day.

The last bullet is: your calculator will often have lots of these numbers that can be summarized for you,

and calculated for you, but sometimes it even gives you more.

Do you remember the first sample mean?

X - bar, that's the mean of a sample. That's called a statistic. And 'mu' ... is the notation for a population mean.

For standard deviation we have a parallel, of sigma versus 's'. Most of the time, I know on TI calculators

when you get your summary measures, sometimes on regular calculators, the little buttons,

they have a sigma button or an 's' button. The sigma is if you had a population of values and you had all those numbers

put into your calculator, then you want the population standard deviation, which is computed a little differently

than the sample one ... and you need to use ... sigma. But most of the time the data that we have is from a sample.

Most of the time, we're computing sample statistics, and NOT true population values,

so we're gonna often pull off the 's' and not the sigma when that comes out from your calculator, so you know that difference.

And the you get to see pictures of the day before you leave.

I have two dogs! First Lily ... and there's Molly.

Lily is a 16 year old Beagle mix that can't hear, can't see but she's still around.

(inaudible talking in background) Molly is a lot of fun! She's kind of a cat-dog. (laugh in background)

She keeps us (inaudible) busy, with walks... I hope you have a good day. We'll see you on Thursday!