Hello. And this video, I want to talk to you about how to build up a chart from the ground up as we think
about the question it's going to try to answer and the pieces that need to go into it.
So the learning outcome for this video is for us to be able to design a chart by thinking first of the questions,
the goals and the data that are going to be in and from the chart.
So a good chart answers a question and the guiding principle for how we design and
how we lay out our chart is to illuminate the question that we want to answer.
And this depends. We need to know what question we want to answer in the first place.
We also need to know precisely how we operationalize that question so we can use that to then inform how we're going into the chart layout.
And we need to know what data that we're using, specifically what variables we're using as a part of this chart.
For example, there's a data set, you'll see it in the notebook that goes with this video for of passengers on the
Titanic and supposedly wanted to examine whether passengers in a higher fare class,
say, first class or more likely to survive than passengers in lower fare classes.
In this analysis, we have an outcome variable zero one,
whether or not the passenger survived the Titanic sinking and a lot of charts are going to have an outcome variable.
We want to we have some outcome variable and we want to see how it responds to or how it differs with some other variable,
which we call the explanatory variable, in this case, the passage class where outcome is survival.
And we want to see how it changes as the the the passengers passage class, the explanatory variable changes.
The outcome variable is also called the response variable or the dependent variable,
because it's what we're trying to measure that's responding to the condition we're trying to analyze.
And the explanatory variable is sometimes called the independent variable because it's changing,
but it's not changing as a function of the other variables in theory.
So we can do this with a bar chart and this bar chart shows the x axis is our steerage class through our passage class first,
second and third, and the Y axis is the average is the fraction of passengers in that class who survive.
We also see some error bars. We're going to see later what those mean and how to how to compute them.
But this lets us see how the outcome survival changes as we age, as the pass or with the different passage classes of the passengers.
And one of the things to note here is that we have our explanatory variable on the X axis and the outcome variable on the Y axis.
That is the general convention. There are some cases where we might want to flip it.
So we've we've got a horizontal bar chart where the explanatory is on the Y and the outcome is on the X,
particularly if we if if it makes the labels more readable.
But the standard convention for most types of charts is to put explanatory on x axis, the horizontal axis and the outcome variable on the Y axis.
And this chart shows the relationship, many charts or relationship,
most of the plot that we're going to be drawing in this class show how some kind of a numeric variable either continues or.
Or integer changes between different values of one or more other variables, and in this case, even though our response was zero one logical.
When we convert it into a rate per per passage class, it became a continuous variable.
And so when we do this, we need to identify a few key things to design our plots.
We need to identify what variable we want to show. That's going to guide a lot of plots that'll be on our y axis.
When it's not, it'll usually be on X and it's going to identifying that variable is,
if anything, probably the most important thing in designing a plot.
We then need to identify what we want to show about this variable.
Do we want to show its value for different data points? Do we want to show a statistic?
The do we want to show, for example, the the mean or the rate?
In the previous when we showed a statistic, the Titanic example, we showed a statistic, the rate of of survival.
Do we want to show its distribution?
And then how do we want to compare that between values of the explanatory variable, particularly, do we want to look at absolute differences?
Do we want to look at relative or proportional differences? And even the histograms follow this kind of a design because they have an outcome,
which is the frequency or the count of the abortion or the density, depending on precisely what kind of histogram we're showing.
And then they have the explanatory variable, which is the value or the.
So we've got a histogram and we've got some beans. And the response variable is how how many values are in that bend and the explanatory is the.
So identifying these then informs the entire pipeline of producing our chart, the data processing the beginning.
We're going to do group aggregation transformation that gets us to the final values we can actually plot.
It's going to affect our choice of plot type and it's going to affect our choice of axis labels, colors, facets, the other aspects of the plot.
So. The type of the variable has a significant impact if the response is numeric or can be transformed,
the response is often numeric or can be transformed to be.
If we're talking about a of categorical value, we usually want the relative frequency of different of different values of that.
So either we're doing it like a histogram and the we're going to transform it so that we're showing just the distribution.
We're going to transform it, so that we're showing that the explanatory becomes the value of the categorical.
And the response is how many or what fraction have it in a logical.
It might if it's a two level categorical, we might turn it into a fraction, just a fraction to have one of the levels versus the other explanatory.
It can be anything. We're going to see how to use numeric explanatory variable is categorical explanatory variables ordinal.
We were just like categorical,
except that the that it's a discrete axis that preserves order and we need to make sure that the order and ordinal data is being preserved.
If you're using pandas ordered category type, it automatically preserves order when you're doing the plot for you.
So if we just have one explanatory variable, this is the easiest case, if our explanatory variable is continuous,
we usually want to scatterplot or align plot for showing individual values.
Sometimes we'll flip the response and explanatory on a scatterplot or will or both might be explanatory.
We want to show where points lie in a two dimensional space.
But generally, if we've got an explanatory, a continuous explanatory variable and we've got a and we're trying to show values,
we're going to use a scatterplot or a line excuse me, we're going to try to show values.
We're going to try to show statistics like a mean at each at each value of the explanatory variable.
We're going to use a scatterplot or a line plott. If the explanatory variable is discrete, then we're going to use a bar chart to show a statistic.
If we want to estimate the relative difference, we want to be able to compare the relative value relatively compared to values,
because a bar, one bar will be twice as high as another.
And a point plot shows a statistic or an individual value, and it emphasizes absolute difference.
You don't have a whole bar in order to to compare heights.
You just have the point. And then if we want to show a distribution, we usually use a box or a violin plot with this discrete explanatory variable.
We don't have great ways to show distributions with continuous explanatory variables.
You can show a variance with an error bar, but that's about where a ribbon.
But that's about it. For too explanatory variables we get into.
Too explanatory variables, we have a couple of options. One is we can do a three, a pseudo 3D display where we do a contour plot or a heat map.
And I'm going to show both of these here. So this is a contour plot.
The left one is a contour plot and it reads like a topographical map.
If you envision your your two explanatory variables in this case, we're going to we're showing a two dimensional distribution.
So one explanatory variable is the score given to a movie by its critics, and another explanatory variable is the score given by its audience.
And then the response variable is how many movies have that combination?
And so we can see here, this is the peak, a contour plot is really good for showing us the peak.
It's going to be that innermost circle and it also shows us the shape because each of these rings is a a a level of decreasing.
Decreasing height in this map, so if the response if we envision that the response variable is this height and we're looking at a two dimensional map,
the rings show us the contours around the mountains of that height.
Good for showing, good for showing shape. The other one of the heat map which uses color.
And so it's usually going to be from a cool color like, say, black here to to a hot orange,
or it's going to be sometimes if you have a bidirectional one, which goes blue to red and it lets us see the highest density is here.
And the as you go out from there, you get lower and lower densities.
Either one can work for a continuous variable heat map, you often have to it in order to.
This is a descriptivist heat map where we have been everything in in bins of of a half a star or a half
a star on the audience score and a four star on the credit score because they're on different scales.
But heat maps also work well for categorical ordinal data.
So. Another way we can do it is we can use other esthetics for secondary variables such as color or shape or size,
sometimes we'll use that to indicate a second response variable,
like you might have a scatterplot where the size of the point is a second response variable, but often it's for multiple explanatory variables.
So this shows us how we can do that. So if we wanted to break down Titanic's survival rates by both class and sex,
we can see we can use we keep our class on the X axis like we did before,
and then we use color for the passenger sex so we can see significantly higher survival rates for women across all three classes.
I'm also showing you here the difference between a bar chart and a point plot.
So the left is the bar chart. The right is the point plot.
And the bar chart lets us compare the heights of the bars. Note that it starts at zero.
Bar charts always start at zero. And because so it lets us compare the height of the bars and we can see that.
It's easy to see from just using our vision that the female passenger first class bar is almost is more than twice as tall as the.
As the female or the male passenger first class bar,
the male passenger first class bar is twice as tall as the male passenger passenger second class bar.
So it lets us compare make relative comparisons between the different values.
This is why it always starts at zero, because the natural thing to do with the bar is compare its height.
If your bar chart does not start at zero, suppose our bar chart started at point one,
then the comparison of height would exaggerate the difference relative to the value.
And what looks twice as tall isn't actually twice as tall because we cut off a bunch of the bottom.
So always start at zero. The point plot. Does not it makes it hard to compare relative difference.
We can't it's difficult for us to tell that the survival rate visually tell.
We can tell if we look at the numbers,
but it's difficult to visually tell that the survival rate of women in first classes is twice as high as the survival rate of men.
But what it does literacy is it lets us see the absolute, absolute difference between these values,
and it makes it easy to compare the difference in the gaps across the three classes.
We can see that the the survival rate by by sex is much higher or is much closer in the third class than it is in the first or in the second class.
So your choice of plot really guides the user to see different things in your choice of plot,
allows you to emphasize different things and you need to decide.
You need to choose and design your plot in such a way that's going to tell the story that you need to tell from the data.
We can also have more than two explanatory variables. It's difficult to have more than one that's numeric or two for doing a contour plot.
We can bend variables that are then going to let us use some more techniques, such as FaceTime.
So if we want to break down by more categorical variables,
so we want let's say we also want to look at a or we want to break down many more variables.
Let's say we also want to look at age. And so we're going to keep sex on the color.
We're going to now use age as the x axis. Since this numeric, it really works better on an axis.
I have bend it into bins of tens that you only have one point for every decade.
But then we use a fassett and the fassett means we draw a different chart for each of the three classes.
The charts all share a y axis so we can directly compare across the row of charts and we can see it lets us see
particularly how does the survival as a function of age change between different different passenger classes,
for example? And so it is, but it lets us start to build up.
And if we had a fourth, we could use rows and columns in the faceted plot.
So we have these mechanisms of building up and we have our x axis or y axis.
We can use esthetics of the lines of the points, particularly color, size, shape,
and then we can use facets to build up even more variables into our plot.
To do fascinating, there's a couple of things you can do, it's built into some of the seabourne row plotting functions.
The plot and cat plot function functions can both do fascinating on their own.
They let you control the statistic. They're very, very flexible functions for a wide range of plot.
The general purpose Fassett Grid allows you to fassett any kind of plot by writing some more Python code on your own.
Very useful if you want to fassett something that doesn't support Facetune built in.
And if you're using Plot nine or the R.G. plot to package Fassett Grid and Fassett wrap a control fassett,
you build that faceted plot you need to pay attention to what variables go where your choice of which variables are going to be on color,
what variables are going to be facets, which variables are going to be on your axes really affect how the reader is going to interpret and understand
your plot and you need to choose them strategically to tell the story that addresses your question.
You also need to do it, though, in a way that is honest and does not mislead your user, your readers.
The chart needs to honestly show the readers what it is that you learned from the data and show that clearly.
Another thing we can do to build up a chart, especially if we have more categorical variables,
if we've got a categorical response variable with more than two levels,
and we want to show how particularly how the the proportion in different categories changes the response to another variable,
a stack chart can be very good. Let's see the differences in composition to see how the parts of a hole change.
And so this chart,
this is a stacked bar chart and it's a horizontal bar chart where I put the explanatory variable on the x axis excuse me, on the Y axis.
Just in part to make the labels easier to read and so are explanatory variable is what data set.
Something came from Locke, M.D. Gry. What those are don't matter for our purposes right now.
The response variable is the distribution of gender's in this case.
These are data sets of books, the genders of the authors of those books in the data set.
And so we have female, we've got mail and we also have codes for we it's ambiguous or unknown or we didn't have data.
And so we can see, for example, the GYŐRI data set has a higher fraction of women and a significantly lower fraction of men.
And we can see quite a few more. Books that we don't know what gender on, and so this the order on this chart is very strategic.
I observed these levels is very strategic. I bunch I batched all of the various kinds of we don't know together so that
you can look at that whole block and see the and see the various types of.
We don't know the gender of the book's author together, but you can also see how they're broken down into individual things.
You can see that UNlinked is a very, very large fraction of of that increase in books where we don't know the author's gender.
So you need to think you need to think about all of these different things in order
to be able to generate a chart that's going to clearly and unambiguously communicate.
You can show either you can show raw values in a stack bar chart at the bars.
Don't all have to be the same height you can show fractions, in which case they will be.
I chose to show fractions in this chart. The code that generates this using raw matplotlib is linked in the notes for the video.
Sometimes we're also going to transform our charts.
We might transform the axis such as doing a log ten scale, in which case the label would transform the axis.
The labels are still in their original value. It's just they're spaced out logarithmically.
We generally won't do this for bars. Reading a bar on a large scale.
You can draw it, but you have to be really, really careful in order to make sure that your readers are going to accurately interpret it.
But for line and scatter plots, log transforms are a lot more common.
Sometimes, though, we're actually going to transform the data itself and we're going to plot a log or a square root or some other rescaling.
And another kargman transformation is to be in the data, somehow democratize it into fixed bins.
By some mechanism or another, so the key decisions that you need to make when you're making one of these charts
are you need to pick the variables and how you're doing their transformations. You need to pick that what's called the esthetics,
how you're going to map the different variables you're looking at to chart features your X and Y axes,
your facets row and column your color, your point marker style.
If you're doing a joint plot, often it's useful to put.
The same esthetic on both color and style, and that way, if you have a reader who's colorblind, they still get different point styles,
even if they can't tell the colors apart or if someone's putting it on a black and white printer.
And then you need the type of the chart line, chart, bar, point box, et cetera.
So you have to make all of these decisions when you're drawing this chart and they're driven by what variables and data you have and what
question you're trying to answer and what story you're trying to tell about that you do need to be careful to avoid excessive complexity.
We can put a different variable on every conceivable esthetic and it's often going to result in a chart that's very difficult to read.
We also have to be careful with color because it's easy to make a chart that has differences
that are difficult for the human eye to distinguish or get obscured by printers,
low quality displays, etc. It's also important to note a good graphic reveals the data and does not distort or obscure the data.
It's easy to create a graphic that manipulates the data to tell a story that's not very well supported.
And we want to avoid that when we're doing data science with honesty and integrity.
So wrap up. You need to identify the variables and relationships that you want to highlight in your chart.
You want to design a plot that illustrates them,
and you're going to need to spend some time studying your plodding library APIs and the Plotting Libraries Gallery.
Any plotting library usually has a gallery of a bunch of different plots and the code that was used to generate them.
Seabourne has this, matplotlib has this.
And so you spending some time with that looking, oh, this looks like this looks like the kind of plot that might display my data well.
And then look and click on it and see what code they use to generate it and borrow it.