Should we get rid of standardized testing? - Arlo Kempf

Download a free audiobook and support TED-Ed's nonprofit mission: http://www.audible.com/teded

View full lesson: https://ed.ted.com/lessons/should-we-get-rid-of-standardized-testing-arlo-kempf

Although standardized testing is a particularly hot topic in education right now, this approach to measurement has been in use for two millennia. And while the results of standardized testing can help us understand some things, they can also be misleading if used incorrectly. So what do these tests actually measure? And are they worthwhile? Arlo Kempf investigates.

Lesson by Arlo Kempf, animation by CUB Animation.

Thank you so much to our patrons for your support! Without you this video would not be possible.
Silas Schwarz, Fabio Peters, MJ Tan Mingjie, Yansong Li, Jason A Saslow, Michael Aquilina, Joanne Luce, Ayaan Heban, Henry Li, Elias Wewel, Nina, Bijan Bayat Mokhtari, Kyle Nguyen, Taylor Hunter, Noa Shore, Lex Azevedo, Merit Gamertsfelder, Bev Millar, John Christian S. Ramos, Rishi Pasham, Jhuval.

Check out our Patreon page here: https://www.patreon.com/teded

The first standardized tests
that we know of

were administered in China
over 2,000 years ago

during the Han dynasty.

Chinese officials used them to determine
aptitude for various government posts.

The subject matter included philosophy,

farming,

and even military tactics.

Standardized tests continued to be used
around the world for the next two millennia,

and today, they're used for everything

from evaluating stair climbs
for firefighters in France

to language examinations
for diplomats in Canada

to students in schools.

Some standardized tests measure scores

only in relation to the results
of other test takers.

Others measure performances on how well
test takers meet predetermined criteria.

So the stair climb for the firefighter

could be measured by comparing
the time of the climb

to that of all other firefighters.

This might be expressed in what
many call a bell curve.

Or it could be evaluated with reference
to set criteria,

such as carrying a certain amount
of weight a certain distance

up a certain number of stairs.

Similarly, the diplomat might be measured
against other test-taking diplomats,

or against a set of fixed criteria,

which demonstrate different levels
of language proficiency.

And all of these results can be expressed
using something called a percentile.

If a diplomat is in the 70th percentile,
70% of test takers scored below her.

If she scored in the 30th percentile,
70% of test takers scored above her.

Although standardized tests
are sometimes controversial,

they're simply a tool.

As a thought experiment,
think of a standardized test as a ruler.

A ruler's usefulness
depends on two things.

First, the job we ask it to do.

Our ruler can't measure
the temperature outside

or how loud someone is singing.

Second, the ruler's usefulness depends
on its design.

Say you need to measure the circumference
of an orange.

Our ruler measures length,
which is the right quantity,

but it hasn't been designed with the
flexibility required for the task at hand.

So, if standardized tests are given
the wrong job,

or aren't designed properly,

they may end up measuring
the wrong things.

In the case of schools,

students with test anxiety may have
trouble performing their best

on a standardized test,

not because they don't know the answers,

but because they're feeling too nervous
to share what they've learned.

Students with reading challenges

may struggle with the wording
of a math problem,

so their test results may better reflect
their literacy

rather than numeracy skills.

And students who were confused by examples

on tests that contain
unfamiliar cultural references

may do poorly,

telling us more about the test taker's
cultural familiarity

than their academic learning.

In these cases, the tests may need
to be designed differently.

Standardized tests can also
have a hard time

measuring abstract
characteristics or skills,

such as creativity, critical thinking,
and collaboration.

If we design a test poorly,

or ask it to do the wrong job,

or a job it's not very good at,

the results may not be reliable or valid.

Reliability and validity
are two critical ideas

for understanding standardized tests.

To understand the difference between them,

we can use the metaphor
of two broken thermometers.

An unreliable thermometer

gives you a different reading
each time you take your temperature,

and the reliable but invalid thermometer
is consistently ten degrees too hot.

Validity also depends on accurate
interpretations of results.

If people say results of a test
mean something they don't,

that test may have a validity problem.

Just as we wouldn't expect a ruler
to tell us how much an elephant weighs,

or what it had for breakfast,

we can't expect standardized tests alone
to reliably tell us how smart someone is,

how diplomats will handle
a tough situation,

or how brave a firefighter
might turn out to be.

So standardized tests may help us learn
a little about a lot of people

in a short time,

but they usually can't tell us a lot
about a single person.

Many social scientists worry about
test scores resulting in sweeping

and often negative changes
for test takers,

sometimes with long-term
life consequences.

We can't blame the tests, though.

It's up to us to use the right tests
for the right jobs,

and to interpret results appropriately.