AI 101 for Teachers: Transforming Learning with AI

Hi. Welcome to session three

of the AI 101 for Teachers
Professional Learning Series.

In this session
we are traveling to the Wharton School

at the University of Pennsylvania
to chat with Dr.

Ethan Mollick, a professor who teaches
innovation and entrepreneurship, and Dr.

Lilach Mollick, who works on
interactive pedagogy and AI research.

They will help us explore
how AI can be combined

with pedagogy to enhance student learning.

Let's go meet the Mollicks.

Hi I’m Ethan Mollick

a professor at Wharton
who has been working on how we democratize

access to education through tools
like games and interactive tools and AI

and I’m Lilach Mollick

I'm director of Pedagogy
at Wharton Interactive,

and I've been working at the intersection

of AI and education,
helping to democratize education

for everyone through effective,
pedagogically sound use of AI.

And we have been working together
on the future of education for a while,

thinking about how to make education
more interactive, to work at scale.

And with the advent of general AI,
we found a powerful new tool

that can really help in the classroom
but also carries some risks.

Today, we'd like to talk a little bit
about the classroom

use of AI upsides and downsides
to give you some examples to work with.

But first, we like to start
with our three guiding principles.

The first is that AI is undetectable.

There are AI tools, detection tools,
but they are not effective.

The second principle
is that AI is ubiquitous.

It's everywhere.

169 countries have access to Bing Chat
and you and

your students have access
to the most powerful AI available.

The third principle
is that AI is transformative.

It will transform how we live,
how we work, and how we teach and learn.

Not only is

AI not going
away, but this is probably the worst AI

you're ever going to use.

So if this feels disruptive
now kind of have bad news, which is that

there isn't a reason to suspect
that AI development will not continue.

And I think people worry

a lot about like the far future
or who knows how far it is where AI

smarter than humans.

But even over the next couple of years,
even with this fact of this academic year,

I would expect AI
to continue to improve.

Five times. Ten times? We have no idea.

But if you're not already thinking
about these systems, what they mean

for education, what they mean for you,
what they mean for your students careers.

I think we have to think about it because
these systems are not going to disappear.

Let me make the very pragmatic case
for why you may want to do this.

The first part of the pragmatic case is
your students are using this anyway.

So you have to come up to speed.

I don't think everybody
wants to be dragged along this technology.

No one asked for education
to be massively disrupted.

But it is.

And unfortunately, we've got to figure out
a way to get around that.

All your homework assignments can be done
by AI now, so you have to think about that.

And then I think the second thing
is a pragmatic argument about how AI

can make your life easier as a teacher.

If you put the hours down,
you get them back later.

And then if you've worked out
a number of prompts

to help make lives easier for teachers.

Yes. So one thing you can do prompts
like give me a lesson

hook prompts like create a lesson plan
or create a quiz for me.

So starting to work with your material
and the different models could get you

give you a really good sense of how the
AI works, what it's good at, what it

what it's not good at. And so
and save you time in the end, right?

So that's where I would be

my push to teachers is A
you have to and B you're going to want to.

I think the other important thing
is just to try it.

They're very simple to use.

They're very intuitive
because they're conversational.

You can continue a conversation
and it feels fairly natural.

And I think the key really
is experimentation.

See how it works with you,
see how it works within your context,

within your topic that you teach.

Our rule of thumb is
you need about 10 hours time with

AI to get what it's good at,
what its limitations are.

So I would actually start by suggesting
that this that the teacher throw

their own assignments into the AI and see
what kind of results they get back.

I would think about asking them
to ask their students

to create an assignment using AI
and then critique that assignment,

potentially even in class, to see

if the students can get a sense
of what the gaps and abilities of AI are.

I have a little bit of freedom
as an instructor

because I'm teaching college
and MBA students entrepreneurship.

So I have a lot of

I have points I want them to make,
but they also are building things

and doing things
and absolutely transformed how that works.

So my assignments now

literally call for students to do this one
impossible thing in class.

If you can't code,
you have to write working programs.

If you have never if you can't do design
work, you have to create a full

graphic design working prototypes
that's literally now part of the class.

So where it used to be, write it write

a little bit of an essay,
do a prototype on paper.

Now you have to create a full working
product.

Every assignment that is written

has to be critiqued
by at least five famous

entrepreneurs through history,
and they use AI to invoke those.

There's a pedagogical reason, too,

which is that entrepreneurs
tend to be overconfident.

So you want feedback
from different sources.

So to me it is
let me teach ten times more than I did.

I used to teach an advanced
intermedia entrepreneurship course.

I can now, in the intermediate
or basic course,

get all the way
past the advanced material and further.

So I think we're going to see that
shake out more in the future.

But some of this is about powering past
what we could do before

and I think that's exciting as well.

Apart from student tutors
as assignments, teachers can certainly use

AI coaches, AI assistants
to help students prepare for discussions,

help students outline, help students
do research, help students get feedback

on assignments, and just help students
develop explanations.

I think there are myriad of approaches
that are pedagogically sound

that teachers can assign to

students and watch their work
and ask for the back and forth interaction

to really see that students are paying
attention to and focusing on the material.

Let's talk a little bit about AI

from a teacher's perspective.

So because of the ubiquity of AI,

you've got some choices to make in terms
of your AI policies in your class.

So do you want to permit
I do want to forbid AI

How are you going
to enforce these sorts of things?

We're going to assume
that you want to use AI

to some extent, and we'll dive
into a little bit of the details here.

So as a instructor,
you should know a few things.

One is there is obviously ongoing
ethical debates about AI,

and those are complicated debates.

There are debates over whether or not

the AI's trained on the right kind of data
about the biases

I might have about the use of AI
and the outcomes for student learning.

And it's worth
acknowledging these sets of things.

But this tool is out there and it is worth
thinking about how you want to use it.

If you decide that that is okay
and how you want to communicate

that information
beyond the initial ethical concerns,

there's also concerns
about how AI actually works.

So the large language models that power

today’s AI don't actually have knowledge
of the world.

They're predicting the next word.

They're predicting the right kinds
of sentences or information to give out.

And as a result, they make stuff up
they hallucinate.

So there are often errors or mistakes.

Now, it's not always clear
those errors or mistakes are worse

than the errors and mistakes
humans would make.

But you need to be aware that there's

going to be
those kind of errors and mistakes.

And then finally,
you need to think about as an instructor

how you're going to be using
AI to aid learning,

which means being really clear about what
you want to accomplish with an AI tool.

They can be used for student learning,

but AI's many possible
uses in the classroom,

so do you want to use them
to have student’s generate ideas,

which I do in my classes and get better
project ideas as a result.

Do you want them to use them as tutors
to explain concepts

to them they don't understand?

Do you want the students
to get feedback from AI

by asking for questions about work
that they're doing.

Do you want to be a writing companion?

Do you want it to explain
why quiz answers might be right or wrong?

And then once you've decided
what who's instructor,

you decide what you're
going to tell your students.

AI detectors don't work.

They just don't work.

You shouldn't use them.
And it's worse than them

not working because they have a high false
positive rate.

That means they select things
that’s AI written that aren't AI written

and that disproportionately falls on
people whose English is a second language.

This is just not something that we can do.

And I think trying to close the barn door
here after it's been opened

and try and detect AI is not the future
for responsibility in classrooms.

The other thing to note too,
is that students

were using shortcuts in the past.

It's not that they weren't using Google,

it's not that they weren't using,
you know, other students essays.

This was happening in the past,
but this is a major disruption,

and I think it does call for a rethinking
of how we do essays.

So thinking a little bit more
about the learning goal for an essay

or the learning goal for any assignment,

One of the things that we're noticing
as we watch teachers do this

is they all feel an obligation
to talk about AI and dive

deep into the ethical implications of AI
and so on.

I think that's important,
but I don't think that needs

to be the theme of every class.

I don't think every class needs
to be a discussion about AI,

just like every class that uses computers
doesn't need to be a discussion

about computers, I think is important
to have that conversation.

And right now

we're all just reacting so it's not clear
who's supposed to have that.

So I totally get teachers
wanting to have AI

discussions,
but it's even harder to get up to speed,

not just on the use of AI,
but how it works.

It's, you know, standards,
its ethical implications.

So I think teachers should feel
a little bit of okay ness

with experimenting with AI without having
to make it the subject of class.

First is
as Ethan mentioned, that AI can fabricate.

That means that any output
that the AI gives a student may be made

up, it may be mistaken,
it may be very subtly mistaken.

And so students should be responsible
for their own work.

They should at the very least check
sources, check any number,

check any facts that the AI gives them
and check them with credible sources.

The second principle is that
the AI is not a person.

It's easy to imbue the AI
with a personality or to feel like you're

talking to a person, but it's not a person
and it doesn't know you.

The third principle is really
to give it a lot of context.

The AI doesn't know you.

It doesn't know your context
or your experience or your expertise.

The more context you give it,
the more useful it'll be for you.

And the fourth principle
is that you're in charge.

Not only should you evaluate

and interrogate its output,
but if it's leading you in a conversation

that is no longer useful to you,
or if it's stuck in a loop,

or if you'd like to change the direction
of the conversation,

you should absolutely
feel free to take charge.

So when we talk about AI

and these generative AI solutions,

we tend to talk about large language
models.

And there's actually only

a few large scale
general purpose, large language models.

There is the models created by OpenAI,
which are GPT 3.5 or GPT 4

GPT 3.5 is the free version
that you get through

through Chat GPT
and GPT 4 is either

through the paid Chat GPT or through
Microsoft Bing in creative mode.

And when we talk about specialized apps,
almost all of them are using

one of these models and providing prompts
and other information on top of it.

I generally think instructors
should get familiar with the models

themselves because those are the models
that are actually producing the answers

and you can manipulate them directly
that way and learn how they work.

So if you're trying to buy
an off the shelf solution,

they're almost certainly
using one of these existing models

and then providing some sort of wrapper
or other information on top of it,

and it's often cheaper and more effective
and gives you more control

to use the foundation models yourself.

But that's a choice you get to make.

So when developing the prompt,
we really and for all of our prompts,

we really look at the science of learning

and try to combine
that with the power of the AI.

So for instance,
a good tutor pushes you for information.

It doesn't just hand to you, a good tutor

finds out what you know
and builds on that prior knowledge.

A good tutor will also find out
a little bit about you.

A good tutor also knows that you need lots
and varied kinds of examples

and analogies,

and a good tutor knows that the way to
you show evidence of mastery

is by being able
to explain something in your own words

to someone else and give an example of it,
which is exactly these are exactly

the steps in the kinds of questions
that we use in the tutor prompt.

But you'll notice
when you look at our prompts

that they do things like provide context
the AI as life has discussed already,

the idea that it asks you who you are,
and we tell the AI who it is.

It's an instructor
with this kind of setting,

you'll notice that it also tells it
exactly

the scientific framework
to use this idea of context matters.

We provide controls.

We ask it to go step by step through
sets of questions to ask, sometimes

not in these prompts, we provide examples
of good output, and then we tested a lot.

You can't do prompting without testing,
and that's one of the great things

about testing your error expertise.

It's cheap to do

and so you get to experiment a lot
and that makes for good prompts.

And we should also say we test it not
just on one model but on several models.

So for instance, these two prompts
we just worked

with ChatGPT 4 for,
they also work with Bing.

Bing will react a little bit differently
and it will

because it's connected to the Internet.

It will also look up citations sometimes.

So the right citation,
sometimes they're not.

But that is available.

It may or may not work
with some of the other models.

So you really have to test it,

I think, as an instructor
before you give it to your students

in the context of the topic
that you're teaching to see how it works.

So this is all very theoretical,
but I think important.

So let's let's get practical.

Let's talk about some examples
of what AI can do.

And again, this isn't a monolithic thing,
a has many possible uses.

As we said, it was transformative earlier.

So we're going to show you
a couple of prompts that we've created

and those will be available to you
as well to work with

and these are just examples of the ways
AI classroom use can work.

So the first one we want to show
you is a prompt that I believe you created

that talks about feedback
that gives proper feedback.

And one of the really interesting things
about the AI side of things

is a more sophisticated prompt

that takes into account
some of the principles

we were talking about
earlier will result in better outcomes.

So students will often
ask for writing advice from an AI

even if you tell them not to do it.

But they're in ask for it in a way
that's fairly unsophisticated

and is going to give them fairly generic
sounding work and possibly more mistakes.

If you give a more elaborate prompt,
you can get more elaborate answers.

So in this case, could you explain what
this prompt does, the feedback prompt. Yes.

So we combine
the principles of good feedback,

which is feedback that takes into account
your prior knowledge

or what you already know from the student
perspective takes into account

who you are, your learning level,
what grade you're in,

whether you're in college
or you're in a professional.

And it also takes into account the idea
that you want to respond to this feedback.

So it is going to be actionable,
it's going to be balanced,

it's going to tell you what's wrong
and what you can improve on

and what you're doing well,
and it's going to keep working with you.

But like any good tutor or coach,
it won't actually give you the answer.

It'll push you in that direction,

ask you to explain,
ask you to construct your own knowledge.

And so you can see the prompt
hopefully on the screen here

and as a place to work from.

You don't need to take this
as an absolute answer.

This is something you can play with,
but let's see it in action.

So let's get started using this prompt.

It says that it's a teaching assistant
because that the instructions we gave it

and it asks us for our grade
level and subject we're studying,

what should we say?

So I think we're studying Macbeth
and we're in 12th grade.

In 12th grade. Okay, great.

Okay.

And so we've told the AI this information
it’s feeding it

into the logic that it's using here.

And it's asking us about a specific assignment

and it's asking if we have a rubric
or other information to work with or what

we're hoping to achieve
with with as much information as possible.

I don't have a huge amount here,
so I'll say I have to write

an analysis of Macbeth.

It is graded

based on

rating style and depth of content

and you'll see what it's going
is it's asking ask questions

and soliciting information from us,

which makes it kind of a good prompt
that you might hand a student

better than one that is just they're just
developing themselves and it's asking

about specific instructions
and ask us to share the assignment.

Here is what I have written so far,

and I

have asked
the I to generate a Macbeth essay.

So here we go.

I'm just pasting that in

and we'll see what it says here.

And you'll notice
it's it's working on the information.

It's saying it's taking time
to carefully read through it.

That's a bit of illusion.

It's obviously not taking extra time,
but it's responding in this method

and you'll see it's giving a set
of strengths and weaknesses.

What's great about, again,

using a tutor that you've built
or a mentor that you built is that it

can give you the kind of feedback
that's educationally valuable

that ties into pedagogy rather than just
students asking Make this essay better.

An example of a working in your favor
as an educator

and not necessarily working against you
and undermining the point you're making.

So you'll notice at the end, by the way,
it gives a question

that for the students to answer.

So how do you plan to revise
your analysis?

Give me a plan
and specific changes are going again.

The kind of thing
we would do as an instructor

in a classroom
soliciting changes or differences.

So I think you can start to see why a

tool like

this can be really useful
when properly applied.

Now, let's also talk about one
other potential use for AI, AI as a tutor.

What are some of the advantages
and disadvantages of that approach?

So an advantage of this approach

is that you're getting students
to actually pay attention to the material.

You're getting them
to read over the rubric, to read over

the purpose of the essay and the audience,
and to really think through it.

A disadvantage is that you

you certainly can ask the AI
to do it for you, but if you work with it

and if you're given guidelines to work
with it, it's one way to get feedback.

But you would then have to evaluate

something else that a teacher could do
is to ask for the interactions

and ask for a reflection
about the interactions.

What about this

feedback was good?

What about this feedback was not as good?

And again, it's a higher order level
thinking about your essay

and your process. Excellent.

And so why don't you do one other example
where the AI acts as a direct instructor

and we have a prompt for that as well.

There are risks associated with asking
the AI to be a direct instructor,

which is that hallucination risk.

It doesn't know your pedagogy,
your your, your perspective.

But I find in my classrooms
that students are increasingly

using the AI as a method of learning
so they don't raise their hands as much.

So when I ask them why, they're like,
Well,

I'd rather not show my ignorance in class,

I could ask the AI to explain
like I'm five.

So they're already engaged
in this behavior.

So something like a tutor
both does a useful thing

of showing you what the future of AI
education might look like.

Like the way Khan Academy is building
AI interactive tutors to work.

And it also might be a tool your students
can use to achieve more in class,

but you should caveat that
with the knowledge

that AI tutors are not 100% there yet.

But let's let's use an example here.

So this tutor is again
trying to take the right kind of format.

It says, Hello there, I'm your AI tutor
and I'm excited to work with you today.

What do we want to learn about today?

Let's learn about opportunity cost.

Opportunity cost.
A concept from economics.

Let's see what happens.

Okay, so we're telling the AI
or the opportunity cost

that saying it is a key concept economics

it’s even throwing in a little emoji here,
which is cute.

Can you ask us about our learning level?

What level are we at here?

11th grade. 11th grade.

Now, I wouldn't get too
tied up on the individual grade.

It's not amazing

at differentiating a 10th grader
from an 11th grader,

but this is part of the context
in which it's working in.

So that pulling from some sort of
universal standards here and it says,

what do we know about opportunity cost?

So we know that it has to do
with alternative choices,

has to do with alternative choices.

That is it.

And of course,
one of the advantages of AI

is this kind of freeform text

and interaction
is the real power of education

and it's something the AI can fake
reasonably well.

Again, not as well as a real human
instructor yet.

And you'll notice it's giving us examples

and explaining things in different ways,
which is a powerful thing that AI can do.

It's very good
at breaking things down in different ways,

but you'll notice that it's
they're starting to ask questions.

It's asking us to make choices.

So one of things
we know from the research on tutoring is

you can't just declaim things to people.

The value of tutoring comes
from soliciting information,

making connections.

And you can see the AI
starting to do this

and asking us for connections
in our own life.

The other thing to mention too,
about the tutor prompt is that it

is not assuming that the student
can judge their own learning. Very often

you'll see in a tutor prompt
that is very simple.

Like explain to me like on ten,

it'll ask you
if you'll understand something instead.

here

It's not asking you to make a judgment

about your own learning,
which we know is inherently flawed.

Instead, it's soliciting, as Ethan said,
soliciting information from you

to find out what you know
and to help you build on your knowledge.

And these sorts of subtle differences
are what separates

using AI in sort of an expert way
in a classroom where we know

what we want to have happen from
just the naive use people are doing,

I think that

there is an advantage
to taking charge of your students AI usage

because they're going to be using it
anyway

and thinking about directing it either by giving them
prompts, having these discussions

and it's a really powerful tool

that in the future will greatly boost
classroom learning

and is not a replacement
or threat for teachers.

It is something that we can use
to improve the outputs of our work,

improve student learning, make our lives
easier while making students lives better

and I think that that's
a very powerful view of the future.

And I hope that you at least embrace
and experiment with AI before deciding

whether you want to use it
or ban it in your classrooms.

Wow. There really are so many ways
to enhance student learning using AI.

With AI

technology advancing rapidly, there
will be more and more tools available.

As with any new tool,
educators have a responsibility to ensure

they're using age appropriate tools,
protecting student privacy

and creating spaces
for students to critically evaluate

the potential pitfalls of the technology
they are using.

Join us in session 4 Ensuring
a Responsible Approach to AI

as we explore these topics.

Thanks for joining us.

See you again In session four.

Visit the AI 101 for Teachers
website at Code.org/ai101

to sign up
for early access

and to explore additional resources
from Code.org.

ETS, ISTE and Khan Academy.

Thanks for joining us.