[Music]
foreign
hi welcome to session three of the AI
101 for teachers professional learning
series
in this session we are traveling to the
Wharton School at the University of
Pennsylvania to chat with Dr Ethan
Mullick a professor who teaches
Innovation and Entrepreneurship and Dr
lilac Mullick who works on interactive
pedagogy and AI research
they will help us explore how AI can be
combined with pedagogy to enhance
student learning
let's go meet the Molex
hi I'm Ethan Malek a professor at
Wharton who has been working on how we
democratize access to education through
tools like games and interactive tools
and AI
Nami and I'm director of pedagogy of
Wharton interactive and I've been
working at the intersection of AI and
education helping to democratize
education for everyone through effective
pedagogically sound use of AI and we
have been working together on the future
of education for a while thinking about
how to make education more interactive
to work at scale and with the Advent of
generative AI we've had a powerful new
tool that can really help in the
classroom but also carries some risks
today we'd like to talk a little bit
about the classroom use of AI upside to
the downside to give you some examples
to work with the first we like to start
with our three guiding principles the
first is that AI is undetectable there
are AI tools detection tools but they
are not effective the second principle
is that AI is ubiquitous it's everywhere
169 countries have access to Bing chat
and you and your students have access to
the most powerful AI available
the third principle is that AI is
transformative it will transform how we
live how we work and how we teach and
learn
foreign
[Music]
not going away but this is probably the
worst AI you're ever going to use so if
this feels disruptive now I kind of have
bad news which is that there isn't a
reason to suspect that AI development
will not continue and I think people
worry a lot about like the far future or
who knows how far it is where AI is
smarter than humans but even over the
next couple years even of the fact of
this Academic Year I would expect AI to
continue to improve five times ten times
we have no idea but if you're not
already thinking about these systems
what they mean for Education what they
mean for you what they mean for your
students careers I think we have to
think about it because these systems are
not going to disappear let me make the
very pragmatic case for why you might
want to do this for the first part of
the pragmatic case is your students will
be using this anyway so you have to come
to speed I don't think everybody wants
to be dragged along this technology no
one asked for education to be massively
disrupted but it is and unfortunately
we've got to figure out a way to get
around that all your homework
assignments can be done by AI now so you
have to think about that and then I
think the second thing is the pragmatic
argument about how AI can make your life
easier as a teacher if you put the hours
down you get them back later and if
you've worked on a number of prompts to
help make lives easier for teachers
yeah so one thing you can do prompts
like give me a lesson hook prompts like
create a lesson plan
um or Create a quiz for me so starting
to work with your material and the
different models can get you give you a
really good sense of how the AI Works
what is good at what it what it's not
good at and so and save you time in the
end right so that's where I would be my
push to teachers is a you have to and B
you're going to want to
I think the other important thing is
just to try it they're very simple to
use they're very intuitive because
they're conversational uh you can
continue a conversation it feels fairly
natural and I think the the key really
is experimentation see how it works with
you see how it works within your context
within your topic that you teach our
rule of thumb is you need about 10 hours
playing with AI to get what it's good
and what its limitations are so I would
actually start by suggesting that this
that the teacher throw their own
assignments into the AI and see what
kind of results they get back I would
think about asking them uh to ask their
students to create an assignment using
Ai and then critique that assignment
potentially even in class to see for the
students can get a sense of what the
gaps and abilities of AI are I have a
little bit of Freedom as an instructor
because I'm teaching college and MBA
students entrepreneurship so I have a
lot of I have points I want them to make
but they also are building things and
doing things and ai's absolutely
transformed how that works so
um my assignments now literally call for
students to do at least one impossible
thing in class if you can't code you
have to write working programs if you
have never if you can't do design work
you have to create a full graphic design
working prototypes that's literally that
part of the class so where it used to be
right up write a little bit of an essay
do a prototype on paper now you have to
create a full working product every
assignment that they turn in has to be
critiqued by at least five famous
entrepreneurs through history and they
use AI to invoke those there's a
pedagogical reason too which is that
entrepreneurs tend to be overconfident
so you want feedback from different
sources so to me it is let me teach 10
times more than I did I used to teach
Advanced intermediate entrepreneurship
course I can now in the intermediate or
basic course get all the way past the
advanced material and further so I think
we're going to see that shake out more
in the future but some of this is about
powering a past what we could do before
and I think that's exciting as well
apart from student tutors as AI
assignments teachers can certainly use
AI coaches and AI assistants to help
students prepare for discussions help
students outline help students do
research help students get feedback on
assignments and just help students
develop explanations I think there are
Myriad of approaches that are
pedagogically sound
that teachers can assign to students and
watch their work and ask for the back
and forth interaction to really see that
students are paying attention to and
focusing on the material
foreign
[Music]
teacher's perspective so because of the
ubiquity of AI you've got some choices
to make in terms of your AI policies in
your class so do you want to permit AI
do you want to forbid AI how are you
going to enforce these sets of things
we're going to assume that you want to
use AI to some extent and we'll dive
into a little bit of the details here so
as a instructor you should know a few
things one is there's obviously ongoing
ethical debates about Ai and those are
complicated debates there are debates
over whether or not the AI is trained on
the right kind of data about the biases
AI might have about the use of AI and
the outcomes for student learning and
it's worth acknowledging these sets of
things but this tool is out there and it
is worth thinking about how you want to
use it if you decide that that is okay
and how you want to communicate that
information beyond the initial ethical
concerns there's also concerns about how
AI actually works so the large language
models that power today's AI don't
actually have knowledge of the world
they're predicting the next word they're
predicting the right kinds of sentences
or information to give out and as a
result they make stuff up they would
call hallucinate so there are often
errors or mistakes now it's not always
clear those errors or mistakes are worse
than the errors and mistakes humans
would make but you need to be aware that
there's going to be those kind of errors
and mistakes and then finally you need
to think about as a instructor how
you're going to be using AI to Aid
learning which means being really clear
about what you want to accomplish with
an AI tool they can be boost for student
learning but ai's many possible uses in
classroom so do you want to use them to
have students generate ideas which I do
in my classes and we get better project
ideas as a result do you want them to
use them as tutors to explain Concepts
to them they don't understand do you
want these students to get feedback from
AI by asking it for questions about work
that they're doing do you want to be a
writing companion do you wanted to
explain why quiz answers might be right
or wrong and then once you've decided
what to do as an instructor you need to
decide what you're going to tell your
students
[Music]
AI detectors don't work they just don't
work you shouldn't use them and it's
worse than them not working because they
have a high false positive rate that
means they flag things as AI written
that aren't AI written and that
disproportionately falls on people whose
English is a second language this is
just not something that we can do and I
think trying to close the barn door here
after it's been opened and try and
detect AI is not the future for
responsibility in classrooms the the
other thing to note too is that students
were using shortcuts in the past it's
not that they weren't using Google it's
so that they weren't using you know
other students essays this was happening
in the past but this is a major
disruption and I think it does call for
a rethinking of how we do essays so
thinking a little bit more about the
learning goal for an essay or the
learning goal for any assignments
[Music]
one of the things that we're noticing as
we watch teachers do this is they all
feel an obligation to talk about Ai and
dive deep into the ethical implications
of AI and so on I think that's important
but I don't think it needs to be the
theme of every class I don't think every
class needs to be a discussion about AI
just like every class that uses
computers doesn't need to be a
discussion about computers I think it's
important to have that conversation and
right now we're all just reacting so
it's not clear who's supposed to have
that so I totally get teachers wanting
to have ai discussions but it's even
harder to come up to speed not just on
the use of AI but how it works it's you
know standards it's ethical implications
so I think teachers should feel a little
bit of um okayness with experimenting
with AI without having to make it the
subject of class
foreign
[Music]
first is as Ethan mentioned that AI can
fabricate that means that any output
that the AI gives a student may be made
up it may be mistaken it may be very
subtly mistaken and so students should
be responsible for their own work they
should at the very least check sources
check any number check any facts that
the AI gives them and check them with
credible sources the second principle is
that the AI is not a person it's easy to
imbue the AI with a personality or to
feel like you're talking to a person but
it's not a person and it doesn't know
you the third principle is really to
give it a lot of context the AI doesn't
know you it doesn't know your context or
your experience or your expertise the
more context you give it the more useful
it'll be for you and the fourth
principle is that you're in charge not
only should you evaluate and interrogate
its output but if it's leading you in a
conversation that is no longer useful to
to you or if it's stuck in a loop or if
you'd like to change the direction of
the conversation you should ask feel
free to take charge
thank you
[Music]
so when we talk about Ai and these
generative AI Solutions we tend to talk
about large language models and there's
actually only a few large-scale general
purpose large language models there is
the models created by openai which are
GPT 3.5 or gbt4 gbd 3.5 is the free
version that you get through uh through
chat CBT and gbt4 is either through the
page chat gbt or through Microsoft Bing
in creative mode and when we talk about
specialized apps almost all of them are
using one of these models and providing
prompts and other information on top of
it I generally think instructors should
get familiar with the models themselves
because those are the models that are
actually producing the answers and you
can manipulate them directly that way
and learn how they work so if you're
trying to buy an off-the-shelf solution
they're almost only using one of these
existing models and then providing some
sort of wrapper or other information on
top of it and it's often cheaper and
more effective and gives you more
control to use the foundation models
yourself but that's a choice you get to
make so when develop helping the prompt
we really and for all of our prompts we
really look at the science of learning
and try to combine that with the power
of the AI so for instance a good tutor
pushes you for information uh it doesn't
just hand it to you a good tutor finds
out what you know and builds on that
prior knowledge
a good tutor will also find out a little
bit about you a good tutor also knows
that you need lots and varied kinds of
examples and analogies and a good tutor
knows that the way that you show
evidence of Mastery is by being able to
explain something in your own words to
someone else and give an example of it
which is exactly these are exactly the
steps and the kinds of questions that we
use in the tutor prompt but you'll
notice when you look at our prompts that
they do things like provide context to
the AI as Leo has discussed already the
idea that it it asks you who you are and
we tell the AI who it is it's an
instructor with this kind of setting
you'll notice that it also tells it
exactly the scientific framework to use
this idea of context matters we provide
controls we ask it to go step by step
through sets of questions to ask
sometimes not in these prompts we
provide examples of good output and then
we test it a lot you can't do prompting
without testing and that's one of the
great things about testing your
expertise it's cheap to do and so you
get to experiment a lot and that makes
for good prompts and and we should also
say we tested not just on one model but
on several models so for instance these
two prompts we just worked with chat
gpt4 they also work with Bing Bing will
react a little bit differently and it
will um because it's connected to the
internet it will also look up citations
sometimes they're the right citations
sometimes they're not but that is
available it may or may not work with
some of the other models so you really
have to test it I think as an instructor
before you give it to your students in
the context of the topic that you're
teaching to see how it works so this is
all very theoretical but I think
important so let's let's get practical
let's talk about some examples of what
AI can do and again this isn't a
monolithical thing a AI has many
possible uses as we said it was
transformative earlier so we're going to
show you a couple of prompts that we've
created and those will be available to
you as well to work with and these are
just examples of the ways AI classroom
use can work so the first one we want to
show you is a prompt that I believe you
created that talks about uh feedback
that gives proper feedback and
one of the really interesting things
about the AI side of things is a more
sophisticated prompt that takes into
account some of the principles we were
talking about earlier will result in
better outcomes so students will often
ask for writing advice from an AI even
if you tell them not to do it but
they're going to ask for it in a way
that's fairly unsophisticated and is
going to give them fairly generic
sounding work and possibly more mistakes
if you give a more elaborate prompt you
can get more elaborate answers so in
this case could you explain what this
prompt does the feedback prompt yes so
we combine the principles of good
feedback which is feedback that takes
into account your prior knowledge or
what you already know from the student
perspective takes into account who you
are your learning level what grade
you're in uh whether you're in college
or you're a professional and it also
takes into account uh the idea that you
want to respond to this feedback so it
is going to be actionable it's going to
be balanced it's going to tell you
what's wrong and what you can improve on
and what you're doing well and it's
going to keep working with you but like
any good tutor or coach it won't
actually give you the answer it'll push
you in that direction ask you to explain
ask you to construct your own knowledge
and so you can see the prompt hopefully
on the screen here uh and uh as a place
to work from you don't need to take this
as an absolute answer this is something
you can play with but let's see it in
action so let's get started using this
prompt it says that it's a teaching
assistant because that's the
instructions we gave it and it asks us
for our grade level and subject we're
studying what should we say so I think
we're studying Macbeth and we're in 12th
grade in 12th grade okay great
okay and so we've told the AI as
information it's feeding it into the
logic that it's using here and it's
asking us about a specific assignment
um and it's asking if we have a rubric
or other information to work with or
what we're hoping to achieve with uh
with as much of Rage as possible I don't
have a huge amount here so I'll say I
have to write an analysis of Macbeth
it is graded
based on
writing style and depth of content
and you'll see what it's going is it's
asking us questions soliciting
information from us which makes it a
kind of a good prompt that you might
hand a student better than one that is
just they're just developing themselves
and it's asking about Specific
Instructions and as gets to share the
assignment uh here is what I have
written so far
and I have asked the AI to generate a
Macbeth essay so here we go I'm just
pasting that in
and we'll see what it says here and
you'll notice it's it's it's working on
the information it's saying it's taking
time to carefully read through it that's
a bit of an illusion it's obviously not
taking any extra time but uh it's
responding in this method and you'll see
it's giving a set of strengths and
weaknesses what's great about again
using a tutor that you've built or a
mentor that you built is that it can
give you the kind of feedback that's
educationally valuable that ties into
pedagogy rather than just students
asking make this essay better a nice
example of AI working in your favor uh
as an educator and not necessarily
working against you and undermining the
points you're making
so you'll notice at the End by the way
it gives a question that you for the
student's answer so how do you plan to
revise your analysis give me a plan on
specific changes you're going to make
again the kind of thing we would do as
an instructor in our classroom
soliciting changes or differences so I
think you can start to see why a uh tool
like this can be really useful when
properly applied now let's also talk
about one other potential use for ai ai
as a tutor what are some of the
advantages of disadvantage of that
approach so an advantage of this
approach is that you're getting students
to actually pay attention to the
material you're getting them to read
over the rubric to read over the purpose
of the essay and the audience and to
really think through it a disadvantage
is that you you certainly can ask the AI
to do it for you but if you work with it
and if you're given guidelines to work
with it it's one way to get feedback
that you would then have to evaluate
something else that a teacher could do
is to ask for the interaction and ask
for a reflection about the interactions
what about this feedback was good what
about this feedback was not as good and
again it's a higher order level thinking
about your essay and and your process
excellent and so I want to show you one
other example where the AI acts as a
direct instructor and we have a prompt
for that as well there are risks
associated with asking the AI to be a
direct instructor which is that
hallucination risk it doesn't know your
pedagogy your your your perspective but
I find in my classrooms that students
are increasingly using the AI as a
method of learning so they don't raise
their hands as much I'm going to ask
them why they're like well I'd rather
not show my evenings in class I could
ask the AI to explain like I'm five so
they're already engaging this Behavior
so something like a tutor both does a
useful thing of showing you what the
future of AI education might look like
uh like the way Khan Academy is building
AI interactive tutors to work and it
also might be a tool your students can
use to achieve more in class but you
should caveat that with the knowledge
that AI tutors are not 100 there yet but
let's let's use an example here so this
tutor is again trying to take the right
kind of format it says hello there I'm
your AI tutor and I'm excited to work
with you today what do we want to learn
about today
opportunity cost the concept from
economics let's see what happens
okay so we're telling the AI where the
opportunity cost it's saying it is a key
concept of Economics that's even throw a
little Emoji here which is cute uh can
you ask us about our learning level what
level are we at here 11th grade 11th
grade
now I wouldn't get too tied up on the
individual grade it's not amazing at
differentiating a 10th grader from 11th
grader but this is part of the context
in which it's working and so it's not
pulling from some sort of universal
standards here and it says what do we
know about opportunity cost
well we know that it has to do with
choices has to do with alternative
choices
that is it and of course the advance one
of the advantages of AI is this kind of
freeform texted interaction is the real
power of education and it's something
the AI can fake reasonably well again
not as well as a real human instructor
yet and you'll notice it's giving us
examples and explaining things in
different ways which is a powerful thing
that AI can do it's very good at
breaking things down in different ways
but you'll notice that it's now starting
to ask questions it's asking us to make
choices so one of the things we know
from the research on tutoring is you
can't just declaim things to people the
advantage of tutoring comes from
soliciting information making
connections and you can see the AI
starting to do this and asking us for
Connections in our own life the other
thing to mention too about the Tudor
prompt is that it is not assuming that
the student can judge their own learning
uh very often you'll see in a tutor
prompt that is very simple like explain
to me like I'm 10 It'll ask you if you
understand something instead here it's
not asking you to make a judgment about
your own learning which we know is
inherently flawed instead it's
soliciting as Ethan said soliciting
information from you to find out what
you know and to help you build on your
knowledge and these sorts of subtle
differences are what separates
using AI in sort of an expert way in a
classroom where we know what we want to
have happen from just the naive use
people are doing
[Music]
I I think that there is an advantage to
taking charge of your students AI usage
because they're going to be using it
anyway and thinking about directing it
either by giving them prompts having
these discussions and it's a really
powerful tool that in the future will
greatly boost classroom learning and is
not a replacement or threat for students
for teachers it is something that we can
use to improve the outputs of our work
improve student learning make our lives
easier while making students lives
better and I think that that's a very
powerful view of the future and I hope
that you at least Embrace that
experiment with AI before deciding
whether you want to use it or ban it in
your classrooms
wow there really are so many ways to
enhance student learning using AI
with AI technology advancing rapidly
there will be more and more tools
available as with any new tool Educators
have a responsibility to ensure they are
using age-appropriate tools
protecting student privacy and creating
spaces for students to critically
evaluate the potential of pitfalls of
the technology they are using join us in
session four ensuring a responsible
approach to AI as we explore these
topics thanks for joining us see you
again in session four visit the AI 101
for teachers website at co.org AI 101 to
sign up for Early Access and to explore
additional resources from code.org ETS
IST and Khan Academy thanks for joining
us