- Thanks, Jennifer.
All right, so it is exactly four o'clock,
so I will go ahead
and get us started to make sure we start on time.
Good afternoon, everyone.
Thank you so much for joining us today.
My name is Frances Barreto.
I am one of the district success managers
that support our amazing partners in Arizona.
And you do have a dedicated team here to support you
as you see on the screen.
That includes Chelsea Hatchard,
who's our district partnership manager,
Sheryl Dossola supports as the rostering specialist.
We have Victoria Cheng,
who is another district success manager,
and then Jennifer Cummings and India Quarles
are your professional learning specialists.
We are especially excited to have Sarah Robertson
from the Khan Academy Writing Coach team with us today,
and she's gonna be sharing
how you can help your students become more confident
and skilled writers using Khanmigo's Writing Coach tool.
This is the first
out of several professional learning opportunities
that you're going to be able to participate in the spring,
so keep an eye out for an email
that should be coming your way March 11th
with the registration links
for our Arizona Spring Learning series.
And remember that if you register,
you'll automatically receive the recording,
even if you can't attend live.
Please feel free to drop any questions
or thoughts in the chat throughout the session.
Jennifer and I will be actively in the chat
and we'll also be sending the recording your way
after this session as well.
So without further ado,
I'm gonna go ahead and pass it on to Sarah.
- Thank you, Frances. - [Frances] You're welcome.
- I'm gonna share my presentation now.
All right, hopefully you can all see
where it says Meet Khanmigo Writing Coach.
Yes, okay, awesome.
I'm gonna try and have the chat open as well
so I can keep an eye on what you all are saying.
Welcome, everyone.
I'm really excited to be here.
A little bit about myself.
My name is Sarah Robertson.
I am a principal product manager here at Khan Academy,
focusing on literacy experiences
and classroom tools to support ELA
and humanities teachers specifically.
And some background of sort of where I came from.
I was a middle school ELA teacher in Boston.
That's a picture of me when I had a lot less gray hair
than I have now.
And when I was a teacher, I taught seventh grade.
Many of my students were significantly below grade level.
I taught a variety of humanities classes
during my time in the classroom,
but the hardest of all the classes
that I taught was writing.
I think as a writing teacher,
I assigned 25 different essays
throughout the course of one single school year.
And like most middle and high school teachers,
I had about 100 students,
maybe more than that,
so that's about 2,500 distinct essay drafts
for me to guide students through producing,
provide timely, detailed, personalized feedback on,
evaluate against a rubric.
I worked easily about 12 hours a day
and nearly every weekend,
and even still, it always felt like I could never get
to all the students in the class
who were asking for my help.
I could never keep a close enough eye
on the students who I knew needed my help,
but weren't asking for it.
I could never deliver personalized feedback
to students quickly enough,
even if I tried really hard.
I never really knew about what my students needed
at any given moment
or how to better support them.
And the burnout from this year
of teaching writing specifically was so real.
And keep in mind this was before COVID,
before learning loss,
and before generative AI,
so I have a lot of empathy for many of you
who are currently in the classroom.
So today I am excited to share with you
the solution that we built
in response to these very real classroom challenges.
So we'll talk a bit about why we built Writing Coach
and our intentions behind its design,
and then I'm gonna dig into how to actually use it
and show you how to find it and how to assign things.
I'll show you what the student experience is like
and what the teacher
and administrator reports are like as well.
And then I will send you off and hopefully you can go
and use it and explore it yourselves.
And then I'll actually give you my email
so you can send me the feedback that you have.
All right, so I had asked if we could make this
a meeting-type webinar
so you all could be a little bit more interactive.
So now we have the chat.
Thank you, Aviv, for setting that up.
I would love for you all
to answer this question in the chat.
I'm wondering if you could just name,
what are some things that you know students need
in order to reach grade-level proficiency in writing?
Take a minute and think
and put your thoughts into the chat.
Kim says, "Vocabulary, sentence fluency,
punctuation, practice
help with sentence structure, grammar, punctuation."
Absolutely.
Format of the writing, yes.
Practice and timely feedback, yes.
I'm seeing a lot of practice, feedback, direct instruction.
Out of curiosity, does anybody know
how much time the National Commission on Writing recommends
that middle and high school students should spend
on writing practice?
Let's say each day.
How many minutes per day do they recommend?
Guess, put it in the chat.
30 minutes.
60 minutes.
30 minutes.
All right, you guys are well informed.
It's 60 minutes per day.
I taught writing,
so in my classroom,
kids were writing a lot of the time
when they weren't watching me model things
or teaching things,
but in many schools,
there isn't a separate writing class
and most students are not actually getting the 60 minutes
of writing practice per day,
as I'm sure you all are well aware.
Other things that I saw mentioned that students need,
they need the direct instruction
specifically in things like the rhetorical situation,
audience, purpose, tone, rhetoric,
text structure, sentence structure, grammar,
the writing process, how we begin to write,
how we think about writing, how to conduct research,
how to incorporate evidence, how to cite evidence.
They also need things like exposure to mentor texts,
they need modeling, they need vocabulary-building practice,
they need feedback
and that feedback has to be specific and actionable
and it also has to be delivered not three weeks
from when they wrote the essay.
And then of course, reading practice and comprehension
is part of that too.
And they also just need to feel motivated,
that's a really big piece.
Again, looking for you all to go to the chat,
what are some of the things
that make teaching writing especially challenging?
Go ahead and put your thoughts in the chat.
Time, yes, for sure.
Yep.
Never enough.
Differentiation.
Yes, Robert.
The feedback piece is, yes, really challenging.
Motivation, yep, yep.
Definitely time for providing that feedback,
especially if you have a lot of students.
Grade-level reading, yep.
Exactly, agree.
Texting all the time.
Yeah, productive struggle, that's a big one.
And I think the motivation piece plays into that too.
So I spend a lot of my time talking to
and learning from ELA teachers,
and writing is consistently one of the hardest parts
of the job as I experienced firsthand.
These are some quotes from interviews
that we've done with ELA teachers
and I also happen to grab some screenshots
from a few of the recent comments
in various ELA teacher Facebook groups that I'm still in.
So yeah, kids are not writing on grade level
and it's really no wonder why.
A big piece of this, as you all mentioned,
is timely, specific, actional feedback is so critical,
but with one teacher to 100-plus students,
it's nearly impossible and teachers are really overwhelmed.
Also, since I've been in the classroom,
which is now, I think, approaching 10 years ago,
it's gotten a lot harder.
My husband is a writing professor,
he teaches college freshmen,
so I hear from his perspective a lot
about how student AI use has affected writing instruction
and practice in his classroom.
I also hear this from many of the teachers that I speak to
through Khan Academy
and I expect that some of you might have had
some personal experience dealing with this too.
And there are various things out there.
Somebody here mentions,
"I've seen several AI checkers online recently.
How accurate are those?"
They're not,
they're not accurate, unfortunately.
We all wish that there was
like one beautiful magical solution to this problem,
but it has been really challenging
for teachers to be able to spot students
who are using this technology.
And it can be, as I'm sure you know,
very difficult to challenge a student
who you think has produced something with ChatGPT,
but you don't have the evidence for that.
Okay, for this one,
I wanted to touch upon another prescient question.
About a year ago I led a workshop with high school students
on AI and writing
and one of the students said to me,
and this was like a very precocious,
I don't know, 14-year-old girl,
she said, "I just don't see the point
in learning to write anymore,
AI can do it for me.
Why do I need to learn to do this thing
that AI can just do for me?"
And I want you to take a minute
and you can answer in the chat,
or if you feel so inclined,
you can come off mute and just say it out loud,
do you agree or disagree?
Do you think that it's still important?
I feel like this is probably a biased group,
but do you think it's important to still learn to write,
for students to still learn to write, and why?
What would you say to that student?
So yes, I'm gonna pause here,
and if you have something to say,
feel free to come off mute and just say or put it in chat.
- Hi Sarah, this is Johna Wallace.
I'm gonna say this out loud,
I think that students do need to learn how to write.
My son works in finance
and he has to be able to communicate to other people
what he has done with his clients,
and if he cannot write coherently,
that communication is not there.
Also with the writing,
it helps clarify students' thinking
and their ability to communicate with one another,
so they need to know how to do that.
- 100%.
You said it so perfectly.
Everything you said.
Writing is thinking,
writing is communicating.
Without knowing how to write,
we can't actually sometimes process
what we even think about something.
We don't have that critical muscle,
we're not building that critical muscle
of processing information and working through information
and communicating it in ways
that reflect our own personal experiences
and our own beliefs.
I came across this Joan Didion quote a while ago
and it sort of gave me exactly
what I think I was trying to convey
when I was presented with these kinds of opinions
from various people,
specifically students who were like,
why do I need to do this anymore?
And Joan Didion says,
"I write entirely to find out what I'm thinking,
what I'm looking at,
what I see and what it means."
And I get chills when I say that every time
because as somebody who,
I consider myself a writer and a writing teacher
and somebody who supports writing teachers now,
and writing is so much more than the written content, right?
It's not,
the process of writing means something
and it's important
and it has real human value.
And I think as teachers,
it's our jobs not just to teach students how to write,
but to teach them about
why they still should be learning to write, right?
There's also a robust body of evidence that shows
that there are real cognitive
and psychological benefits from writing.
It helps students, again, process emotions,
process their thinking,
figure out what they feel,
figure out what they believe.
It's a really wonderful tool
that we all have in our toolbox as human beings.
So I agree,
I don't think that large language models or AI
is a reason that we should not teach writing anymore.
I don't think anybody would be surprised by that.
And if anything, I actually think it's more important now
than it has ever been,
but it is still very critical
to educate students about AI and large language models
and to use it ethically and responsibly in the classroom.
And we could do an entire webinar
specifically on this topic of effective
and ethical AI use in classrooms alone,
but I sort of jotted down a list of some of the things
to consider when we are incorporating AI in the classroom.
The first thing is,
is the use of this AI actually helping students learn
or are we using it as a shortcut?
Will students become reliant on the AI?
And if it's being used as some sort of scaffolding,
how can we gradually remove that scaffolding over time?
Do students understand
how the content from the AI is actually being generated?
Where it comes from,
when to trust it, when definitely not to trust it,
when to say something about the fact
that they might have used it in some way.
And are they using AI in ways
that allow them to focus more of their efforts
on higher-order thinking skills
and also on the human element of writing too?
There are many opinions on these topics,
but I do believe that there are many ways
in which even if we use AI for things like brainstorming,
I think it depends on the situation,
but in some ways some might argue
that that's actually taking away
some of the really important thinking skills
that students do when they brainstorm.
It doesn't hurt maybe to have somebody
to bounce ideas off of or to jumpstart things,
but worlds in which AI generates a first draft
or AI generates a list of ideas,
we should just be thinking about what is the AI doing
that a student previously could have been doing,
and are we taking something away of value to that student?
Okay.
So with all of that in mind,
let's learn about how and why we made Writing Coach
and then I'm going to show you how to use it.
Khan Academy is a well-known company.
It is not necessarily as well-known for writing and reading
and ELA or humanities as it is for STEM,
but our mission has always been
to provide a free world-class education
to anyone and anywhere,
and we've always wanted to provide
that same level of support in the humanities.
So when large language models became broadly available
in late 2022,
we immediately began exploring ways
that we could leverage the technology
to achieve that mission.
And it's probably no surprise that we found LLMs
to be especially helpful in the writing space specifically.
So like all great products,
Writing Coach was also designed
as a solution to some very real problems.
So what we know about writing
is that, of course, students are expected
to meet grade-level standards.
They need to be writing 60 minutes per day.
They need support,
not just at the end of the writing process
when they've produced a draft and they need feedback,
but they actually need help getting started,
they need help outlining, they need help drafting,
and they need to learn
how to do those different parts of the process.
And then, of course, as we all mentioned,
they need specific, actionable,
personalized, timely feedback in order to improve.
But in reality, only about a quarter of secondary students
meet grade-level writing standards.
The vast majority of students are not clocking even close
to their recommended hour of practice a day.
Most teachers have way more students
than they can give personalized writing process support to.
And I mention this math set all the time.
If the average teacher has 100 students
and limits herself to 10 minutes of feedback per essay,
which is something I would do,
I would set my timer 10 minutes per essay,
that's 17 hours for one draft of one assignment
for me to give feedback to every kid,
17 hours for one assignment.
And if kids are writing an hour a day,
I just don't,
the math isn't mathing,
as the kids say these days,
it's just not really possible.
So in 2023, we started working on a solution
and then last fall we launched Writing Coach
to our school and district partners.
And when I talk about Writing Coach,
I think oftentimes people hear AI, writing,
and they sort of have a picture in their head
of what this is.
So I like to kind of explain what it is
against what it's not
and how it's different from some of the other things
that people might be familiar with.
So you might be familiar with tools,
for example, like Grammarly.
I use Grammarly,
Grammarly is great.
Grammarly gives AI-generated feedback to students,
but it typically provides more sentence
or phrase-level suggestions
and allows them to click a button and accept the suggestion
without really having to figure out
how to change it themselves.
You might be familiar with other tools
who can kind of generate feedback quickly
for the teacher using AI or even for the students.
Some of these things are paid, some are free,
but usually the feedback is rather limited,
there's not that much feedback,
or the teacher gets the feedback,
but then has to figure out how to get it to the student.
And to this day, I haven't really found anybody else
who guides students through the actual writing process
stage by stage in the way that we do.
And if you're familiar with ChatGPT,
which I'm assuming all of you are,
it's obviously always more than happy
to just rewrite a student's essay for them
or produce it from scratch.
And there's positive use cases
for all of these different kinds of tools,
but the primary purpose behind them
is usually either to save time or to produce better writing,
not to produce better writers.
And that is really how Writing Coach is different.
The intention behind Writing Coach is to help teachers
create better writers and not just better writing.
So it is an instructional tool
with the primary purpose of student learning
and teacher transparency.
It's a tool designed to guide students
through the full essay-writing process
while preventing cheating.
It's a way for teachers to give students
more writing practice and real-time support.
And we have data dashboards for teachers
that have at-a-glance and in-depth insights
into their students' writing processes.
So it's not just a digital essay-writing platform,
it's not something that grades or evaluate student writing,
it gives feedback, it doesn't score,
and that's really on purpose
because I could talk about this for a long time,
but I think if we're telling students
that their voices matter,
we should have a human reading what they have to say
at some point.
It's not a student productivity tool,
it's not a feedback generator,
and it's not a tool that provides
easy-to-accept suggestions
for improving things like grammar and mechanics.
So just really quick,
we launched Writing Coach now about six months ago or so
and the feedback that we've gotten
has been very, very positive
from the teachers and the students who have used it so far.
We had one teacher talk about how the feedback
that Khanmigo generated was very, very close
to what they would have written themselves
on student writing.
In some cases it was more detailed or more specific.
Another teacher said that they used this
for an IB essay prep course
and they had seen this make the biggest difference
in helping students with that essay
than any any tool they've ever used.
We had one teacher in a district who used this consistently
and her class had every single student pass
that state writing assessment.
And so we were really excited about what this is
and how it works.
So let's just dive into what it looks like.
Okay, so the first step in using Writing Coach
is to create a Writing Coach assignment,
and a Writing Coach assignment is an essay assignment.
To begin, you should start by logging in to Khan Academy.
So you can access Writing Coach
directly through Khan Academy.
As you can see here in the top of every Khan Academy screen,
you see a Khanmigo drop-down
and you can click on Writing Coach and get there that way.
And if you're logged in to a Khan Academy account,
you'll be logged in to a Writing Coach account,
Writing Coach is part of Khan Academy,
but you can also just go to khanmigo.ai/writingcoach
and that will also take you directly to Writing Coach.
Once you are in Writing Coach as a teacher,
you will see three tabs on the left,
one says Assignments,
one says My Essays,
and one says About Writing Coach.
Assignments, self-explanatory,
that is where you create assignments for students.
The My Essays space is kind of like a playground for you
to play around with sample essays that we've created.
You can test out or demo essays
that you might have assigned to your students
so that you can model
how to use it for the first time with them.
But to access your assignments and create assignments,
you would do that from your Assignments tab.
And you just should be looking
for this Create Assignment button
in the top right of the screen.
So a Writing Coach essay assignment starts with a prompt
and it should be a prompt that you are providing.
The essay instructions, I can't emphasize this enough,
are the most important part.
The more details you include,
the more of a stickler Khanmigo will be when, they,
Khanmigo is guiding students through the writing process.
So if you say things like,
make sure in your instructions,
make sure you include three pieces of textual evidence,
Khanmigo will take that information
and check that the student has that amount of evidence
whenever it's giving feedback.
If you say you need to use MLA in-text citations
or APA in-text citations,
Khanmigo will check to make sure that students did that.
If you say that they need to have
a concession/rebuttal paragraph
in their argumentative essay,
again, Khanmigo will have that context
and then make sure students are meeting those requirements.
Also, if you are doing an essay
that is in order to prep students for the ACT
or another test, an AP test or something,
you can say in the instruction specifically,
this is an AP practice test
or this is an ACT practice essay,
and that also helps Khanmigo understand,
okay, now I have all of the information in the world
about what makes a great ACT essay
and I know what an exemplary ACT essay looks like
and I know what the rubric looks like.
Just telling Khanmigo this is an ACT essay,
Khanmigo will take that context
and tailor its interactions accordingly.
One thing I do wanna point out,
'cause this is a common source of confusion,
the grade level drop-down here,
you can select any grade level you want.
Right now we have fifth/sixth,
which is kind of like a beta level
because it will still be a little bit advanced.
And then we have the seventh all the way through 12th.
We use this to get Khanmigo
to tailor the language that it's using
when it's chatting with students,
but also to tailor the feedback that it gives.
So if you select fifth/sixth,
it might give less feedback or less harsh feedback
than if you select 12th grade for the same essay.
And that's, again, in order to make sure
that Khanmigo is leveling itself appropriately.
Students will not see the grade level that you select.
So if you teach 11th graders
and it's the beginning of the year
and you're kind of building things up,
you can select seventh grade as their level
if you really want the feedback
to be a little bit more just broken down
or to kind of build them up
to the higher grade levels of feedback throughout the year.
So that's a scaffolding tool that I like to mention
because it's not always clear.
Students won't see the grade level,
so they're not gonna feel demoralized
if you choose a level that's below
the level they're actually in.
Another thing I like to tell people.
So when you are designing your essay instructions,
again, for the assignment,
things that you can use are details from the actual rubrics
that you will be using or that are relevant.
So for example, if you're preparing students
for the Arizona six through eight argumentative essay,
you're assigning something,
you can use former released essay prompts, for example.
One thing you can do in your essay instructions is to say,
this is a writing practice test
for the ASA seventh grade essay.
Students will be evaluated on purpose,
focus and organization,
evidence and elaboration, conventions.
And you can even say things from that score four,
so you can literally copy and paste.
The response should have a strongly maintained claim
with little or no,
wait, yeah,
loosely related material.
So you're pulling the exact information
from that score four.
So that means that when Khanmigo is helping the student
through the essay writing,
drafting and revising and all of that,
it will have that in mind that those are the expectations
that will make this a great essay.
So if you wanna get more information in there,
that's usually always very helpful
to make sure that Khanmigo is preparing students
for the essay.
No, you cannot upload the rubric yet,
that is something that we are hoping to do,
but instead you can copy and paste details
from this part of the rubric.
Again, you really only need to tell it
this is what you would need to do to get a four.
You don't need to provide the rest of it
because, again, this isn't an evaluative tool,
it's not gonna score students,
so it's really about giving them feedback
that will help them get the highest score possible.
And then for ACT, I think I remember,
am I remembering correctly
that high schools in Arizona use the ACT?
Okay.
You don't have to do that level of detail,
you can just say this is an ACT practice essay
because Khanmigo knows ACT practice essay
and knows what a great ACT practice essay looks like,
it knows what will get you the highest score,
and it will make sure that it's giving you feedback
and tailoring directions
according to those expectations for ACT.
So once you have created your assignment,
I think the only other details we ask for are due date
and then you can create a class if you haven't yet
or assign it to a class you already have on Khan Academy.
Your students can access it
from their Khan Academy main dashboard
or they can go directly to Writing Coach themselves.
You'll also, when you create your assignment,
you'll see a link that you can share with them,
you can put it in Google Classroom or Canvas
or whatever you use,
and they can go straight to the assignment.
Students can also go to khanmigo.ai/writingcoach
and log in to their Khan Academy account from there.
But they should be able to access their assignment
directly in Writing Coach or from Khan Academy
or from the link that you share.
So let's walk through what this looks like for students.
On the left here,
students will see the assignment details
that you provided in the essay instructions field
when you were creating the assignment.
And so they'll see the title of the essay,
they will see the type of essay that you selected,
and right now we support argumentative, persuasive,
expository, explanatory, and literary analysis.
And then we also will be adding soon
AP Language and Composition and SAT.
We don't have an ACT one yet,
but again, you can just choose persuasive, argumentative,
or anything relevant,
and then you can specifically say it's an ACT essay
in the instructions.
And then students see the essay prompt
and instructions that you provided
and then they have Khanmigo on the side.
So for understanding,
it's really just about breaking down the prompt,
making sure the student understands the requirements,
making sure they can ask questions
about anything that they don't get that you put in there.
So they can do things like ask Khanmigo
to define specific terms or break things down,
or even if you provided a lot of information,
it can kind of help them search through that
to kind of find specific answers to their questions.
And they can come back to this at any point.
So if at any point they're like,
what were those directions again,
they can come back to this and ask questions,
but they can also access these instructions
throughout the whole rest of the writing process.
Okay.
When students are ready to get started,
and some additional information
that I don't think I mentioned earlier
is that if in cases where the essay requires
that students do readings
or they have like the source packet
or they have a text
that they are supposed to be gathering evidence from,
that is something that they can either do that
before they start their Writing Coach assignment
or they can do it between understanding and outlining.
Outlining works best,
I mean, it can actually help them
with making sure they're selecting relevant evidence,
but it works best if they've already read the text
or they've already looked at the text
and they're familiar with the topic.
And during the understanding phase,
Khanmigo will kind of nudge them to figure out,
have you read the text yet?
Do you need to go back?
And it will kind of give them advice for what to do next
before they start outlining.
But when students are ready to get started,
this is the basic outline template,
probably looks relatively familiar.
It starts out as a basic kind of five-paragraph essay,
but we made it flexible
so that students can remove paragraphs,
they can add additional paragraphs,
they can do things like add additional evidence
or reasoning,
they can add or remove the source fields,
they can add,
I think we have, yeah, reasoning explanation,
I might have mentioned that,
and move paragraphs around.
So we tried to design it so that it would support
many different kinds of academic essays.
Even if you have a one-paragraph essay,
an extended open response,
it will work that way as well
as long as you're clear in the essay instructions
about the format of what you want students to do.
For each of the outline templates,
we also provide some pretty simple exemplar sentences
for each of the fields.
So if you chose persuasive or argumentative,
we will have an example thesis
related to from like a persuasive or argumentative essay
of a completely different topic.
They're all Khan Academy-related,
so I don't think we have any risk of picking the same topic
as you would choose for your students.
But then Khanmigo, again, is on the side
and Khanmigo can help students get started.
If the student gets started and is stuck,
it's there to help them with that,
Khanmigo can see their outline,
it can check the outline
against the assignment instructions,
again, that the teacher has provided.
So if the student doesn't know what to do next,
Khanmigo will offer some suggestions,
but Khanmigo will never ever provide
something for the student
to actually put into their outline.
So Khanmigo is not gonna give them evidence,
it's not going to write a thesis statement for them,
it's not going to provide examples or main points
or any of that,
it will look at what the student has
and it will give the student advice and tips
and guidance for how they can make progress
and what they're supposed to be doing.
It can also,
oh yes,
when the student is finished outlining,
one of the other kind of interesting things about this
is you might have some students
who just don't really wanna talk to Khanmigo
or they don't really feel like chatting
or they don't really know how to chat.
Even if you never talk to Khanmigo,
when the student moves from outlining to drafting,
we check it anyway.
So we have Khanmigo, again, look at the outline
and look at the assignment instructions,
and at this step,
we're really just checking for pretty high-level basic,
not like, is this a well crafted essay?
It's like, does it have a thesis statement?
And is the thesis statement actually related
to what the essay prompt is asking?
Does the student have enough evidence
according to what the teacher asked for?
Does it have main points that actually support
or could reasonably support the thesis?
I sometimes had seventh graders
who would start with a thesis on one side of an argument,
and then halfway through, they'd switch sides
and wouldn't even realize that they did that.
So that's one of the things that we check for.
And again, if you ask for something like a counterargument
in your instructions,
we'll check for that here.
And if the student has not met
those very basic requirements,
there will be kind of a stopping point where we say,
it looks like your teacher asked for this
and you didn't quite do this part,
so it will let them go back
and they can fix what they didn't do correctly
and then they will move on to drafting at that point.
During the drafting phase,
we try to keep it pretty simple,
we don't want it to be too much distraction.
So again, the student can access the assignment instructions
whenever they need to.
They can also access the outline that they just wrote
from the previous step,
and from that outline,
they can copy and paste the different contents
of their essay that they worked on.
They can take a piece of evidence and put that in
or their thesis statement, et cetera.
They can also just hide this completely
and just have a nice blank drafting area if they want.
But they can chat with Khanmigo if they want to.
And again, Khanmigo in the drafting phase
is going to focus on helping the student get their words
on the paper,
looking at their outline at the appropriate points,
and pulling the information
that the student has already written out in their outline
and helping them kind of begin to craft their arguments
and their points and their prose.
It's not necessarily going to be giving a lot of feedback
at this point because, again,
the drafting phase is really about
get your words on the paper,
get your thoughts on the paper,
get your ideas on the paper,
and then the revising stage will come next.
So Khanmigo at this point is just helping students
get a first draft together
and not too much focusing on the perfection,
trying to make sure every word is perfect.
And again, Khanmigo won't write for them.
You can see here the student might ask them to,
they might ask them to nicely,
and Khanmigo will gently, politely decline
and help them kind of work
through whatever it is they're struggling with.
So when the student has their first draft,
they will at that point move on to the revising stage.
And in revising, the first thing that happens
is Khanmigo will generate feedback in five categories,
which you can see up here.
Introduction and claim is the first one,
then we have evidence and reasoning,
and then there's structure and organization,
conclusion, and style and tone.
Note that in Writing Coach,
Khanmigo will give students feedback
in all of the same areas
as the AASA and ACT writing rubrics,
but it might be categorized slightly differently.
So for this grade six through eight AASA essays
for example,
feedback related to purpose, focus, and organization
that would fall under either introduction or structure.
Evidence and elaboration feedback
would fall under evidence and reasoning.
Conventions would fall under style and tone, et cetera.
And then same for SAT.
Feedback related to development and support
would be under evidence and reasoning,
organization would be under structure, et cetera.
So that the labels are different,
but the feedback is the same.
But I think one of the most important things to know
about the revising stage of Writing Coach
is that it's not just about the feedback.
So of course, in this example,
the student has 20 suggestions
under each of these five categories total
and we let them kind of focus on one area at a time,
but we don't just give them the feedback
and then say, good luck, hope you do well,
we built in revising tools.
So we give, at some point, praise or positive feedback,
but for the critical feedback and the suggestions,
there are actual actionable steps that students can take
to make the changes on the draft right there on the page
or ask Khanmigo for follow-up guidance or support.
So some of the things that students can do are ask Khanmigo
to give them an example,
and this is, I think, one of the most popular things
a student would choose to do.
So if Khanmigo says
your introduction doesn't have enough context
for the reader,
that's, I think, for a very common piece of feedback
for introduction,
the student can say, give me an example,
and Khanmigo will take an essay
on a completely different topic
and then give them an example
of what it looks like not to have context
and then it would look like to have that context,
and that we have seen has been really, really valuable
for students who get a piece of feedback
and they're just like, I don't get it.
So that's very, very helpful.
Then we have explain suggestion,
and that's really helpful.
If Khanmigo is being a little bit wordy,
they can ask it to explain it
and that'll usually result in Khanmigo breaking it down.
If students are struggling readers,
you can also have them say
after they've explained it,
you say, make it simpler, say it simpler,
just keep kind of pushing Khanmigo if they need to.
Students can just ask a general question as well.
But then if they go in
and they change their essay based on the feedback,
they can then immediately just say, check my revision,
and Khanmigo will look at that piece of feedback
and at the change the student just made
and let them know if they've fixed it,
let them know if they've addressed it or not,
and then Khanmigo will say,
you made a little bit of an improvement,
but you could still make it clear
or you can still kind of provide
a little bit more information here
or give them whatever follow-up guidance is necessary
until the student has what they need.
And what else?
They also have access to their outline again
during revising.
And as they are working through their feedback,
they can mark them as resolved
and keep track of where they're at in their revision stage.
As their teacher,
and you'll see this in a minute,
you can see how much feedback they got
and how much they've resolved,
so you can keep track of how close they are
to being done with this second draft.
And when they are finished,
the student can click mark as complete
and that's to let you know that they're done
and happy with their draft.
And they get the option of exporting it,
so they can export it to Microsoft Word or PDF
or even save it as a Google doc.
One other thing to note is that we don't currently support
a work cited page.
Usually for test prep, that's not a concern,
but for longer research essays or whatever,
if you want students to include work cited,
I would have them do that after they've exported it to a doc
or a Google doc,
they can add the work cited there
because this isn't built to kind of support feedback
on that end page.
You can have students provide in-text citations
and it will give them feedback on the in-text citations,
just not the work cited page or the bibliography.
All right, let's look at the teacher report.
So this is the class report for an assignment.
So if you created a persuasive essay assignment
for a given class,
from your Assignments page in your Writing Coach dashboard,
you can click on that and you can see this report.
We designed this report to be a way for you, as a teacher,
to get an at-a-glance view of where students are
and where you might want to dig in
to pay a little bit more close attention
to what happened with a specific student's essay.
So the information that we show you here
is the step of the process that the student is in.
So it'll tell you if a student hasn't started yet,
if they are in the understanding phase or outlining,
if they're drafting or revising or if they're completed.
We also have little flags
that will tell you the student is completed,
but they edited their essay after the due date came,
so you might wanna just see
how much they actually did by the due date
if you have some procrastinators,
or if students have past due date assignments,
you can see that as well.
You can see the last time that they updated it
or how much time they spent overall,
and this is actively spent.
So if they have the tab open
and then go to sleep and come back the next day,
it's not gonna count those hours that they were sleeping.
This is time actively spent on the page,
chatting with amigo or typing or drafting.
Word count is the current word count
for their current latest draft.
Then this is where you would see
of the 21 pieces of feedback Khanmigo gave them,
they've resolved this many of them.
And then these are the originality flags.
Originality flags are designed to help you figure out,
again, where to drill down.
So if you see a critical flag and you hover over it,
then you'll see what exactly happened and where it happened
and then you can click into that
and see exactly what went down.
And I'll show you a live view of that in a moment.
This is also a great place to figure out
if I have a student who didn't get much feedback
or got a lot of feedback and didn't resolve much of it
or didn't spend very much time writing,
but has a really high word count,
those are the kinds of things too
that you might wanna be like,
let me just see exactly what happened here.
So it's really designed to be a way to document
at a high level parts of the writing process
and let you figure out where to drill down into that.
So when you do drill down into one student,
this is an example
of where I would've clicked on an originality flag,
a critical flag.
So it'll take me straight to the moment where this happened.
In this case, it was a student
who, during the drafting phase,
pasted 58 words from somewhere that was not their outline.
And I can see the words that they pasted,
those words are highlighted.
I can also see their chat
at the time that they were drafting.
So I can see here some clues
that maybe the student wasn't following
academic integrity rules.
I can just have a little bit more information
about why it was flagged
and then give me the opportunity
to follow up with that student.
But there's no originality flags,
I might not even need to drill down into this report,
but I can if I want to
because it has the whole record
of the student's writing process.
So I could click on Understanding
and I can see the chat the student had with Khanmigo.
I can click into Outlining
and see their whole outlining history.
I can see their whole drafting history
and all of their chats.
I can see for feedback
all the feedback Khanmigo gave.
I can see the revision student made.
I can see the chats students had with Khanmigo
about their revisions,
what their essay looked like at the time it was due,
what it looked like when they marked it as complete
and all of that kind of record information.
So again, obviously we don't expect teachers
to review all of this information for every single student,
we're not trying to make more work for writing teachers,
but it is here for the cases when you do want to drill down
or you do need to drill down
and you wanna kind of get a sense of what's happening.
And we are also mid process of exploring other ways
that we can help surface more high-level insights
for teachers at the assignment level
so that they know about other information
around when they might wanna drill down
or instructional insights and things like that.
And then quickly,
if you are an administrator for a Khan Academy District,
you also have administrator reporting
and this includes data such as Writing Coach usage.
So that includes the time spent
using Writing Coach among your district,
so how much time students have spent or teachers,
and then also the percent and the total number of students
who have used Writing Coach during a given timeframe.
So I'm going to now show you,
'cause we have a few minutes,
a live view.
Let me change.
Oh, can you see?
Wait, no.
One moment.
- Yeah, we're still seeing the report, Sarah.
- Okay.
Oh, do you see assignment review?
- Yes, assignment review. - Okay, okay, that did work.
So this is an actual assignment report
with fake student data, of course.
In this case I assigned
a school start time persuasive essay.
I can view the essay instructions
that I provided to Khanmigo and the students here.
And this is what it looks like to view the student progress.
So again, I can see who is at which stage,
how far they've gotten.
And then for the originality flags,
if I hover over this,
I get kind of a high-level understanding
of what was flagged.
Critical flags are shown
in cases where students are pasting a lot of text
from outside of their outline
or they're specifically doing this in drafting or revising.
Questionable flags are they might have pasted a phrase
or they might have put things into their outline
that isn't evidence or a source
that you might wanna look at.
And you can kind of get a sense
of what the different levels are here
and how those things are triggered.
And right now it is just paste events.
So what we are looking at are students using the content
that they outlined with Khanmigo.
So if they are creating an outline with Khanmigo
and then they're drafting
and they are pasting an information
that's not from the outline,
that will trigger a flag,
but if they're pasting from their outline
or if it's coming from their outline,
that's fine.
We know that Khanmigo worked with them on it,
we know that this was something that they did
in our record of the experience,
and that will not be flagged.
Let me click into another one of these students.
So if I click on this student again,
you can see first the final draft.
So if I'm just here
and I wanna look at the student's final draft and grade that
or just see what it looked like at the very end,
I can jump straight to that.
But I can also click to understanding
and I can see how much time they spent,
what they talked to Khanmigo about during this stage,
the whole conversation if I want to.
I can also see, again, their record of outlining.
So I can see where they provided a thesis statement,
here they have some evidence,
they started filling in some more details.
And then I can see their whole chat
during the whole evidence,
or sorry, during the whole outlining stage.
And then for drafting,
you can see that the students started
by asking Khanmigo to help them get started.
It prompted them to start with their introduction
and then the student asked Khanmigo for feedback
and Khanmigo recommended they provide more context.
And also giving it too specific feedback about the thesis,
but again,
this stage is not really about the specific feedback,
it's just about get the words on the page.
So I can see, again, any revisions that they made
while they were drafting
and then their final draft.
And then for revising, same thing,
I can see all the feedback they got from Khanmigo.
I can see what they chatted with Khanmigo about
when it came to the feedback that they got.
Here I can see that the student asked Khanmigo
to check that they did their MLA in-text citation correctly.
And then I can see where they've done revisions.
The student got a questionable flag here,
so I can look at that.
But I can see that the flag
was related to a new piece of evidence
that the student pasted.
So I know that that's a non-issue.
You're pasting evidence from somewhere,
I know you didn't write it 'cause it's evidence,
so that's completely fine.
And all the way down to where their final draft was.
So that is the teacher report
and the individual student report.
Let me go back to my deck.
All right, so we have a few minutes left.
These are the instructions for you
to get started on this now
if you're interested in trying this out.
Again, if you already have a Khan Academy account,
then you already have a Writing Coach account.
So you can go to Khan Academy
or you can go to khanmigo.ai/writingcoach
and you can get to Writing Coach from there.
And log in to your Khan Academy account,
and again, that will take you right into Writing Coach.
Some things if you're just getting started
that you could do.
You can start by exploring the student experience.
So if you are in a teacher account,
let me go back to this.
I mentioned these three tabs here.
If you click on My Essays,
this will let you experience some of the student experience.
Note, however,
if you are not a district partner with Khanmigo,
then your students won't be able
to just write their own essay whenever they want,
that's only available to district partners,
but all students will be able
to access assignments from here.
And a thing that you have as a teacher
that your students don't have are these sample essays.
So if you are just trying to get a sense
of the student experience,
these are pretty helpful
because you can click into them
and it'll start you off at understanding
and you can chat with Khanmigo
as if it were like a fresh assignment.
You can go through outlining
and we have pre-filled in content
so you don't have to write an entire outline for yourself,
you can fill it in
and then kind of see the drafting experience.
Again, we pre-fill in an essay for you
so you can try that out
and then you can get Khanmigo to generate feedback
and do the revising stage.
The sample essay tool is also very helpful
if you are demoing Writing Coach with your students
for the first time
because it helps them see
what they will see when they're working on their assignment,
and you can walk them through the stages
and how to use Khanmigo
and how to access all of the different tabs at the bottom
when they're on various stages.
So I'd recommend checking out the student experience first,
and then when you're ready,
you can create your first Writing Coach essay.
Again, you go straight to your Assignments tab
and you click Create Assignment
and you fill in the information from there.
And I am very eager to hear any and all feedback
that anyone has about this.
So if you use it yourself,
if you share it with another teacher and they use it,
I have no problem sharing my email address,
please reach out to me,
it's sarahrobertson@khanacademy.org.
Let me know what your experience was like,
send me your ideas,
send me your critical feedback,
send me anything that you have that could help us continue
to make Writing Coach a really valuable experience
for you and your students.
And yeah, that is it for me.
I will pass it back, I think, to Frances or Aviv.
I don't know if we have room or time for questions.
- Yeah, we had a couple of questions.
One is around,
I know you talked about right now
we don't have the ability to upload rubrics.
The question was asked around uploading sources,
because, how will Khanmigo know if the student-
- [Sarah] Great question.
- making up evidence or the evidence is accurate?
- Yeah, that's a great question.
So we don't have a way right now
to just kind of attach a PDF or upload a reference text,
however, if the reference text is a well-known text
and you include the name of that text
in the actual essay instructions,
Khanmigo will know about the text,
it will be able to kind of provide,
spot any issues with the text that are wildly inaccurate
or just very off base.
It is not really designed to do things
like check that the evidence is legit evidence,
that's not really something that it does.
What it can do is make sure the evidence
that the students provided is aligned
to the points that they're trying to make,
to the argument that they're making,
make sure that if they choose a piece of evidence,
that they are explaining how it helps them prove the point
that they're trying to make.
It can help them do things
like fix their introductions to evidence,
I remember that was one of the biggest things
that I had to deal with with my middle schoolers,
and then their citations as well.
But it won't be able to be like,
hey, that's not a real quote,
'cause it wasn't designed to do that.
But in the future when we do have things
like the ability for you to attach a reference text,
that is something reasonable
that we would be able to support.
- Thank you, Sarah.
- [Sarah] Yeah.
- There was another question around,
can you assign an essay to a class that you created
separate from your roster?
For example, if you had a combined group for tutoring.
- Yes.
- [Frances] Yes. - You can, yep.
It just has to be a Khan Academy class,
which you can actually do.
So when you are,
I'll show you actually,
when you are in your teacher experience
and you are creating your assignment,
you will be able to assign it
to an existing class that you have.
Or oh, this might just be because,
oh, I think I'm in a different,
you should be able to see any other classes that you have
that maybe are not even Khanmigo-rostered classes.
So if you have another class on Khan Academy
that isn't a district-rostered class,
like a tutoring group,
you can assign an essay to that class.
In some cases you can do it straight from here.
There's an option sometimes that is create a new class,
I'm not sure why that's not showing up here.
But if you are on Khan Academy
and you're creating a class there, like a manual class,
you will see that in the list here
'cause this will show you
any of your Khan Academy classes that you have.
- That's right.
- [Sarah] Mm-hmm.
- All right.
Is the feedback able to be differentiated
by student ability,
but just still at same grade level?
For example, some feedback simpler,
but still at grade level?
- Yes and no.
We do have some features that are student-specific
where a student can change, for example,
their Khanmigo reading level.
It's not super expansive,
there's like basic,
which is actually the default for all students,
and then there's maybe two other levels that are higher.
So if you have really advanced students,
you can have them increase their Khanmigo level,
but most students will start out at the basic level.
If you have students who are English learners,
you also can have them,
if you want, this isn't required,
but you can have them change their Khanmigo chat language
to their native language,
and then they can talk to Khanmigo in their native language,
but they're still looking at an English essay
that they're writing
and Khanmigo will still be able
to kind of communicate with them
and look at their essay in English and help them with that.
Other than that,
the scaffolding is really at that point up to the student.
So if the student needs help,
Khanmigo will be there to help them.
And there is that checkpoint between outlining and drafting
where it's checking to see, did they meet the requirements?
So if you have students who are way behind,
then that is a point where they might spend a lot more time
outlining with Khanmigo,
and it will do that automatically
in that if they really are off topic
or they're missing a lot of pieces,
Khanmigo will repeatedly kind of help them through that
until they have a good outline,
whereas other students
would just speed straight through to drafting
'cause they wouldn't have those checkpoints to worry about.
But yeah, we hear this a lot,
there's a lot of concerns,
especially from the past few years
where it just sounds
like students need a lot more scaffolding
than they have in previous years.
And so one of our biggest priorities
is figuring out better ways to support kiddos
and especially find ways to differentiate
to the students who need it the most.
So if you have more ideas,
please send them to me.
- Awesome, thank you.
Thank you, Sarah, so much.
Thank you, everybody that joined us.
I've also dropped in my email in the chat.
In case you guys think of any other questions,
please feel free to email Sarah,
email me.
We are here to help
and support in any way that you guys need
and we truly value your feedback,
your ideas and your feedback.
So thank you, thank you.
- Thank you all so much.