Khan Academy Writing Coach Symposium

- I am excited that you're here.

I'm excited about Khanmigo.

I'm excited because, well,
for a number of reasons.

Again, I know technology can sometimes be

a little bit scary, right?

Like, oh my gosh, where's this
going? What are we gonna do?

How's this gonna work out?

But at the same time, there
are opportunities for it

to really give you a
boost in your classrooms.

And there's a lot of products out there.

Like I literally,

I don't wanna put this
to the Khanmigo folks,

as I told Jason recently, like,

I think I probably get like
two or three calls a week

from vendors who are selling
a variety of products, really.

And truthfully, they just
feel like output machines,

you know, it's like just
having a ChatGPT engine,

but they put a nice pretty interface on it

as opposed to this is a platform
that is interactive, right?

And the Writing Coach is
one of my favorite elements

that you guys are gonna
be dealing with today

because the number one, you
know, one of the things that

you always hear people say like,

"Well, the kids are
just gonna be cheating."

Well, actually we've kind
of solved some of that,

haven't we? As you learn
how some of this works.

And so you can get the
best of both worlds.

You can like harness the
power of AI, you know,

but also allow kids
and allow kids to learn

and stuff like that,
but not have the whole

like, well, they're just using it to cheat

and these kids aren't
gonna be able to write

or anything else like that.

So I really appreciate the
fact that you all are willing

to come here and to learn about this.

Enjoy your time, I know
you're in good hands.

Thank you.

Thank you.
- Thank you.

Hello everyone. Thank
you for welcoming me.

As they said, I'm a
Principal Product Manager

at Khan Academy, so I'm very
excited to be with you all.

I'm a former teacher.

A little bit about me, I
taught middle school, ELA

and humanities in Roxbury in Boston.

I did some curriculum writing,
I did some teacher training.

I got my master's in education
from Boston University.

And after that, when I left the classroom,

I co-founded another company
called CommonLit.org.

Any CommonLit users in here?

And then after that I joined Khan Academy

where I have been focused on
developing literacy resources

for Khan, specifically with Khanmigo.

So when I was a teacher in Boston,

I taught a variety of subjects.

I taught reading, I taught
a little bit of US history

when somebody went on maternity leave

and I taught writing.

And writing was by and
far the hardest subject

that I had to teach.

And I spent at least 10
hours a day just teaching,

working, grading, planning,
and then weekends.

And it was exhausting and it definitely

was one of the reasons why
when I left the classroom,

I felt a sense of relief

because teaching writing was impossible.

It just felt really impossible.

I'd go back to teaching
reading, reading was lovely,

but even reading involves writing as well.

And whenever you assign an essay,

anybody who's taught writing before,

whether it be in history
class, a science class,

an ELA class, you know the time it takes

to provide feedback to students.

So when I came to Khan Academy,

one of the things that we
wanted to do was to try

and figure out a solution
to that specific problem.

So today, what we're gonna be doing

is we're gonna be talking about
why we built Writing Coach

at Khan Academy.

And we're also gonna be talking about

how we designed it specifically

for an instructional purpose,

which is a little bit different

from some other tools out there.

And then we're gonna teach you how

to assign a Writing Coach essay

and we're gonna show you what
the student experience is like

and the teacher reporting as well.

And you all are gonna get
some time to play with it

and experiment with it on your own.

Can anybody tell me how much
time the National Commission

on Writing expects students
to write regularly?

Like what's the time
expectation for student writing

from the National Commission on Writing?

60 minutes a day, which is a lot.

And I think, I mean, I
don't know about you,

but I rarely see that when
I do go into classrooms,

that's pretty rare.

So I heard many of these things mentioned,

60 minutes of writing practice
per day is what's recommended

for students in middle and high school.

They need direct instruction in everything

from audience and purpose and tone

to how to conduct research
and incorporate evidence.

They need exposure to mentor texts.

They need somebody to model
good writing for them.

They need strong vocabulary
and practice vocabulary.

They need feedback that
is specific and actionable

and delivered in a timely manner.

And reading is a big part of that too.

Reading comprehension skills.

And they also need motivation.

They need to kind of care about it

and want to do the task
in order to do it well.

Now, let's do this one out loud.

What are some of the things

that make teaching
writing particularly hard?

Yes.
- Time.

- Tell me more about that.

- [Attendee] Well, if we're talking

about 60 minutes per day,
balancing that against

all the other course skills
that you're trying to build

in regular practice
with, it's challenging.

It's almost unsustainable.

- Yeah. And I think the 60 minutes per day

doesn't necessarily mean 60 minutes

of sustained writing per day.

It might be like 10
minutes in science class,

20 minutes in history class, you know,

and you spread it out
across the various subjects.

I saw another hand, yep.

- [Attendee] There's one of
me and there's 20 of them

and you know, per class.

So getting around the room
to like the management

of teaching writing, right?

I have kids over here
who need a lot more help

and I've got kids who are goofing off

and I've got kids who
are just stuck, you know?

So one of the biggest challenges

that makes it difficult
is meeting the needs

of all the different
kids at the same time.

- And you're talking about
the writing process itself,

not even just once they
turn in their drafts,

the effort required for one teacher

with however many students total

to give the feedback on
all those essays, yes.

- [Attendee] The
accessibility of plagiarism.

- Yes. Absolutely.

Can you, can you talk more about that?

- I'm sorry.
- Can you share more

about your experience with that?

- [Attendee] You know, we as a department,

we've had a diagnostic
so that we have a basis

and we can see the drastic
difference in language

that it is like a red flag,

but it's the time it takes
a way to investigate.

Is this fairly being marked
as plagiarism or not?

And it disrupts the
whole learning process.

And I keep preaching the philosophy.

This is the place to make the mistake.

This is normal part of
writing process is the mess

before you get to clarity.

- How many of you have
experienced something similar

with students using AI for,

purportedly using AI for that purpose?

Okay, I met a teacher in Indiana who said

that she was using Google
draft history, Draftback

to look at like almost
every student's draft

to check that they were,

she even had to make
her outline so specific

that she could compare
exactly what was drafted

to the outline she specified
so that she could make sure

that it wasn't ChatGPT.

And she said this added in addition

to the already really hard
work of being a teacher,

this added extra time to her day.

So I spend a lot of my time talking to

and learning from ELA
teachers specifically.

And again, writing is
one of the hardest parts,

the hardest jobs when
it comes to teaching.

These are some quotes on
the left from interviews

that we've done recently with ELA teachers

before we developed writing
coaching during that process.

And on the right are some screenshots

of recent comments in various
ELA teacher Facebook groups

that I'm still in that
I just wanted to show

because it's such a range of challenges

in a writing classroom.

You have the struggle of
just teaching writing itself,

that alone is very hard.

How do you teach students
to write well without,

you know, it being way too formulaic.

You wanna encourage voice, but
like it's gotta be academic.

And then on top of that,

you have students who are
below grade-level typically.

And getting those minutes
in every day is really hard.

Getting the feedback to students

in a timely manner is very hard.

Supporting them through the
entire writing process is hard.

And then you have AI on top of
that, teachers are exhausted.

It is a struggle.

Now that it's 2025, we now
have to contend with ChatGPT

and DeepSeek and all these
other variety of AI platforms

where students can turn to, to
have AI do the work for them.

I was in another classroom once

where the students were
using the Snapchat AI feature

to do their writing for them.

So they'll find all kind of creative ways.

Another quick turn and talk.

I wanna touch upon another question

that I hear come up sometimes.

And I led a workshop with a high school,

just with a group of
students in a high school

about a year ago.

And one of those students said to me,

I just don't see the point
in learning to write anymore

'cause AI can do it for me.

Like, why do I even need
to learn this anymore?

So I want you to talk amongst your table.

What would you say to
that student in this age

where we have large language models

where AI can produce writing
that can pass for human.

Why is it still important
to learn writing skills

if you think it's still important?

I hope you do. Go ahead.

I would love to hear some
of the reasons shared

at your table.

Do you agree with that
student or not? And why?

Over here.

- [Attendee] Well, not to get to 1984,

but writing is taught, language is taught

any way that you reduce language,

whether into a large language model

or just the reduction of
even being able to break down

your ideas and set critical thinking,

you're losing your own freedom
of thinking and expression.

- Completely agree.

Can everybody hear in either sides? Okay.

Repeat it? Okay.

So what she had said was
that writing is thinking

and taught, and so if you
remove the process of learning

to write, then that's
a detriment to students

who are also using writing
as a way of thinking.

What else?

I heard some people talk about,

oh, you need to be able to
write something to the AI

to prompt it to do what you wanted to do.

That's true, yes.

- [Attendee] Yeah. I mean, we also talked

about good uses for AI.

So for instance, if I have kids writing

a creative writing assignment, and like,

you know, some kids may
not consider themselves

particularly creative, Hey,
have AI generate a prompt

for you rather than
writing the entire thing.

Now at the end of the day,

we're still there to
help support you through

the writing process.

Just because you think
you're not good at writing

doesn't mean you can't not be good at it.

You can always be good at something,

just how you put your mindset towards it.

- Agreed. So that was teaching students

about kind of more ethical,
responsible ways of using AI,

for example, with creative writing,

asking it to give you a prompt

to help spark your creativity.

So in these past couple
of AI-crazed years,

I often return to this
quote from Joan Didion

in which he says, "I
write entirely to find out

what I am thinking, what I am looking at,

what I see and what it means."

And so to go back to
what you said over here,

the process of writing
is a process of learning

and thinking critical in and of itself.

There's also, on top of that
just a robust body of evidence

that shows that real cognitive

and psychological benefits come from

the process of writing too.

Students use writing as a way to process

what they think about things,
what they feel about things

to work through, things to
develop, how they even believe,

like what their argument is just sitting.

If you've ever sat down and
written something, oftentimes

you come out on the other end
thinking something different

just from the process of
having to put your thoughts

on paper and organize them.

So no, I do not personally
see large language models

as a reason not to teach writing anymore.

Did anybody agree with
the student, by the way,

out of curiosity, we don't
need to teach writing anymore?

Wonderful. Okay, great.

I'm in the right room.

So if anything, I think
it's actually more important

maybe now than it ever has been

because it's like thinking as a human

it's a such a deeply
human thing and writing

is such a big part of that.

But it is critical, I
think, to your point,

to educate students about AI
and large language models,

how they work, how to use them ethically

and responsibly in a classroom.

And we could do an entire
symposium on this topic alone,

but I did include some thoughts here

to consider when you're
incorporating AI in the classroom.

So one of the first things I think about

and as you're thinking about these things,

consider other AI tools too.

How many of those tools
are these things true for?

So will the AI actually
help students learn

Or is it more of a shortcut?

Will students become reliant on the AI

if it's being used as scaffolding,

is there a way that we
can gradually remove

that scaffolding?

Do students actually understand

how large language models work

and how that content is
generated and when not to trust

the content that is being generated?

And are they actually using AI in ways

that encourage high order thinking skills

versus kind of put that on the AI itself?

So these are the questions

that I think are important when
we are trying to figure out

should we be using this
AI in the classroom.

So with all of that, let's
talk about Writing Coach.

How writing Coach came to be,

why it is different from other tools.

We went over a little bit of this before,

but some people think Khan Academy

and they think test prep or
math and science content, right?

It's rarely kind of
seen as the ELA resource

or the tool for teaching writing.

But our mission at Khan
Academy has always been

about providing a world class
education to anyone anywhere.

So that has always
included the humanities.

I think we've always just
been trying to figure out

how do we break into the humanities?

How do we support teachers
with that particularly?

So when we started exploring AI solutions

back in 2022, one of the
things that immediately came

to mind was, this is really
good with writing-related tasks,

for obvious reasons, it's
a large language model.

And so we were exploring
what are some of the problems

that we could solve with
this new technology?

And as a former writing teacher,

I knew that the problems
with teaching writing

were some of the hardest to
solve problems out there.

And this is because have to
meet grade level standards.

They're supposed to be
practicing 60 minutes a day,

apparently, many students need support

through the entire writing process,

not just getting that feedback
piece and then delivering

that feedback in a way that's timely

and specific and actionable is a big job.

So in reality, what we often
see in writing classrooms

is that about a quarter of
students actually end up

meeting grade level writing standards

in middle and high
school, a tiny percentage

of students are actually
getting 60 minutes

of writing practice per day.

And because ELA secondary teachers,

middle and high school
teachers tend to have

a hundred or more students often

that means that for a teacher
to deliver timely, specific,

actionable feedback to
all of their students,

if I have a hundred
students and I limit myself

to 10 minutes of feedback per essay,

which I would literally do,
I would set my phone timer

and be like, I'm stopping
after 10 minutes for each one.

That's 17 hours for one
draft of one essay assignment

to get feedback to all of my students.

And we're supposed to
be assigning 16 minutes

of writing per day.

It seemed at the time to be
kind of an intractable problem,

but we decided to try
and tackle it with AI.

And last fall, in September,
we launched the what ultimately

is now the Khanmigo
Writing Coach experience.

It started off as a
prototype that we built.

We did a feedback activity,

we did a college admissions essay activity

that's still available on Khanmigo too.

And then this past fall
is when we launched

the full Writing Coach experience.

So I wanted to go over,

because again, there's a
many, many AI tools out there,

as the commissioner mentioned,

he's getting a lot of phone calls

about all these different tools

and I think you're definitely
familiar with ChatGPT,

it has its usage, but it's
also always more than happy

to write a student's
entire essay for them.

And so there's positive use
cases for all of these tools,

but the primary purpose behind
them it's either to save time

for the student or the teacher,

or it's to produce better writing.

And with Writing Coach, the purpose is

to produce better writers.

And so here's how Writing
Coach is different.

Writing Coach is an instructional tool

with the primary purpose
of student learning

and teacher transparency.

It's to designed to guide students

through the entire process of writing

from the very beginning of
getting that essay prompt

through outlining,
drafting, getting feedback,

revising, submitting to the teacher.

It's also a way for teachers
to get more writing practice

in the classroom and real-time
support for students.

And then it also includes data dashboards

for teachers with at-a-glance,
high-level insights

at the class level and
then in-depth insights

into the actual student writing processes.

So it is not a digital
essay writing platform,

it's not Google Docs, it's
not an evaluative tool

or a grading tool,

we're not gonna just spit
out scores for every kid.

It isn't a student
student productivity tool

like Grammarly where they can
just accept the suggestions.

It doesn't just generate
feedback, although it does do that

and it doesn't focus on
grammar and mechanics

and kind of let students just
accept those suggestions.

So we launched Writing Coach

and some of the feedback
that we've gotten so far

has been related to the
quality of the feedback.

I think this was one of the things

that initially really set it apart

from some of those other tools.

When we developed Writing Coach,

we were a little bit obsessive

about the quality of the feedback.

I think because it was built by a team

of former writing teachers and tutors,

were not gonna settle for
anything that wasn't better than,

you know, what a student teacher
would be able to do, right?

And so we ended up creating
multiple AI prompts

to focus on various parts
of the writing process

as well as various feedback prompts

for different feedback categories.

And we meticulously
tested and refined these

over the course of over a year.

And we're still doing this to make sure

that the feedback is targeted,

it's aligned to what grade
level expectations are

and it's actually actionable and not just,

oh, you did good on this
and maybe you should work

on your organization.

It helps the student through the process

of understanding what's wrong.

So that was one piece of feedback we got

is that the feedback from
Khanmigo is really excellent.

And the other piece is that it has helped

actually improve student writing.

So students who are using Writing Coach,

we had one teacher in Florida, I think,

who used it very often last
fall and every single one,

or she was part of a pilot that
we did actually last spring

and every single one of
her students in her class

passed the state test.

And we think that this is because
it's an instructional tool

and it puts the onus on the
student to have to do all

of the thinking themselves,
all of the work themselves,

we think that that is where
it holds its most promise.

So the first thing to do when
working with Writing Coach

is to create the assignment

and to create the
Writing Coach assignment,

you would start by
logging into Khan Academy

and clicking on the Khanmigo dropdown.

Can you raise your hand
if you are familiar

with the Khan Academy website
and that Khanmigo dropdown?

Okay, if not, once you log in,

you should all have this dropdown.

So a Writing Coach assignment,

when you click that assign button,

let me go back to just show you
where that assign button is.

Once you click on Writing Coach,

you should see an assign button

that's blue in the top right corner.

And every Writing Coach
assignment starts with a prompt.

You provide the prompt, the
teacher provides the prompt

to the student and you can use

any academic essay prompt that you want.

A few things to note about how
Writing Coach was designed,

it was designed for secondary
humanities classrooms.

So we typically say grade
seven and up, 7 through 12.

However, if you can put
fifth, sixth graders on it,

it might just be a little
bit advanced for them.

It struggles a little bit to
kind of bring that level down.

And it's trained on, we've tweaked it

and made it work for middle
and high school essays

starting around grade seven.

So those are the grade
levels we officially support.

But again, you can use it
for fifth and sixth grade.

The three essay types that we
officially support right now,

we're gonna be adding more,
are persuasive, argumentative,

expository, explanatory
and literary analysis.

And so what I recommend,

if you are planning on using Writing Coach

for general writing practice,
those are very good ways

to kind of select the kind of writing

that you want students to do.

But if you're using it for
something like SAT essay prep

or the middle school essay prep,

choose the closest one
to what the prompt is.

So in some cases that's
gonna be literary analysis.

Usually it's argumentative, persuasive.

So just give your best guess

as to the essay type that it is.

And what is really important
about the essay instructions

is that you are as specific as possible.

In every stage of Writing Coach,

Khanmigo is going to
look at the instructions

that you provided and it's
going to be a stickler

for the student about those instructions.

So if you say write an
essay about whatever,

it's not gonna push the student
to include textual evidence

or a certain amount of textual evidence

or even maybe the student
will give examples,

but not actually textual evidence.

So if you want students to
include at least three pieces

of textual evidence, say that.

So be really explicit about
what you want them to do.

If you want them to include
a concession rebuttal

or counterargument paragraph, say so.

If you want students
to cite their evidence

in a specific format, say which one.

You can even tell it exactly what rubric

that you're going to use.

We don't have a feature yet
where you can upload the rubric,

but all you have to do is say,

what I would actually recommend

is this is from the New
Hampshire SAS rubric

from middle school, you can literally take

the entire section from
all of the score fours

and put that into the essay instructions.

Say this essay should
meet these expectations

and Khanmigo will check
the student's essay

to make sure it's meeting
those expectations.

For SAT, similarly, you
can take the same text here

and plot that into the essay instructions

and it's going to pay
attention to what you want.

You can also just say this
is an SAT practice test

for the essay.

And because it's in large
language model, it's gonna know

what that means and it's
gonna know generally

what the expectations are for that.

So if you're in a hurry,

you can just say this is
an SAT essay practice test

and it will kind of have
those expectations in mind.

Another word about
these SAS and SAT tests,

some of them, or I think
most of them actually require

that students read a reading passage.

For SAT, because they're
typically pretty short,

you can actually paste the reading passage

like the entire prompt directly
into essay instructions

into that field there.

If it's a well-known text,

you don't even really have to do that.

It just sort of will be aware of the text

that is being referenced.

If it is longer,

you also don't really
need to include it at all.

I looked at one in this case it says,

"Explain how Paul Bogart
builds an argument

to persuade his audience."

You don't need to put in the
actual reading passage here.

You can just let students
do the reading part outside

of Writing Coach and then
Writing Coach will push them

to include evidence to make
sure the evidence is aligned.

But if it's a not super well-known passage

or it's too long to include

in the essay instructions itself,

Khanmigo won't necessarily be able to say,

oh, that evidence is not
actually from that text

or it might not be able to, you know,

to verify that it is a
real piece of evidence

if it's not from a well known text

or if you don't provide the text.

Does that make sense? Yes.

- [Attendee] Sorry, you
said is there a limit

to how much text you
can put into the prompt?

- I believe there is not.

I just checked to see like-
- If they're doing

a rhetorical analysis of a
of a speech, for example,

and let's say the speech
is like three pages,

would it be able to fit
that whole speech in?

- You could. I'm pretty sure you could.

The one caveat is sometimes
large language models

have this thing called a context window.

And if it's too long,

if you put way too much
information in there,

it might get confused and
it might not work as well.

So if it's a speech that's well-known,

you can just reference
the name of the speech

and that's perfectly fine.

If it's like a random article from some,

I dunno, random publication

that is maybe not super
Googleable and it's short,

you might wanna include it.

But otherwise, I probably
wouldn't include it.

- Okay, thank you.
- Yeah.

Another point about the reading passages.

So if you are gonna be
using this for test prep,

I would have students start by
reading the passages on paper

or wherever you want them
to read them in class.

We do not have anything in Writing Coach

that will do the reading with students.

We're thinking about it, we're
exploring a reading coach,

but for now, have them
do the reading passages

before they start the essay, okay?

Once they get in to their accounts,

students will again
log into their accounts

whether they get there
through Clever or ClassLink,

or whatever it is that you guys
use, log into their account,

they will see their assignment right

on their assignments page,

but they can also do the
same thing that you did.

Go to their Khanmigo learner activities,

go to Writing Coach and
they'll see it there too.

But it's listed under their assignment

so they should see it there.

What Writing Coach does for students,

and I'll show you what this
looks like in a minute,

is it starts by breaking down the prompt

with understanding the assignment.

It helps them formulate the outline,

it helps them get started on a draft

and then it helps 'em with
feedback and revising.

So for understanding the assignment,

if you look up here, I know
there's multiple screens.

We have understandings outlining,
drafting, and revising.

So if your teachers are
doing writing class,

writing coaching class,
you can walk around,

you can see what stage they're in.

But there's also a dashboard for teachers

that I'll show you shortly.

That also tells you exactly

what stage students each
student is working on.

So you can get a sense of how far

in the assignment students are.

In this first stage,
understanding Khanmigo

is looking at the prompt that you gave,

it's looking at the
information you provided

and it's engaging the student

in a conversation about the assignment.

It gives a couple of initial prompts.

Do you want me to explain the essay type?

Do you wanna know what
the requirements are?

Do you want me to summarize
them? Do you have questions?

So students can use this as an opportunity

to ask about, you know, defining terms.

What is the rebuttal again,
like can you remind me

how I cite in text and citations for MLA?

Or can you kind of tell me
what I should do now, right?

And sometimes Khanmigo ask the student,

have you done the readings yet?

If not, go to the readings.

Here's what you should be looking for.

And it's an opportunity for the student

to kind of ask any questions
they need, feel safe,

feel like they can, you
know, share their confusion

with the AI and get anything answered

before they begin the
actual writing process.

So the second step is outlining.

We provide students with a
default flexible outline.

Students can modify this,
they can add body paragraphs,

remove body paragraphs, add
evidence, add explanations,

add reasoning, add sources,
whatever they need,

but we do have a default
three paragraph essay outline.

It also comes with rough
kind of exemplar sentences

for each section.

So students can kind of see an example

of what an in-text citation looks like

or a thesis statement in that's
related to the type of essay

that you specified.

During the stage, the student
again, can chat with Khanmigo,

the student can start by
just saying, I'm stuck,

or you know, they can just dive right in.

They don't have to engage
with Khanmigo at all

if they don't want to.

And one of the nice
things about Writing Coach

is even if they don't
chat with Khanmigo at all,

Khanmigo will still check their work.

So once they move from
outlining to drafting,

without even being asked,
Khanmigo's gonna be like,

I'm just gonna make sure
that your outline is matching

what the instructions are.

Because again, Khanmigo can see

the teacher's assignment
instructions at every stage.

So once the student tries
to move on to drafting,

Khanmigo performs an initial check

to make sure that the thesis

is actually answered the question,

answering the question that is being asked

in the assignment instructions.

It's checking to make sure the main points

have something to do with the thesis.

This is not the part where it's gonna be

like giving a ton of feedback,

it's just doing basic
alignment checks here

and completion checks.

It's checking to make sure the evidence

is related to the points they're making,

that the evidence meets
the assignment expectation.

So again, if you said I need three pieces

of textual evidence and the
student provided one piece

of textual evidence, a random
example from their life

and nothing else, then
it's going to say, hold up,

that's not what your teacher asked for.

And it's gonna give
them a chance to go back

and fix it before they move on.

It's also gonna just check
to make sure they didn't put,

you know, one paragraph before another

that didn't make any sense.

It's looking for, you know,
logical reasoning here.

And if you say they
need a counter argument

and a rebuttal and any of
those kind of core elements

of an argumentative essay

and the student didn't include that,

it's gonna point that out too.

So this is one of those
kind of checks that we do

to make sure that the student
is on the right track.

When we get to drafting,
we have... (chuckles)

Yeah, this is an example of how Khanmigo

is maybe less helpful
than ChatGPT in some ways.

So here, the student has
access to the outline

that they just made with Khanmigo.

So if you see the tabs here at the bottom,

their assignment instructions
are always at the bottom.

They can always chat with Khanmigo

and when they're drafting
they can get to their outline.

So when they're using their outline,

we let them copy and paste
right from their outline

into their draft, they can keep drafting.

At any point, they can
talk to Khanmigo about,

am I doing this right or what comes next?

Or I'm stuck or I don't know what to put.

But obviously, if the
student is asking Khanmigo

to do any of the writing for
them to suggest evidence,

to write a hook, to write a conclusion,

Khanmigo will not do that.

And you'll see this later,

but you also as the teacher,
will have full visibility

and to all of these interactions.

And students, we tell students that too,

so it's not a surprise for them.

They know that this chat
is entirely viewable

to the teacher.

So once the student has a first draft,

they can then submit it for feedback.

Once they submit it for
feedback, what we do

is we generate suggestions in
these five categories up here.

Introduction, evidence
and reasoning, structure

and organization, conclusion
and style and tone.

And again, these are each
meticulously crafted to make sure

that the feedback is
grade level appropriate.

Because again, we know
what grade level you put in

for the students, we know

what the assignment instructions were.

If you said, you know, make
sure there's MLA citations

and make sure that there's this or that.

It's going to put that feedback
under these categories.

And these categories might be different

from the rubrics that
you're used to working with,

but all of the feedback
that you would get,

for example, on the New Hampshire SAS test

would still be reflected
in these categories.

So introduction will
contain feedback on things

like statement of purpose,
evidence and reasoning,

will have things like
evidence and elaboration.

Structure and conclusion
will contain feedback related

to focus and organization

and style and tone will
contain conventions

and editing feedback.

And for SAT, that feedback
also will be reflected

across those different categories.

One thing to note about
reading with SAT, once again,

if you don't either include
the short passage itself

in the essay prompt and instructions

or that the prompt is kind of,

or the essay itself is too long,

or the reading passage rather is too long,

or it's kind of a not
super well-known passage,

it might struggle with the
reading feedback part for SAT

because it doesn't know
what the heck this story

or passage is that you're talking about.

So if you have an example of a passage

that's short enough to
include in the prompt

or is well-known enough to just
be referenced in the prompt,

then it will give feedback related

to your analysis of the reading.

A couple of other things to point out

in our feedback interface.

For every piece of feedback,

including even positive phrase,
which you don't always get,

not every kid gets positive
praise in every category,

but if they do something well,
we try to point that out.

Before any feedback that's
critical or actionable,

what we noticed when we
first launched this version

of the product is that many students

didn't really know what to do next.

They saw the feedback and they
were like, okay, now what?

And we let them chat
with Khanmigo about it,

but they didn't know how to
chat with Khanmigo about it.

So we added these different
examples of how you can talk

to Khanmigo about the
feedback that you're getting.

So some of the things
that we let students do,

and they can ask any question they want

as long as it's related
to improving their essay,

but we let them ask for an example

from another essay prompt.

So if the feedback is related

to adding more context
in your introduction,

then it'll give an example of an essay

on a completely different topic

where it kind of shows
them what what it means

by giving more context to the
reader in the introduction.

We let them ask a follow
up question to explain

the suggestion, so if the
student just doesn't really get

what it's saying, they can
click explain suggestion

and it'll reword it for them.

Or the student can say, break
it down for me even more,

or be more specific and
it'll respond to that.

And then the other thing that it does,

which I think is probably the
most useful is the student can

actually edit their essay in that box.

So they can go back at any point

and they can change the
sentence, they can erase,

they can revise, they can
edit anything they want

and then they can say, check my revision.

And Khanmigo will reference the feedback

that the student is talking about

and it will look at the essay before

and after the student asked
it to check their revision

and it will actually
reassess did they fix it?

Like did they actually
add enough context now

that it makes sense to the reader

what they're talking about.

And Khanmigo will kind
of give them a thumbs up

or you know, it's not quite there yet

and help them work through that.

So anytime a student is, you know,

uncertain about is that
better, did it fix it?

That's a really useful tool

to have them do the revision live

in the interface and then ask
Khanmigo if they fixed it.

And as they're making
the revisions and edits,

they can mark each different
piece of feedback as resolved.

Once they're done revising,
they can export their essay

to PDF or Microsoft Word, or Google Drive.

If you all use any of
those kind of platforms

and you wanna do your grading
there of their final draft

or whatever, those are some methods

where you can have your students

kind of export their final
drafts to those places.

But at the same time,
you will also have access

to final draft directly in Writing Coach

in your kind of teacher dashboard.

So this is what the teacher
experience looks like.

For every writing coach
assignment that a teacher creates,

there is a class assignment report.

The class assignment report
includes the information

about the assignment that you added.

So you can expand it to
view the essay instructions

that you wrote.

But then this is where you can see

for all of the students in your class,

where in the writing process they are?

So if students are haven't started yet

or maybe they're still
chatting with Khanmigo

and understanding or if they're outlining,

if they're actively drafting,
or if they are revising,

or if they've marked it as complete,

you can see it from this dashboard

to get a sense of the
progress that's being made.

We also have flags to let you know

if somebody submitted
something by the due date,

but then edit it after
the due date passed.

So you can go in and kind
of see what exactly they did

after the due date passed.

And obviously, if they didn't
submit it by the due date,

we'll also flag that as past due.

Other things you can
see from this dashboard,

how much time students spent.

This is active time,

this isn't just like they
had their browser open

and they went to sleep like
this is actively scrolling

on the page or talking to
Khanmigo or writing, or revising,

or any of the things that you
could be doing on the page.

So that's the active time that they spent.

And what we typically
find with Writing Coach

for the partners who
have been using it so far

is that the average time spent

is about two hours for one essay.

And that includes everything
from understanding

all the way to completed.

The column under writing feedback,

that's where you can see the number

of suggestions Khanmigo gave.

So you know, if they got 20
suggestions versus like five,

that'll tell you a little bit about

how good their first draft was.

And you can also see how
much of that feedback

they have resolved, if
they've resolved any of it.

So if they've submitted it,

but they didn't resolve any feedback,

maybe they just didn't click the button

but maybe they didn't
actually make any revisions,

I would click in there and
see exactly what happened.

Word count is another piece
of data that you have here.

And then the originality flags,

these are where Khanmigo will tell you

if the student pasted something

that wasn't from their
outline into their draft

or into their revised draft.

So if they're pasting from
their outline, that's fine.

If they're pasting into their
outline evidence, that's fine.

If they're pasting, you know,

more than a couple of words
into their main points

or their thesis, or their
evidence elaboration,

not evidence, but their
reasoning and elaboration

of their outline, that'll get flagged.

And if they're pasting,
I think more than five

or 10 words, I forget
exactly into their draft,

or in their revisions,

that's also gonna raise a flag for you.

And of course, this is
not foolproof, right?

Students might reasonably
be pacing in a new piece

of evidence that they found,

maybe Khanmigo gave them
feedback during revising

that their evidence wasn't that great,

so they went and found a new piece.

You then can go in and check
to see is this a real issue

or was this flagged

because they pasted in new
evidence or something like that.

And of course, students
will always have their ways,

but we do think that this helps

with a lot of the transparency
and from what we've seen,

it has caught a lot of students.

So this is what it looks
like if you go into the,

what we call the individual
student essay report.

So this is when you click
on one of those students

or you click on one of
those originality flags,

you get the entire
student writing process.

So you can see everything
from every single chat

that they had with Khanmigo
from the very beginning.

You can see their entire
outlining history.

You can see their chats
while they were outlining.

You can see their draft history,

their chats while they were drafting.

You can see the first
draft that they submitted

for Khanmigo for feedback.

You can see the feedback
that Khanmigo gave.

You can see when they chatted
with Khanmigo about feedback,

you can see the revisions that they made.

You can see if they edited
after the due date passed.

You can see what the essay looked like

when the due date passed.

So we provide a lot of transparency

into the writing process here.

We do not expect every teacher to dig in

and look at this, all of
this data for every kid

that would increase the amount of work

that we'd be expecting teachers to do.

But this is here for when you need it.

So the reason why this
page is so important

is this page kind of tells
you where to drill down.

So again, if you have
students who wrote an essay

from start to finish in 30
minutes, maybe we check that kid.

If you have a student who
got 20 pieces of feedback

and resolved none of it, maybe
drill down into that one.

Anytime you see an originality flag,

that would be a good reason to drill down.

So there's a lot of data here,

but the intent is to make
it easier to figure out

who should we really look at?

Who should we drill down into?

And in cases where you do have

to investigate potential
issues of plagiarism,

there's a lot of information here

that can help you understand
what exactly happened

in the whole process.

All right. Oh, and real quick,

for those of you who are school
or district administrators,

there is also some
Writing Coach related data

that you can find in
the KAD admin dashboard.

So any time that students
spend working on Writing Coach,

that active time spent that you saw

that gets reflected in learning minutes

and actual student
engagement with Writing Coach

would be reflected as an activity

under the Khanmigo usage section.

So you can get a sense of the number

of students using Khanmigo Writing Coach

or the percentage of students

who are using Khanmigo Writing Coach

and, you know, drill that
down into different schools

or whatever you're looking at.

So I'm gonna actually switch

'cause I have a couple of minutes.

I'm gonna switch over
to the live experience

'cause next I'm gonna
give you all a chance

to log into your accounts
and play around with this.

So again, when you're logged
into your Khan Academy account,

you wanna click on the Khanmigo dropdown

and go to Learner activities,
which is this page.

I'll just show you.

Okay, when you click
on Learner Activities,

it'll take you here.

You're gonna scroll down on
the left hand side menu here

and click on Writing Coach.

And again, your students
can do this as well.

Students can't assign and they
have a different button here.

They can actually use
Writing Coach on their own.

If they just want extra
help with an essay,

you don't need to assign it to them.

But if you assign it to them,
then you get that class report

and all of those useful insights.

So you'll see three tabs here.

The two most important
tabs are your assignments

that you've created for your
classes and your essays.

Your essays are, is kind
of like your playground

for Writing Coach.

So if you wanted to model

how to use Writing coach in a classroom,

you can create your own essay

and that will show up under my essays.

So if you click Try Student Experience,

that's gonna create an essay
under your essays, okay?

So if you wanted to demo for
students how to use this,

you could use that tool
to kind of show them

and work through it together.

But your assignments will show up

under the My Assignments tab.

One of the things that I would
encourage you all to explore

once I let you go off and try this out,

we have two different sample
essays under my essays.

So when you log in, obviously,

writing an essay, it takes a long time.

So even if you are
experimenting with Writing Coach

or just trying it out,

we don't want you to
have to actually outline

and write an entire essay
just to see how it works.

So these sample essays
have pre-filled in outlines

and drafts so that you can
just see what it looks like.

So if I go into, let's see,

Daylight Savings Time persuasive essay,

I'm gonna go back to the beginning.

This is the student preview,
this is not a real essay,

but you can see that there's
the essay instructions

over here and then Khanmigo go over here,

you can chat with it, you
can move on to outlining

and it's already pre-filled
in with some outlining.

I would recommend that you play around

with the leading stuff

or making totally weird
random evidence like

that has nothing to do with the assignment

and talk to Khanmigo and see what happens.

You can try out, you know,
asking it general questions

about what to do next.

You know, if you leave
a bunch of things blank

and see how it responds there.

When you get into drafting,

you can see that it's gonna do the check

and I haven't deleted anything,

so hopefully all these boxes are...

Oh, it still thinks.

Okay, so there are
suggestions with the default.

So this is what it looks
like if you do not meet

those basic expectations, it
will give you the examples

of what you need to fix.

And technically, if you're
like, eh, I disagree,

you can ignore and move on.

Or you can go back down here

and you can revise your outline.

I'm gonna just ignore
and move on to show you

the drafting experience.

Again, we've pre-filled
in a draft here for you.

When a student starts on this
page, it's gonna be blank,

but just to show you, this
is the outline on this side.

So if the student is
pasting from their outline,

they're just gonna click copy

for whatever section they wanna paste in

and paste it into their draft field.

And again, down here that's
where they can switch to chat

or just view the assignment
instructions again.

And when we move to revising,

it's gonna generate the
feedback for you in real-time.

And you can see as it's generating

the different pieces of
feedback that you get.

And again, like you can interact
with all of these chats.

When you're done with the sample essay,

if you wanna just delete
it and start over,

you can press delete and
that's just gonna erase

the sample essay and you can
see a fresh new one down here

at the bottom and you can start over.

So this is also really helpful

if you're just showing students
how to use Writing Coach

so you don't have to actually type in

maybe all of the outlines or the drafts.

This is kind of a quick way
to try out Writing Coach,

so you can play around with that.

You can also play around
with actually creating

a Writing Coach assignment

if your Khan Academy account has classes

just to see what that looks like.

Or you can create your own essay

that has a prompt of your choice.

So those are some of the different ways

that you can do that.

I will also, as you guys are doing that,

I will leave up one of those
sample teacher reports up here.

So if you are curious and
you wanna take a closer look

at what that looks like,

I can kind of show you
what that looks like too.

All right, are there any questions?

Let me go back to the deck about, yes.

- So if a student, you
know, ignores, ignores,

ignores, submits, is
there a function in there

for the teacher to kick
it right back track,

try it again?
- That's a great question.

As long as the due date hasn't passed,

they can go right back.

They can go back and and do it again, but-

- [Attendee] But is there a specific,

like the teacher can send it back

or the kid can just go in?
- The kid can go in

and actually, so they can
go back if they haven't,

the only difference...

Okay, let me think about this.

They can go back at any point

and they can modify their
outline, their draft

or their revised draft at any point

if the due date hasn't passed.

If they have already generated feedback

on their first draft, Khanmigo
can't generate feedback again

on that same assignment.

They only get one chance
to generate feedback.

But what you can do is
have them do it with

the start your own essay and
have them do it again there,

it's not gonna be part of the assignment.

That's one of the things that

we've heard a few people be like,

that be really nice, if I
could just be like, start over.

- [Attendee] You like
to encourage them to,

okay, let's improve upon what we've got.

- Yes.
- I'm gonna send this

back to you right now.

This is what it's going in as,

but we're gonna keep working on this.

And I know for me I would have a due date.

And I guess that was the due date,

but as a middle school teacher especially

that doesn't mean I'm done
working with you. (chuckles)

I wanna give you an opportunity
to review and improve

and continue to learn and grow and so-

- Agree.
- I want 'em to be able

to edit-
- I think actually,

I'm wrong about you
can't, that's why we added

the edited after due date.

You can edit after the due date.

We just tell the teacher
when they did that.

- [Attendee] It would really
be nice to have us a function

that the teacher has sent it back.

- Yes. I agree.
- Because a lot of times

otherwise the kids are gonna
be like whatever I submitted.

- Yeah, I agree.

And actually later on,
after we do all of this,

Bernadette and Barbara are
gonna put up chart paper

where you're all going to
share all of your feedback,

all of your ideas, we wanna hear from you,

how can we make this
better? What's missing?

What are some of the things

that would make this more
useful for all of you?

So save those nuggets and please add them

to the chart paper when
we get to that, yes.

- If I'm a teacher and
I create an assignment

and I put all my stuff in and
then I assign it to my kids

and then let's call it a week later

while they're working on it,

I recognize that I
forgot to add something,

I go back and I add, oh, it
should have been APA not MLA.

Will the feedback for the students

that have already in the
outlining and drafting,

will the feedback then adjust?

- No, not yet.
- Okay.

- But we're working on that, yes.

- So is the feedback only

for writing style structure elaboration

or does it also take
into account the content?

- It does take into account the contents,

but only if the student is
writing about literature

or a reading passage
and the reading passage

isn't super well-known, it might struggle

to like the student could
technically make up, you know,

an article or something.

There's no way for the LLM to know

that that's a made up article.

- [Attendee] Because
that's what I'm noticing

that I need to, if I need
to score for the content

- Yes.
- But it's giving structure,

but not the content.
- Yeah. If you're assigning

an essay about Lord of
the Flies or you know,

any kind of well-known book-
- If we're doing

Revolutionary War or-
- Yes, exactly.

- I'm experiment in science.
- Exactly.

If you're referencing well-known things,

it will give feedback on content.

It'll say, you know,
that's the wrong character.

It might point out like issues with that,

but it has to be kind of a well-known

or commonly known passage.

Or again, if it's a short passage,

you can just put the whole passage

in the assignment instructions for the AI

to have as a reference, yeah.

- [Attendee] I have two short questions.

First, if when it exports
it to Google Drive,

do they have to go in and
reformat it to look like

an MLA essay, double-spacing
time, all that kind of stuff?

- They probably will have
to do some reformatting.

- [Attendee] Okay. My other question is,

where do you see peer review
kind of fitting in to this?

Like let's say they're doing
their first draft via this,

they're getting legit feedback.

Would you then say, after
you've gotten Khanmigo feedback,

still give it to a
partner? Or would you say

do that before-
- I would do that.

Yeah, I would have them
export to Google Drive

and then share it with a
partner and have that partner

kind of annotate it with
Google Doc comments.

And then, I mean, you
could also have them...

One of the things that we don't do yet,

but we wanna do is allow
you to submit a draft

for feedback again.

So what we're actually
planning on doing this year

is letting you just do the feedback part.

So if you just want
students to put a draft in

and you don't really care
about the whole writing process

being documented in Writing
Coach and getting that support,

you just want them to get a feedback,

a round of feedback on a draft,

they will be able to do that.

So you could have them do that

after they did one
round with Writing Coach

and then Google Docs with a peer

or something after.
- If they're getting

conflicting comments from their partner

versus what they got from Khanmigo,

would you say it's then
on the student to decide

what is appropriate feedback to take?

- I think so.

You could also go to Tutor Me Humanities,

which is another Khanmigo
activity and have a student ask.

- Sure.
- Kind go what they think.

But yeah, I think that's a good,

like higher order skill for the kid

is just to figure out, you know,

should I trust Khanmigo's
feedback or my peers feedback?

Or maybe that's a time
to just ask the teacher.

- Thank you.
- Yeah. Other questions?

Okay. All right, so I'm gonna
leave these directions up here

for you all, and we have
until 11:30, I believe.

Yep. You have 'em until 11:30 for you all

to try out these different things,

log into your Khan Academy
account, play around,

create assignments, create essays,

play with the sample essays.

Just go ahead and we'll tell
you when the 15 minutes is up.

- Just trying to put in

like what a student response would be

and then what are some of
that feedback that comes back.

And literally, it came up
with I think 16 different

kind of prompts and things to resolve.

And as a student, we had to go through

and we were kinda doing
this work together,

put in a sample piece of writing,

looking at all those suggestions,

and then you have to go
through and like do the work.

It's not doing any of the work for you.

It continues to come back and
say, you can have an example.

Here's an example, what
we're talking about.

Do it yourself.

So I think teachers can be confident

that it's student work in the end.

And that's what we want
to know is that students,

they're learning the process,
there's modeling there,

there's examples and exemplars there,

but in the end it comes
back to it's gotta be

that student's work, which I
think is a piece that a lot

of teachers are gonna
have reluctance with.

But our first kind of kick of the tires

and testing was that this is good,

this is gonna be really great
for teachers and for students.

I love the ability for
that instant feedback.

Teachers are very excited that students

have the ability to get that
feedback on their writing,

their topic, their level, their
struggles, whatever it is.

Instantaneously, one of our
teachers said, oh my gosh,

this feedback would take me
days to give back to students.

They're getting it instantaneously.

It is critical.

And that's something we as teachers

never have enough time for.

And it's feedback, the most
important piece is as fast

as you can get it back to students.

Accurate, yes.

And, you know, personalized,

but timely is that one critical factor.

And Writing Coach is as timely
as you can possibly get,

it's almost in instantaneous.

So that's exciting.

I really wanna support
teachers in adopting

those smart AI tools that
helps make teaching better,

makes it better for student learning.

At the end, it's all
about student learning.

And this is an example of AI where really,

the focus is all on student learning.

I see very few downsides to this,

AI deployed in this way.

It's not just send 'em to ChatGPT

and say go for it and hope for the best.

This is very structured.

It's within Khan, we have oversight.

Teachers can see the whole process

and guide that whole process

and see what student are
getting back from that.

So if any teachers are kind
of on that fence, whether,

you know, how's AI gonna help?

This is a tool that they should be trying,

- That makes me feel better knowing

that I'm giving them a tool
as opposed to something

that'll help them cheat
or just think for them.

So it definitely helps
them guide their thinking

and a source that you can
trust that's reliable.

It's gonna give them
good quality information

instead of just a bot
auto-populating answers.

It'll free up a lot of
time for teachers to meet

with the students who
really need their help

and they can gauge which ones are grasping

the concepts more easily
and spend more time on that

rather than checking all of
the papers, all the steps.

- The Writing Coach is an
extremely powerful tool

that will help students and teachers

begin the process of writing.

Not only in the feedback process,

but 20 years in the classroom,

one of the most common questions
I get from students is,

how do I start this?

And the writing tool for Khan
Academy really takes them

through that process of how to start it.

And if they have that
question, how do I start this?

The fields are there, Khanmigo is there

to help them brainstorm,
begin that process.

The assignment is there,
so it's all in one place,

which I absolutely love.

Plus it's probably in
whatever learning system

that they have, like Google Classrooms.

So I think that is super,
super helpful for teachers

and students alike.

One-on-one because there
is feedback being given,

it gives the teacher some time

to have more directed one-on-one time

into the deeper elements
of the paper as opposed

to more superficial stuff
like format or paragraphs,

or those things.