- I am excited that you're here.
I'm excited about Khanmigo.
I'm excited because, well, for a number of reasons.
Again, I know technology can sometimes be
a little bit scary, right?
Like, oh my gosh, where's this going? What are we gonna do?
How's this gonna work out?
But at the same time, there are opportunities for it
to really give you a boost in your classrooms.
And there's a lot of products out there.
Like I literally,
I don't wanna put this to the Khanmigo folks,
as I told Jason recently, like,
I think I probably get like two or three calls a week
from vendors who are selling a variety of products, really.
And truthfully, they just feel like output machines,
you know, it's like just having a ChatGPT engine,
but they put a nice pretty interface on it
as opposed to this is a platform that is interactive, right?
And the Writing Coach is one of my favorite elements
that you guys are gonna be dealing with today
because the number one, you know, one of the things that
you always hear people say like,
"Well, the kids are just gonna be cheating."
Well, actually we've kind of solved some of that,
haven't we? As you learn how some of this works.
And so you can get the best of both worlds.
You can like harness the power of AI, you know,
but also allow kids and allow kids to learn
and stuff like that, but not have the whole
like, well, they're just using it to cheat
and these kids aren't gonna be able to write
or anything else like that.
So I really appreciate the fact that you all are willing
to come here and to learn about this.
Enjoy your time, I know you're in good hands.
Thank you.
- Thank you.
Hello everyone. Thank you for welcoming me.
As they said, I'm a Principal Product Manager
at Khan Academy, so I'm very excited to be with you all.
I'm a former teacher.
A little bit about me, I taught middle school, ELA
and humanities in Roxbury in Boston.
I did some curriculum writing, I did some teacher training.
I got my master's in education from Boston University.
And after that, when I left the classroom,
I co-founded another company called CommonLit.org.
Any CommonLit users in here?
And then after that I joined Khan Academy
where I have been focused on developing literacy resources
for Khan, specifically with Khanmigo.
So when I was a teacher in Boston,
I taught a variety of subjects.
I taught reading, I taught a little bit of US history
when somebody went on maternity leave
and I taught writing.
And writing was by and far the hardest subject
that I had to teach.
And I spent at least 10 hours a day just teaching,
working, grading, planning, and then weekends.
And it was exhausting and it definitely
was one of the reasons why when I left the classroom,
I felt a sense of relief
because teaching writing was impossible.
It just felt really impossible.
I'd go back to teaching reading, reading was lovely,
but even reading involves writing as well.
And whenever you assign an essay,
anybody who's taught writing before,
whether it be in history class, a science class,
an ELA class, you know the time it takes
to provide feedback to students.
So when I came to Khan Academy,
one of the things that we wanted to do was to try
and figure out a solution to that specific problem.
So today, what we're gonna be doing
is we're gonna be talking about why we built Writing Coach
at Khan Academy.
And we're also gonna be talking about
how we designed it specifically
for an instructional purpose,
which is a little bit different
from some other tools out there.
And then we're gonna teach you how
to assign a Writing Coach essay
and we're gonna show you what the student experience is like
and the teacher reporting as well.
And you all are gonna get some time to play with it
and experiment with it on your own.
Can anybody tell me how much time the National Commission
on Writing expects students to write regularly?
Like what's the time expectation for student writing
from the National Commission on Writing?
60 minutes a day, which is a lot.
And I think, I mean, I don't know about you,
but I rarely see that when I do go into classrooms,
that's pretty rare.
So I heard many of these things mentioned,
60 minutes of writing practice per day is what's recommended
for students in middle and high school.
They need direct instruction in everything
from audience and purpose and tone
to how to conduct research and incorporate evidence.
They need exposure to mentor texts.
They need somebody to model good writing for them.
They need strong vocabulary and practice vocabulary.
They need feedback that is specific and actionable
and delivered in a timely manner.
And reading is a big part of that too.
Reading comprehension skills.
And they also need motivation.
They need to kind of care about it
and want to do the task in order to do it well.
Now, let's do this one out loud.
What are some of the things
that make teaching writing particularly hard?
Yes. - Time.
- Tell me more about that.
- [Attendee] Well, if we're talking
about 60 minutes per day, balancing that against
all the other course skills that you're trying to build
in regular practice with, it's challenging.
It's almost unsustainable.
- Yeah. And I think the 60 minutes per day
doesn't necessarily mean 60 minutes
of sustained writing per day.
It might be like 10 minutes in science class,
20 minutes in history class, you know,
and you spread it out across the various subjects.
I saw another hand, yep.
- [Attendee] There's one of me and there's 20 of them
and you know, per class.
So getting around the room to like the management
of teaching writing, right?
I have kids over here who need a lot more help
and I've got kids who are goofing off
and I've got kids who are just stuck, you know?
So one of the biggest challenges
that makes it difficult is meeting the needs
of all the different kids at the same time.
- And you're talking about the writing process itself,
not even just once they turn in their drafts,
the effort required for one teacher
with however many students total
to give the feedback on all those essays, yes.
- [Attendee] The accessibility of plagiarism.
- Yes. Absolutely.
Can you, can you talk more about that?
- I'm sorry. - Can you share more
about your experience with that?
- [Attendee] You know, we as a department,
we've had a diagnostic so that we have a basis
and we can see the drastic difference in language
that it is like a red flag,
but it's the time it takes a way to investigate.
Is this fairly being marked as plagiarism or not?
And it disrupts the whole learning process.
And I keep preaching the philosophy.
This is the place to make the mistake.
This is normal part of writing process is the mess
before you get to clarity.
- How many of you have experienced something similar
with students using AI for,
purportedly using AI for that purpose?
Okay, I met a teacher in Indiana who said
that she was using Google draft history, Draftback
to look at like almost every student's draft
to check that they were,
she even had to make her outline so specific
that she could compare exactly what was drafted
to the outline she specified so that she could make sure
that it wasn't ChatGPT.
And she said this added in addition
to the already really hard work of being a teacher,
this added extra time to her day.
So I spend a lot of my time talking to
and learning from ELA teachers specifically.
And again, writing is one of the hardest parts,
the hardest jobs when it comes to teaching.
These are some quotes on the left from interviews
that we've done recently with ELA teachers
before we developed writing coaching during that process.
And on the right are some screenshots
of recent comments in various ELA teacher Facebook groups
that I'm still in that I just wanted to show
because it's such a range of challenges
in a writing classroom.
You have the struggle of just teaching writing itself,
that alone is very hard.
How do you teach students to write well without,
you know, it being way too formulaic.
You wanna encourage voice, but like it's gotta be academic.
And then on top of that,
you have students who are below grade-level typically.
And getting those minutes in every day is really hard.
Getting the feedback to students
in a timely manner is very hard.
Supporting them through the entire writing process is hard.
And then you have AI on top of that, teachers are exhausted.
It is a struggle.
Now that it's 2025, we now have to contend with ChatGPT
and DeepSeek and all these other variety of AI platforms
where students can turn to, to have AI do the work for them.
I was in another classroom once
where the students were using the Snapchat AI feature
to do their writing for them.
So they'll find all kind of creative ways.
Another quick turn and talk.
I wanna touch upon another question
that I hear come up sometimes.
And I led a workshop with a high school,
just with a group of students in a high school
about a year ago.
And one of those students said to me,
I just don't see the point in learning to write anymore
'cause AI can do it for me.
Like, why do I even need to learn this anymore?
So I want you to talk amongst your table.
What would you say to that student in this age
where we have large language models
where AI can produce writing that can pass for human.
Why is it still important to learn writing skills
if you think it's still important?
I hope you do. Go ahead.
I would love to hear some of the reasons shared
at your table.
Do you agree with that student or not? And why?
Over here.
- [Attendee] Well, not to get to 1984,
but writing is taught, language is taught
any way that you reduce language,
whether into a large language model
or just the reduction of even being able to break down
your ideas and set critical thinking,
you're losing your own freedom of thinking and expression.
- Completely agree.
Can everybody hear in either sides? Okay.
Repeat it? Okay.
So what she had said was that writing is thinking
and taught, and so if you remove the process of learning
to write, then that's a detriment to students
who are also using writing as a way of thinking.
What else?
I heard some people talk about,
oh, you need to be able to write something to the AI
to prompt it to do what you wanted to do.
That's true, yes.
- [Attendee] Yeah. I mean, we also talked
about good uses for AI.
So for instance, if I have kids writing
a creative writing assignment, and like,
you know, some kids may not consider themselves
particularly creative, Hey, have AI generate a prompt
for you rather than writing the entire thing.
Now at the end of the day,
we're still there to help support you through
the writing process.
Just because you think you're not good at writing
doesn't mean you can't not be good at it.
You can always be good at something,
just how you put your mindset towards it.
- Agreed. So that was teaching students
about kind of more ethical, responsible ways of using AI,
for example, with creative writing,
asking it to give you a prompt
to help spark your creativity.
So in these past couple of AI-crazed years,
I often return to this quote from Joan Didion
in which he says, "I write entirely to find out
what I am thinking, what I am looking at,
what I see and what it means."
And so to go back to what you said over here,
the process of writing is a process of learning
and thinking critical in and of itself.
There's also, on top of that just a robust body of evidence
that shows that real cognitive
and psychological benefits come from
the process of writing too.
Students use writing as a way to process
what they think about things, what they feel about things
to work through, things to develop, how they even believe,
like what their argument is just sitting.
If you've ever sat down and written something, oftentimes
you come out on the other end thinking something different
just from the process of having to put your thoughts
on paper and organize them.
So no, I do not personally see large language models
as a reason not to teach writing anymore.
Did anybody agree with the student, by the way,
out of curiosity, we don't need to teach writing anymore?
Wonderful. Okay, great.
I'm in the right room.
So if anything, I think it's actually more important
maybe now than it ever has been
because it's like thinking as a human
it's a such a deeply human thing and writing
is such a big part of that.
But it is critical, I think, to your point,
to educate students about AI and large language models,
how they work, how to use them ethically
and responsibly in a classroom.
And we could do an entire symposium on this topic alone,
but I did include some thoughts here
to consider when you're incorporating AI in the classroom.
So one of the first things I think about
and as you're thinking about these things,
consider other AI tools too.
How many of those tools are these things true for?
So will the AI actually help students learn
Or is it more of a shortcut?
Will students become reliant on the AI
if it's being used as scaffolding,
is there a way that we can gradually remove
that scaffolding?
Do students actually understand
how large language models work
and how that content is generated and when not to trust
the content that is being generated?
And are they actually using AI in ways
that encourage high order thinking skills
versus kind of put that on the AI itself?
So these are the questions
that I think are important when we are trying to figure out
should we be using this AI in the classroom.
So with all of that, let's talk about Writing Coach.
How writing Coach came to be,
why it is different from other tools.
We went over a little bit of this before,
but some people think Khan Academy
and they think test prep or math and science content, right?
It's rarely kind of seen as the ELA resource
or the tool for teaching writing.
But our mission at Khan Academy has always been
about providing a world class education to anyone anywhere.
So that has always included the humanities.
I think we've always just been trying to figure out
how do we break into the humanities?
How do we support teachers with that particularly?
So when we started exploring AI solutions
back in 2022, one of the things that immediately came
to mind was, this is really good with writing-related tasks,
for obvious reasons, it's a large language model.
And so we were exploring what are some of the problems
that we could solve with this new technology?
And as a former writing teacher,
I knew that the problems with teaching writing
were some of the hardest to solve problems out there.
And this is because have to meet grade level standards.
They're supposed to be practicing 60 minutes a day,
apparently, many students need support
through the entire writing process,
not just getting that feedback piece and then delivering
that feedback in a way that's timely
and specific and actionable is a big job.
So in reality, what we often see in writing classrooms
is that about a quarter of students actually end up
meeting grade level writing standards
in middle and high school, a tiny percentage
of students are actually getting 60 minutes
of writing practice per day.
And because ELA secondary teachers,
middle and high school teachers tend to have
a hundred or more students often
that means that for a teacher to deliver timely, specific,
actionable feedback to all of their students,
if I have a hundred students and I limit myself
to 10 minutes of feedback per essay,
which I would literally do, I would set my phone timer
and be like, I'm stopping after 10 minutes for each one.
That's 17 hours for one draft of one essay assignment
to get feedback to all of my students.
And we're supposed to be assigning 16 minutes
of writing per day.
It seemed at the time to be kind of an intractable problem,
but we decided to try and tackle it with AI.
And last fall, in September, we launched the what ultimately
is now the Khanmigo Writing Coach experience.
It started off as a prototype that we built.
We did a feedback activity,
we did a college admissions essay activity
that's still available on Khanmigo too.
And then this past fall is when we launched
the full Writing Coach experience.
So I wanted to go over,
because again, there's a many, many AI tools out there,
as the commissioner mentioned,
he's getting a lot of phone calls
about all these different tools
and I think you're definitely familiar with ChatGPT,
it has its usage, but it's also always more than happy
to write a student's entire essay for them.
And so there's positive use cases for all of these tools,
but the primary purpose behind them it's either to save time
for the student or the teacher,
or it's to produce better writing.
And with Writing Coach, the purpose is
to produce better writers.
And so here's how Writing Coach is different.
Writing Coach is an instructional tool
with the primary purpose of student learning
and teacher transparency.
It's to designed to guide students
through the entire process of writing
from the very beginning of getting that essay prompt
through outlining, drafting, getting feedback,
revising, submitting to the teacher.
It's also a way for teachers to get more writing practice
in the classroom and real-time support for students.
And then it also includes data dashboards
for teachers with at-a-glance, high-level insights
at the class level and then in-depth insights
into the actual student writing processes.
So it is not a digital essay writing platform,
it's not Google Docs, it's not an evaluative tool
or a grading tool,
we're not gonna just spit out scores for every kid.
It isn't a student student productivity tool
like Grammarly where they can just accept the suggestions.
It doesn't just generate feedback, although it does do that
and it doesn't focus on grammar and mechanics
and kind of let students just accept those suggestions.
So we launched Writing Coach
and some of the feedback that we've gotten so far
has been related to the quality of the feedback.
I think this was one of the things
that initially really set it apart
from some of those other tools.
When we developed Writing Coach,
we were a little bit obsessive
about the quality of the feedback.
I think because it was built by a team
of former writing teachers and tutors,
were not gonna settle for anything that wasn't better than,
you know, what a student teacher would be able to do, right?
And so we ended up creating multiple AI prompts
to focus on various parts of the writing process
as well as various feedback prompts
for different feedback categories.
And we meticulously tested and refined these
over the course of over a year.
And we're still doing this to make sure
that the feedback is targeted,
it's aligned to what grade level expectations are
and it's actually actionable and not just,
oh, you did good on this and maybe you should work
on your organization.
It helps the student through the process
of understanding what's wrong.
So that was one piece of feedback we got
is that the feedback from Khanmigo is really excellent.
And the other piece is that it has helped
actually improve student writing.
So students who are using Writing Coach,
we had one teacher in Florida, I think,
who used it very often last fall and every single one,
or she was part of a pilot that we did actually last spring
and every single one of her students in her class
passed the state test.
And we think that this is because it's an instructional tool
and it puts the onus on the student to have to do all
of the thinking themselves, all of the work themselves,
we think that that is where it holds its most promise.
So the first thing to do when working with Writing Coach
is to create the assignment
and to create the Writing Coach assignment,
you would start by logging into Khan Academy
and clicking on the Khanmigo dropdown.
Can you raise your hand if you are familiar
with the Khan Academy website and that Khanmigo dropdown?
Okay, if not, once you log in,
you should all have this dropdown.
So a Writing Coach assignment,
when you click that assign button,
let me go back to just show you where that assign button is.
Once you click on Writing Coach,
you should see an assign button
that's blue in the top right corner.
And every Writing Coach assignment starts with a prompt.
You provide the prompt, the teacher provides the prompt
to the student and you can use
any academic essay prompt that you want.
A few things to note about how Writing Coach was designed,
it was designed for secondary humanities classrooms.
So we typically say grade seven and up, 7 through 12.
However, if you can put fifth, sixth graders on it,
it might just be a little bit advanced for them.
It struggles a little bit to kind of bring that level down.
And it's trained on, we've tweaked it
and made it work for middle and high school essays
starting around grade seven.
So those are the grade levels we officially support.
But again, you can use it for fifth and sixth grade.
The three essay types that we officially support right now,
we're gonna be adding more, are persuasive, argumentative,
expository, explanatory and literary analysis.
And so what I recommend,
if you are planning on using Writing Coach
for general writing practice, those are very good ways
to kind of select the kind of writing
that you want students to do.
But if you're using it for something like SAT essay prep
or the middle school essay prep,
choose the closest one to what the prompt is.
So in some cases that's gonna be literary analysis.
Usually it's argumentative, persuasive.
So just give your best guess
as to the essay type that it is.
And what is really important about the essay instructions
is that you are as specific as possible.
In every stage of Writing Coach,
Khanmigo is going to look at the instructions
that you provided and it's going to be a stickler
for the student about those instructions.
So if you say write an essay about whatever,
it's not gonna push the student to include textual evidence
or a certain amount of textual evidence
or even maybe the student will give examples,
but not actually textual evidence.
So if you want students to include at least three pieces
of textual evidence, say that.
So be really explicit about what you want them to do.
If you want them to include a concession rebuttal
or counterargument paragraph, say so.
If you want students to cite their evidence
in a specific format, say which one.
You can even tell it exactly what rubric
that you're going to use.
We don't have a feature yet where you can upload the rubric,
but all you have to do is say,
what I would actually recommend
is this is from the New Hampshire SAS rubric
from middle school, you can literally take
the entire section from all of the score fours
and put that into the essay instructions.
Say this essay should meet these expectations
and Khanmigo will check the student's essay
to make sure it's meeting those expectations.
For SAT, similarly, you can take the same text here
and plot that into the essay instructions
and it's going to pay attention to what you want.
You can also just say this is an SAT practice test
for the essay.
And because it's in large language model, it's gonna know
what that means and it's gonna know generally
what the expectations are for that.
So if you're in a hurry,
you can just say this is an SAT essay practice test
and it will kind of have those expectations in mind.
Another word about these SAS and SAT tests,
some of them, or I think most of them actually require
that students read a reading passage.
For SAT, because they're typically pretty short,
you can actually paste the reading passage
like the entire prompt directly into essay instructions
into that field there.
If it's a well-known text,
you don't even really have to do that.
It just sort of will be aware of the text
that is being referenced.
If it is longer,
you also don't really need to include it at all.
I looked at one in this case it says,
"Explain how Paul Bogart builds an argument
to persuade his audience."
You don't need to put in the actual reading passage here.
You can just let students do the reading part outside
of Writing Coach and then Writing Coach will push them
to include evidence to make sure the evidence is aligned.
But if it's a not super well-known passage
or it's too long to include
in the essay instructions itself,
Khanmigo won't necessarily be able to say,
oh, that evidence is not actually from that text
or it might not be able to, you know,
to verify that it is a real piece of evidence
if it's not from a well known text
or if you don't provide the text.
Does that make sense? Yes.
- [Attendee] Sorry, you said is there a limit
to how much text you can put into the prompt?
- I believe there is not.
I just checked to see like- - If they're doing
a rhetorical analysis of a of a speech, for example,
and let's say the speech is like three pages,
would it be able to fit that whole speech in?
- You could. I'm pretty sure you could.
The one caveat is sometimes large language models
have this thing called a context window.
And if it's too long,
if you put way too much information in there,
it might get confused and it might not work as well.
So if it's a speech that's well-known,
you can just reference the name of the speech
and that's perfectly fine.
If it's like a random article from some,
I dunno, random publication
that is maybe not super Googleable and it's short,
you might wanna include it.
But otherwise, I probably wouldn't include it.
- Okay, thank you. - Yeah.
Another point about the reading passages.
So if you are gonna be using this for test prep,
I would have students start by reading the passages on paper
or wherever you want them to read them in class.
We do not have anything in Writing Coach
that will do the reading with students.
We're thinking about it, we're exploring a reading coach,
but for now, have them do the reading passages
before they start the essay, okay?
Once they get in to their accounts,
students will again log into their accounts
whether they get there through Clever or ClassLink,
or whatever it is that you guys use, log into their account,
they will see their assignment right
on their assignments page,
but they can also do the same thing that you did.
Go to their Khanmigo learner activities,
go to Writing Coach and they'll see it there too.
But it's listed under their assignment
so they should see it there.
What Writing Coach does for students,
and I'll show you what this looks like in a minute,
is it starts by breaking down the prompt
with understanding the assignment.
It helps them formulate the outline,
it helps them get started on a draft
and then it helps 'em with feedback and revising.
So for understanding the assignment,
if you look up here, I know there's multiple screens.
We have understandings outlining, drafting, and revising.
So if your teachers are doing writing class,
writing coaching class, you can walk around,
you can see what stage they're in.
But there's also a dashboard for teachers
that I'll show you shortly.
That also tells you exactly
what stage students each student is working on.
So you can get a sense of how far
in the assignment students are.
In this first stage, understanding Khanmigo
is looking at the prompt that you gave,
it's looking at the information you provided
and it's engaging the student
in a conversation about the assignment.
It gives a couple of initial prompts.
Do you want me to explain the essay type?
Do you wanna know what the requirements are?
Do you want me to summarize them? Do you have questions?
So students can use this as an opportunity
to ask about, you know, defining terms.
What is the rebuttal again, like can you remind me
how I cite in text and citations for MLA?
Or can you kind of tell me what I should do now, right?
And sometimes Khanmigo ask the student,
have you done the readings yet?
If not, go to the readings.
Here's what you should be looking for.
And it's an opportunity for the student
to kind of ask any questions they need, feel safe,
feel like they can, you know, share their confusion
with the AI and get anything answered
before they begin the actual writing process.
So the second step is outlining.
We provide students with a default flexible outline.
Students can modify this, they can add body paragraphs,
remove body paragraphs, add evidence, add explanations,
add reasoning, add sources, whatever they need,
but we do have a default three paragraph essay outline.
It also comes with rough kind of exemplar sentences
for each section.
So students can kind of see an example
of what an in-text citation looks like
or a thesis statement in that's related to the type of essay
that you specified.
During the stage, the student again, can chat with Khanmigo,
the student can start by just saying, I'm stuck,
or you know, they can just dive right in.
They don't have to engage with Khanmigo at all
if they don't want to.
And one of the nice things about Writing Coach
is even if they don't chat with Khanmigo at all,
Khanmigo will still check their work.
So once they move from outlining to drafting,
without even being asked, Khanmigo's gonna be like,
I'm just gonna make sure that your outline is matching
what the instructions are.
Because again, Khanmigo can see
the teacher's assignment instructions at every stage.
So once the student tries to move on to drafting,
Khanmigo performs an initial check
to make sure that the thesis
is actually answered the question,
answering the question that is being asked
in the assignment instructions.
It's checking to make sure the main points
have something to do with the thesis.
This is not the part where it's gonna be
like giving a ton of feedback,
it's just doing basic alignment checks here
and completion checks.
It's checking to make sure the evidence
is related to the points they're making,
that the evidence meets the assignment expectation.
So again, if you said I need three pieces
of textual evidence and the student provided one piece
of textual evidence, a random example from their life
and nothing else, then it's going to say, hold up,
that's not what your teacher asked for.
And it's gonna give them a chance to go back
and fix it before they move on.
It's also gonna just check to make sure they didn't put,
you know, one paragraph before another
that didn't make any sense.
It's looking for, you know, logical reasoning here.
And if you say they need a counter argument
and a rebuttal and any of those kind of core elements
of an argumentative essay
and the student didn't include that,
it's gonna point that out too.
So this is one of those kind of checks that we do
to make sure that the student is on the right track.
When we get to drafting, we have... (chuckles)
Yeah, this is an example of how Khanmigo
is maybe less helpful than ChatGPT in some ways.
So here, the student has access to the outline
that they just made with Khanmigo.
So if you see the tabs here at the bottom,
their assignment instructions are always at the bottom.
They can always chat with Khanmigo
and when they're drafting they can get to their outline.
So when they're using their outline,
we let them copy and paste right from their outline
into their draft, they can keep drafting.
At any point, they can talk to Khanmigo about,
am I doing this right or what comes next?
Or I'm stuck or I don't know what to put.
But obviously, if the student is asking Khanmigo
to do any of the writing for them to suggest evidence,
to write a hook, to write a conclusion,
Khanmigo will not do that.
And you'll see this later,
but you also as the teacher, will have full visibility
and to all of these interactions.
And students, we tell students that too,
so it's not a surprise for them.
They know that this chat is entirely viewable
to the teacher.
So once the student has a first draft,
they can then submit it for feedback.
Once they submit it for feedback, what we do
is we generate suggestions in these five categories up here.
Introduction, evidence and reasoning, structure
and organization, conclusion and style and tone.
And again, these are each meticulously crafted to make sure
that the feedback is grade level appropriate.
Because again, we know what grade level you put in
for the students, we know
what the assignment instructions were.
If you said, you know, make sure there's MLA citations
and make sure that there's this or that.
It's going to put that feedback under these categories.
And these categories might be different
from the rubrics that you're used to working with,
but all of the feedback that you would get,
for example, on the New Hampshire SAS test
would still be reflected in these categories.
So introduction will contain feedback on things
like statement of purpose, evidence and reasoning,
will have things like evidence and elaboration.
Structure and conclusion will contain feedback related
to focus and organization
and style and tone will contain conventions
and editing feedback.
And for SAT, that feedback also will be reflected
across those different categories.
One thing to note about reading with SAT, once again,
if you don't either include the short passage itself
in the essay prompt and instructions
or that the prompt is kind of,
or the essay itself is too long,
or the reading passage rather is too long,
or it's kind of a not super well-known passage,
it might struggle with the reading feedback part for SAT
because it doesn't know what the heck this story
or passage is that you're talking about.
So if you have an example of a passage
that's short enough to include in the prompt
or is well-known enough to just be referenced in the prompt,
then it will give feedback related
to your analysis of the reading.
A couple of other things to point out
in our feedback interface.
For every piece of feedback,
including even positive phrase, which you don't always get,
not every kid gets positive praise in every category,
but if they do something well, we try to point that out.
Before any feedback that's critical or actionable,
what we noticed when we first launched this version
of the product is that many students
didn't really know what to do next.
They saw the feedback and they were like, okay, now what?
And we let them chat with Khanmigo about it,
but they didn't know how to chat with Khanmigo about it.
So we added these different examples of how you can talk
to Khanmigo about the feedback that you're getting.
So some of the things that we let students do,
and they can ask any question they want
as long as it's related to improving their essay,
but we let them ask for an example
from another essay prompt.
So if the feedback is related
to adding more context in your introduction,
then it'll give an example of an essay
on a completely different topic
where it kind of shows them what what it means
by giving more context to the reader in the introduction.
We let them ask a follow up question to explain
the suggestion, so if the student just doesn't really get
what it's saying, they can click explain suggestion
and it'll reword it for them.
Or the student can say, break it down for me even more,
or be more specific and it'll respond to that.
And then the other thing that it does,
which I think is probably the most useful is the student can
actually edit their essay in that box.
So they can go back at any point
and they can change the sentence, they can erase,
they can revise, they can edit anything they want
and then they can say, check my revision.
And Khanmigo will reference the feedback
that the student is talking about
and it will look at the essay before
and after the student asked it to check their revision
and it will actually reassess did they fix it?
Like did they actually add enough context now
that it makes sense to the reader
what they're talking about.
And Khanmigo will kind of give them a thumbs up
or you know, it's not quite there yet
and help them work through that.
So anytime a student is, you know,
uncertain about is that better, did it fix it?
That's a really useful tool
to have them do the revision live
in the interface and then ask Khanmigo if they fixed it.
And as they're making the revisions and edits,
they can mark each different piece of feedback as resolved.
Once they're done revising, they can export their essay
to PDF or Microsoft Word, or Google Drive.
If you all use any of those kind of platforms
and you wanna do your grading there of their final draft
or whatever, those are some methods
where you can have your students
kind of export their final drafts to those places.
But at the same time, you will also have access
to final draft directly in Writing Coach
in your kind of teacher dashboard.
So this is what the teacher experience looks like.
For every writing coach assignment that a teacher creates,
there is a class assignment report.
The class assignment report includes the information
about the assignment that you added.
So you can expand it to view the essay instructions
that you wrote.
But then this is where you can see
for all of the students in your class,
where in the writing process they are?
So if students are haven't started yet
or maybe they're still chatting with Khanmigo
and understanding or if they're outlining,
if they're actively drafting, or if they are revising,
or if they've marked it as complete,
you can see it from this dashboard
to get a sense of the progress that's being made.
We also have flags to let you know
if somebody submitted something by the due date,
but then edit it after the due date passed.
So you can go in and kind of see what exactly they did
after the due date passed.
And obviously, if they didn't submit it by the due date,
we'll also flag that as past due.
Other things you can see from this dashboard,
how much time students spent.
This is active time,
this isn't just like they had their browser open
and they went to sleep like this is actively scrolling
on the page or talking to Khanmigo or writing, or revising,
or any of the things that you could be doing on the page.
So that's the active time that they spent.
And what we typically find with Writing Coach
for the partners who have been using it so far
is that the average time spent
is about two hours for one essay.
And that includes everything from understanding
all the way to completed.
The column under writing feedback,
that's where you can see the number
of suggestions Khanmigo gave.
So you know, if they got 20 suggestions versus like five,
that'll tell you a little bit about
how good their first draft was.
And you can also see how much of that feedback
they have resolved, if they've resolved any of it.
So if they've submitted it,
but they didn't resolve any feedback,
maybe they just didn't click the button
but maybe they didn't actually make any revisions,
I would click in there and see exactly what happened.
Word count is another piece of data that you have here.
And then the originality flags,
these are where Khanmigo will tell you
if the student pasted something
that wasn't from their outline into their draft
or into their revised draft.
So if they're pasting from their outline, that's fine.
If they're pasting into their outline evidence, that's fine.
If they're pasting, you know,
more than a couple of words into their main points
or their thesis, or their evidence elaboration,
not evidence, but their reasoning and elaboration
of their outline, that'll get flagged.
And if they're pasting, I think more than five
or 10 words, I forget exactly into their draft,
or in their revisions,
that's also gonna raise a flag for you.
And of course, this is not foolproof, right?
Students might reasonably be pacing in a new piece
of evidence that they found,
maybe Khanmigo gave them feedback during revising
that their evidence wasn't that great,
so they went and found a new piece.
You then can go in and check to see is this a real issue
or was this flagged
because they pasted in new evidence or something like that.
And of course, students will always have their ways,
but we do think that this helps
with a lot of the transparency and from what we've seen,
it has caught a lot of students.
So this is what it looks like if you go into the,
what we call the individual student essay report.
So this is when you click on one of those students
or you click on one of those originality flags,
you get the entire student writing process.
So you can see everything from every single chat
that they had with Khanmigo from the very beginning.
You can see their entire outlining history.
You can see their chats while they were outlining.
You can see their draft history,
their chats while they were drafting.
You can see the first draft that they submitted
for Khanmigo for feedback.
You can see the feedback that Khanmigo gave.
You can see when they chatted with Khanmigo about feedback,
you can see the revisions that they made.
You can see if they edited after the due date passed.
You can see what the essay looked like
when the due date passed.
So we provide a lot of transparency
into the writing process here.
We do not expect every teacher to dig in
and look at this, all of this data for every kid
that would increase the amount of work
that we'd be expecting teachers to do.
But this is here for when you need it.
So the reason why this page is so important
is this page kind of tells you where to drill down.
So again, if you have students who wrote an essay
from start to finish in 30 minutes, maybe we check that kid.
If you have a student who got 20 pieces of feedback
and resolved none of it, maybe drill down into that one.
Anytime you see an originality flag,
that would be a good reason to drill down.
So there's a lot of data here,
but the intent is to make it easier to figure out
who should we really look at?
Who should we drill down into?
And in cases where you do have
to investigate potential issues of plagiarism,
there's a lot of information here
that can help you understand what exactly happened
in the whole process.
All right. Oh, and real quick,
for those of you who are school or district administrators,
there is also some Writing Coach related data
that you can find in the KAD admin dashboard.
So any time that students spend working on Writing Coach,
that active time spent that you saw
that gets reflected in learning minutes
and actual student engagement with Writing Coach
would be reflected as an activity
under the Khanmigo usage section.
So you can get a sense of the number
of students using Khanmigo Writing Coach
or the percentage of students
who are using Khanmigo Writing Coach
and, you know, drill that down into different schools
or whatever you're looking at.
So I'm gonna actually switch
'cause I have a couple of minutes.
I'm gonna switch over to the live experience
'cause next I'm gonna give you all a chance
to log into your accounts and play around with this.
So again, when you're logged into your Khan Academy account,
you wanna click on the Khanmigo dropdown
and go to Learner activities, which is this page.
I'll just show you.
Okay, when you click on Learner Activities,
it'll take you here.
You're gonna scroll down on the left hand side menu here
and click on Writing Coach.
And again, your students can do this as well.
Students can't assign and they have a different button here.
They can actually use Writing Coach on their own.
If they just want extra help with an essay,
you don't need to assign it to them.
But if you assign it to them, then you get that class report
and all of those useful insights.
So you'll see three tabs here.
The two most important tabs are your assignments
that you've created for your classes and your essays.
Your essays are, is kind of like your playground
for Writing Coach.
So if you wanted to model
how to use Writing coach in a classroom,
you can create your own essay
and that will show up under my essays.
So if you click Try Student Experience,
that's gonna create an essay under your essays, okay?
So if you wanted to demo for students how to use this,
you could use that tool to kind of show them
and work through it together.
But your assignments will show up
under the My Assignments tab.
One of the things that I would encourage you all to explore
once I let you go off and try this out,
we have two different sample essays under my essays.
So when you log in, obviously,
writing an essay, it takes a long time.
So even if you are experimenting with Writing Coach
or just trying it out,
we don't want you to have to actually outline
and write an entire essay just to see how it works.
So these sample essays have pre-filled in outlines
and drafts so that you can just see what it looks like.
So if I go into, let's see,
Daylight Savings Time persuasive essay,
I'm gonna go back to the beginning.
This is the student preview, this is not a real essay,
but you can see that there's the essay instructions
over here and then Khanmigo go over here,
you can chat with it, you can move on to outlining
and it's already pre-filled in with some outlining.
I would recommend that you play around
with the leading stuff
or making totally weird random evidence like
that has nothing to do with the assignment
and talk to Khanmigo and see what happens.
You can try out, you know, asking it general questions
about what to do next.
You know, if you leave a bunch of things blank
and see how it responds there.
When you get into drafting,
you can see that it's gonna do the check
and I haven't deleted anything,
so hopefully all these boxes are...
Oh, it still thinks.
Okay, so there are suggestions with the default.
So this is what it looks like if you do not meet
those basic expectations, it will give you the examples
of what you need to fix.
And technically, if you're like, eh, I disagree,
you can ignore and move on.
Or you can go back down here
and you can revise your outline.
I'm gonna just ignore and move on to show you
the drafting experience.
Again, we've pre-filled in a draft here for you.
When a student starts on this page, it's gonna be blank,
but just to show you, this is the outline on this side.
So if the student is pasting from their outline,
they're just gonna click copy
for whatever section they wanna paste in
and paste it into their draft field.
And again, down here that's where they can switch to chat
or just view the assignment instructions again.
And when we move to revising,
it's gonna generate the feedback for you in real-time.
And you can see as it's generating
the different pieces of feedback that you get.
And again, like you can interact with all of these chats.
When you're done with the sample essay,
if you wanna just delete it and start over,
you can press delete and that's just gonna erase
the sample essay and you can see a fresh new one down here
at the bottom and you can start over.
So this is also really helpful
if you're just showing students how to use Writing Coach
so you don't have to actually type in
maybe all of the outlines or the drafts.
This is kind of a quick way to try out Writing Coach,
so you can play around with that.
You can also play around with actually creating
a Writing Coach assignment
if your Khan Academy account has classes
just to see what that looks like.
Or you can create your own essay
that has a prompt of your choice.
So those are some of the different ways
that you can do that.
I will also, as you guys are doing that,
I will leave up one of those sample teacher reports up here.
So if you are curious and you wanna take a closer look
at what that looks like,
I can kind of show you what that looks like too.
All right, are there any questions?
Let me go back to the deck about, yes.
- So if a student, you know, ignores, ignores,
ignores, submits, is there a function in there
for the teacher to kick it right back track,
try it again? - That's a great question.
As long as the due date hasn't passed,
they can go right back.
They can go back and and do it again, but-
- [Attendee] But is there a specific,
like the teacher can send it back
or the kid can just go in? - The kid can go in
and actually, so they can go back if they haven't,
the only difference...
Okay, let me think about this.
They can go back at any point
and they can modify their outline, their draft
or their revised draft at any point
if the due date hasn't passed.
If they have already generated feedback
on their first draft, Khanmigo can't generate feedback again
on that same assignment.
They only get one chance to generate feedback.
But what you can do is have them do it with
the start your own essay and have them do it again there,
it's not gonna be part of the assignment.
That's one of the things that
we've heard a few people be like,
that be really nice, if I could just be like, start over.
- [Attendee] You like to encourage them to,
okay, let's improve upon what we've got.
- Yes. - I'm gonna send this
back to you right now.
This is what it's going in as,
but we're gonna keep working on this.
And I know for me I would have a due date.
And I guess that was the due date,
but as a middle school teacher especially
that doesn't mean I'm done working with you. (chuckles)
I wanna give you an opportunity to review and improve
and continue to learn and grow and so-
- Agree. - I want 'em to be able
to edit- - I think actually,
I'm wrong about you can't, that's why we added
the edited after due date.
You can edit after the due date.
We just tell the teacher when they did that.
- [Attendee] It would really be nice to have us a function
that the teacher has sent it back.
- Yes. I agree. - Because a lot of times
otherwise the kids are gonna be like whatever I submitted.
- Yeah, I agree.
And actually later on, after we do all of this,
Bernadette and Barbara are gonna put up chart paper
where you're all going to share all of your feedback,
all of your ideas, we wanna hear from you,
how can we make this better? What's missing?
What are some of the things
that would make this more useful for all of you?
So save those nuggets and please add them
to the chart paper when we get to that, yes.
- If I'm a teacher and I create an assignment
and I put all my stuff in and then I assign it to my kids
and then let's call it a week later
while they're working on it,
I recognize that I forgot to add something,
I go back and I add, oh, it should have been APA not MLA.
Will the feedback for the students
that have already in the outlining and drafting,
will the feedback then adjust?
- No, not yet. - Okay.
- But we're working on that, yes.
- So is the feedback only
for writing style structure elaboration
or does it also take into account the content?
- It does take into account the contents,
but only if the student is writing about literature
or a reading passage and the reading passage
isn't super well-known, it might struggle
to like the student could technically make up, you know,
an article or something.
There's no way for the LLM to know
that that's a made up article.
- [Attendee] Because that's what I'm noticing
that I need to, if I need to score for the content
- Yes. - But it's giving structure,
but not the content. - Yeah. If you're assigning
an essay about Lord of the Flies or you know,
any kind of well-known book- - If we're doing
Revolutionary War or- - Yes, exactly.
- I'm experiment in science. - Exactly.
If you're referencing well-known things,
it will give feedback on content.
It'll say, you know, that's the wrong character.
It might point out like issues with that,
but it has to be kind of a well-known
or commonly known passage.
Or again, if it's a short passage,
you can just put the whole passage
in the assignment instructions for the AI
to have as a reference, yeah.
- [Attendee] I have two short questions.
First, if when it exports it to Google Drive,
do they have to go in and reformat it to look like
an MLA essay, double-spacing time, all that kind of stuff?
- They probably will have to do some reformatting.
- [Attendee] Okay. My other question is,
where do you see peer review kind of fitting in to this?
Like let's say they're doing their first draft via this,
they're getting legit feedback.
Would you then say, after you've gotten Khanmigo feedback,
still give it to a partner? Or would you say
do that before- - I would do that.
Yeah, I would have them export to Google Drive
and then share it with a partner and have that partner
kind of annotate it with Google Doc comments.
And then, I mean, you could also have them...
One of the things that we don't do yet,
but we wanna do is allow you to submit a draft
for feedback again.
So what we're actually planning on doing this year
is letting you just do the feedback part.
So if you just want students to put a draft in
and you don't really care about the whole writing process
being documented in Writing Coach and getting that support,
you just want them to get a feedback,
a round of feedback on a draft,
they will be able to do that.
So you could have them do that
after they did one round with Writing Coach
and then Google Docs with a peer
or something after. - If they're getting
conflicting comments from their partner
versus what they got from Khanmigo,
would you say it's then on the student to decide
what is appropriate feedback to take?
- I think so.
You could also go to Tutor Me Humanities,
which is another Khanmigo activity and have a student ask.
- Sure. - Kind go what they think.
But yeah, I think that's a good,
like higher order skill for the kid
is just to figure out, you know,
should I trust Khanmigo's feedback or my peers feedback?
Or maybe that's a time to just ask the teacher.
- Thank you. - Yeah. Other questions?
Okay. All right, so I'm gonna leave these directions up here
for you all, and we have until 11:30, I believe.
Yep. You have 'em until 11:30 for you all
to try out these different things,
log into your Khan Academy account, play around,
create assignments, create essays,
play with the sample essays.
Just go ahead and we'll tell you when the 15 minutes is up.
- Just trying to put in
like what a student response would be
and then what are some of that feedback that comes back.
And literally, it came up with I think 16 different
kind of prompts and things to resolve.
And as a student, we had to go through
and we were kinda doing this work together,
put in a sample piece of writing,
looking at all those suggestions,
and then you have to go through and like do the work.
It's not doing any of the work for you.
It continues to come back and say, you can have an example.
Here's an example, what we're talking about.
Do it yourself.
So I think teachers can be confident
that it's student work in the end.
And that's what we want to know is that students,
they're learning the process, there's modeling there,
there's examples and exemplars there,
but in the end it comes back to it's gotta be
that student's work, which I think is a piece that a lot
of teachers are gonna have reluctance with.
But our first kind of kick of the tires
and testing was that this is good,
this is gonna be really great for teachers and for students.
I love the ability for that instant feedback.
Teachers are very excited that students
have the ability to get that feedback on their writing,
their topic, their level, their struggles, whatever it is.
Instantaneously, one of our teachers said, oh my gosh,
this feedback would take me days to give back to students.
They're getting it instantaneously.
It is critical.
And that's something we as teachers
never have enough time for.
And it's feedback, the most important piece is as fast
as you can get it back to students.
Accurate, yes.
And, you know, personalized,
but timely is that one critical factor.
And Writing Coach is as timely as you can possibly get,
it's almost in instantaneous.
So that's exciting.
I really wanna support teachers in adopting
those smart AI tools that helps make teaching better,
makes it better for student learning.
At the end, it's all about student learning.
And this is an example of AI where really,
the focus is all on student learning.
I see very few downsides to this,
AI deployed in this way.
It's not just send 'em to ChatGPT
and say go for it and hope for the best.
This is very structured.
It's within Khan, we have oversight.
Teachers can see the whole process
and guide that whole process
and see what student are getting back from that.
So if any teachers are kind of on that fence, whether,
you know, how's AI gonna help?
This is a tool that they should be trying,
- That makes me feel better knowing
that I'm giving them a tool as opposed to something
that'll help them cheat or just think for them.
So it definitely helps them guide their thinking
and a source that you can trust that's reliable.
It's gonna give them good quality information
instead of just a bot auto-populating answers.
It'll free up a lot of time for teachers to meet
with the students who really need their help
and they can gauge which ones are grasping
the concepts more easily and spend more time on that
rather than checking all of the papers, all the steps.
- The Writing Coach is an extremely powerful tool
that will help students and teachers
begin the process of writing.
Not only in the feedback process,
but 20 years in the classroom,
one of the most common questions I get from students is,
how do I start this?
And the writing tool for Khan Academy really takes them
through that process of how to start it.
And if they have that question, how do I start this?
The fields are there, Khanmigo is there
to help them brainstorm, begin that process.
The assignment is there, so it's all in one place,
which I absolutely love.
Plus it's probably in whatever learning system
that they have, like Google Classrooms.
So I think that is super, super helpful for teachers
and students alike.
One-on-one because there is feedback being given,
it gives the teacher some time
to have more directed one-on-one time
into the deeper elements of the paper as opposed
to more superficial stuff like format or paragraphs,
or those things.