1
00:00:00,330 --> 00:00:01,830
- I am excited that you're here.

2
00:00:01,830 --> 00:00:03,510
I'm excited about Khanmigo.

3
00:00:03,510 --> 00:00:06,870
I'm excited because, well,
for a number of reasons.

4
00:00:06,870 --> 00:00:09,330
Again, I know technology can sometimes be

5
00:00:09,330 --> 00:00:10,620
a little bit scary, right?

6
00:00:10,620 --> 00:00:12,870
Like, oh my gosh, where's this
going? What are we gonna do?

7
00:00:12,870 --> 00:00:14,760
How's this gonna work out?

8
00:00:14,760 --> 00:00:17,880
But at the same time, there
are opportunities for it

9
00:00:17,880 --> 00:00:22,880
to really give you a
boost in your classrooms.

10
00:00:23,220 --> 00:00:26,790
And there's a lot of products out there.

11
00:00:26,790 --> 00:00:28,019
Like I literally,

12
00:00:28,019 --> 00:00:30,360
I don't wanna put this
to the Khanmigo folks,

13
00:00:30,360 --> 00:00:32,430
as I told Jason recently, like,

14
00:00:32,430 --> 00:00:35,430
I think I probably get like
two or three calls a week

15
00:00:35,430 --> 00:00:38,640
from vendors who are selling
a variety of products, really.

16
00:00:38,640 --> 00:00:42,270
And truthfully, they just
feel like output machines,

17
00:00:42,270 --> 00:00:45,030
you know, it's like just
having a ChatGPT engine,

18
00:00:45,030 --> 00:00:47,220
but they put a nice pretty interface on it

19
00:00:47,220 --> 00:00:51,270
as opposed to this is a platform
that is interactive, right?

20
00:00:51,270 --> 00:00:54,450
And the Writing Coach is
one of my favorite elements

21
00:00:54,450 --> 00:00:57,090
that you guys are gonna
be dealing with today

22
00:00:57,090 --> 00:00:59,670
because the number one, you
know, one of the things that

23
00:00:59,670 --> 00:01:01,207
you always hear people say like,

24
00:01:01,207 --> 00:01:03,210
"Well, the kids are
just gonna be cheating."

25
00:01:03,210 --> 00:01:04,890
Well, actually we've kind
of solved some of that,

26
00:01:04,890 --> 00:01:07,227
haven't we? As you learn
how some of this works.

27
00:01:07,227 --> 00:01:09,180
And so you can get the
best of both worlds.

28
00:01:09,180 --> 00:01:14,180
You can like harness the
power of AI, you know,

29
00:01:14,550 --> 00:01:17,370
but also allow kids
and allow kids to learn

30
00:01:17,370 --> 00:01:19,620
and stuff like that,
but not have the whole

31
00:01:19,620 --> 00:01:21,360
like, well, they're just using it to cheat

32
00:01:21,360 --> 00:01:22,800
and these kids aren't
gonna be able to write

33
00:01:22,800 --> 00:01:23,891
or anything else like that.

34
00:01:23,891 --> 00:01:27,540
So I really appreciate the
fact that you all are willing

35
00:01:27,540 --> 00:01:29,760
to come here and to learn about this.

36
00:01:29,760 --> 00:01:32,446
Enjoy your time, I know
you're in good hands.

37
00:01:32,446 --> 00:01:33,724
Thank you.

38
00:01:33,724 --> 00:01:34,830
Thank you.
- Thank you.

39
00:01:34,830 --> 00:01:37,710
Hello everyone. Thank
you for welcoming me.

40
00:01:37,710 --> 00:01:40,020
As they said, I'm a
Principal Product Manager

41
00:01:40,020 --> 00:01:42,300
at Khan Academy, so I'm very
excited to be with you all.

42
00:01:42,300 --> 00:01:44,070
I'm a former teacher.

43
00:01:44,070 --> 00:01:47,070
A little bit about me, I
taught middle school, ELA

44
00:01:47,070 --> 00:01:49,923
and humanities in Roxbury in Boston.

45
00:01:50,760 --> 00:01:54,090
I did some curriculum writing,
I did some teacher training.

46
00:01:54,090 --> 00:01:57,030
I got my master's in education
from Boston University.

47
00:01:57,030 --> 00:01:59,640
And after that, when I left the classroom,

48
00:01:59,640 --> 00:02:02,730
I co-founded another company
called CommonLit.org.

49
00:02:02,730 --> 00:02:04,263
Any CommonLit users in here?

50
00:02:05,460 --> 00:02:07,410
And then after that I joined Khan Academy

51
00:02:07,410 --> 00:02:10,170
where I have been focused on
developing literacy resources

52
00:02:10,170 --> 00:02:12,630
for Khan, specifically with Khanmigo.

53
00:02:12,630 --> 00:02:15,510
So when I was a teacher in Boston,

54
00:02:15,510 --> 00:02:17,040
I taught a variety of subjects.

55
00:02:17,040 --> 00:02:19,560
I taught reading, I taught
a little bit of US history

56
00:02:19,560 --> 00:02:21,900
when somebody went on maternity leave

57
00:02:21,900 --> 00:02:23,340
and I taught writing.

58
00:02:23,340 --> 00:02:27,210
And writing was by and
far the hardest subject

59
00:02:27,210 --> 00:02:28,530
that I had to teach.

60
00:02:28,530 --> 00:02:33,530
And I spent at least 10
hours a day just teaching,

61
00:02:34,050 --> 00:02:37,140
working, grading, planning,
and then weekends.

62
00:02:37,140 --> 00:02:39,360
And it was exhausting and it definitely

63
00:02:39,360 --> 00:02:42,120
was one of the reasons why
when I left the classroom,

64
00:02:42,120 --> 00:02:44,340
I felt a sense of relief

65
00:02:44,340 --> 00:02:47,280
because teaching writing was impossible.

66
00:02:47,280 --> 00:02:49,590
It just felt really impossible.

67
00:02:49,590 --> 00:02:52,170
I'd go back to teaching
reading, reading was lovely,

68
00:02:52,170 --> 00:02:54,240
but even reading involves writing as well.

69
00:02:54,240 --> 00:02:55,770
And whenever you assign an essay,

70
00:02:55,770 --> 00:02:57,330
anybody who's taught writing before,

71
00:02:57,330 --> 00:03:00,810
whether it be in history
class, a science class,

72
00:03:00,810 --> 00:03:04,110
an ELA class, you know the time it takes

73
00:03:04,110 --> 00:03:06,570
to provide feedback to students.

74
00:03:06,570 --> 00:03:10,620
So when I came to Khan Academy,

75
00:03:10,620 --> 00:03:13,080
one of the things that we
wanted to do was to try

76
00:03:13,080 --> 00:03:15,540
and figure out a solution
to that specific problem.

77
00:03:15,540 --> 00:03:17,370
So today, what we're gonna be doing

78
00:03:17,370 --> 00:03:19,980
is we're gonna be talking about
why we built Writing Coach

79
00:03:19,980 --> 00:03:21,240
at Khan Academy.

80
00:03:21,240 --> 00:03:24,120
And we're also gonna be talking about

81
00:03:24,120 --> 00:03:26,220
how we designed it specifically

82
00:03:26,220 --> 00:03:28,500
for an instructional purpose,

83
00:03:28,500 --> 00:03:29,430
which is a little bit different

84
00:03:29,430 --> 00:03:30,990
from some other tools out there.

85
00:03:30,990 --> 00:03:32,250
And then we're gonna teach you how

86
00:03:32,250 --> 00:03:34,290
to assign a Writing Coach essay

87
00:03:34,290 --> 00:03:36,447
and we're gonna show you what
the student experience is like

88
00:03:36,447 --> 00:03:38,700
and the teacher reporting as well.

89
00:03:38,700 --> 00:03:41,280
And you all are gonna get
some time to play with it

90
00:03:41,280 --> 00:03:43,560
and experiment with it on your own.

91
00:03:43,560 --> 00:03:46,770
Can anybody tell me how much
time the National Commission

92
00:03:46,770 --> 00:03:51,630
on Writing expects students
to write regularly?

93
00:03:51,630 --> 00:03:54,840
Like what's the time
expectation for student writing

94
00:03:54,840 --> 00:03:56,220
from the National Commission on Writing?

95
00:03:56,220 --> 00:03:59,940
60 minutes a day, which is a lot.

96
00:03:59,940 --> 00:04:02,820
And I think, I mean, I
don't know about you,

97
00:04:02,820 --> 00:04:05,190
but I rarely see that when
I do go into classrooms,

98
00:04:05,190 --> 00:04:06,570
that's pretty rare.

99
00:04:06,570 --> 00:04:08,760
So I heard many of these things mentioned,

100
00:04:08,760 --> 00:04:11,490
60 minutes of writing practice
per day is what's recommended

101
00:04:11,490 --> 00:04:13,470
for students in middle and high school.

102
00:04:13,470 --> 00:04:16,320
They need direct instruction in everything

103
00:04:16,320 --> 00:04:18,060
from audience and purpose and tone

104
00:04:18,060 --> 00:04:21,090
to how to conduct research
and incorporate evidence.

105
00:04:21,090 --> 00:04:22,770
They need exposure to mentor texts.

106
00:04:22,770 --> 00:04:26,250
They need somebody to model
good writing for them.

107
00:04:26,250 --> 00:04:30,000
They need strong vocabulary
and practice vocabulary.

108
00:04:30,000 --> 00:04:33,180
They need feedback that
is specific and actionable

109
00:04:33,180 --> 00:04:35,250
and delivered in a timely manner.

110
00:04:35,250 --> 00:04:37,140
And reading is a big part of that too.

111
00:04:37,140 --> 00:04:38,700
Reading comprehension skills.

112
00:04:38,700 --> 00:04:40,110
And they also need motivation.

113
00:04:40,110 --> 00:04:41,640
They need to kind of care about it

114
00:04:41,640 --> 00:04:44,940
and want to do the task
in order to do it well.

115
00:04:44,940 --> 00:04:46,770
Now, let's do this one out loud.

116
00:04:46,770 --> 00:04:48,120
What are some of the things

117
00:04:48,120 --> 00:04:51,003
that make teaching
writing particularly hard?

118
00:04:52,020 --> 00:04:53,373
Yes.
- Time.

119
00:04:54,210 --> 00:04:55,350
- Tell me more about that.

120
00:04:55,350 --> 00:04:56,730
- [Attendee] Well, if we're talking

121
00:04:56,730 --> 00:05:00,960
about 60 minutes per day,
balancing that against

122
00:05:00,960 --> 00:05:03,870
all the other course skills
that you're trying to build

123
00:05:03,870 --> 00:05:07,950
in regular practice
with, it's challenging.

124
00:05:07,950 --> 00:05:09,330
It's almost unsustainable.

125
00:05:09,330 --> 00:05:11,790
- Yeah. And I think the 60 minutes per day

126
00:05:11,790 --> 00:05:13,350
doesn't necessarily mean 60 minutes

127
00:05:13,350 --> 00:05:15,180
of sustained writing per day.

128
00:05:15,180 --> 00:05:17,490
It might be like 10
minutes in science class,

129
00:05:17,490 --> 00:05:19,227
20 minutes in history class, you know,

130
00:05:19,227 --> 00:05:22,410
and you spread it out
across the various subjects.

131
00:05:22,410 --> 00:05:23,610
I saw another hand, yep.

132
00:05:24,510 --> 00:05:28,557
- [Attendee] There's one of
me and there's 20 of them

133
00:05:28,557 --> 00:05:29,850
and you know, per class.

134
00:05:29,850 --> 00:05:34,620
So getting around the room
to like the management

135
00:05:34,620 --> 00:05:35,700
of teaching writing, right?

136
00:05:35,700 --> 00:05:39,750
I have kids over here
who need a lot more help

137
00:05:39,750 --> 00:05:41,460
and I've got kids who are goofing off

138
00:05:41,460 --> 00:05:43,740
and I've got kids who
are just stuck, you know?

139
00:05:43,740 --> 00:05:46,050
So one of the biggest challenges

140
00:05:46,050 --> 00:05:51,050
that makes it difficult
is meeting the needs

141
00:05:51,300 --> 00:05:53,280
of all the different
kids at the same time.

142
00:05:53,280 --> 00:05:55,680
- And you're talking about
the writing process itself,

143
00:05:55,680 --> 00:05:58,320
not even just once they
turn in their drafts,

144
00:05:58,320 --> 00:06:00,180
the effort required for one teacher

145
00:06:00,180 --> 00:06:02,250
with however many students total

146
00:06:02,250 --> 00:06:04,980
to give the feedback on
all those essays, yes.

147
00:06:04,980 --> 00:06:07,410
- [Attendee] The
accessibility of plagiarism.

148
00:06:07,410 --> 00:06:09,000
- Yes. Absolutely.

149
00:06:09,000 --> 00:06:10,710
Can you, can you talk more about that?

150
00:06:10,710 --> 00:06:11,970
- I'm sorry.
- Can you share more

151
00:06:11,970 --> 00:06:14,124
about your experience with that?

152
00:06:14,124 --> 00:06:19,020
- [Attendee] You know, we as a department,

153
00:06:19,020 --> 00:06:24,020
we've had a diagnostic
so that we have a basis

154
00:06:24,690 --> 00:06:29,690
and we can see the drastic
difference in language

155
00:06:30,030 --> 00:06:31,980
that it is like a red flag,

156
00:06:31,980 --> 00:06:36,000
but it's the time it takes
a way to investigate.

157
00:06:36,000 --> 00:06:39,270
Is this fairly being marked
as plagiarism or not?

158
00:06:39,270 --> 00:06:42,540
And it disrupts the
whole learning process.

159
00:06:42,540 --> 00:06:45,150
And I keep preaching the philosophy.

160
00:06:45,150 --> 00:06:47,700
This is the place to make the mistake.

161
00:06:47,700 --> 00:06:52,470
This is normal part of
writing process is the mess

162
00:06:52,470 --> 00:06:54,540
before you get to clarity.

163
00:06:54,540 --> 00:06:57,120
- How many of you have
experienced something similar

164
00:06:57,120 --> 00:06:58,803
with students using AI for,

165
00:06:59,760 --> 00:07:01,350
purportedly using AI for that purpose?

166
00:07:01,350 --> 00:07:04,920
Okay, I met a teacher in Indiana who said

167
00:07:04,920 --> 00:07:07,530
that she was using Google
draft history, Draftback

168
00:07:07,530 --> 00:07:10,230
to look at like almost
every student's draft

169
00:07:10,230 --> 00:07:11,550
to check that they were,

170
00:07:11,550 --> 00:07:14,940
she even had to make
her outline so specific

171
00:07:14,940 --> 00:07:18,330
that she could compare
exactly what was drafted

172
00:07:18,330 --> 00:07:20,640
to the outline she specified
so that she could make sure

173
00:07:20,640 --> 00:07:22,170
that it wasn't ChatGPT.

174
00:07:22,170 --> 00:07:24,150
And she said this added in addition

175
00:07:24,150 --> 00:07:25,890
to the already really hard
work of being a teacher,

176
00:07:25,890 --> 00:07:28,833
this added extra time to her day.

177
00:07:30,000 --> 00:07:32,400
So I spend a lot of my time talking to

178
00:07:32,400 --> 00:07:35,040
and learning from ELA
teachers specifically.

179
00:07:35,040 --> 00:07:37,890
And again, writing is
one of the hardest parts,

180
00:07:37,890 --> 00:07:40,200
the hardest jobs when
it comes to teaching.

181
00:07:40,200 --> 00:07:42,270
These are some quotes on
the left from interviews

182
00:07:42,270 --> 00:07:44,550
that we've done recently with ELA teachers

183
00:07:44,550 --> 00:07:47,340
before we developed writing
coaching during that process.

184
00:07:47,340 --> 00:07:49,110
And on the right are some screenshots

185
00:07:49,110 --> 00:07:52,950
of recent comments in various
ELA teacher Facebook groups

186
00:07:52,950 --> 00:07:55,920
that I'm still in that
I just wanted to show

187
00:07:55,920 --> 00:07:58,860
because it's such a range of challenges

188
00:07:58,860 --> 00:08:00,180
in a writing classroom.

189
00:08:00,180 --> 00:08:03,300
You have the struggle of
just teaching writing itself,

190
00:08:03,300 --> 00:08:04,410
that alone is very hard.

191
00:08:04,410 --> 00:08:07,290
How do you teach students
to write well without,

192
00:08:07,290 --> 00:08:09,000
you know, it being way too formulaic.

193
00:08:09,000 --> 00:08:12,000
You wanna encourage voice, but
like it's gotta be academic.

194
00:08:12,000 --> 00:08:13,050
And then on top of that,

195
00:08:13,050 --> 00:08:16,170
you have students who are
below grade-level typically.

196
00:08:16,170 --> 00:08:19,560
And getting those minutes
in every day is really hard.

197
00:08:19,560 --> 00:08:21,180
Getting the feedback to students

198
00:08:21,180 --> 00:08:22,620
in a timely manner is very hard.

199
00:08:22,620 --> 00:08:25,470
Supporting them through the
entire writing process is hard.

200
00:08:25,470 --> 00:08:29,580
And then you have AI on top of
that, teachers are exhausted.

201
00:08:29,580 --> 00:08:31,503
It is a struggle.

202
00:08:34,080 --> 00:08:38,247
Now that it's 2025, we now
have to contend with ChatGPT

203
00:08:38,247 --> 00:08:42,390
and DeepSeek and all these
other variety of AI platforms

204
00:08:42,390 --> 00:08:47,040
where students can turn to, to
have AI do the work for them.

205
00:08:47,040 --> 00:08:48,540
I was in another classroom once

206
00:08:48,540 --> 00:08:52,260
where the students were
using the Snapchat AI feature

207
00:08:52,260 --> 00:08:54,360
to do their writing for them.

208
00:08:54,360 --> 00:08:57,240
So they'll find all kind of creative ways.

209
00:08:57,240 --> 00:08:59,940
Another quick turn and talk.

210
00:08:59,940 --> 00:09:01,860
I wanna touch upon another question

211
00:09:01,860 --> 00:09:03,810
that I hear come up sometimes.

212
00:09:03,810 --> 00:09:07,170
And I led a workshop with a high school,

213
00:09:07,170 --> 00:09:09,180
just with a group of
students in a high school

214
00:09:09,180 --> 00:09:10,530
about a year ago.

215
00:09:10,530 --> 00:09:12,930
And one of those students said to me,

216
00:09:12,930 --> 00:09:15,510
I just don't see the point
in learning to write anymore

217
00:09:15,510 --> 00:09:16,680
'cause AI can do it for me.

218
00:09:16,680 --> 00:09:18,540
Like, why do I even need
to learn this anymore?

219
00:09:18,540 --> 00:09:21,480
So I want you to talk amongst your table.

220
00:09:21,480 --> 00:09:24,270
What would you say to
that student in this age

221
00:09:24,270 --> 00:09:25,770
where we have large language models

222
00:09:25,770 --> 00:09:29,370
where AI can produce writing
that can pass for human.

223
00:09:29,370 --> 00:09:32,610
Why is it still important
to learn writing skills

224
00:09:32,610 --> 00:09:33,660
if you think it's still important?

225
00:09:33,660 --> 00:09:34,860
I hope you do. Go ahead.

226
00:09:36,193 --> 00:09:38,610
I would love to hear some
of the reasons shared

227
00:09:38,610 --> 00:09:39,960
at your table.

228
00:09:39,960 --> 00:09:43,143
Do you agree with that
student or not? And why?

229
00:09:44,400 --> 00:09:45,233
Over here.

230
00:09:45,233 --> 00:09:46,750
- [Attendee] Well, not to get to 1984,

231
00:09:47,940 --> 00:09:52,320
but writing is taught, language is taught

232
00:09:52,320 --> 00:09:54,270
any way that you reduce language,

233
00:09:54,270 --> 00:09:56,550
whether into a large language model

234
00:09:56,550 --> 00:09:59,160
or just the reduction of
even being able to break down

235
00:09:59,160 --> 00:10:00,990
your ideas and set critical thinking,

236
00:10:00,990 --> 00:10:04,170
you're losing your own freedom
of thinking and expression.

237
00:10:04,170 --> 00:10:05,340
- Completely agree.

238
00:10:05,340 --> 00:10:08,520
Can everybody hear in either sides? Okay.

239
00:10:08,520 --> 00:10:09,453
Repeat it? Okay.

240
00:10:10,320 --> 00:10:13,260
So what she had said was
that writing is thinking

241
00:10:13,260 --> 00:10:16,260
and taught, and so if you
remove the process of learning

242
00:10:16,260 --> 00:10:18,570
to write, then that's
a detriment to students

243
00:10:18,570 --> 00:10:21,363
who are also using writing
as a way of thinking.

244
00:10:22,590 --> 00:10:23,423
What else?

245
00:10:26,310 --> 00:10:27,600
I heard some people talk about,

246
00:10:27,600 --> 00:10:30,030
oh, you need to be able to
write something to the AI

247
00:10:30,030 --> 00:10:32,080
to prompt it to do what you wanted to do.

248
00:10:33,060 --> 00:10:35,280
That's true, yes.

249
00:10:35,280 --> 00:10:37,560
- [Attendee] Yeah. I mean, we also talked

250
00:10:37,560 --> 00:10:39,360
about good uses for AI.

251
00:10:39,360 --> 00:10:41,310
So for instance, if I have kids writing

252
00:10:41,310 --> 00:10:43,290
a creative writing assignment, and like,

253
00:10:43,290 --> 00:10:45,330
you know, some kids may
not consider themselves

254
00:10:45,330 --> 00:10:48,570
particularly creative, Hey,
have AI generate a prompt

255
00:10:48,570 --> 00:10:51,030
for you rather than
writing the entire thing.

256
00:10:51,030 --> 00:10:52,020
Now at the end of the day,

257
00:10:52,020 --> 00:10:54,390
we're still there to
help support you through

258
00:10:54,390 --> 00:10:55,680
the writing process.

259
00:10:55,680 --> 00:10:57,540
Just because you think
you're not good at writing

260
00:10:57,540 --> 00:11:00,690
doesn't mean you can't not be good at it.

261
00:11:00,690 --> 00:11:02,310
You can always be good at something,

262
00:11:02,310 --> 00:11:04,800
just how you put your mindset towards it.

263
00:11:04,800 --> 00:11:07,440
- Agreed. So that was teaching students

264
00:11:07,440 --> 00:11:11,040
about kind of more ethical,
responsible ways of using AI,

265
00:11:11,040 --> 00:11:12,270
for example, with creative writing,

266
00:11:12,270 --> 00:11:13,740
asking it to give you a prompt

267
00:11:13,740 --> 00:11:15,900
to help spark your creativity.

268
00:11:15,900 --> 00:11:19,560
So in these past couple
of AI-crazed years,

269
00:11:19,560 --> 00:11:22,800
I often return to this
quote from Joan Didion

270
00:11:22,800 --> 00:11:25,770
in which he says, "I
write entirely to find out

271
00:11:25,770 --> 00:11:28,350
what I am thinking, what I am looking at,

272
00:11:28,350 --> 00:11:30,660
what I see and what it means."

273
00:11:30,660 --> 00:11:33,240
And so to go back to
what you said over here,

274
00:11:33,240 --> 00:11:36,720
the process of writing
is a process of learning

275
00:11:36,720 --> 00:11:39,210
and thinking critical in and of itself.

276
00:11:39,210 --> 00:11:42,870
There's also, on top of that
just a robust body of evidence

277
00:11:42,870 --> 00:11:44,910
that shows that real cognitive

278
00:11:44,910 --> 00:11:46,500
and psychological benefits come from

279
00:11:46,500 --> 00:11:47,940
the process of writing too.

280
00:11:47,940 --> 00:11:50,850
Students use writing as a way to process

281
00:11:50,850 --> 00:11:53,280
what they think about things,
what they feel about things

282
00:11:53,280 --> 00:11:56,700
to work through, things to
develop, how they even believe,

283
00:11:56,700 --> 00:11:58,440
like what their argument is just sitting.

284
00:11:58,440 --> 00:12:01,170
If you've ever sat down and
written something, oftentimes

285
00:12:01,170 --> 00:12:03,600
you come out on the other end
thinking something different

286
00:12:03,600 --> 00:12:06,000
just from the process of
having to put your thoughts

287
00:12:06,000 --> 00:12:08,490
on paper and organize them.

288
00:12:08,490 --> 00:12:12,210
So no, I do not personally
see large language models

289
00:12:12,210 --> 00:12:13,890
as a reason not to teach writing anymore.

290
00:12:13,890 --> 00:12:16,080
Did anybody agree with
the student, by the way,

291
00:12:16,080 --> 00:12:19,500
out of curiosity, we don't
need to teach writing anymore?

292
00:12:19,500 --> 00:12:20,443
Wonderful. Okay, great.

293
00:12:20,443 --> 00:12:22,053
I'm in the right room.

294
00:12:23,160 --> 00:12:26,220
So if anything, I think
it's actually more important

295
00:12:26,220 --> 00:12:28,690
maybe now than it ever has been

296
00:12:29,550 --> 00:12:31,800
because it's like thinking as a human

297
00:12:31,800 --> 00:12:35,850
it's a such a deeply
human thing and writing

298
00:12:35,850 --> 00:12:37,773
is such a big part of that.

299
00:12:38,820 --> 00:12:40,830
But it is critical, I
think, to your point,

300
00:12:40,830 --> 00:12:44,430
to educate students about AI
and large language models,

301
00:12:44,430 --> 00:12:46,080
how they work, how to use them ethically

302
00:12:46,080 --> 00:12:47,940
and responsibly in a classroom.

303
00:12:47,940 --> 00:12:50,580
And we could do an entire
symposium on this topic alone,

304
00:12:50,580 --> 00:12:52,890
but I did include some thoughts here

305
00:12:52,890 --> 00:12:55,740
to consider when you're
incorporating AI in the classroom.

306
00:12:55,740 --> 00:12:57,870
So one of the first things I think about

307
00:12:57,870 --> 00:13:00,090
and as you're thinking about these things,

308
00:13:00,090 --> 00:13:01,620
consider other AI tools too.

309
00:13:01,620 --> 00:13:04,350
How many of those tools
are these things true for?

310
00:13:04,350 --> 00:13:07,320
So will the AI actually
help students learn

311
00:13:07,320 --> 00:13:09,330
Or is it more of a shortcut?

312
00:13:09,330 --> 00:13:12,180
Will students become reliant on the AI

313
00:13:12,180 --> 00:13:14,130
if it's being used as scaffolding,

314
00:13:14,130 --> 00:13:15,960
is there a way that we
can gradually remove

315
00:13:15,960 --> 00:13:17,460
that scaffolding?

316
00:13:17,460 --> 00:13:18,990
Do students actually understand

317
00:13:18,990 --> 00:13:20,550
how large language models work

318
00:13:20,550 --> 00:13:24,930
and how that content is
generated and when not to trust

319
00:13:24,930 --> 00:13:27,270
the content that is being generated?

320
00:13:27,270 --> 00:13:30,150
And are they actually using AI in ways

321
00:13:30,150 --> 00:13:32,610
that encourage high order thinking skills

322
00:13:32,610 --> 00:13:36,870
versus kind of put that on the AI itself?

323
00:13:36,870 --> 00:13:38,040
So these are the questions

324
00:13:38,040 --> 00:13:41,940
that I think are important when
we are trying to figure out

325
00:13:41,940 --> 00:13:44,190
should we be using this
AI in the classroom.

326
00:13:44,190 --> 00:13:47,760
So with all of that, let's
talk about Writing Coach.

327
00:13:47,760 --> 00:13:49,620
How writing Coach came to be,

328
00:13:49,620 --> 00:13:51,510
why it is different from other tools.

329
00:13:51,510 --> 00:13:54,360
We went over a little bit of this before,

330
00:13:54,360 --> 00:13:56,280
but some people think Khan Academy

331
00:13:56,280 --> 00:14:01,020
and they think test prep or
math and science content, right?

332
00:14:01,020 --> 00:14:03,990
It's rarely kind of
seen as the ELA resource

333
00:14:03,990 --> 00:14:06,600
or the tool for teaching writing.

334
00:14:06,600 --> 00:14:08,760
But our mission at Khan
Academy has always been

335
00:14:08,760 --> 00:14:12,510
about providing a world class
education to anyone anywhere.

336
00:14:12,510 --> 00:14:15,000
So that has always
included the humanities.

337
00:14:15,000 --> 00:14:17,280
I think we've always just
been trying to figure out

338
00:14:17,280 --> 00:14:18,780
how do we break into the humanities?

339
00:14:18,780 --> 00:14:21,870
How do we support teachers
with that particularly?

340
00:14:21,870 --> 00:14:24,450
So when we started exploring AI solutions

341
00:14:24,450 --> 00:14:28,050
back in 2022, one of the
things that immediately came

342
00:14:28,050 --> 00:14:32,010
to mind was, this is really
good with writing-related tasks,

343
00:14:32,010 --> 00:14:34,530
for obvious reasons, it's
a large language model.

344
00:14:34,530 --> 00:14:37,440
And so we were exploring
what are some of the problems

345
00:14:37,440 --> 00:14:40,200
that we could solve with
this new technology?

346
00:14:40,200 --> 00:14:42,540
And as a former writing teacher,

347
00:14:42,540 --> 00:14:47,070
I knew that the problems
with teaching writing

348
00:14:47,070 --> 00:14:49,920
were some of the hardest to
solve problems out there.

349
00:14:49,920 --> 00:14:53,190
And this is because have to
meet grade level standards.

350
00:14:53,190 --> 00:14:55,740
They're supposed to be
practicing 60 minutes a day,

351
00:14:55,740 --> 00:14:58,860
apparently, many students need support

352
00:14:58,860 --> 00:15:00,570
through the entire writing process,

353
00:15:00,570 --> 00:15:03,270
not just getting that feedback
piece and then delivering

354
00:15:03,270 --> 00:15:05,610
that feedback in a way that's timely

355
00:15:05,610 --> 00:15:09,510
and specific and actionable is a big job.

356
00:15:09,510 --> 00:15:13,620
So in reality, what we often
see in writing classrooms

357
00:15:13,620 --> 00:15:16,770
is that about a quarter of
students actually end up

358
00:15:16,770 --> 00:15:18,540
meeting grade level writing standards

359
00:15:18,540 --> 00:15:21,930
in middle and high
school, a tiny percentage

360
00:15:21,930 --> 00:15:24,990
of students are actually
getting 60 minutes

361
00:15:24,990 --> 00:15:27,030
of writing practice per day.

362
00:15:27,030 --> 00:15:29,820
And because ELA secondary teachers,

363
00:15:29,820 --> 00:15:32,220
middle and high school
teachers tend to have

364
00:15:32,220 --> 00:15:34,440
a hundred or more students often

365
00:15:34,440 --> 00:15:38,040
that means that for a teacher
to deliver timely, specific,

366
00:15:38,040 --> 00:15:40,200
actionable feedback to
all of their students,

367
00:15:40,200 --> 00:15:42,780
if I have a hundred
students and I limit myself

368
00:15:42,780 --> 00:15:45,630
to 10 minutes of feedback per essay,

369
00:15:45,630 --> 00:15:47,760
which I would literally do,
I would set my phone timer

370
00:15:47,760 --> 00:15:50,370
and be like, I'm stopping
after 10 minutes for each one.

371
00:15:50,370 --> 00:15:55,200
That's 17 hours for one
draft of one essay assignment

372
00:15:55,200 --> 00:15:57,150
to get feedback to all of my students.

373
00:15:57,150 --> 00:15:59,430
And we're supposed to
be assigning 16 minutes

374
00:15:59,430 --> 00:16:00,974
of writing per day.

375
00:16:00,974 --> 00:16:04,623
It seemed at the time to be
kind of an intractable problem,

376
00:16:05,520 --> 00:16:10,083
but we decided to try
and tackle it with AI.

377
00:16:10,920 --> 00:16:15,600
And last fall, in September,
we launched the what ultimately

378
00:16:15,600 --> 00:16:17,970
is now the Khanmigo
Writing Coach experience.

379
00:16:17,970 --> 00:16:20,850
It started off as a
prototype that we built.

380
00:16:20,850 --> 00:16:22,560
We did a feedback activity,

381
00:16:22,560 --> 00:16:24,600
we did a college admissions essay activity

382
00:16:24,600 --> 00:16:27,270
that's still available on Khanmigo too.

383
00:16:27,270 --> 00:16:28,740
And then this past fall
is when we launched

384
00:16:28,740 --> 00:16:30,420
the full Writing Coach experience.

385
00:16:30,420 --> 00:16:32,400
So I wanted to go over,

386
00:16:32,400 --> 00:16:35,250
because again, there's a
many, many AI tools out there,

387
00:16:35,250 --> 00:16:36,540
as the commissioner mentioned,

388
00:16:36,540 --> 00:16:38,010
he's getting a lot of phone calls

389
00:16:38,010 --> 00:16:39,450
about all these different tools

390
00:16:39,450 --> 00:16:41,760
and I think you're definitely
familiar with ChatGPT,

391
00:16:41,760 --> 00:16:45,360
it has its usage, but it's
also always more than happy

392
00:16:45,360 --> 00:16:47,523
to write a student's
entire essay for them.

393
00:16:48,480 --> 00:16:52,320
And so there's positive use
cases for all of these tools,

394
00:16:52,320 --> 00:16:57,320
but the primary purpose behind
them it's either to save time

395
00:16:58,170 --> 00:16:59,580
for the student or the teacher,

396
00:16:59,580 --> 00:17:02,700
or it's to produce better writing.

397
00:17:02,700 --> 00:17:04,770
And with Writing Coach, the purpose is

398
00:17:04,770 --> 00:17:07,170
to produce better writers.

399
00:17:07,170 --> 00:17:09,720
And so here's how Writing
Coach is different.

400
00:17:09,720 --> 00:17:11,760
Writing Coach is an instructional tool

401
00:17:11,760 --> 00:17:14,550
with the primary purpose
of student learning

402
00:17:14,550 --> 00:17:16,500
and teacher transparency.

403
00:17:16,500 --> 00:17:18,060
It's to designed to guide students

404
00:17:18,060 --> 00:17:19,890
through the entire process of writing

405
00:17:19,890 --> 00:17:22,290
from the very beginning of
getting that essay prompt

406
00:17:22,290 --> 00:17:24,720
through outlining,
drafting, getting feedback,

407
00:17:24,720 --> 00:17:27,030
revising, submitting to the teacher.

408
00:17:27,030 --> 00:17:30,900
It's also a way for teachers
to get more writing practice

409
00:17:30,900 --> 00:17:34,530
in the classroom and real-time
support for students.

410
00:17:34,530 --> 00:17:37,050
And then it also includes data dashboards

411
00:17:37,050 --> 00:17:40,050
for teachers with at-a-glance,
high-level insights

412
00:17:40,050 --> 00:17:42,750
at the class level and
then in-depth insights

413
00:17:42,750 --> 00:17:44,880
into the actual student writing processes.

414
00:17:44,880 --> 00:17:47,790
So it is not a digital
essay writing platform,

415
00:17:47,790 --> 00:17:51,150
it's not Google Docs, it's
not an evaluative tool

416
00:17:51,150 --> 00:17:52,140
or a grading tool,

417
00:17:52,140 --> 00:17:54,753
we're not gonna just spit
out scores for every kid.

418
00:17:55,590 --> 00:17:57,630
It isn't a student
student productivity tool

419
00:17:57,630 --> 00:18:00,180
like Grammarly where they can
just accept the suggestions.

420
00:18:00,180 --> 00:18:03,310
It doesn't just generate
feedback, although it does do that

421
00:18:04,230 --> 00:18:07,410
and it doesn't focus on
grammar and mechanics

422
00:18:07,410 --> 00:18:11,730
and kind of let students just
accept those suggestions.

423
00:18:11,730 --> 00:18:13,080
So we launched Writing Coach

424
00:18:13,080 --> 00:18:15,300
and some of the feedback
that we've gotten so far

425
00:18:15,300 --> 00:18:18,990
has been related to the
quality of the feedback.

426
00:18:18,990 --> 00:18:20,160
I think this was one of the things

427
00:18:20,160 --> 00:18:22,980
that initially really set it apart

428
00:18:22,980 --> 00:18:24,330
from some of those other tools.

429
00:18:24,330 --> 00:18:26,370
When we developed Writing Coach,

430
00:18:26,370 --> 00:18:28,590
we were a little bit obsessive

431
00:18:28,590 --> 00:18:30,120
about the quality of the feedback.

432
00:18:30,120 --> 00:18:31,680
I think because it was built by a team

433
00:18:31,680 --> 00:18:33,690
of former writing teachers and tutors,

434
00:18:33,690 --> 00:18:37,380
were not gonna settle for
anything that wasn't better than,

435
00:18:37,380 --> 00:18:40,830
you know, what a student teacher
would be able to do, right?

436
00:18:40,830 --> 00:18:45,210
And so we ended up creating
multiple AI prompts

437
00:18:45,210 --> 00:18:47,790
to focus on various parts
of the writing process

438
00:18:47,790 --> 00:18:50,310
as well as various feedback prompts

439
00:18:50,310 --> 00:18:52,860
for different feedback categories.

440
00:18:52,860 --> 00:18:55,650
And we meticulously
tested and refined these

441
00:18:55,650 --> 00:18:57,480
over the course of over a year.

442
00:18:57,480 --> 00:19:00,090
And we're still doing this to make sure

443
00:19:00,090 --> 00:19:01,950
that the feedback is targeted,

444
00:19:01,950 --> 00:19:05,310
it's aligned to what grade
level expectations are

445
00:19:05,310 --> 00:19:07,770
and it's actually actionable and not just,

446
00:19:07,770 --> 00:19:09,900
oh, you did good on this
and maybe you should work

447
00:19:09,900 --> 00:19:11,430
on your organization.

448
00:19:11,430 --> 00:19:13,560
It helps the student through the process

449
00:19:13,560 --> 00:19:15,330
of understanding what's wrong.

450
00:19:15,330 --> 00:19:16,800
So that was one piece of feedback we got

451
00:19:16,800 --> 00:19:21,060
is that the feedback from
Khanmigo is really excellent.

452
00:19:21,060 --> 00:19:24,600
And the other piece is that it has helped

453
00:19:24,600 --> 00:19:26,370
actually improve student writing.

454
00:19:26,370 --> 00:19:28,020
So students who are using Writing Coach,

455
00:19:28,020 --> 00:19:30,240
we had one teacher in Florida, I think,

456
00:19:30,240 --> 00:19:34,020
who used it very often last
fall and every single one,

457
00:19:34,020 --> 00:19:36,780
or she was part of a pilot that
we did actually last spring

458
00:19:36,780 --> 00:19:38,700
and every single one of
her students in her class

459
00:19:38,700 --> 00:19:40,023
passed the state test.

460
00:19:41,010 --> 00:19:44,160
And we think that this is because
it's an instructional tool

461
00:19:44,160 --> 00:19:46,280
and it puts the onus on the
student to have to do all

462
00:19:46,280 --> 00:19:49,290
of the thinking themselves,
all of the work themselves,

463
00:19:49,290 --> 00:19:51,990
we think that that is where
it holds its most promise.

464
00:19:52,920 --> 00:19:57,050
So the first thing to do when
working with Writing Coach

465
00:19:57,050 --> 00:19:59,490
is to create the assignment

466
00:19:59,490 --> 00:20:01,470
and to create the
Writing Coach assignment,

467
00:20:01,470 --> 00:20:03,630
you would start by
logging into Khan Academy

468
00:20:03,630 --> 00:20:05,370
and clicking on the Khanmigo dropdown.

469
00:20:05,370 --> 00:20:07,770
Can you raise your hand
if you are familiar

470
00:20:07,770 --> 00:20:12,770
with the Khan Academy website
and that Khanmigo dropdown?

471
00:20:13,020 --> 00:20:15,150
Okay, if not, once you log in,

472
00:20:15,150 --> 00:20:17,640
you should all have this dropdown.

473
00:20:17,640 --> 00:20:18,960
So a Writing Coach assignment,

474
00:20:18,960 --> 00:20:20,250
when you click that assign button,

475
00:20:20,250 --> 00:20:23,550
let me go back to just show you
where that assign button is.

476
00:20:23,550 --> 00:20:25,590
Once you click on Writing Coach,

477
00:20:25,590 --> 00:20:27,390
you should see an assign button

478
00:20:27,390 --> 00:20:29,760
that's blue in the top right corner.

479
00:20:29,760 --> 00:20:33,633
And every Writing Coach
assignment starts with a prompt.

480
00:20:35,160 --> 00:20:37,500
You provide the prompt, the
teacher provides the prompt

481
00:20:37,500 --> 00:20:40,890
to the student and you can use

482
00:20:40,890 --> 00:20:43,560
any academic essay prompt that you want.

483
00:20:43,560 --> 00:20:47,370
A few things to note about how
Writing Coach was designed,

484
00:20:47,370 --> 00:20:51,150
it was designed for secondary
humanities classrooms.

485
00:20:51,150 --> 00:20:55,293
So we typically say grade
seven and up, 7 through 12.

486
00:20:56,430 --> 00:21:00,810
However, if you can put
fifth, sixth graders on it,

487
00:21:00,810 --> 00:21:03,000
it might just be a little
bit advanced for them.

488
00:21:03,000 --> 00:21:06,060
It struggles a little bit to
kind of bring that level down.

489
00:21:06,060 --> 00:21:08,610
And it's trained on, we've tweaked it

490
00:21:08,610 --> 00:21:11,820
and made it work for middle
and high school essays

491
00:21:11,820 --> 00:21:14,970
starting around grade seven.

492
00:21:14,970 --> 00:21:17,430
So those are the grade
levels we officially support.

493
00:21:17,430 --> 00:21:20,490
But again, you can use it
for fifth and sixth grade.

494
00:21:20,490 --> 00:21:24,300
The three essay types that we
officially support right now,

495
00:21:24,300 --> 00:21:27,630
we're gonna be adding more,
are persuasive, argumentative,

496
00:21:27,630 --> 00:21:32,520
expository, explanatory
and literary analysis.

497
00:21:32,520 --> 00:21:34,260
And so what I recommend,

498
00:21:34,260 --> 00:21:36,450
if you are planning on using Writing Coach

499
00:21:36,450 --> 00:21:40,290
for general writing practice,
those are very good ways

500
00:21:40,290 --> 00:21:44,220
to kind of select the kind of writing

501
00:21:44,220 --> 00:21:46,080
that you want students to do.

502
00:21:46,080 --> 00:21:49,410
But if you're using it for
something like SAT essay prep

503
00:21:49,410 --> 00:21:52,140
or the middle school essay prep,

504
00:21:52,140 --> 00:21:54,870
choose the closest one
to what the prompt is.

505
00:21:54,870 --> 00:21:57,510
So in some cases that's
gonna be literary analysis.

506
00:21:57,510 --> 00:22:00,510
Usually it's argumentative, persuasive.

507
00:22:00,510 --> 00:22:01,890
So just give your best guess

508
00:22:01,890 --> 00:22:04,710
as to the essay type that it is.

509
00:22:04,710 --> 00:22:08,520
And what is really important
about the essay instructions

510
00:22:08,520 --> 00:22:11,430
is that you are as specific as possible.

511
00:22:11,430 --> 00:22:13,470
In every stage of Writing Coach,

512
00:22:13,470 --> 00:22:15,930
Khanmigo is going to
look at the instructions

513
00:22:15,930 --> 00:22:19,200
that you provided and it's
going to be a stickler

514
00:22:19,200 --> 00:22:21,300
for the student about those instructions.

515
00:22:21,300 --> 00:22:25,110
So if you say write an
essay about whatever,

516
00:22:25,110 --> 00:22:28,350
it's not gonna push the student
to include textual evidence

517
00:22:28,350 --> 00:22:30,390
or a certain amount of textual evidence

518
00:22:30,390 --> 00:22:32,730
or even maybe the student
will give examples,

519
00:22:32,730 --> 00:22:34,680
but not actually textual evidence.

520
00:22:34,680 --> 00:22:37,920
So if you want students to
include at least three pieces

521
00:22:37,920 --> 00:22:40,020
of textual evidence, say that.

522
00:22:40,020 --> 00:22:42,690
So be really explicit about
what you want them to do.

523
00:22:42,690 --> 00:22:45,060
If you want them to include
a concession rebuttal

524
00:22:45,060 --> 00:22:47,580
or counterargument paragraph, say so.

525
00:22:47,580 --> 00:22:52,000
If you want students
to cite their evidence

526
00:22:52,983 --> 00:22:56,280
in a specific format, say which one.

527
00:22:56,280 --> 00:22:59,550
You can even tell it exactly what rubric

528
00:22:59,550 --> 00:23:00,383
that you're going to use.

529
00:23:00,383 --> 00:23:03,060
We don't have a feature yet
where you can upload the rubric,

530
00:23:03,060 --> 00:23:04,860
but all you have to do is say,

531
00:23:04,860 --> 00:23:06,300
what I would actually recommend

532
00:23:06,300 --> 00:23:10,470
is this is from the New
Hampshire SAS rubric

533
00:23:10,470 --> 00:23:13,980
from middle school, you can literally take

534
00:23:13,980 --> 00:23:17,940
the entire section from
all of the score fours

535
00:23:17,940 --> 00:23:19,650
and put that into the essay instructions.

536
00:23:19,650 --> 00:23:22,650
Say this essay should
meet these expectations

537
00:23:22,650 --> 00:23:25,530
and Khanmigo will check
the student's essay

538
00:23:25,530 --> 00:23:28,350
to make sure it's meeting
those expectations.

539
00:23:28,350 --> 00:23:33,350
For SAT, similarly, you
can take the same text here

540
00:23:34,890 --> 00:23:37,530
and plot that into the essay instructions

541
00:23:37,530 --> 00:23:40,260
and it's going to pay
attention to what you want.

542
00:23:40,260 --> 00:23:43,740
You can also just say this
is an SAT practice test

543
00:23:43,740 --> 00:23:45,930
for the essay.

544
00:23:45,930 --> 00:23:48,210
And because it's in large
language model, it's gonna know

545
00:23:48,210 --> 00:23:49,920
what that means and it's
gonna know generally

546
00:23:49,920 --> 00:23:51,360
what the expectations are for that.

547
00:23:51,360 --> 00:23:52,320
So if you're in a hurry,

548
00:23:52,320 --> 00:23:55,620
you can just say this is
an SAT essay practice test

549
00:23:55,620 --> 00:23:58,650
and it will kind of have
those expectations in mind.

550
00:23:58,650 --> 00:24:03,650
Another word about
these SAS and SAT tests,

551
00:24:05,790 --> 00:24:07,950
some of them, or I think
most of them actually require

552
00:24:07,950 --> 00:24:10,230
that students read a reading passage.

553
00:24:10,230 --> 00:24:13,470
For SAT, because they're
typically pretty short,

554
00:24:13,470 --> 00:24:15,840
you can actually paste the reading passage

555
00:24:15,840 --> 00:24:18,930
like the entire prompt directly
into essay instructions

556
00:24:18,930 --> 00:24:20,043
into that field there.

557
00:24:22,020 --> 00:24:23,370
If it's a well-known text,

558
00:24:23,370 --> 00:24:24,990
you don't even really have to do that.

559
00:24:24,990 --> 00:24:26,910
It just sort of will be aware of the text

560
00:24:26,910 --> 00:24:28,680
that is being referenced.

561
00:24:28,680 --> 00:24:30,210
If it is longer,

562
00:24:30,210 --> 00:24:32,700
you also don't really
need to include it at all.

563
00:24:32,700 --> 00:24:34,477
I looked at one in this case it says,

564
00:24:34,477 --> 00:24:36,180
"Explain how Paul Bogart
builds an argument

565
00:24:36,180 --> 00:24:37,470
to persuade his audience."

566
00:24:37,470 --> 00:24:40,563
You don't need to put in the
actual reading passage here.

567
00:24:41,400 --> 00:24:45,360
You can just let students
do the reading part outside

568
00:24:45,360 --> 00:24:47,850
of Writing Coach and then
Writing Coach will push them

569
00:24:47,850 --> 00:24:50,700
to include evidence to make
sure the evidence is aligned.

570
00:24:50,700 --> 00:24:53,190
But if it's a not super well-known passage

571
00:24:53,190 --> 00:24:54,210
or it's too long to include

572
00:24:54,210 --> 00:24:55,920
in the essay instructions itself,

573
00:24:55,920 --> 00:24:58,110
Khanmigo won't necessarily be able to say,

574
00:24:58,110 --> 00:25:00,360
oh, that evidence is not
actually from that text

575
00:25:00,360 --> 00:25:03,000
or it might not be able to, you know,

576
00:25:03,000 --> 00:25:05,280
to verify that it is a
real piece of evidence

577
00:25:05,280 --> 00:25:06,480
if it's not from a well known text

578
00:25:06,480 --> 00:25:07,500
or if you don't provide the text.

579
00:25:07,500 --> 00:25:09,030
Does that make sense? Yes.

580
00:25:09,030 --> 00:25:10,290
- [Attendee] Sorry, you
said is there a limit

581
00:25:10,290 --> 00:25:12,900
to how much text you
can put into the prompt?

582
00:25:12,900 --> 00:25:14,880
- I believe there is not.

583
00:25:14,880 --> 00:25:16,170
I just checked to see like-
- If they're doing

584
00:25:16,170 --> 00:25:20,070
a rhetorical analysis of a
of a speech, for example,

585
00:25:20,070 --> 00:25:21,900
and let's say the speech
is like three pages,

586
00:25:21,900 --> 00:25:24,240
would it be able to fit
that whole speech in?

587
00:25:24,240 --> 00:25:26,370
- You could. I'm pretty sure you could.

588
00:25:26,370 --> 00:25:29,640
The one caveat is sometimes
large language models

589
00:25:29,640 --> 00:25:31,620
have this thing called a context window.

590
00:25:31,620 --> 00:25:33,180
And if it's too long,

591
00:25:33,180 --> 00:25:34,800
if you put way too much
information in there,

592
00:25:34,800 --> 00:25:37,140
it might get confused and
it might not work as well.

593
00:25:37,140 --> 00:25:39,000
So if it's a speech that's well-known,

594
00:25:39,000 --> 00:25:40,800
you can just reference
the name of the speech

595
00:25:40,800 --> 00:25:42,390
and that's perfectly fine.

596
00:25:42,390 --> 00:25:45,360
If it's like a random article from some,

597
00:25:45,360 --> 00:25:47,040
I dunno, random publication

598
00:25:47,040 --> 00:25:50,280
that is maybe not super
Googleable and it's short,

599
00:25:50,280 --> 00:25:51,210
you might wanna include it.

600
00:25:51,210 --> 00:25:52,827
But otherwise, I probably
wouldn't include it.

601
00:25:52,827 --> 00:25:54,177
- Okay, thank you.
- Yeah.

602
00:25:55,590 --> 00:25:57,390
Another point about the reading passages.

603
00:25:57,390 --> 00:25:59,640
So if you are gonna be
using this for test prep,

604
00:25:59,640 --> 00:26:03,360
I would have students start by
reading the passages on paper

605
00:26:03,360 --> 00:26:06,270
or wherever you want them
to read them in class.

606
00:26:06,270 --> 00:26:08,460
We do not have anything in Writing Coach

607
00:26:08,460 --> 00:26:10,380
that will do the reading with students.

608
00:26:10,380 --> 00:26:13,440
We're thinking about it, we're
exploring a reading coach,

609
00:26:13,440 --> 00:26:16,110
but for now, have them
do the reading passages

610
00:26:16,110 --> 00:26:18,303
before they start the essay, okay?

611
00:26:19,500 --> 00:26:24,330
Once they get in to their accounts,

612
00:26:24,330 --> 00:26:26,400
students will again
log into their accounts

613
00:26:26,400 --> 00:26:28,490
whether they get there
through Clever or ClassLink,

614
00:26:28,490 --> 00:26:31,680
or whatever it is that you guys
use, log into their account,

615
00:26:31,680 --> 00:26:33,240
they will see their assignment right

616
00:26:33,240 --> 00:26:34,920
on their assignments page,

617
00:26:34,920 --> 00:26:37,620
but they can also do the
same thing that you did.

618
00:26:37,620 --> 00:26:39,630
Go to their Khanmigo learner activities,

619
00:26:39,630 --> 00:26:42,300
go to Writing Coach and
they'll see it there too.

620
00:26:42,300 --> 00:26:44,070
But it's listed under their assignment

621
00:26:44,070 --> 00:26:45,470
so they should see it there.

622
00:26:46,740 --> 00:26:48,780
What Writing Coach does for students,

623
00:26:48,780 --> 00:26:51,540
and I'll show you what this
looks like in a minute,

624
00:26:51,540 --> 00:26:53,760
is it starts by breaking down the prompt

625
00:26:53,760 --> 00:26:55,800
with understanding the assignment.

626
00:26:55,800 --> 00:26:57,930
It helps them formulate the outline,

627
00:26:57,930 --> 00:27:00,180
it helps them get started on a draft

628
00:27:00,180 --> 00:27:02,543
and then it helps 'em with
feedback and revising.

629
00:27:05,790 --> 00:27:07,440
So for understanding the assignment,

630
00:27:07,440 --> 00:27:11,640
if you look up here, I know
there's multiple screens.

631
00:27:11,640 --> 00:27:15,150
We have understandings outlining,
drafting, and revising.

632
00:27:15,150 --> 00:27:17,940
So if your teachers are
doing writing class,

633
00:27:17,940 --> 00:27:19,620
writing coaching class,
you can walk around,

634
00:27:19,620 --> 00:27:21,690
you can see what stage they're in.

635
00:27:21,690 --> 00:27:24,030
But there's also a dashboard for teachers

636
00:27:24,030 --> 00:27:25,350
that I'll show you shortly.

637
00:27:25,350 --> 00:27:26,820
That also tells you exactly

638
00:27:26,820 --> 00:27:28,487
what stage students each
student is working on.

639
00:27:28,487 --> 00:27:30,300
So you can get a sense of how far

640
00:27:30,300 --> 00:27:32,656
in the assignment students are.

641
00:27:32,656 --> 00:27:35,160
In this first stage,
understanding Khanmigo

642
00:27:35,160 --> 00:27:37,170
is looking at the prompt that you gave,

643
00:27:37,170 --> 00:27:39,240
it's looking at the
information you provided

644
00:27:39,240 --> 00:27:40,530
and it's engaging the student

645
00:27:40,530 --> 00:27:43,620
in a conversation about the assignment.

646
00:27:43,620 --> 00:27:45,630
It gives a couple of initial prompts.

647
00:27:45,630 --> 00:27:47,730
Do you want me to explain the essay type?

648
00:27:47,730 --> 00:27:49,375
Do you wanna know what
the requirements are?

649
00:27:49,375 --> 00:27:50,640
Do you want me to summarize
them? Do you have questions?

650
00:27:50,640 --> 00:27:52,470
So students can use this as an opportunity

651
00:27:52,470 --> 00:27:54,840
to ask about, you know, defining terms.

652
00:27:54,840 --> 00:27:57,000
What is the rebuttal again,
like can you remind me

653
00:27:57,000 --> 00:28:00,570
how I cite in text and citations for MLA?

654
00:28:00,570 --> 00:28:04,380
Or can you kind of tell me
what I should do now, right?

655
00:28:04,380 --> 00:28:06,270
And sometimes Khanmigo ask the student,

656
00:28:06,270 --> 00:28:07,440
have you done the readings yet?

657
00:28:07,440 --> 00:28:08,940
If not, go to the readings.

658
00:28:08,940 --> 00:28:10,500
Here's what you should be looking for.

659
00:28:10,500 --> 00:28:11,850
And it's an opportunity for the student

660
00:28:11,850 --> 00:28:14,940
to kind of ask any questions
they need, feel safe,

661
00:28:14,940 --> 00:28:18,210
feel like they can, you
know, share their confusion

662
00:28:18,210 --> 00:28:21,780
with the AI and get anything answered

663
00:28:21,780 --> 00:28:24,900
before they begin the
actual writing process.

664
00:28:24,900 --> 00:28:27,480
So the second step is outlining.

665
00:28:27,480 --> 00:28:32,480
We provide students with a
default flexible outline.

666
00:28:32,820 --> 00:28:35,700
Students can modify this,
they can add body paragraphs,

667
00:28:35,700 --> 00:28:38,970
remove body paragraphs, add
evidence, add explanations,

668
00:28:38,970 --> 00:28:41,370
add reasoning, add sources,
whatever they need,

669
00:28:41,370 --> 00:28:44,433
but we do have a default
three paragraph essay outline.

670
00:28:45,780 --> 00:28:49,770
It also comes with rough
kind of exemplar sentences

671
00:28:49,770 --> 00:28:51,330
for each section.

672
00:28:51,330 --> 00:28:52,920
So students can kind of see an example

673
00:28:52,920 --> 00:28:54,960
of what an in-text citation looks like

674
00:28:54,960 --> 00:28:59,040
or a thesis statement in that's
related to the type of essay

675
00:28:59,040 --> 00:29:00,003
that you specified.

676
00:29:01,590 --> 00:29:04,860
During the stage, the student
again, can chat with Khanmigo,

677
00:29:04,860 --> 00:29:06,660
the student can start by
just saying, I'm stuck,

678
00:29:06,660 --> 00:29:09,390
or you know, they can just dive right in.

679
00:29:09,390 --> 00:29:11,490
They don't have to engage
with Khanmigo at all

680
00:29:11,490 --> 00:29:12,690
if they don't want to.

681
00:29:12,690 --> 00:29:14,640
And one of the nice
things about Writing Coach

682
00:29:14,640 --> 00:29:19,350
is even if they don't
chat with Khanmigo at all,

683
00:29:19,350 --> 00:29:21,180
Khanmigo will still check their work.

684
00:29:21,180 --> 00:29:24,750
So once they move from
outlining to drafting,

685
00:29:24,750 --> 00:29:27,000
without even being asked,
Khanmigo's gonna be like,

686
00:29:27,000 --> 00:29:30,930
I'm just gonna make sure
that your outline is matching

687
00:29:30,930 --> 00:29:31,920
what the instructions are.

688
00:29:31,920 --> 00:29:33,600
Because again, Khanmigo can see

689
00:29:33,600 --> 00:29:37,080
the teacher's assignment
instructions at every stage.

690
00:29:37,080 --> 00:29:39,843
So once the student tries
to move on to drafting,

691
00:29:40,710 --> 00:29:43,410
Khanmigo performs an initial check

692
00:29:43,410 --> 00:29:45,510
to make sure that the thesis

693
00:29:45,510 --> 00:29:47,970
is actually answered the question,

694
00:29:47,970 --> 00:29:50,880
answering the question that is being asked

695
00:29:50,880 --> 00:29:52,710
in the assignment instructions.

696
00:29:52,710 --> 00:29:55,290
It's checking to make sure the main points

697
00:29:55,290 --> 00:29:57,030
have something to do with the thesis.

698
00:29:57,030 --> 00:29:58,350
This is not the part where it's gonna be

699
00:29:58,350 --> 00:29:59,460
like giving a ton of feedback,

700
00:29:59,460 --> 00:30:01,620
it's just doing basic
alignment checks here

701
00:30:01,620 --> 00:30:03,000
and completion checks.

702
00:30:03,000 --> 00:30:04,530
It's checking to make sure the evidence

703
00:30:04,530 --> 00:30:07,020
is related to the points they're making,

704
00:30:07,020 --> 00:30:09,480
that the evidence meets
the assignment expectation.

705
00:30:09,480 --> 00:30:12,270
So again, if you said I need three pieces

706
00:30:12,270 --> 00:30:15,570
of textual evidence and the
student provided one piece

707
00:30:15,570 --> 00:30:18,600
of textual evidence, a random
example from their life

708
00:30:18,600 --> 00:30:21,180
and nothing else, then
it's going to say, hold up,

709
00:30:21,180 --> 00:30:22,470
that's not what your teacher asked for.

710
00:30:22,470 --> 00:30:24,180
And it's gonna give
them a chance to go back

711
00:30:24,180 --> 00:30:25,833
and fix it before they move on.

712
00:30:26,670 --> 00:30:29,310
It's also gonna just check
to make sure they didn't put,

713
00:30:29,310 --> 00:30:31,320
you know, one paragraph before another

714
00:30:31,320 --> 00:30:32,370
that didn't make any sense.

715
00:30:32,370 --> 00:30:35,910
It's looking for, you know,
logical reasoning here.

716
00:30:35,910 --> 00:30:38,160
And if you say they
need a counter argument

717
00:30:38,160 --> 00:30:41,460
and a rebuttal and any of
those kind of core elements

718
00:30:41,460 --> 00:30:42,630
of an argumentative essay

719
00:30:42,630 --> 00:30:44,430
and the student didn't include that,

720
00:30:44,430 --> 00:30:46,020
it's gonna point that out too.

721
00:30:46,020 --> 00:30:48,930
So this is one of those
kind of checks that we do

722
00:30:48,930 --> 00:30:51,360
to make sure that the student
is on the right track.

723
00:30:51,360 --> 00:30:55,650
When we get to drafting,
we have... (chuckles)

724
00:30:55,650 --> 00:30:58,410
Yeah, this is an example of how Khanmigo

725
00:30:58,410 --> 00:31:01,143
is maybe less helpful
than ChatGPT in some ways.

726
00:31:02,520 --> 00:31:05,400
So here, the student has
access to the outline

727
00:31:05,400 --> 00:31:07,650
that they just made with Khanmigo.

728
00:31:07,650 --> 00:31:10,140
So if you see the tabs here at the bottom,

729
00:31:10,140 --> 00:31:13,350
their assignment instructions
are always at the bottom.

730
00:31:13,350 --> 00:31:15,180
They can always chat with Khanmigo

731
00:31:15,180 --> 00:31:18,210
and when they're drafting
they can get to their outline.

732
00:31:18,210 --> 00:31:20,250
So when they're using their outline,

733
00:31:20,250 --> 00:31:23,220
we let them copy and paste
right from their outline

734
00:31:23,220 --> 00:31:25,800
into their draft, they can keep drafting.

735
00:31:25,800 --> 00:31:28,950
At any point, they can
talk to Khanmigo about,

736
00:31:28,950 --> 00:31:32,400
am I doing this right or what comes next?

737
00:31:32,400 --> 00:31:34,290
Or I'm stuck or I don't know what to put.

738
00:31:34,290 --> 00:31:36,480
But obviously, if the
student is asking Khanmigo

739
00:31:36,480 --> 00:31:40,260
to do any of the writing for
them to suggest evidence,

740
00:31:40,260 --> 00:31:42,690
to write a hook, to write a conclusion,

741
00:31:42,690 --> 00:31:44,403
Khanmigo will not do that.

742
00:31:45,420 --> 00:31:47,940
And you'll see this later,

743
00:31:47,940 --> 00:31:51,210
but you also as the teacher,
will have full visibility

744
00:31:51,210 --> 00:31:53,190
and to all of these interactions.

745
00:31:53,190 --> 00:31:55,050
And students, we tell students that too,

746
00:31:55,050 --> 00:31:56,670
so it's not a surprise for them.

747
00:31:56,670 --> 00:31:58,770
They know that this chat
is entirely viewable

748
00:31:58,770 --> 00:32:00,120
to the teacher.

749
00:32:00,120 --> 00:32:05,120
So once the student has a first draft,

750
00:32:05,130 --> 00:32:08,880
they can then submit it for feedback.

751
00:32:08,880 --> 00:32:11,790
Once they submit it for
feedback, what we do

752
00:32:11,790 --> 00:32:16,790
is we generate suggestions in
these five categories up here.

753
00:32:17,160 --> 00:32:20,370
Introduction, evidence
and reasoning, structure

754
00:32:20,370 --> 00:32:24,210
and organization, conclusion
and style and tone.

755
00:32:24,210 --> 00:32:28,050
And again, these are each
meticulously crafted to make sure

756
00:32:28,050 --> 00:32:31,140
that the feedback is
grade level appropriate.

757
00:32:31,140 --> 00:32:34,050
Because again, we know
what grade level you put in

758
00:32:34,050 --> 00:32:35,250
for the students, we know

759
00:32:35,250 --> 00:32:36,780
what the assignment instructions were.

760
00:32:36,780 --> 00:32:39,930
If you said, you know, make
sure there's MLA citations

761
00:32:39,930 --> 00:32:42,630
and make sure that there's this or that.

762
00:32:42,630 --> 00:32:46,020
It's going to put that feedback
under these categories.

763
00:32:46,020 --> 00:32:47,400
And these categories might be different

764
00:32:47,400 --> 00:32:49,170
from the rubrics that
you're used to working with,

765
00:32:49,170 --> 00:32:51,630
but all of the feedback
that you would get,

766
00:32:51,630 --> 00:32:56,010
for example, on the New Hampshire SAS test

767
00:32:56,010 --> 00:32:57,930
would still be reflected
in these categories.

768
00:32:57,930 --> 00:33:00,750
So introduction will
contain feedback on things

769
00:33:00,750 --> 00:33:03,900
like statement of purpose,
evidence and reasoning,

770
00:33:03,900 --> 00:33:06,930
will have things like
evidence and elaboration.

771
00:33:06,930 --> 00:33:09,060
Structure and conclusion
will contain feedback related

772
00:33:09,060 --> 00:33:10,770
to focus and organization

773
00:33:10,770 --> 00:33:13,320
and style and tone will
contain conventions

774
00:33:13,320 --> 00:33:15,180
and editing feedback.

775
00:33:15,180 --> 00:33:18,720
And for SAT, that feedback
also will be reflected

776
00:33:18,720 --> 00:33:20,670
across those different categories.

777
00:33:20,670 --> 00:33:23,730
One thing to note about
reading with SAT, once again,

778
00:33:23,730 --> 00:33:27,030
if you don't either include
the short passage itself

779
00:33:27,030 --> 00:33:29,010
in the essay prompt and instructions

780
00:33:29,010 --> 00:33:31,440
or that the prompt is kind of,

781
00:33:31,440 --> 00:33:33,390
or the essay itself is too long,

782
00:33:33,390 --> 00:33:35,640
or the reading passage rather is too long,

783
00:33:35,640 --> 00:33:39,360
or it's kind of a not
super well-known passage,

784
00:33:39,360 --> 00:33:42,600
it might struggle with the
reading feedback part for SAT

785
00:33:42,600 --> 00:33:44,670
because it doesn't know
what the heck this story

786
00:33:44,670 --> 00:33:46,380
or passage is that you're talking about.

787
00:33:46,380 --> 00:33:49,020
So if you have an example of a passage

788
00:33:49,020 --> 00:33:51,030
that's short enough to
include in the prompt

789
00:33:51,030 --> 00:33:54,300
or is well-known enough to just
be referenced in the prompt,

790
00:33:54,300 --> 00:33:55,650
then it will give feedback related

791
00:33:55,650 --> 00:33:57,250
to your analysis of the reading.

792
00:33:59,520 --> 00:34:01,770
A couple of other things to point out

793
00:34:01,770 --> 00:34:04,140
in our feedback interface.

794
00:34:04,140 --> 00:34:05,883
For every piece of feedback,

795
00:34:06,960 --> 00:34:10,950
including even positive phrase,
which you don't always get,

796
00:34:10,950 --> 00:34:13,260
not every kid gets positive
praise in every category,

797
00:34:13,260 --> 00:34:17,010
but if they do something well,
we try to point that out.

798
00:34:17,010 --> 00:34:20,670
Before any feedback that's
critical or actionable,

799
00:34:20,670 --> 00:34:23,340
what we noticed when we
first launched this version

800
00:34:23,340 --> 00:34:26,100
of the product is that many students

801
00:34:26,100 --> 00:34:27,480
didn't really know what to do next.

802
00:34:27,480 --> 00:34:31,470
They saw the feedback and they
were like, okay, now what?

803
00:34:31,470 --> 00:34:33,900
And we let them chat
with Khanmigo about it,

804
00:34:33,900 --> 00:34:36,390
but they didn't know how to
chat with Khanmigo about it.

805
00:34:36,390 --> 00:34:39,870
So we added these different
examples of how you can talk

806
00:34:39,870 --> 00:34:42,630
to Khanmigo about the
feedback that you're getting.

807
00:34:42,630 --> 00:34:44,910
So some of the things
that we let students do,

808
00:34:44,910 --> 00:34:46,680
and they can ask any question they want

809
00:34:46,680 --> 00:34:49,650
as long as it's related
to improving their essay,

810
00:34:49,650 --> 00:34:52,500
but we let them ask for an example

811
00:34:52,500 --> 00:34:53,970
from another essay prompt.

812
00:34:53,970 --> 00:34:55,740
So if the feedback is related

813
00:34:55,740 --> 00:34:58,920
to adding more context
in your introduction,

814
00:34:58,920 --> 00:35:01,110
then it'll give an example of an essay

815
00:35:01,110 --> 00:35:02,340
on a completely different topic

816
00:35:02,340 --> 00:35:04,650
where it kind of shows
them what what it means

817
00:35:04,650 --> 00:35:07,833
by giving more context to the
reader in the introduction.

818
00:35:08,820 --> 00:35:12,240
We let them ask a follow
up question to explain

819
00:35:12,240 --> 00:35:14,880
the suggestion, so if the
student just doesn't really get

820
00:35:14,880 --> 00:35:17,460
what it's saying, they can
click explain suggestion

821
00:35:17,460 --> 00:35:18,900
and it'll reword it for them.

822
00:35:18,900 --> 00:35:21,990
Or the student can say, break
it down for me even more,

823
00:35:21,990 --> 00:35:25,140
or be more specific and
it'll respond to that.

824
00:35:25,140 --> 00:35:27,090
And then the other thing that it does,

825
00:35:27,090 --> 00:35:30,570
which I think is probably the
most useful is the student can

826
00:35:30,570 --> 00:35:34,020
actually edit their essay in that box.

827
00:35:34,020 --> 00:35:36,210
So they can go back at any point

828
00:35:36,210 --> 00:35:38,610
and they can change the
sentence, they can erase,

829
00:35:38,610 --> 00:35:42,060
they can revise, they can
edit anything they want

830
00:35:42,060 --> 00:35:44,610
and then they can say, check my revision.

831
00:35:44,610 --> 00:35:47,490
And Khanmigo will reference the feedback

832
00:35:47,490 --> 00:35:49,350
that the student is talking about

833
00:35:49,350 --> 00:35:51,150
and it will look at the essay before

834
00:35:51,150 --> 00:35:54,720
and after the student asked
it to check their revision

835
00:35:54,720 --> 00:35:57,540
and it will actually
reassess did they fix it?

836
00:35:57,540 --> 00:35:59,643
Like did they actually
add enough context now

837
00:35:59,643 --> 00:36:01,530
that it makes sense to the reader

838
00:36:01,530 --> 00:36:02,880
what they're talking about.

839
00:36:02,880 --> 00:36:04,980
And Khanmigo will kind
of give them a thumbs up

840
00:36:04,980 --> 00:36:06,330
or you know, it's not quite there yet

841
00:36:06,330 --> 00:36:08,250
and help them work through that.

842
00:36:08,250 --> 00:36:10,530
So anytime a student is, you know,

843
00:36:10,530 --> 00:36:13,590
uncertain about is that
better, did it fix it?

844
00:36:13,590 --> 00:36:15,090
That's a really useful tool

845
00:36:15,090 --> 00:36:16,830
to have them do the revision live

846
00:36:16,830 --> 00:36:19,860
in the interface and then ask
Khanmigo if they fixed it.

847
00:36:19,860 --> 00:36:22,260
And as they're making
the revisions and edits,

848
00:36:22,260 --> 00:36:25,830
they can mark each different
piece of feedback as resolved.

849
00:36:25,830 --> 00:36:29,310
Once they're done revising,
they can export their essay

850
00:36:29,310 --> 00:36:33,420
to PDF or Microsoft Word, or Google Drive.

851
00:36:33,420 --> 00:36:36,840
If you all use any of
those kind of platforms

852
00:36:36,840 --> 00:36:39,420
and you wanna do your grading
there of their final draft

853
00:36:39,420 --> 00:36:41,070
or whatever, those are some methods

854
00:36:41,070 --> 00:36:42,420
where you can have your students

855
00:36:42,420 --> 00:36:44,850
kind of export their final
drafts to those places.

856
00:36:44,850 --> 00:36:46,860
But at the same time,
you will also have access

857
00:36:46,860 --> 00:36:48,980
to final draft directly in Writing Coach

858
00:36:48,980 --> 00:36:51,903
in your kind of teacher dashboard.

859
00:36:53,400 --> 00:36:57,723
So this is what the teacher
experience looks like.

860
00:36:59,310 --> 00:37:01,800
For every writing coach
assignment that a teacher creates,

861
00:37:01,800 --> 00:37:04,320
there is a class assignment report.

862
00:37:04,320 --> 00:37:07,380
The class assignment report
includes the information

863
00:37:07,380 --> 00:37:08,700
about the assignment that you added.

864
00:37:08,700 --> 00:37:10,890
So you can expand it to
view the essay instructions

865
00:37:10,890 --> 00:37:11,723
that you wrote.

866
00:37:12,660 --> 00:37:13,980
But then this is where you can see

867
00:37:13,980 --> 00:37:16,140
for all of the students in your class,

868
00:37:16,140 --> 00:37:17,940
where in the writing process they are?

869
00:37:17,940 --> 00:37:20,430
So if students are haven't started yet

870
00:37:20,430 --> 00:37:22,257
or maybe they're still
chatting with Khanmigo

871
00:37:22,257 --> 00:37:24,420
and understanding or if they're outlining,

872
00:37:24,420 --> 00:37:28,500
if they're actively drafting,
or if they are revising,

873
00:37:28,500 --> 00:37:29,880
or if they've marked it as complete,

874
00:37:29,880 --> 00:37:31,080
you can see it from this dashboard

875
00:37:31,080 --> 00:37:35,040
to get a sense of the
progress that's being made.

876
00:37:35,040 --> 00:37:36,660
We also have flags to let you know

877
00:37:36,660 --> 00:37:39,240
if somebody submitted
something by the due date,

878
00:37:39,240 --> 00:37:42,067
but then edit it after
the due date passed.

879
00:37:42,067 --> 00:37:44,100
So you can go in and kind
of see what exactly they did

880
00:37:44,100 --> 00:37:45,570
after the due date passed.

881
00:37:45,570 --> 00:37:48,990
And obviously, if they didn't
submit it by the due date,

882
00:37:48,990 --> 00:37:50,703
we'll also flag that as past due.

883
00:37:51,870 --> 00:37:54,750
Other things you can
see from this dashboard,

884
00:37:54,750 --> 00:37:56,940
how much time students spent.

885
00:37:56,940 --> 00:37:58,590
This is active time,

886
00:37:58,590 --> 00:38:01,260
this isn't just like they
had their browser open

887
00:38:01,260 --> 00:38:04,620
and they went to sleep like
this is actively scrolling

888
00:38:04,620 --> 00:38:08,220
on the page or talking to
Khanmigo or writing, or revising,

889
00:38:08,220 --> 00:38:10,980
or any of the things that you
could be doing on the page.

890
00:38:10,980 --> 00:38:13,080
So that's the active time that they spent.

891
00:38:13,080 --> 00:38:15,357
And what we typically
find with Writing Coach

892
00:38:15,357 --> 00:38:17,730
for the partners who
have been using it so far

893
00:38:17,730 --> 00:38:20,070
is that the average time spent

894
00:38:20,070 --> 00:38:23,850
is about two hours for one essay.

895
00:38:23,850 --> 00:38:25,470
And that includes everything
from understanding

896
00:38:25,470 --> 00:38:27,063
all the way to completed.

897
00:38:29,430 --> 00:38:32,490
The column under writing feedback,

898
00:38:32,490 --> 00:38:34,470
that's where you can see the number

899
00:38:34,470 --> 00:38:36,570
of suggestions Khanmigo gave.

900
00:38:36,570 --> 00:38:41,130
So you know, if they got 20
suggestions versus like five,

901
00:38:41,130 --> 00:38:42,420
that'll tell you a little bit about

902
00:38:42,420 --> 00:38:44,730
how good their first draft was.

903
00:38:44,730 --> 00:38:46,650
And you can also see how
much of that feedback

904
00:38:46,650 --> 00:38:49,170
they have resolved, if
they've resolved any of it.

905
00:38:49,170 --> 00:38:50,490
So if they've submitted it,

906
00:38:50,490 --> 00:38:52,320
but they didn't resolve any feedback,

907
00:38:52,320 --> 00:38:53,730
maybe they just didn't click the button

908
00:38:53,730 --> 00:38:56,460
but maybe they didn't
actually make any revisions,

909
00:38:56,460 --> 00:38:59,820
I would click in there and
see exactly what happened.

910
00:38:59,820 --> 00:39:03,510
Word count is another piece
of data that you have here.

911
00:39:03,510 --> 00:39:05,730
And then the originality flags,

912
00:39:05,730 --> 00:39:08,220
these are where Khanmigo will tell you

913
00:39:08,220 --> 00:39:11,190
if the student pasted something

914
00:39:11,190 --> 00:39:13,560
that wasn't from their
outline into their draft

915
00:39:13,560 --> 00:39:15,690
or into their revised draft.

916
00:39:15,690 --> 00:39:18,240
So if they're pasting from
their outline, that's fine.

917
00:39:18,240 --> 00:39:23,100
If they're pasting into their
outline evidence, that's fine.

918
00:39:23,100 --> 00:39:24,720
If they're pasting, you know,

919
00:39:24,720 --> 00:39:27,720
more than a couple of words
into their main points

920
00:39:27,720 --> 00:39:32,280
or their thesis, or their
evidence elaboration,

921
00:39:32,280 --> 00:39:34,650
not evidence, but their
reasoning and elaboration

922
00:39:34,650 --> 00:39:36,660
of their outline, that'll get flagged.

923
00:39:36,660 --> 00:39:39,240
And if they're pasting,
I think more than five

924
00:39:39,240 --> 00:39:41,910
or 10 words, I forget
exactly into their draft,

925
00:39:41,910 --> 00:39:43,470
or in their revisions,

926
00:39:43,470 --> 00:39:45,780
that's also gonna raise a flag for you.

927
00:39:45,780 --> 00:39:49,140
And of course, this is
not foolproof, right?

928
00:39:49,140 --> 00:39:51,930
Students might reasonably
be pacing in a new piece

929
00:39:51,930 --> 00:39:52,920
of evidence that they found,

930
00:39:52,920 --> 00:39:55,140
maybe Khanmigo gave them
feedback during revising

931
00:39:55,140 --> 00:39:57,330
that their evidence wasn't that great,

932
00:39:57,330 --> 00:39:59,400
so they went and found a new piece.

933
00:39:59,400 --> 00:40:03,030
You then can go in and check
to see is this a real issue

934
00:40:03,030 --> 00:40:04,290
or was this flagged

935
00:40:04,290 --> 00:40:07,500
because they pasted in new
evidence or something like that.

936
00:40:07,500 --> 00:40:10,500
And of course, students
will always have their ways,

937
00:40:10,500 --> 00:40:11,880
but we do think that this helps

938
00:40:11,880 --> 00:40:14,400
with a lot of the transparency
and from what we've seen,

939
00:40:14,400 --> 00:40:17,970
it has caught a lot of students.

940
00:40:17,970 --> 00:40:20,550
So this is what it looks
like if you go into the,

941
00:40:20,550 --> 00:40:23,400
what we call the individual
student essay report.

942
00:40:23,400 --> 00:40:25,440
So this is when you click
on one of those students

943
00:40:25,440 --> 00:40:28,230
or you click on one of
those originality flags,

944
00:40:28,230 --> 00:40:31,290
you get the entire
student writing process.

945
00:40:31,290 --> 00:40:35,070
So you can see everything
from every single chat

946
00:40:35,070 --> 00:40:37,500
that they had with Khanmigo
from the very beginning.

947
00:40:37,500 --> 00:40:40,710
You can see their entire
outlining history.

948
00:40:40,710 --> 00:40:42,930
You can see their chats
while they were outlining.

949
00:40:42,930 --> 00:40:44,430
You can see their draft history,

950
00:40:44,430 --> 00:40:46,080
their chats while they were drafting.

951
00:40:46,080 --> 00:40:49,170
You can see the first
draft that they submitted

952
00:40:49,170 --> 00:40:50,730
for Khanmigo for feedback.

953
00:40:50,730 --> 00:40:53,160
You can see the feedback
that Khanmigo gave.

954
00:40:53,160 --> 00:40:55,680
You can see when they chatted
with Khanmigo about feedback,

955
00:40:55,680 --> 00:40:57,330
you can see the revisions that they made.

956
00:40:57,330 --> 00:40:59,490
You can see if they edited
after the due date passed.

957
00:40:59,490 --> 00:41:00,840
You can see what the essay looked like

958
00:41:00,840 --> 00:41:02,070
when the due date passed.

959
00:41:02,070 --> 00:41:04,950
So we provide a lot of transparency

960
00:41:04,950 --> 00:41:06,513
into the writing process here.

961
00:41:07,380 --> 00:41:11,100
We do not expect every teacher to dig in

962
00:41:11,100 --> 00:41:13,230
and look at this, all of
this data for every kid

963
00:41:13,230 --> 00:41:15,360
that would increase the amount of work

964
00:41:15,360 --> 00:41:17,280
that we'd be expecting teachers to do.

965
00:41:17,280 --> 00:41:19,290
But this is here for when you need it.

966
00:41:19,290 --> 00:41:23,550
So the reason why this
page is so important

967
00:41:23,550 --> 00:41:26,640
is this page kind of tells
you where to drill down.

968
00:41:26,640 --> 00:41:29,880
So again, if you have
students who wrote an essay

969
00:41:29,880 --> 00:41:33,840
from start to finish in 30
minutes, maybe we check that kid.

970
00:41:33,840 --> 00:41:37,320
If you have a student who
got 20 pieces of feedback

971
00:41:37,320 --> 00:41:40,230
and resolved none of it, maybe
drill down into that one.

972
00:41:40,230 --> 00:41:42,360
Anytime you see an originality flag,

973
00:41:42,360 --> 00:41:44,880
that would be a good reason to drill down.

974
00:41:44,880 --> 00:41:47,010
So there's a lot of data here,

975
00:41:47,010 --> 00:41:50,250
but the intent is to make
it easier to figure out

976
00:41:50,250 --> 00:41:51,630
who should we really look at?

977
00:41:51,630 --> 00:41:53,130
Who should we drill down into?

978
00:41:53,130 --> 00:41:54,300
And in cases where you do have

979
00:41:54,300 --> 00:41:57,630
to investigate potential
issues of plagiarism,

980
00:41:57,630 --> 00:41:58,860
there's a lot of information here

981
00:41:58,860 --> 00:42:01,710
that can help you understand
what exactly happened

982
00:42:01,710 --> 00:42:02,883
in the whole process.

983
00:42:05,220 --> 00:42:07,620
All right. Oh, and real quick,

984
00:42:07,620 --> 00:42:12,300
for those of you who are school
or district administrators,

985
00:42:12,300 --> 00:42:15,570
there is also some
Writing Coach related data

986
00:42:15,570 --> 00:42:19,620
that you can find in
the KAD admin dashboard.

987
00:42:19,620 --> 00:42:22,920
So any time that students
spend working on Writing Coach,

988
00:42:22,920 --> 00:42:24,780
that active time spent that you saw

989
00:42:24,780 --> 00:42:26,970
that gets reflected in learning minutes

990
00:42:26,970 --> 00:42:29,760
and actual student
engagement with Writing Coach

991
00:42:29,760 --> 00:42:32,250
would be reflected as an activity

992
00:42:32,250 --> 00:42:33,720
under the Khanmigo usage section.

993
00:42:33,720 --> 00:42:34,920
So you can get a sense of the number

994
00:42:34,920 --> 00:42:37,200
of students using Khanmigo Writing Coach

995
00:42:37,200 --> 00:42:38,127
or the percentage of students

996
00:42:38,127 --> 00:42:40,260
who are using Khanmigo Writing Coach

997
00:42:40,260 --> 00:42:43,710
and, you know, drill that
down into different schools

998
00:42:43,710 --> 00:42:45,600
or whatever you're looking at.

999
00:42:45,600 --> 00:42:49,110
So I'm gonna actually switch

1000
00:42:49,110 --> 00:42:52,410
'cause I have a couple of minutes.

1001
00:42:52,410 --> 00:42:57,410
I'm gonna switch over
to the live experience

1002
00:42:57,690 --> 00:42:59,910
'cause next I'm gonna
give you all a chance

1003
00:42:59,910 --> 00:43:04,260
to log into your accounts
and play around with this.

1004
00:43:04,260 --> 00:43:08,163
So again, when you're logged
into your Khan Academy account,

1005
00:43:09,060 --> 00:43:11,060
you wanna click on the Khanmigo dropdown

1006
00:43:12,030 --> 00:43:15,600
and go to Learner activities,
which is this page.

1007
00:43:15,600 --> 00:43:16,683
I'll just show you.

1008
00:43:22,710 --> 00:43:24,510
Okay, when you click
on Learner Activities,

1009
00:43:24,510 --> 00:43:25,713
it'll take you here.

1010
00:43:26,940 --> 00:43:30,630
You're gonna scroll down on
the left hand side menu here

1011
00:43:30,630 --> 00:43:31,983
and click on Writing Coach.

1012
00:43:33,990 --> 00:43:36,090
And again, your students
can do this as well.

1013
00:43:36,090 --> 00:43:38,850
Students can't assign and they
have a different button here.

1014
00:43:38,850 --> 00:43:41,670
They can actually use
Writing Coach on their own.

1015
00:43:41,670 --> 00:43:43,950
If they just want extra
help with an essay,

1016
00:43:43,950 --> 00:43:45,720
you don't need to assign it to them.

1017
00:43:45,720 --> 00:43:48,480
But if you assign it to them,
then you get that class report

1018
00:43:48,480 --> 00:43:50,370
and all of those useful insights.

1019
00:43:50,370 --> 00:43:52,950
So you'll see three tabs here.

1020
00:43:52,950 --> 00:43:55,620
The two most important
tabs are your assignments

1021
00:43:55,620 --> 00:43:59,100
that you've created for your
classes and your essays.

1022
00:43:59,100 --> 00:44:02,550
Your essays are, is kind
of like your playground

1023
00:44:02,550 --> 00:44:03,383
for Writing Coach.

1024
00:44:03,383 --> 00:44:04,800
So if you wanted to model

1025
00:44:04,800 --> 00:44:07,230
how to use Writing coach in a classroom,

1026
00:44:07,230 --> 00:44:09,570
you can create your own essay

1027
00:44:09,570 --> 00:44:12,510
and that will show up under my essays.

1028
00:44:12,510 --> 00:44:15,570
So if you click Try Student Experience,

1029
00:44:15,570 --> 00:44:18,720
that's gonna create an essay
under your essays, okay?

1030
00:44:18,720 --> 00:44:22,320
So if you wanted to demo for
students how to use this,

1031
00:44:22,320 --> 00:44:25,050
you could use that tool
to kind of show them

1032
00:44:25,050 --> 00:44:27,060
and work through it together.

1033
00:44:27,060 --> 00:44:29,130
But your assignments will show up

1034
00:44:29,130 --> 00:44:30,990
under the My Assignments tab.

1035
00:44:30,990 --> 00:44:34,290
One of the things that I would
encourage you all to explore

1036
00:44:34,290 --> 00:44:36,990
once I let you go off and try this out,

1037
00:44:36,990 --> 00:44:40,290
we have two different sample
essays under my essays.

1038
00:44:40,290 --> 00:44:41,910
So when you log in, obviously,

1039
00:44:41,910 --> 00:44:43,350
writing an essay, it takes a long time.

1040
00:44:43,350 --> 00:44:45,990
So even if you are
experimenting with Writing Coach

1041
00:44:45,990 --> 00:44:47,340
or just trying it out,

1042
00:44:47,340 --> 00:44:49,980
we don't want you to
have to actually outline

1043
00:44:49,980 --> 00:44:52,410
and write an entire essay
just to see how it works.

1044
00:44:52,410 --> 00:44:56,010
So these sample essays
have pre-filled in outlines

1045
00:44:56,010 --> 00:44:58,770
and drafts so that you can
just see what it looks like.

1046
00:44:58,770 --> 00:45:00,900
So if I go into, let's see,

1047
00:45:00,900 --> 00:45:03,570
Daylight Savings Time persuasive essay,

1048
00:45:03,570 --> 00:45:05,373
I'm gonna go back to the beginning.

1049
00:45:06,810 --> 00:45:09,600
This is the student preview,
this is not a real essay,

1050
00:45:09,600 --> 00:45:12,030
but you can see that there's
the essay instructions

1051
00:45:12,030 --> 00:45:14,760
over here and then Khanmigo go over here,

1052
00:45:14,760 --> 00:45:18,780
you can chat with it, you
can move on to outlining

1053
00:45:18,780 --> 00:45:22,230
and it's already pre-filled
in with some outlining.

1054
00:45:22,230 --> 00:45:23,673
I would recommend that you play around

1055
00:45:23,673 --> 00:45:25,080
with the leading stuff

1056
00:45:25,080 --> 00:45:28,320
or making totally weird
random evidence like

1057
00:45:28,320 --> 00:45:30,540
that has nothing to do with the assignment

1058
00:45:30,540 --> 00:45:33,210
and talk to Khanmigo and see what happens.

1059
00:45:33,210 --> 00:45:36,660
You can try out, you know,
asking it general questions

1060
00:45:36,660 --> 00:45:37,740
about what to do next.

1061
00:45:37,740 --> 00:45:39,120
You know, if you leave
a bunch of things blank

1062
00:45:39,120 --> 00:45:41,250
and see how it responds there.

1063
00:45:41,250 --> 00:45:43,260
When you get into drafting,

1064
00:45:43,260 --> 00:45:45,720
you can see that it's gonna do the check

1065
00:45:45,720 --> 00:45:46,980
and I haven't deleted anything,

1066
00:45:46,980 --> 00:45:48,167
so hopefully all these boxes are...

1067
00:45:48,167 --> 00:45:49,710
Oh, it still thinks.

1068
00:45:49,710 --> 00:45:52,290
Okay, so there are
suggestions with the default.

1069
00:45:52,290 --> 00:45:54,480
So this is what it looks
like if you do not meet

1070
00:45:54,480 --> 00:45:59,480
those basic expectations, it
will give you the examples

1071
00:45:59,850 --> 00:46:01,650
of what you need to fix.

1072
00:46:01,650 --> 00:46:03,750
And technically, if you're
like, eh, I disagree,

1073
00:46:03,750 --> 00:46:05,190
you can ignore and move on.

1074
00:46:05,190 --> 00:46:07,260
Or you can go back down here

1075
00:46:07,260 --> 00:46:09,693
and you can revise your outline.

1076
00:46:11,100 --> 00:46:13,470
I'm gonna just ignore
and move on to show you

1077
00:46:13,470 --> 00:46:15,360
the drafting experience.

1078
00:46:15,360 --> 00:46:18,090
Again, we've pre-filled
in a draft here for you.

1079
00:46:18,090 --> 00:46:20,700
When a student starts on this
page, it's gonna be blank,

1080
00:46:20,700 --> 00:46:24,510
but just to show you, this
is the outline on this side.

1081
00:46:24,510 --> 00:46:26,760
So if the student is
pasting from their outline,

1082
00:46:26,760 --> 00:46:28,350
they're just gonna click copy

1083
00:46:28,350 --> 00:46:30,510
for whatever section they wanna paste in

1084
00:46:30,510 --> 00:46:32,880
and paste it into their draft field.

1085
00:46:32,880 --> 00:46:35,850
And again, down here that's
where they can switch to chat

1086
00:46:35,850 --> 00:46:39,030
or just view the assignment
instructions again.

1087
00:46:39,030 --> 00:46:41,223
And when we move to revising,

1088
00:46:42,720 --> 00:46:46,503
it's gonna generate the
feedback for you in real-time.

1089
00:46:47,400 --> 00:46:51,030
And you can see as it's generating

1090
00:46:51,030 --> 00:46:52,950
the different pieces of
feedback that you get.

1091
00:46:52,950 --> 00:46:55,740
And again, like you can interact
with all of these chats.

1092
00:46:55,740 --> 00:46:58,290
When you're done with the sample essay,

1093
00:46:58,290 --> 00:47:00,090
if you wanna just delete
it and start over,

1094
00:47:00,090 --> 00:47:03,240
you can press delete and
that's just gonna erase

1095
00:47:03,240 --> 00:47:06,120
the sample essay and you can
see a fresh new one down here

1096
00:47:06,120 --> 00:47:07,890
at the bottom and you can start over.

1097
00:47:07,890 --> 00:47:09,060
So this is also really helpful

1098
00:47:09,060 --> 00:47:12,330
if you're just showing students
how to use Writing Coach

1099
00:47:12,330 --> 00:47:14,430
so you don't have to actually type in

1100
00:47:14,430 --> 00:47:16,530
maybe all of the outlines or the drafts.

1101
00:47:16,530 --> 00:47:18,780
This is kind of a quick way
to try out Writing Coach,

1102
00:47:18,780 --> 00:47:20,520
so you can play around with that.

1103
00:47:20,520 --> 00:47:23,100
You can also play around
with actually creating

1104
00:47:23,100 --> 00:47:24,330
a Writing Coach assignment

1105
00:47:24,330 --> 00:47:27,300
if your Khan Academy account has classes

1106
00:47:27,300 --> 00:47:28,950
just to see what that looks like.

1107
00:47:30,540 --> 00:47:33,480
Or you can create your own essay

1108
00:47:33,480 --> 00:47:35,550
that has a prompt of your choice.

1109
00:47:35,550 --> 00:47:37,200
So those are some of the different ways

1110
00:47:37,200 --> 00:47:38,520
that you can do that.

1111
00:47:38,520 --> 00:47:41,400
I will also, as you guys are doing that,

1112
00:47:41,400 --> 00:47:45,630
I will leave up one of those
sample teacher reports up here.

1113
00:47:45,630 --> 00:47:48,540
So if you are curious and
you wanna take a closer look

1114
00:47:48,540 --> 00:47:49,980
at what that looks like,

1115
00:47:49,980 --> 00:47:53,428
I can kind of show you
what that looks like too.

1116
00:47:53,428 --> 00:47:57,030
All right, are there any questions?

1117
00:47:57,030 --> 00:48:02,030
Let me go back to the deck about, yes.

1118
00:48:02,250 --> 00:48:04,980
- So if a student, you
know, ignores, ignores,

1119
00:48:04,980 --> 00:48:08,160
ignores, submits, is
there a function in there

1120
00:48:08,160 --> 00:48:10,020
for the teacher to kick
it right back track,

1121
00:48:10,020 --> 00:48:12,060
try it again?
- That's a great question.

1122
00:48:12,060 --> 00:48:13,500
As long as the due date hasn't passed,

1123
00:48:13,500 --> 00:48:15,330
they can go right back.

1124
00:48:15,330 --> 00:48:17,700
They can go back and and do it again, but-

1125
00:48:17,700 --> 00:48:19,740
- [Attendee] But is there a specific,

1126
00:48:19,740 --> 00:48:20,940
like the teacher can send it back

1127
00:48:20,940 --> 00:48:23,820
or the kid can just go in?
- The kid can go in

1128
00:48:23,820 --> 00:48:28,820
and actually, so they can
go back if they haven't,

1129
00:48:29,460 --> 00:48:30,390
the only difference...

1130
00:48:30,390 --> 00:48:31,560
Okay, let me think about this.

1131
00:48:31,560 --> 00:48:33,540
They can go back at any point

1132
00:48:33,540 --> 00:48:36,810
and they can modify their
outline, their draft

1133
00:48:36,810 --> 00:48:39,090
or their revised draft at any point

1134
00:48:39,090 --> 00:48:41,340
if the due date hasn't passed.

1135
00:48:41,340 --> 00:48:44,310
If they have already generated feedback

1136
00:48:44,310 --> 00:48:47,970
on their first draft, Khanmigo
can't generate feedback again

1137
00:48:47,970 --> 00:48:49,050
on that same assignment.

1138
00:48:49,050 --> 00:48:51,000
They only get one chance
to generate feedback.

1139
00:48:51,000 --> 00:48:53,160
But what you can do is
have them do it with

1140
00:48:53,160 --> 00:48:58,160
the start your own essay and
have them do it again there,

1141
00:48:58,230 --> 00:48:59,880
it's not gonna be part of the assignment.

1142
00:48:59,880 --> 00:49:01,560
That's one of the things that

1143
00:49:01,560 --> 00:49:02,673
we've heard a few people be like,

1144
00:49:02,673 --> 00:49:04,770
that be really nice, if I
could just be like, start over.

1145
00:49:04,770 --> 00:49:06,330
- [Attendee] You like
to encourage them to,

1146
00:49:06,330 --> 00:49:08,610
okay, let's improve upon what we've got.

1147
00:49:08,610 --> 00:49:10,920
- Yes.
- I'm gonna send this

1148
00:49:10,920 --> 00:49:11,753
back to you right now.

1149
00:49:11,753 --> 00:49:12,780
This is what it's going in as,

1150
00:49:12,780 --> 00:49:15,090
but we're gonna keep working on this.

1151
00:49:15,090 --> 00:49:18,840
And I know for me I would have a due date.

1152
00:49:18,840 --> 00:49:20,340
And I guess that was the due date,

1153
00:49:20,340 --> 00:49:22,390
but as a middle school teacher especially

1154
00:49:23,310 --> 00:49:26,190
that doesn't mean I'm done
working with you. (chuckles)

1155
00:49:26,190 --> 00:49:28,230
I wanna give you an opportunity
to review and improve

1156
00:49:28,230 --> 00:49:31,637
and continue to learn and grow and so-

1157
00:49:31,637 --> 00:49:32,910
- Agree.
- I want 'em to be able

1158
00:49:32,910 --> 00:49:35,040
to edit-
- I think actually,

1159
00:49:35,040 --> 00:49:38,370
I'm wrong about you
can't, that's why we added

1160
00:49:38,370 --> 00:49:39,720
the edited after due date.

1161
00:49:39,720 --> 00:49:41,790
You can edit after the due date.

1162
00:49:41,790 --> 00:49:43,990
We just tell the teacher
when they did that.

1163
00:49:44,940 --> 00:49:47,610
- [Attendee] It would really
be nice to have us a function

1164
00:49:47,610 --> 00:49:49,080
that the teacher has sent it back.

1165
00:49:49,080 --> 00:49:50,820
- Yes. I agree.
- Because a lot of times

1166
00:49:50,820 --> 00:49:53,100
otherwise the kids are gonna
be like whatever I submitted.

1167
00:49:53,100 --> 00:49:53,933
- Yeah, I agree.

1168
00:49:53,933 --> 00:49:56,850
And actually later on,
after we do all of this,

1169
00:49:56,850 --> 00:49:59,280
Bernadette and Barbara are
gonna put up chart paper

1170
00:49:59,280 --> 00:50:01,710
where you're all going to
share all of your feedback,

1171
00:50:01,710 --> 00:50:04,350
all of your ideas, we wanna hear from you,

1172
00:50:04,350 --> 00:50:06,660
how can we make this
better? What's missing?

1173
00:50:06,660 --> 00:50:07,493
What are some of the things

1174
00:50:07,493 --> 00:50:09,060
that would make this more
useful for all of you?

1175
00:50:09,060 --> 00:50:11,880
So save those nuggets and please add them

1176
00:50:11,880 --> 00:50:14,670
to the chart paper when
we get to that, yes.

1177
00:50:14,670 --> 00:50:17,100
- If I'm a teacher and
I create an assignment

1178
00:50:17,100 --> 00:50:22,100
and I put all my stuff in and
then I assign it to my kids

1179
00:50:22,290 --> 00:50:23,970
and then let's call it a week later

1180
00:50:23,970 --> 00:50:25,770
while they're working on it,

1181
00:50:25,770 --> 00:50:28,143
I recognize that I
forgot to add something,

1182
00:50:29,700 --> 00:50:33,930
I go back and I add, oh, it
should have been APA not MLA.

1183
00:50:33,930 --> 00:50:36,420
Will the feedback for the students

1184
00:50:36,420 --> 00:50:38,340
that have already in the
outlining and drafting,

1185
00:50:38,340 --> 00:50:39,933
will the feedback then adjust?

1186
00:50:41,460 --> 00:50:43,590
- No, not yet.
- Okay.

1187
00:50:43,590 --> 00:50:46,020
- But we're working on that, yes.

1188
00:50:46,020 --> 00:50:48,000
- So is the feedback only

1189
00:50:48,000 --> 00:50:52,140
for writing style structure elaboration

1190
00:50:52,140 --> 00:50:55,470
or does it also take
into account the content?

1191
00:50:55,470 --> 00:50:58,200
- It does take into account the contents,

1192
00:50:58,200 --> 00:51:03,200
but only if the student is
writing about literature

1193
00:51:03,630 --> 00:51:06,300
or a reading passage
and the reading passage

1194
00:51:06,300 --> 00:51:08,610
isn't super well-known, it might struggle

1195
00:51:08,610 --> 00:51:10,820
to like the student could
technically make up, you know,

1196
00:51:10,820 --> 00:51:11,880
an article or something.

1197
00:51:11,880 --> 00:51:13,290
There's no way for the LLM to know

1198
00:51:13,290 --> 00:51:14,340
that that's a made up article.

1199
00:51:14,340 --> 00:51:16,260
- [Attendee] Because
that's what I'm noticing

1200
00:51:16,260 --> 00:51:19,995
that I need to, if I need
to score for the content

1201
00:51:19,995 --> 00:51:22,768
- Yes.
- But it's giving structure,

1202
00:51:22,768 --> 00:51:24,030
but not the content.
- Yeah. If you're assigning

1203
00:51:24,030 --> 00:51:26,880
an essay about Lord of
the Flies or you know,

1204
00:51:26,880 --> 00:51:28,620
any kind of well-known book-
- If we're doing

1205
00:51:28,620 --> 00:51:30,545
Revolutionary War or-
- Yes, exactly.

1206
00:51:30,545 --> 00:51:32,400
- I'm experiment in science.
- Exactly.

1207
00:51:32,400 --> 00:51:34,620
If you're referencing well-known things,

1208
00:51:34,620 --> 00:51:36,240
it will give feedback on content.

1209
00:51:36,240 --> 00:51:40,440
It'll say, you know,
that's the wrong character.

1210
00:51:40,440 --> 00:51:42,663
It might point out like issues with that,

1211
00:51:43,680 --> 00:51:45,900
but it has to be kind of a well-known

1212
00:51:45,900 --> 00:51:47,160
or commonly known passage.

1213
00:51:47,160 --> 00:51:49,440
Or again, if it's a short passage,

1214
00:51:49,440 --> 00:51:50,970
you can just put the whole passage

1215
00:51:50,970 --> 00:51:53,160
in the assignment instructions for the AI

1216
00:51:53,160 --> 00:51:54,527
to have as a reference, yeah.

1217
00:51:54,527 --> 00:51:55,920
- [Attendee] I have two short questions.

1218
00:51:55,920 --> 00:51:58,860
First, if when it exports
it to Google Drive,

1219
00:51:58,860 --> 00:52:02,250
do they have to go in and
reformat it to look like

1220
00:52:02,250 --> 00:52:05,880
an MLA essay, double-spacing
time, all that kind of stuff?

1221
00:52:05,880 --> 00:52:08,310
- They probably will have
to do some reformatting.

1222
00:52:08,310 --> 00:52:10,410
- [Attendee] Okay. My other question is,

1223
00:52:10,410 --> 00:52:15,090
where do you see peer review
kind of fitting in to this?

1224
00:52:15,090 --> 00:52:18,120
Like let's say they're doing
their first draft via this,

1225
00:52:18,120 --> 00:52:20,283
they're getting legit feedback.

1226
00:52:21,510 --> 00:52:25,020
Would you then say, after
you've gotten Khanmigo feedback,

1227
00:52:25,020 --> 00:52:26,910
still give it to a
partner? Or would you say

1228
00:52:26,910 --> 00:52:28,470
do that before-
- I would do that.

1229
00:52:28,470 --> 00:52:30,960
Yeah, I would have them
export to Google Drive

1230
00:52:30,960 --> 00:52:33,030
and then share it with a
partner and have that partner

1231
00:52:33,030 --> 00:52:35,763
kind of annotate it with
Google Doc comments.

1232
00:52:36,960 --> 00:52:40,250
And then, I mean, you
could also have them...

1233
00:52:41,250 --> 00:52:42,810
One of the things that we don't do yet,

1234
00:52:42,810 --> 00:52:46,650
but we wanna do is allow
you to submit a draft

1235
00:52:46,650 --> 00:52:47,940
for feedback again.

1236
00:52:47,940 --> 00:52:51,510
So what we're actually
planning on doing this year

1237
00:52:51,510 --> 00:52:53,730
is letting you just do the feedback part.

1238
00:52:53,730 --> 00:52:57,120
So if you just want
students to put a draft in

1239
00:52:57,120 --> 00:52:59,490
and you don't really care
about the whole writing process

1240
00:52:59,490 --> 00:53:02,280
being documented in Writing
Coach and getting that support,

1241
00:53:02,280 --> 00:53:03,990
you just want them to get a feedback,

1242
00:53:03,990 --> 00:53:05,730
a round of feedback on a draft,

1243
00:53:05,730 --> 00:53:07,350
they will be able to do that.

1244
00:53:07,350 --> 00:53:08,640
So you could have them do that

1245
00:53:08,640 --> 00:53:11,460
after they did one
round with Writing Coach

1246
00:53:11,460 --> 00:53:13,410
and then Google Docs with a peer

1247
00:53:13,410 --> 00:53:15,140
or something after.
- If they're getting

1248
00:53:15,140 --> 00:53:17,610
conflicting comments from their partner

1249
00:53:17,610 --> 00:53:19,560
versus what they got from Khanmigo,

1250
00:53:19,560 --> 00:53:22,410
would you say it's then
on the student to decide

1251
00:53:22,410 --> 00:53:25,230
what is appropriate feedback to take?

1252
00:53:25,230 --> 00:53:26,460
- I think so.

1253
00:53:26,460 --> 00:53:28,380
You could also go to Tutor Me Humanities,

1254
00:53:28,380 --> 00:53:31,170
which is another Khanmigo
activity and have a student ask.

1255
00:53:31,170 --> 00:53:33,922
- Sure.
- Kind go what they think.

1256
00:53:33,922 --> 00:53:34,800
But yeah, I think that's a good,

1257
00:53:34,800 --> 00:53:36,720
like higher order skill for the kid

1258
00:53:36,720 --> 00:53:38,880
is just to figure out, you know,

1259
00:53:38,880 --> 00:53:41,460
should I trust Khanmigo's
feedback or my peers feedback?

1260
00:53:41,460 --> 00:53:43,800
Or maybe that's a time
to just ask the teacher.

1261
00:53:43,800 --> 00:53:46,473
- Thank you.
- Yeah. Other questions?

1262
00:53:48,930 --> 00:53:52,470
Okay. All right, so I'm gonna
leave these directions up here

1263
00:53:52,470 --> 00:53:57,470
for you all, and we have
until 11:30, I believe.

1264
00:53:58,080 --> 00:54:00,090
Yep. You have 'em until 11:30 for you all

1265
00:54:00,090 --> 00:54:01,440
to try out these different things,

1266
00:54:01,440 --> 00:54:03,403
log into your Khan Academy
account, play around,

1267
00:54:03,403 --> 00:54:06,120
create assignments, create essays,

1268
00:54:06,120 --> 00:54:07,593
play with the sample essays.

1269
00:54:07,593 --> 00:54:11,110
Just go ahead and we'll tell
you when the 15 minutes is up.

1270
00:54:13,320 --> 00:54:14,250
- Just trying to put in

1271
00:54:14,250 --> 00:54:16,380
like what a student response would be

1272
00:54:16,380 --> 00:54:18,870
and then what are some of
that feedback that comes back.

1273
00:54:18,870 --> 00:54:23,200
And literally, it came up
with I think 16 different

1274
00:54:24,990 --> 00:54:27,450
kind of prompts and things to resolve.

1275
00:54:27,450 --> 00:54:29,520
And as a student, we had to go through

1276
00:54:29,520 --> 00:54:32,100
and we were kinda doing
this work together,

1277
00:54:32,100 --> 00:54:33,990
put in a sample piece of writing,

1278
00:54:33,990 --> 00:54:36,180
looking at all those suggestions,

1279
00:54:36,180 --> 00:54:38,670
and then you have to go
through and like do the work.

1280
00:54:38,670 --> 00:54:40,410
It's not doing any of the work for you.

1281
00:54:40,410 --> 00:54:43,950
It continues to come back and
say, you can have an example.

1282
00:54:43,950 --> 00:54:45,420
Here's an example, what
we're talking about.

1283
00:54:45,420 --> 00:54:46,740
Do it yourself.

1284
00:54:46,740 --> 00:54:49,320
So I think teachers can be confident

1285
00:54:49,320 --> 00:54:51,630
that it's student work in the end.

1286
00:54:51,630 --> 00:54:53,790
And that's what we want
to know is that students,

1287
00:54:53,790 --> 00:54:56,730
they're learning the process,
there's modeling there,

1288
00:54:56,730 --> 00:54:59,280
there's examples and exemplars there,

1289
00:54:59,280 --> 00:55:01,410
but in the end it comes
back to it's gotta be

1290
00:55:01,410 --> 00:55:04,470
that student's work, which I
think is a piece that a lot

1291
00:55:04,470 --> 00:55:06,690
of teachers are gonna
have reluctance with.

1292
00:55:06,690 --> 00:55:09,180
But our first kind of kick of the tires

1293
00:55:09,180 --> 00:55:12,570
and testing was that this is good,

1294
00:55:12,570 --> 00:55:16,320
this is gonna be really great
for teachers and for students.

1295
00:55:16,320 --> 00:55:19,500
I love the ability for
that instant feedback.

1296
00:55:19,500 --> 00:55:23,130
Teachers are very excited that students

1297
00:55:23,130 --> 00:55:26,670
have the ability to get that
feedback on their writing,

1298
00:55:26,670 --> 00:55:31,080
their topic, their level, their
struggles, whatever it is.

1299
00:55:31,080 --> 00:55:34,680
Instantaneously, one of our
teachers said, oh my gosh,

1300
00:55:34,680 --> 00:55:38,640
this feedback would take me
days to give back to students.

1301
00:55:38,640 --> 00:55:39,930
They're getting it instantaneously.

1302
00:55:39,930 --> 00:55:42,060
It is critical.

1303
00:55:42,060 --> 00:55:43,650
And that's something we as teachers

1304
00:55:43,650 --> 00:55:45,033
never have enough time for.

1305
00:55:46,110 --> 00:55:50,010
And it's feedback, the most
important piece is as fast

1306
00:55:50,010 --> 00:55:51,810
as you can get it back to students.

1307
00:55:51,810 --> 00:55:52,890
Accurate, yes.

1308
00:55:52,890 --> 00:55:55,503
And, you know, personalized,

1309
00:55:56,730 --> 00:55:58,860
but timely is that one critical factor.

1310
00:55:58,860 --> 00:56:02,100
And Writing Coach is as timely
as you can possibly get,

1311
00:56:02,100 --> 00:56:03,810
it's almost in instantaneous.

1312
00:56:03,810 --> 00:56:05,130
So that's exciting.

1313
00:56:05,130 --> 00:56:07,830
I really wanna support
teachers in adopting

1314
00:56:07,830 --> 00:56:11,970
those smart AI tools that
helps make teaching better,

1315
00:56:11,970 --> 00:56:13,220
makes it better for student learning.

1316
00:56:13,220 --> 00:56:15,750
At the end, it's all
about student learning.

1317
00:56:15,750 --> 00:56:19,560
And this is an example of AI where really,

1318
00:56:19,560 --> 00:56:22,110
the focus is all on student learning.

1319
00:56:22,110 --> 00:56:25,770
I see very few downsides to this,

1320
00:56:25,770 --> 00:56:27,630
AI deployed in this way.

1321
00:56:27,630 --> 00:56:30,090
It's not just send 'em to ChatGPT

1322
00:56:30,090 --> 00:56:32,880
and say go for it and hope for the best.

1323
00:56:32,880 --> 00:56:34,350
This is very structured.

1324
00:56:34,350 --> 00:56:37,230
It's within Khan, we have oversight.

1325
00:56:37,230 --> 00:56:39,330
Teachers can see the whole process

1326
00:56:39,330 --> 00:56:40,710
and guide that whole process

1327
00:56:40,710 --> 00:56:43,950
and see what student are
getting back from that.

1328
00:56:43,950 --> 00:56:47,700
So if any teachers are kind
of on that fence, whether,

1329
00:56:47,700 --> 00:56:50,280
you know, how's AI gonna help?

1330
00:56:50,280 --> 00:56:52,770
This is a tool that they should be trying,

1331
00:56:52,770 --> 00:56:53,790
- That makes me feel better knowing

1332
00:56:53,790 --> 00:56:56,160
that I'm giving them a tool
as opposed to something

1333
00:56:56,160 --> 00:56:58,260
that'll help them cheat
or just think for them.

1334
00:56:58,260 --> 00:57:01,620
So it definitely helps
them guide their thinking

1335
00:57:01,620 --> 00:57:04,710
and a source that you can
trust that's reliable.

1336
00:57:04,710 --> 00:57:07,380
It's gonna give them
good quality information

1337
00:57:07,380 --> 00:57:10,500
instead of just a bot
auto-populating answers.

1338
00:57:10,500 --> 00:57:12,540
It'll free up a lot of
time for teachers to meet

1339
00:57:12,540 --> 00:57:14,940
with the students who
really need their help

1340
00:57:14,940 --> 00:57:16,740
and they can gauge which ones are grasping

1341
00:57:16,740 --> 00:57:20,250
the concepts more easily
and spend more time on that

1342
00:57:20,250 --> 00:57:23,820
rather than checking all of
the papers, all the steps.

1343
00:57:23,820 --> 00:57:28,230
- The Writing Coach is an
extremely powerful tool

1344
00:57:28,230 --> 00:57:31,920
that will help students and teachers

1345
00:57:31,920 --> 00:57:34,200
begin the process of writing.

1346
00:57:34,200 --> 00:57:36,210
Not only in the feedback process,

1347
00:57:36,210 --> 00:57:40,680
but 20 years in the classroom,

1348
00:57:40,680 --> 00:57:44,640
one of the most common questions
I get from students is,

1349
00:57:44,640 --> 00:57:45,663
how do I start this?

1350
00:57:46,673 --> 00:57:51,330
And the writing tool for Khan
Academy really takes them

1351
00:57:51,330 --> 00:57:53,370
through that process of how to start it.

1352
00:57:53,370 --> 00:57:56,850
And if they have that
question, how do I start this?

1353
00:57:56,850 --> 00:57:59,670
The fields are there, Khanmigo is there

1354
00:57:59,670 --> 00:58:02,790
to help them brainstorm,
begin that process.

1355
00:58:02,790 --> 00:58:06,000
The assignment is there,
so it's all in one place,

1356
00:58:06,000 --> 00:58:08,100
which I absolutely love.

1357
00:58:08,100 --> 00:58:11,670
Plus it's probably in
whatever learning system

1358
00:58:11,670 --> 00:58:14,100
that they have, like Google Classrooms.

1359
00:58:14,100 --> 00:58:16,700
So I think that is super,
super helpful for teachers

1360
00:58:17,700 --> 00:58:18,720
and students alike.

1361
00:58:18,720 --> 00:58:23,070
One-on-one because there
is feedback being given,

1362
00:58:23,070 --> 00:58:25,110
it gives the teacher some time

1363
00:58:25,110 --> 00:58:28,530
to have more directed one-on-one time

1364
00:58:28,530 --> 00:58:32,520
into the deeper elements
of the paper as opposed

1365
00:58:32,520 --> 00:58:36,060
to more superficial stuff
like format or paragraphs,

1366
00:58:36,060 --> 00:58:37,503
or those things.