0:00:07.173,0:00:09.134 Hi. Welcome to session three 0:00:09.134,0:00:13.346 of the AI 101 for Teachers[br]Professional Learning Series. 0:00:13.430,0:00:16.182 In this session[br]we are traveling to the Wharton School 0:00:16.182,0:00:19.269 at the University of Pennsylvania[br]to chat with Dr. 0:00:19.269,0:00:24.482 Ethan Mollick, a professor who teaches[br]innovation and entrepreneurship, and Dr. 0:00:24.482,0:00:29.279 Lilach Mollick, who works on[br]interactive pedagogy and AI research. 0:00:29.362,0:00:32.615 They will help us explore[br]how AI can be combined 0:00:32.615,0:00:36.119 with pedagogy to enhance student learning. 0:00:36.202,0:00:37.495 Let's go meet the Mollicks. 0:00:43.752,0:00:45.128 Hi I’m Ethan Mollick 0:00:45.128,0:00:48.882 a professor at Wharton[br]who has been working on how we democratize 0:00:48.882,0:00:54.054 access to education through tools[br]like games and interactive tools and AI 0:00:54.137,0:00:55.180 and I’m Lilach Mollick 0:00:55.180,0:00:57.265 I'm director of Pedagogy[br]at Wharton Interactive, 0:00:57.265,0:00:58.892 and I've been working at the intersection 0:00:58.892,0:01:02.812 of AI and education,[br]helping to democratize education 0:01:02.812,0:01:06.858 for everyone through effective,[br]pedagogically sound use of AI. 0:01:06.941,0:01:11.112 And we have been working together[br]on the future of education for a while, 0:01:11.112,0:01:14.908 thinking about how to make education[br]more interactive, to work at scale. 0:01:15.116,0:01:18.828 And with the advent of general AI,[br]we found a powerful new tool 0:01:18.995,0:01:22.248 that can really help in the classroom[br]but also carries some risks. 0:01:22.457,0:01:24.876 Today, we'd like to talk a little bit[br]about the classroom 0:01:24.876,0:01:28.755 use of AI upsides and downsides[br]to give you some examples to work with. 0:01:28.838,0:01:31.758 But first, we like to start[br]with our three guiding principles. 0:01:31.758,0:01:33.968 The first is that AI is undetectable. 0:01:33.968,0:01:38.431 There are AI tools, detection tools,[br]but they are not effective. 0:01:38.515,0:01:41.768 The second principle[br]is that AI is ubiquitous. 0:01:41.768,0:01:43.353 It's everywhere. 0:01:43.353,0:01:46.731 169 countries have access to Bing Chat[br]and you and 0:01:46.731,0:01:50.610 your students have access[br]to the most powerful AI available. 0:01:50.693,0:01:53.822 The third principle[br]is that AI is transformative. 0:01:53.863,0:01:58.576 It will transform how we live,[br]how we work, and how we teach and learn. 0:02:04.958,0:02:06.251 Not only is 0:02:06.251,0:02:09.796 AI not going[br]away, but this is probably the worst AI 0:02:09.796,0:02:10.672 you're ever going to use. 0:02:10.672,0:02:14.509 So if this feels disruptive[br]now kind of have bad news, which is that 0:02:14.509,0:02:17.137 there isn't a reason to suspect[br]that AI development will not continue. 0:02:17.137,0:02:17.887 And I think people worry 0:02:17.887,0:02:21.850 a lot about like the far future[br]or who knows how far it is where AI 0:02:21.850,0:02:22.600 smarter than humans. 0:02:22.600,0:02:26.563 But even over the next couple of years,[br]even with this fact of this academic year, 0:02:26.688,0:02:28.898 I would expect AI[br]to continue to improve. 0:02:28.898,0:02:31.151 Five times. Ten times? We have no idea. 0:02:31.151,0:02:34.112 But if you're not already thinking[br]about these systems, what they mean 0:02:34.112,0:02:37.115 for education, what they mean for you,[br]what they mean for your students careers. 0:02:37.115,0:02:40.493 I think we have to think about it because[br]these systems are not going to disappear. 0:02:40.618,0:02:44.581 Let me make the very pragmatic case[br]for why you may want to do this. 0:02:44.664,0:02:47.917 The first part of the pragmatic case is[br]your students are using this anyway. 0:02:48.126,0:02:49.377 So you have to come up to speed. 0:02:49.377,0:02:51.796 I don't think everybody[br]wants to be dragged along this technology. 0:02:51.796,0:02:54.132 No one asked for education[br]to be massively disrupted. 0:02:54.132,0:02:54.966 But it is. 0:02:54.966,0:02:57.385 And unfortunately, we've got to figure out[br]a way to get around that. 0:02:57.385,0:03:00.471 All your homework assignments can be done[br]by AI now, so you have to think about that. 0:03:00.471,0:03:04.642 And then I think the second thing[br]is a pragmatic argument about how AI 0:03:04.642,0:03:06.060 can make your life easier as a teacher. 0:03:06.060,0:03:08.021 If you put the hours down,[br]you get them back later. 0:03:08.021,0:03:09.439 And then if you've worked out[br]a number of prompts 0:03:09.439,0:03:11.649 to help make lives easier for teachers. 0:03:11.649,0:03:14.777 Yes. So one thing you can do prompts[br]like give me a lesson 0:03:14.777,0:03:19.574 hook prompts like create a lesson plan[br]or create a quiz for me. 0:03:19.782,0:03:24.078 So starting to work with your material[br]and the different models could get you 0:03:24.120,0:03:28.082 give you a really good sense of how the[br]AI works, what it's good at, what it 0:03:28.166,0:03:32.003 what it's not good at. And so[br]and save you time in the end, right? 0:03:32.003,0:03:33.171 So that's where I would be 0:03:33.171,0:03:37.383 my push to teachers is A[br]you have to and B you're going to want to. 0:03:37.467,0:03:40.470 I think the other important thing[br]is just to try it. 0:03:40.678,0:03:42.055 They're very simple to use. 0:03:42.055,0:03:45.391 They're very intuitive[br]because they're conversational. 0:03:45.475,0:03:49.270 You can continue a conversation[br]and it feels fairly natural. 0:03:49.479,0:03:52.690 And I think the key really[br]is experimentation. 0:03:52.774,0:03:56.527 See how it works with you,[br]see how it works within your context, 0:03:56.611,0:03:58.613 within your topic that you teach. 0:03:58.613,0:04:01.115 Our rule of thumb is[br]you need about 10 hours time with 0:04:01.115,0:04:03.952 AI to get what it's good at,[br]what its limitations are. 0:04:03.952,0:04:07.705 So I would actually start by suggesting[br]that this that the teacher throw 0:04:07.705,0:04:10.708 their own assignments into the AI and see[br]what kind of results they get back. 0:04:10.875,0:04:14.254 I would think about asking them[br]to ask their students 0:04:14.254,0:04:18.049 to create an assignment using AI[br]and then critique that assignment, 0:04:18.216,0:04:20.176 potentially even in class, to see 0:04:20.176,0:04:23.680 if the students can get a sense[br]of what the gaps and abilities of AI are. 0:04:23.721,0:04:26.099 I have a little bit of freedom[br]as an instructor 0:04:26.099,0:04:29.727 because I'm teaching college[br]and MBA students entrepreneurship. 0:04:29.727,0:04:30.728 So I have a lot of 0:04:30.728,0:04:33.106 I have points I want them to make,[br]but they also are building things 0:04:33.106,0:04:36.943 and doing things[br]and absolutely transformed how that works. 0:04:36.943,0:04:38.695 So my assignments now 0:04:38.695,0:04:41.781 literally call for students to do this one[br]impossible thing in class. 0:04:41.781,0:04:44.367 If you can't code,[br]you have to write working programs. 0:04:44.367,0:04:47.245 If you have never if you can't do design[br]work, you have to create a full 0:04:47.245,0:04:50.748 graphic design working prototypes[br]that's literally now part of the class. 0:04:50.748,0:04:52.333 So where it used to be, write it write 0:04:52.333,0:04:54.669 a little bit of an essay,[br]do a prototype on paper. 0:04:54.669,0:04:56.713 Now you have to create a full working[br]product. 0:04:56.713,0:04:58.381 Every assignment that is written 0:04:58.381,0:05:00.216 has to be critiqued[br]by at least five famous 0:05:00.216,0:05:02.969 entrepreneurs through history,[br]and they use AI to invoke those. 0:05:02.969,0:05:04.304 There's a pedagogical reason, too, 0:05:04.304,0:05:06.597 which is that entrepreneurs[br]tend to be overconfident. 0:05:06.597,0:05:08.725 So you want feedback[br]from different sources. 0:05:08.725,0:05:12.186 So to me it is[br]let me teach ten times more than I did. 0:05:12.186,0:05:14.897 I used to teach an advanced[br]intermedia entrepreneurship course. 0:05:14.897,0:05:17.233 I can now, in the intermediate[br]or basic course, 0:05:17.233,0:05:19.652 get all the way[br]past the advanced material and further. 0:05:19.652,0:05:22.155 So I think we're going to see that[br]shake out more in the future. 0:05:22.155,0:05:25.116 But some of this is about powering past[br]what we could do before 0:05:25.116,0:05:26.993 and I think that's exciting as well. 0:05:26.993,0:05:32.040 Apart from student tutors[br]as assignments, teachers can certainly use 0:05:32.123,0:05:36.753 AI coaches, AI assistants[br]to help students prepare for discussions, 0:05:37.003,0:05:41.591 help students outline, help students[br]do research, help students get feedback 0:05:41.591,0:05:44.719 on assignments, and just help students[br]develop explanations. 0:05:44.719,0:05:50.183 I think there are myriad of approaches[br]that are pedagogically sound 0:05:50.266,0:05:51.768 that teachers can assign to 0:05:51.768,0:05:56.481 students and watch their work[br]and ask for the back and forth interaction 0:05:56.481,0:06:00.109 to really see that students are paying[br]attention to and focusing on the material. 0:06:05.907,0:06:07.325 Let's talk a little bit about AI 0:06:07.325,0:06:08.409 from a teacher's perspective. 0:06:08.409,0:06:11.162 So because of the ubiquity of AI, 0:06:11.162,0:06:14.874 you've got some choices to make in terms[br]of your AI policies in your class. 0:06:14.874,0:06:18.127 So do you want to permit[br]I do want to forbid AI 0:06:18.252,0:06:20.004 How are you going[br]to enforce these sorts of things? 0:06:20.004,0:06:21.756 We're going to assume[br]that you want to use AI 0:06:21.756,0:06:25.134 to some extent, and we'll dive[br]into a little bit of the details here. 0:06:25.134,0:06:27.720 So as a instructor,[br]you should know a few things. 0:06:27.720,0:06:30.973 One is there is obviously ongoing[br]ethical debates about AI, 0:06:30.973,0:06:32.266 and those are complicated debates. 0:06:32.266,0:06:34.143 There are debates over whether or not 0:06:34.143,0:06:37.063 the AI's trained on the right kind of data[br]about the biases 0:06:37.063,0:06:41.984 I might have about the use of AI[br]and the outcomes for student learning. 0:06:42.193,0:06:44.153 And it's worth[br]acknowledging these sets of things. 0:06:44.153,0:06:48.032 But this tool is out there and it is worth[br]thinking about how you want to use it. 0:06:48.116,0:06:51.077 If you decide that that is okay[br]and how you want to communicate 0:06:51.077,0:06:54.205 that information[br]beyond the initial ethical concerns, 0:06:54.372,0:06:56.707 there's also concerns[br]about how AI actually works. 0:06:56.707,0:07:00.128 So the large language models that power 0:07:00.128,0:07:03.256 today’s AI don't actually have knowledge[br]of the world. 0:07:03.256,0:07:04.632 They're predicting the next word. 0:07:04.632,0:07:07.718 They're predicting the right kinds[br]of sentences or information to give out. 0:07:07.927,0:07:11.639 And as a result, they make stuff up[br]they hallucinate. 0:07:11.639,0:07:13.641 So there are often errors or mistakes. 0:07:13.641,0:07:15.643 Now, it's not always clear[br]those errors or mistakes are worse 0:07:15.643,0:07:17.353 than the errors and mistakes[br]humans would make. 0:07:17.353,0:07:18.855 But you need to be aware that there's 0:07:18.855,0:07:20.690 going to be[br]those kind of errors and mistakes. 0:07:20.690,0:07:23.860 And then finally,[br]you need to think about as an instructor 0:07:24.068,0:07:26.946 how you're going to be using[br]AI to aid learning, 0:07:26.946,0:07:30.825 which means being really clear about what[br]you want to accomplish with an AI tool. 0:07:31.075,0:07:33.244 They can be used for student learning, 0:07:33.244,0:07:35.413 but AI's many possible[br]uses in the classroom, 0:07:35.413,0:07:37.790 so do you want to use them[br]to have student’s generate ideas, 0:07:37.790,0:07:41.085 which I do in my classes and get better[br]project ideas as a result. 0:07:41.169,0:07:44.589 Do you want them to use them as tutors[br]to explain concepts 0:07:44.589,0:07:45.756 to them they don't understand? 0:07:45.756,0:07:48.426 Do you want the students[br]to get feedback from AI 0:07:48.426,0:07:51.679 by asking for questions about work[br]that they're doing. 0:07:51.804,0:07:53.556 Do you want to be a writing companion? 0:07:53.556,0:07:56.559 Do you want it to explain[br]why quiz answers might be right or wrong? 0:07:56.684,0:07:58.728 And then once you've decided[br]what who's instructor, 0:07:58.728,0:08:02.231 you decide what you're[br]going to tell your students. 0:08:07.570,0:08:09.572 AI detectors don't work. 0:08:09.572,0:08:10.281 They just don't work. 0:08:10.281,0:08:12.116 You shouldn't use them.[br]And it's worse than them 0:08:12.116,0:08:15.286 not working because they have a high false[br]positive rate. 0:08:15.286,0:08:18.581 That means they select things[br]that’s AI written that aren't AI written 0:08:18.748,0:08:23.169 and that disproportionately falls on[br]people whose English is a second language. 0:08:23.252,0:08:24.795 This is just not something that we can do. 0:08:24.795,0:08:28.299 And I think trying to close the barn door[br]here after it's been opened 0:08:28.382,0:08:33.471 and try and detect AI is not the future[br]for responsibility in classrooms. 0:08:33.554,0:08:37.058 The other thing to note too,[br]is that students 0:08:37.141,0:08:39.060 were using shortcuts in the past. 0:08:39.060,0:08:40.603 It's not that they weren't using Google, 0:08:40.603,0:08:43.105 it's not that they weren't using,[br]you know, other students essays. 0:08:43.105,0:08:46.150 This was happening in the past,[br]but this is a major disruption, 0:08:46.359,0:08:50.446 and I think it does call for a rethinking[br]of how we do essays. 0:08:50.446,0:08:54.200 So thinking a little bit more[br]about the learning goal for an essay 0:08:54.200,0:08:55.993 or the learning goal for any assignment, 0:09:02.416,0:09:05.586 One of the things that we're noticing[br]as we watch teachers do this 0:09:05.586,0:09:09.298 is they all feel an obligation[br]to talk about AI and dive 0:09:09.298,0:09:12.301 deep into the ethical implications of AI[br]and so on. 0:09:12.343,0:09:14.554 I think that's important,[br]but I don't think that needs 0:09:14.554,0:09:15.930 to be the theme of every class. 0:09:15.930,0:09:18.933 I don't think every class needs[br]to be a discussion about AI, 0:09:19.016,0:09:21.686 just like every class that uses computers[br]doesn't need to be a discussion 0:09:21.686,0:09:24.272 about computers, I think is important[br]to have that conversation. 0:09:24.272,0:09:25.189 And right now 0:09:25.189,0:09:28.234 we're all just reacting so it's not clear[br]who's supposed to have that. 0:09:28.234,0:09:30.361 So I totally get teachers[br]wanting to have AI 0:09:30.361,0:09:33.906 discussions,[br]but it's even harder to get up to speed, 0:09:33.906,0:09:36.075 not just on the use of AI,[br]but how it works. 0:09:36.075,0:09:39.161 It's, you know, standards,[br]its ethical implications. 0:09:39.412,0:09:43.332 So I think teachers should feel[br]a little bit of okay ness 0:09:43.332,0:09:46.794 with experimenting with AI without having[br]to make it the subject of class. 0:09:53.301,0:09:56.304 First is[br]as Ethan mentioned, that AI can fabricate. 0:09:56.304,0:10:00.099 That means that any output[br]that the AI gives a student may be made 0:10:00.099,0:10:04.395 up, it may be mistaken,[br]it may be very subtly mistaken. 0:10:04.395,0:10:07.481 And so students should be responsible[br]for their own work. 0:10:07.690,0:10:11.027 They should at the very least check[br]sources, check any number, 0:10:11.027,0:10:14.947 check any facts that the AI gives them[br]and check them with credible sources. 0:10:15.031,0:10:18.326 The second principle is that[br]the AI is not a person. 0:10:18.326,0:10:21.996 It's easy to imbue the AI[br]with a personality or to feel like you're 0:10:21.996,0:10:25.958 talking to a person, but it's not a person[br]and it doesn't know you. 0:10:26.042,0:10:29.378 The third principle is really[br]to give it a lot of context. 0:10:29.378,0:10:30.921 The AI doesn't know you. 0:10:30.921,0:10:34.342 It doesn't know your context[br]or your experience or your expertise. 0:10:34.425,0:10:38.012 The more context you give it,[br]the more useful it'll be for you. 0:10:38.054,0:10:40.806 And the fourth principle[br]is that you're in charge. 0:10:40.806,0:10:42.391 Not only should you evaluate 0:10:42.391,0:10:46.312 and interrogate its output,[br]but if it's leading you in a conversation 0:10:46.395,0:10:49.732 that is no longer useful to you,[br]or if it's stuck in a loop, 0:10:49.815,0:10:52.693 or if you'd like to change the direction[br]of the conversation, 0:10:52.693,0:10:55.196 you should absolutely[br]feel free to take charge. 0:11:01.786,0:11:04.413 So when we talk about AI 0:11:04.413,0:11:07.333 and these generative AI solutions, 0:11:07.333,0:11:09.335 we tend to talk about large language[br]models. 0:11:09.335,0:11:10.461 And there's actually only 0:11:10.461,0:11:13.923 a few large scale[br]general purpose, large language models. 0:11:13.923,0:11:18.928 There is the models created by OpenAI,[br]which are GPT 3.5 or GPT 4 0:11:19.136,0:11:23.057 GPT 3.5 is the free version[br]that you get through 0:11:23.140,0:11:26.352 through Chat GPT[br]and GPT 4 is either 0:11:26.352,0:11:29.647 through the paid Chat GPT or through[br]Microsoft Bing in creative mode. 0:11:29.855,0:11:34.527 And when we talk about specialized apps,[br]almost all of them are using 0:11:34.527,0:11:38.739 one of these models and providing prompts[br]and other information on top of it. 0:11:38.823,0:11:42.368 I generally think instructors[br]should get familiar with the models 0:11:42.368,0:11:46.664 themselves because those are the models[br]that are actually producing the answers 0:11:46.831,0:11:49.375 and you can manipulate them directly[br]that way and learn how they work. 0:11:49.375,0:11:51.377 So if you're trying to buy[br]an off the shelf solution, 0:11:51.377,0:11:53.796 they're almost certainly[br]using one of these existing models 0:11:53.796,0:11:57.258 and then providing some sort of wrapper[br]or other information on top of it, 0:11:57.466,0:12:00.761 and it's often cheaper and more effective[br]and gives you more control 0:12:00.886,0:12:02.680 to use the foundation models yourself. 0:12:02.680,0:12:04.223 But that's a choice you get to make. 0:12:04.223,0:12:09.311 So when developing the prompt,[br]we really and for all of our prompts, 0:12:09.520,0:12:11.814 we really look at the science of learning 0:12:11.814,0:12:14.817 and try to combine[br]that with the power of the AI. 0:12:14.900,0:12:19.071 So for instance,[br]a good tutor pushes you for information. 0:12:19.155,0:12:21.657 It doesn't just hand to you, a good tutor 0:12:21.657,0:12:26.287 finds out what you know[br]and builds on that prior knowledge. 0:12:26.370,0:12:29.623 A good tutor will also find out[br]a little bit about you. 0:12:29.623,0:12:34.128 A good tutor also knows that you need lots[br]and varied kinds of examples 0:12:34.211,0:12:35.838 and analogies, 0:12:35.838,0:12:40.509 and a good tutor knows that the way to[br]you show evidence of mastery 0:12:40.509,0:12:43.512 is by being able[br]to explain something in your own words 0:12:43.512,0:12:48.392 to someone else and give an example of it,[br]which is exactly these are exactly 0:12:48.392,0:12:52.146 the steps in the kinds of questions[br]that we use in the tutor prompt. 0:12:52.396,0:12:54.398 But you'll notice[br]when you look at our prompts 0:12:54.398,0:12:58.569 that they do things like provide context[br]the AI as life has discussed already, 0:12:58.569,0:13:02.656 the idea that it asks you who you are,[br]and we tell the AI who it is. 0:13:02.656,0:13:04.867 It's an instructor[br]with this kind of setting, 0:13:04.867,0:13:07.703 you'll notice that it also tells it[br]exactly 0:13:07.703,0:13:11.415 the scientific framework[br]to use this idea of context matters. 0:13:11.499,0:13:12.666 We provide controls. 0:13:12.666,0:13:16.629 We ask it to go step by step through[br]sets of questions to ask, sometimes 0:13:16.629,0:13:20.049 not in these prompts, we provide examples[br]of good output, and then we tested a lot. 0:13:20.257,0:13:23.803 You can't do prompting without testing,[br]and that's one of the great things 0:13:23.803,0:13:25.387 about testing your error expertise. 0:13:25.387,0:13:26.680 It's cheap to do 0:13:26.680,0:13:30.142 and so you get to experiment a lot[br]and that makes for good prompts. 0:13:30.309,0:13:34.563 And we should also say we test it not[br]just on one model but on several models. 0:13:34.563,0:13:37.608 So for instance, these two prompts[br]we just worked 0:13:37.608,0:13:40.778 with ChatGPT 4 for,[br]they also work with Bing. 0:13:40.903,0:13:44.865 Bing will react a little bit differently[br]and it will 0:13:44.949,0:13:46.408 because it's connected to the Internet. 0:13:46.408,0:13:48.369 It will also look up citations sometimes. 0:13:48.369,0:13:50.246 So the right citation,[br]sometimes they're not. 0:13:50.246,0:13:51.997 But that is available. 0:13:51.997,0:13:54.416 It may or may not work[br]with some of the other models. 0:13:54.416,0:13:55.626 So you really have to test it, 0:13:55.626,0:13:58.629 I think, as an instructor[br]before you give it to your students 0:13:58.838,0:14:02.383 in the context of the topic[br]that you're teaching to see how it works. 0:14:02.466,0:14:05.386 So this is all very theoretical,[br]but I think important. 0:14:05.386,0:14:06.512 So let's let's get practical. 0:14:06.512,0:14:08.806 Let's talk about some examples[br]of what AI can do. 0:14:08.806,0:14:13.227 And again, this isn't a monolithic thing,[br]a has many possible uses. 0:14:13.227,0:14:14.812 As we said, it was transformative earlier. 0:14:14.812,0:14:17.523 So we're going to show you[br]a couple of prompts that we've created 0:14:17.523,0:14:21.193 and those will be available to you[br]as well to work with 0:14:21.277,0:14:25.489 and these are just examples of the ways[br]AI classroom use can work. 0:14:25.489,0:14:29.577 So the first one we want to show[br]you is a prompt that I believe you created 0:14:29.660,0:14:33.122 that talks about feedback[br]that gives proper feedback. 0:14:33.205,0:14:36.750 And one of the really interesting things[br]about the AI side of things 0:14:36.750,0:14:38.168 is a more sophisticated prompt 0:14:38.168,0:14:39.712 that takes into account[br]some of the principles 0:14:39.712,0:14:41.922 we were talking about[br]earlier will result in better outcomes. 0:14:41.922,0:14:44.425 So students will often[br]ask for writing advice from an AI 0:14:44.425,0:14:45.968 even if you tell them not to do it. 0:14:45.968,0:14:48.220 But they're in ask for it in a way[br]that's fairly unsophisticated 0:14:48.220,0:14:51.849 and is going to give them fairly generic[br]sounding work and possibly more mistakes. 0:14:51.932,0:14:54.810 If you give a more elaborate prompt,[br]you can get more elaborate answers. 0:14:54.810,0:14:58.814 So in this case, could you explain what[br]this prompt does, the feedback prompt. Yes. 0:14:58.814,0:15:03.068 So we combine[br]the principles of good feedback, 0:15:03.068,0:15:06.363 which is feedback that takes into account[br]your prior knowledge 0:15:06.363,0:15:10.075 or what you already know from the student[br]perspective takes into account 0:15:10.075,0:15:13.537 who you are, your learning level,[br]what grade you're in, 0:15:13.621,0:15:16.624 whether you're in college[br]or you're in a professional. 0:15:16.665,0:15:21.337 And it also takes into account the idea[br]that you want to respond to this feedback. 0:15:21.337,0:15:24.548 So it is going to be actionable,[br]it's going to be balanced, 0:15:24.548,0:15:28.260 it's going to tell you what's wrong[br]and what you can improve on 0:15:28.344,0:15:31.180 and what you're doing well,[br]and it's going to keep working with you. 0:15:31.180,0:15:35.351 But like any good tutor or coach,[br]it won't actually give you the answer. 0:15:35.517,0:15:37.102 It'll push you in that direction, 0:15:37.102,0:15:40.773 ask you to explain,[br]ask you to construct your own knowledge. 0:15:40.856,0:15:44.234 And so you can see the prompt[br]hopefully on the screen here 0:15:44.318,0:15:46.904 and as a place to work from. 0:15:46.904,0:15:49.573 You don't need to take this[br]as an absolute answer. 0:15:49.573,0:15:52.076 This is something you can play with,[br]but let's see it in action. 0:15:52.076,0:15:53.953 So let's get started using this prompt. 0:15:53.953,0:15:57.206 It says that it's a teaching assistant[br]because that the instructions we gave it 0:15:57.373,0:16:00.793 and it asks us for our grade[br]level and subject we're studying, 0:16:00.793,0:16:01.961 what should we say? 0:16:01.961,0:16:05.255 So I think we're studying Macbeth[br]and we're in 12th grade. 0:16:05.339,0:16:11.387 In 12th grade. Okay, great. 0:16:11.470,0:16:12.262 Okay. 0:16:12.262,0:16:14.974 And so we've told the AI this information[br]it’s feeding it 0:16:14.974,0:16:17.977 into the logic that it's using here. 0:16:18.060,0:16:20.771 And it's asking us about a specific assignment 0:16:20.771,0:16:23.440 and it's asking if we have a rubric[br]or other information to work with or what 0:16:23.440,0:16:26.860 we're hoping to achieve[br]with with as much information as possible. 0:16:26.860,0:16:30.948 I don't have a huge amount here,[br]so I'll say I have to write 0:16:31.031,0:16:35.411 an analysis of Macbeth. 0:16:35.494,0:16:38.664 It is graded 0:16:38.747,0:16:40.916 based on 0:16:40.916,0:16:45.921 rating style and depth of content 0:16:46.005,0:16:48.799 and you'll see what it's going[br]is it's asking ask questions 0:16:48.799,0:16:50.259 and soliciting information from us, 0:16:50.259,0:16:53.262 which makes it kind of a good prompt[br]that you might hand a student 0:16:53.303,0:16:56.974 better than one that is just they're just[br]developing themselves and it's asking 0:16:56.974,0:17:00.352 about specific instructions[br]and ask us to share the assignment. 0:17:00.436,0:17:05.190 Here is what I have written so far, 0:17:05.274,0:17:05.566 and I 0:17:05.566,0:17:08.777 have asked[br]the I to generate a Macbeth essay. 0:17:08.777,0:17:10.696 So here we go. 0:17:10.696,0:17:13.323 I'm just pasting that in 0:17:13.323,0:17:14.283 and we'll see what it says here. 0:17:14.283,0:17:17.453 And you'll notice[br]it's it's working on the information. 0:17:17.453,0:17:19.455 It's saying it's taking time[br]to carefully read through it. 0:17:19.455,0:17:20.706 That's a bit of illusion. 0:17:20.706,0:17:24.043 It's obviously not taking extra time,[br]but it's responding in this method 0:17:24.043,0:17:26.378 and you'll see it's giving a set[br]of strengths and weaknesses. 0:17:26.378,0:17:27.629 What's great about, again, 0:17:27.629,0:17:31.008 using a tutor that you've built[br]or a mentor that you built is that it 0:17:31.008,0:17:33.677 can give you the kind of feedback[br]that's educationally valuable 0:17:33.677,0:17:37.598 that ties into pedagogy rather than just[br]students asking Make this essay better. 0:17:37.806,0:17:42.394 An example of a working in your favor[br]as an educator 0:17:42.394,0:17:46.148 and not necessarily working against you[br]and undermining the point you're making. 0:17:46.231,0:17:49.359 So you'll notice at the end, by the way,[br]it gives a question 0:17:49.359,0:17:51.695 that for the students to answer. 0:17:51.695,0:17:54.490 So how do you plan to revise[br]your analysis? 0:17:54.490,0:17:56.825 Give me a plan[br]and specific changes are going again. 0:17:56.825,0:17:58.452 The kind of thing[br]we would do as an instructor 0:17:58.452,0:18:01.538 in a classroom[br]soliciting changes or differences. 0:18:01.538,0:18:05.751 So I think you can start to see why a 0:18:05.834,0:18:06.543 tool like 0:18:06.543,0:18:09.546 this can be really useful[br]when properly applied. 0:18:09.713,0:18:13.509 Now, let's also talk about one[br]other potential use for AI, AI as a tutor. 0:18:13.509,0:18:16.178 What are some of the advantages[br]and disadvantages of that approach? 0:18:16.178,0:18:17.471 So an advantage of this approach 0:18:17.471,0:18:21.016 is that you're getting students[br]to actually pay attention to the material. 0:18:21.225,0:18:24.728 You're getting them[br]to read over the rubric, to read over 0:18:24.853,0:18:29.108 the purpose of the essay and the audience,[br]and to really think through it. 0:18:29.191,0:18:31.860 A disadvantage is that you 0:18:31.860,0:18:36.448 you certainly can ask the AI[br]to do it for you, but if you work with it 0:18:36.448,0:18:40.452 and if you're given guidelines to work[br]with it, it's one way to get feedback. 0:18:40.452,0:18:42.746 But you would then have to evaluate 0:18:42.746,0:18:46.375 something else that a teacher could do[br]is to ask for the interactions 0:18:46.375,0:18:48.585 and ask for a reflection[br]about the interactions. 0:18:48.585,0:18:49.211 What about this 0:18:49.211,0:18:50.129 feedback was good? 0:18:50.129,0:18:52.297 What about this feedback was not as good? 0:18:52.297,0:18:56.385 And again, it's a higher order level[br]thinking about your essay 0:18:56.468,0:18:59.346 and your process. Excellent. 0:18:59.346,0:19:04.143 And so why don't you do one other example[br]where the AI acts as a direct instructor 0:19:04.226,0:19:06.019 and we have a prompt for that as well. 0:19:06.019,0:19:09.648 There are risks associated with asking[br]the AI to be a direct instructor, 0:19:09.648,0:19:11.441 which is that hallucination risk. 0:19:11.441,0:19:14.695 It doesn't know your pedagogy,[br]your your, your perspective. 0:19:14.945,0:19:17.948 But I find in my classrooms[br]that students are increasingly 0:19:17.948,0:19:22.369 using the AI as a method of learning[br]so they don't raise their hands as much. 0:19:22.369,0:19:23.620 So when I ask them why, they're like,[br]Well, 0:19:23.620,0:19:25.205 I'd rather not show my ignorance in class, 0:19:25.205,0:19:27.916 I could ask the AI to explain[br]like I'm five. 0:19:27.916,0:19:29.626 So they're already engaged[br]in this behavior. 0:19:29.626,0:19:32.880 So something like a tutor[br]both does a useful thing 0:19:32.880,0:19:36.216 of showing you what the future of AI[br]education might look like. 0:19:36.300,0:19:39.636 Like the way Khan Academy is building[br]AI interactive tutors to work. 0:19:39.803,0:19:43.432 And it also might be a tool your students[br]can use to achieve more in class, 0:19:43.682,0:19:45.475 but you should caveat that[br]with the knowledge 0:19:45.475,0:19:48.353 that AI tutors are not 100% there yet. 0:19:48.353,0:19:49.771 But let's let's use an example here. 0:19:49.771,0:19:52.441 So this tutor is again[br]trying to take the right kind of format. 0:19:52.441,0:19:55.444 It says, Hello there, I'm your AI tutor[br]and I'm excited to work with you today. 0:19:55.694,0:19:57.237 What do we want to learn about today? 0:19:57.237,0:19:59.114 Let's learn about opportunity cost. 0:19:59.114,0:20:02.117 Opportunity cost.[br]A concept from economics. 0:20:02.201,0:20:04.286 Let's see what happens. 0:20:04.328,0:20:06.622 Okay, so we're telling the AI[br]or the opportunity cost 0:20:06.622,0:20:09.625 that saying it is a key concept economics 0:20:09.666,0:20:12.461 it’s even throwing in a little emoji here,[br]which is cute. 0:20:12.461,0:20:14.129 Can you ask us about our learning level? 0:20:14.129,0:20:15.547 What level are we at here? 0:20:15.547,0:20:18.342 11th grade. 11th grade. 0:20:18.342,0:20:21.428 Now, I wouldn't get too[br]tied up on the individual grade. 0:20:21.428,0:20:22.471 It's not amazing 0:20:22.471,0:20:24.556 at differentiating a 10th grader[br]from an 11th grader, 0:20:24.556,0:20:26.308 but this is part of the context[br]in which it's working in. 0:20:26.308,0:20:31.104 So that pulling from some sort of[br]universal standards here and it says, 0:20:31.146,0:20:33.899 what do we know about opportunity cost? 0:20:33.899,0:20:37.527 So we know that it has to do[br]with alternative choices, 0:20:37.611,0:20:41.865 has to do with alternative choices. 0:20:41.949,0:20:43.158 That is it. 0:20:43.158,0:20:44.826 And of course, [br]one of the advantages of AI 0:20:44.826,0:20:46.078 is this kind of freeform text 0:20:46.078,0:20:49.665 and interaction[br]is the real power of education 0:20:49.748,0:20:51.917 and it's something the AI can fake[br]reasonably well. 0:20:51.917,0:20:54.878 Again, not as well as a real human[br]instructor yet. 0:20:54.878,0:20:57.881 And you'll notice it's giving us examples 0:20:58.006,0:21:02.386 and explaining things in different ways,[br]which is a powerful thing that AI can do. 0:21:02.386,0:21:04.721 It's very good[br]at breaking things down in different ways, 0:21:04.721,0:21:07.182 but you'll notice that it's[br]they're starting to ask questions. 0:21:07.182,0:21:09.184 It's asking us to make choices. 0:21:09.184,0:21:11.603 So one of things[br]we know from the research on tutoring is 0:21:11.603,0:21:13.855 you can't just declaim things to people. 0:21:13.855,0:21:16.441 The value of tutoring comes[br]from soliciting information, 0:21:16.441,0:21:17.276 making connections. 0:21:17.276,0:21:18.860 And you can see the AI[br]starting to do this 0:21:18.860,0:21:21.071 and asking us for connections[br]in our own life. 0:21:21.071,0:21:24.366 The other thing to mention too,[br]about the tutor prompt is that it 0:21:24.366,0:21:29.830 is not assuming that the student[br]can judge their own learning. Very often 0:21:29.955,0:21:32.708 you'll see in a tutor prompt[br]that is very simple. 0:21:32.708,0:21:34.251 Like explain to me like on ten, 0:21:34.251,0:21:36.837 it'll ask you[br]if you'll understand something instead. 0:21:36.837,0:21:37.546 here 0:21:37.546,0:21:39.131 It's not asking you to make a judgment 0:21:39.131,0:21:41.800 about your own learning,[br]which we know is inherently flawed. 0:21:41.800,0:21:45.387 Instead, it's soliciting, as Ethan said,[br]soliciting information from you 0:21:45.554,0:21:49.182 to find out what you know[br]and to help you build on your knowledge. 0:21:49.391,0:21:53.478 And these sorts of subtle differences[br]are what separates 0:21:53.562,0:21:57.399 using AI in sort of an expert way[br]in a classroom where we know 0:21:57.482,0:22:00.986 what we want to have happen from[br]just the naive use people are doing, 0:22:05.949,0:22:06.742 I think that 0:22:06.742,0:22:10.537 there is an advantage[br]to taking charge of your students AI usage 0:22:10.537,0:22:12.122 because they're going to be using it[br]anyway 0:22:12.122,0:22:15.417 and thinking about directing it either by giving them[br]prompts, having these discussions 0:22:15.625,0:22:17.044 and it's a really powerful tool 0:22:17.044,0:22:20.047 that in the future will greatly boost[br]classroom learning 0:22:20.047,0:22:23.383 and is not a replacement[br]or threat for teachers. 0:22:23.383,0:22:27.095 It is something that we can use[br]to improve the outputs of our work, 0:22:27.220,0:22:31.558 improve student learning, make our lives[br]easier while making students lives better 0:22:31.683,0:22:34.686 and I think that that's[br]a very powerful view of the future. 0:22:34.686,0:22:37.981 And I hope that you at least embrace[br]and experiment with AI before deciding 0:22:37.981,0:22:40.359 whether you want to use it[br]or ban it in your classrooms. 0:22:41.234,0:22:46.156 Wow. There really are so many ways[br]to enhance student learning using AI. 0:22:46.239,0:22:46.865 With AI 0:22:46.865,0:22:51.286 technology advancing rapidly, there[br]will be more and more tools available. 0:22:51.370,0:22:55.415 As with any new tool,[br]educators have a responsibility to ensure 0:22:55.415,0:23:00.170 they're using age appropriate tools,[br]protecting student privacy 0:23:00.253,0:23:03.590 and creating spaces[br]for students to critically evaluate 0:23:03.673,0:23:07.177 the potential pitfalls of the technology[br]they are using. 0:23:07.260,0:23:11.223 Join us in session 4 Ensuring[br]a Responsible Approach to AI 0:23:11.390,0:23:13.392 as we explore these topics. 0:23:13.392,0:23:14.434 Thanks for joining us. 0:23:14.434,0:23:16.103 See you again In session four. 0:23:16.103,0:23:21.900 Visit the AI 101 for Teachers[br]website at Code.org/ai101 0:23:22.109,0:23:23.735 to sign up[br]for early access 0:23:23.860,0:23:27.072 and to explore additional resources[br]from Code.org. 0:23:27.155,0:23:29.950 ETS, ISTE and Khan Academy. 0:23:29.950,0:23:30.909 Thanks for joining us.