WEBVTT 00:00:07.173 --> 00:00:09.134 Hi. Welcome to session three 00:00:09.134 --> 00:00:13.346 of the AI 101 for Teachers Professional Learning Series. 00:00:13.430 --> 00:00:16.182 In this session we are traveling to the Wharton School 00:00:16.182 --> 00:00:19.269 at the University of Pennsylvania to chat with Dr. 00:00:19.269 --> 00:00:24.482 Ethan Mollick, a professor who teaches innovation and entrepreneurship, and Dr. 00:00:24.482 --> 00:00:29.279 Lilach Mollick, who works on interactive pedagogy and AI research. 00:00:29.362 --> 00:00:32.615 They will help us explore how AI can be combined 00:00:32.615 --> 00:00:36.119 with pedagogy to enhance student learning. 00:00:36.202 --> 00:00:37.495 Let's go meet the Mollicks. 00:00:43.752 --> 00:00:45.128 Hi I’m Ethan Mollick 00:00:45.128 --> 00:00:48.882 a professor at Wharton who has been working on how we democratize 00:00:48.882 --> 00:00:54.054 access to education through tools like games and interactive tools and AI 00:00:54.137 --> 00:00:55.180 and I’m Lilach Mollick 00:00:55.180 --> 00:00:57.265 I'm director of Pedagogy at Wharton Interactive, 00:00:57.265 --> 00:00:58.892 and I've been working at the intersection 00:00:58.892 --> 00:01:02.812 of AI and education, helping to democratize education 00:01:02.812 --> 00:01:06.858 for everyone through effective, pedagogically sound use of AI. 00:01:06.941 --> 00:01:11.112 And we have been working together on the future of education for a while, 00:01:11.112 --> 00:01:14.908 thinking about how to make education more interactive, to work at scale. 00:01:15.116 --> 00:01:18.828 And with the advent of general AI, we found a powerful new tool 00:01:18.995 --> 00:01:22.248 that can really help in the classroom but also carries some risks. 00:01:22.457 --> 00:01:24.876 Today, we'd like to talk a little bit about the classroom 00:01:24.876 --> 00:01:28.755 use of AI upsides and downsides to give you some examples to work with. 00:01:28.838 --> 00:01:31.758 But first, we like to start with our three guiding principles. 00:01:31.758 --> 00:01:33.968 The first is that AI is undetectable. 00:01:33.968 --> 00:01:38.431 There are AI tools, detection tools, but they are not effective. 00:01:38.515 --> 00:01:41.768 The second principle is that AI is ubiquitous. 00:01:41.768 --> 00:01:43.353 It's everywhere. 00:01:43.353 --> 00:01:46.731 169 countries have access to Bing Chat and you and 00:01:46.731 --> 00:01:50.610 your students have access to the most powerful AI available. 00:01:50.693 --> 00:01:53.822 The third principle is that AI is transformative. 00:01:53.863 --> 00:01:58.576 It will transform how we live, how we work, and how we teach and learn. 00:02:04.958 --> 00:02:06.251 Not only is 00:02:06.251 --> 00:02:09.796 AI not going away, but this is probably the worst AI 00:02:09.796 --> 00:02:10.672 you're ever going to use. 00:02:10.672 --> 00:02:14.509 So if this feels disruptive now kind of have bad news, which is that 00:02:14.509 --> 00:02:17.137 there isn't a reason to suspect that AI development will not continue. 00:02:17.137 --> 00:02:17.887 And I think people worry 00:02:17.887 --> 00:02:21.850 a lot about like the far future or who knows how far it is where AI 00:02:21.850 --> 00:02:22.600 smarter than humans. 00:02:22.600 --> 00:02:26.563 But even over the next couple of years, even with this fact of this academic year, 00:02:26.688 --> 00:02:28.898 I would expect AI to continue to improve. 00:02:28.898 --> 00:02:31.151 Five times. Ten times? We have no idea. 00:02:31.151 --> 00:02:34.112 But if you're not already thinking about these systems, what they mean 00:02:34.112 --> 00:02:37.115 for education, what they mean for you, what they mean for your students careers. 00:02:37.115 --> 00:02:40.493 I think we have to think about it because these systems are not going to disappear. 00:02:40.618 --> 00:02:44.581 Let me make the very pragmatic case for why you may want to do this. 00:02:44.664 --> 00:02:47.917 The first part of the pragmatic case is your students are using this anyway. 00:02:48.126 --> 00:02:49.377 So you have to come up to speed. 00:02:49.377 --> 00:02:51.796 I don't think everybody wants to be dragged along this technology. 00:02:51.796 --> 00:02:54.132 No one asked for education to be massively disrupted. 00:02:54.132 --> 00:02:54.966 But it is. 00:02:54.966 --> 00:02:57.385 And unfortunately, we've got to figure out a way to get around that. 00:02:57.385 --> 00:03:00.471 All your homework assignments can be done by AI now, so you have to think about that. 00:03:00.471 --> 00:03:04.642 And then I think the second thing is a pragmatic argument about how AI 00:03:04.642 --> 00:03:06.060 can make your life easier as a teacher. 00:03:06.060 --> 00:03:08.021 If you put the hours down, you get them back later. 00:03:08.021 --> 00:03:09.439 And then if you've worked out a number of prompts 00:03:09.439 --> 00:03:11.649 to help make lives easier for teachers. 00:03:11.649 --> 00:03:14.777 Yes. So one thing you can do prompts like give me a lesson 00:03:14.777 --> 00:03:19.574 hook prompts like create a lesson plan or create a quiz for me. 00:03:19.782 --> 00:03:24.078 So starting to work with your material and the different models could get you 00:03:24.120 --> 00:03:28.082 give you a really good sense of how the AI works, what it's good at, what it 00:03:28.166 --> 00:03:32.003 what it's not good at. And so and save you time in the end, right? 00:03:32.003 --> 00:03:33.171 So that's where I would be 00:03:33.171 --> 00:03:37.383 my push to teachers is A you have to and B you're going to want to. 00:03:37.467 --> 00:03:40.470 I think the other important thing is just to try it. 00:03:40.678 --> 00:03:42.055 They're very simple to use. 00:03:42.055 --> 00:03:45.391 They're very intuitive because they're conversational. 00:03:45.475 --> 00:03:49.270 You can continue a conversation and it feels fairly natural. 00:03:49.479 --> 00:03:52.690 And I think the key really is experimentation. 00:03:52.774 --> 00:03:56.527 See how it works with you, see how it works within your context, 00:03:56.611 --> 00:03:58.613 within your topic that you teach. 00:03:58.613 --> 00:04:01.115 Our rule of thumb is you need about 10 hours time with 00:04:01.115 --> 00:04:03.952 AI to get what it's good at, what its limitations are. 00:04:03.952 --> 00:04:07.705 So I would actually start by suggesting that this that the teacher throw 00:04:07.705 --> 00:04:10.708 their own assignments into the AI and see what kind of results they get back. 00:04:10.875 --> 00:04:14.254 I would think about asking them to ask their students 00:04:14.254 --> 00:04:18.049 to create an assignment using AI and then critique that assignment, 00:04:18.216 --> 00:04:20.176 potentially even in class, to see 00:04:20.176 --> 00:04:23.680 if the students can get a sense of what the gaps and abilities of AI are. 00:04:23.721 --> 00:04:26.099 I have a little bit of freedom as an instructor 00:04:26.099 --> 00:04:29.727 because I'm teaching college and MBA students entrepreneurship. 00:04:29.727 --> 00:04:30.728 So I have a lot of 00:04:30.728 --> 00:04:33.106 I have points I want them to make, but they also are building things 00:04:33.106 --> 00:04:36.943 and doing things and absolutely transformed how that works. 00:04:36.943 --> 00:04:38.695 So my assignments now 00:04:38.695 --> 00:04:41.781 literally call for students to do this one impossible thing in class. 00:04:41.781 --> 00:04:44.367 If you can't code, you have to write working programs. 00:04:44.367 --> 00:04:47.245 If you have never if you can't do design work, you have to create a full 00:04:47.245 --> 00:04:50.748 graphic design working prototypes that's literally now part of the class. 00:04:50.748 --> 00:04:52.333 So where it used to be, write it write 00:04:52.333 --> 00:04:54.669 a little bit of an essay, do a prototype on paper. 00:04:54.669 --> 00:04:56.713 Now you have to create a full working product. 00:04:56.713 --> 00:04:58.381 Every assignment that is written 00:04:58.381 --> 00:05:00.216 has to be critiqued by at least five famous 00:05:00.216 --> 00:05:02.969 entrepreneurs through history, and they use AI to invoke those. 00:05:02.969 --> 00:05:04.304 There's a pedagogical reason, too, 00:05:04.304 --> 00:05:06.597 which is that entrepreneurs tend to be overconfident. 00:05:06.597 --> 00:05:08.725 So you want feedback from different sources. 00:05:08.725 --> 00:05:12.186 So to me it is let me teach ten times more than I did. 00:05:12.186 --> 00:05:14.897 I used to teach an advanced intermedia entrepreneurship course. 00:05:14.897 --> 00:05:17.233 I can now, in the intermediate or basic course, 00:05:17.233 --> 00:05:19.652 get all the way past the advanced material and further. 00:05:19.652 --> 00:05:22.155 So I think we're going to see that shake out more in the future. 00:05:22.155 --> 00:05:25.116 But some of this is about powering past what we could do before 00:05:25.116 --> 00:05:26.993 and I think that's exciting as well. 00:05:26.993 --> 00:05:32.040 Apart from student tutors as assignments, teachers can certainly use 00:05:32.123 --> 00:05:36.753 AI coaches, AI assistants to help students prepare for discussions, 00:05:37.003 --> 00:05:41.591 help students outline, help students do research, help students get feedback 00:05:41.591 --> 00:05:44.719 on assignments, and just help students develop explanations. 00:05:44.719 --> 00:05:50.183 I think there are myriad of approaches that are pedagogically sound 00:05:50.266 --> 00:05:51.768 that teachers can assign to 00:05:51.768 --> 00:05:56.481 students and watch their work and ask for the back and forth interaction 00:05:56.481 --> 00:06:00.109 to really see that students are paying attention to and focusing on the material. 00:06:05.907 --> 00:06:07.325 Let's talk a little bit about AI 00:06:07.325 --> 00:06:08.409 from a teacher's perspective. 00:06:08.409 --> 00:06:11.162 So because of the ubiquity of AI, 00:06:11.162 --> 00:06:14.874 you've got some choices to make in terms of your AI policies in your class. 00:06:14.874 --> 00:06:18.127 So do you want to permit I do want to forbid AI 00:06:18.252 --> 00:06:20.004 How are you going to enforce these sorts of things? 00:06:20.004 --> 00:06:21.756 We're going to assume that you want to use AI 00:06:21.756 --> 00:06:25.134 to some extent, and we'll dive into a little bit of the details here. 00:06:25.134 --> 00:06:27.720 So as a instructor, you should know a few things. 00:06:27.720 --> 00:06:30.973 One is there is obviously ongoing ethical debates about AI, 00:06:30.973 --> 00:06:32.266 and those are complicated debates. 00:06:32.266 --> 00:06:34.143 There are debates over whether or not 00:06:34.143 --> 00:06:37.063 the AI's trained on the right kind of data about the biases 00:06:37.063 --> 00:06:41.984 I might have about the use of AI and the outcomes for student learning. 00:06:42.193 --> 00:06:44.153 And it's worth acknowledging these sets of things. 00:06:44.153 --> 00:06:48.032 But this tool is out there and it is worth thinking about how you want to use it. 00:06:48.116 --> 00:06:51.077 If you decide that that is okay and how you want to communicate 00:06:51.077 --> 00:06:54.205 that information beyond the initial ethical concerns, 00:06:54.372 --> 00:06:56.707 there's also concerns about how AI actually works. 00:06:56.707 --> 00:07:00.128 So the large language models that power 00:07:00.128 --> 00:07:03.256 today’s AI don't actually have knowledge of the world. 00:07:03.256 --> 00:07:04.632 They're predicting the next word. 00:07:04.632 --> 00:07:07.718 They're predicting the right kinds of sentences or information to give out. 00:07:07.927 --> 00:07:11.639 And as a result, they make stuff up they hallucinate. 00:07:11.639 --> 00:07:13.641 So there are often errors or mistakes. 00:07:13.641 --> 00:07:15.643 Now, it's not always clear those errors or mistakes are worse 00:07:15.643 --> 00:07:17.353 than the errors and mistakes humans would make. 00:07:17.353 --> 00:07:18.855 But you need to be aware that there's 00:07:18.855 --> 00:07:20.690 going to be those kind of errors and mistakes. 00:07:20.690 --> 00:07:23.860 And then finally, you need to think about as an instructor 00:07:24.068 --> 00:07:26.946 how you're going to be using AI to aid learning, 00:07:26.946 --> 00:07:30.825 which means being really clear about what you want to accomplish with an AI tool. 00:07:31.075 --> 00:07:33.244 They can be used for student learning, 00:07:33.244 --> 00:07:35.413 but AI's many possible uses in the classroom, 00:07:35.413 --> 00:07:37.790 so do you want to use them to have student’s generate ideas, 00:07:37.790 --> 00:07:41.085 which I do in my classes and get better project ideas as a result. 00:07:41.169 --> 00:07:44.589 Do you want them to use them as tutors to explain concepts 00:07:44.589 --> 00:07:45.756 to them they don't understand? 00:07:45.756 --> 00:07:48.426 Do you want the students to get feedback from AI 00:07:48.426 --> 00:07:51.679 by asking for questions about work that they're doing. 00:07:51.804 --> 00:07:53.556 Do you want to be a writing companion? 00:07:53.556 --> 00:07:56.559 Do you want it to explain why quiz answers might be right or wrong? 00:07:56.684 --> 00:07:58.728 And then once you've decided what who's instructor, 00:07:58.728 --> 00:08:02.231 you decide what you're going to tell your students. 00:08:07.570 --> 00:08:09.572 AI detectors don't work. 00:08:09.572 --> 00:08:10.281 They just don't work. 00:08:10.281 --> 00:08:12.116 You shouldn't use them. And it's worse than them 00:08:12.116 --> 00:08:15.286 not working because they have a high false positive rate. 00:08:15.286 --> 00:08:18.581 That means they select things that’s AI written that aren't AI written 00:08:18.748 --> 00:08:23.169 and that disproportionately falls on people whose English is a second language. 00:08:23.252 --> 00:08:24.795 This is just not something that we can do. 00:08:24.795 --> 00:08:28.299 And I think trying to close the barn door here after it's been opened 00:08:28.382 --> 00:08:33.471 and try and detect AI is not the future for responsibility in classrooms. 00:08:33.554 --> 00:08:37.058 The other thing to note too, is that students 00:08:37.141 --> 00:08:39.060 were using shortcuts in the past. 00:08:39.060 --> 00:08:40.603 It's not that they weren't using Google, 00:08:40.603 --> 00:08:43.105 it's not that they weren't using, you know, other students essays. 00:08:43.105 --> 00:08:46.150 This was happening in the past, but this is a major disruption, 00:08:46.359 --> 00:08:50.446 and I think it does call for a rethinking of how we do essays. 00:08:50.446 --> 00:08:54.200 So thinking a little bit more about the learning goal for an essay 00:08:54.200 --> 00:08:55.993 or the learning goal for any assignment, 00:09:02.416 --> 00:09:05.586 One of the things that we're noticing as we watch teachers do this 00:09:05.586 --> 00:09:09.298 is they all feel an obligation to talk about AI and dive 00:09:09.298 --> 00:09:12.301 deep into the ethical implications of AI and so on. 00:09:12.343 --> 00:09:14.554 I think that's important, but I don't think that needs 00:09:14.554 --> 00:09:15.930 to be the theme of every class. 00:09:15.930 --> 00:09:18.933 I don't think every class needs to be a discussion about AI, 00:09:19.016 --> 00:09:21.686 just like every class that uses computers doesn't need to be a discussion 00:09:21.686 --> 00:09:24.272 about computers, I think is important to have that conversation. 00:09:24.272 --> 00:09:25.189 And right now 00:09:25.189 --> 00:09:28.234 we're all just reacting so it's not clear who's supposed to have that. 00:09:28.234 --> 00:09:30.361 So I totally get teachers wanting to have AI 00:09:30.361 --> 00:09:33.906 discussions, but it's even harder to get up to speed, 00:09:33.906 --> 00:09:36.075 not just on the use of AI, but how it works. 00:09:36.075 --> 00:09:39.161 It's, you know, standards, its ethical implications. 00:09:39.412 --> 00:09:43.332 So I think teachers should feel a little bit of okay ness 00:09:43.332 --> 00:09:46.794 with experimenting with AI without having to make it the subject of class. 00:09:53.301 --> 00:09:56.304 First is as Ethan mentioned, that AI can fabricate. 00:09:56.304 --> 00:10:00.099 That means that any output that the AI gives a student may be made 00:10:00.099 --> 00:10:04.395 up, it may be mistaken, it may be very subtly mistaken. 00:10:04.395 --> 00:10:07.481 And so students should be responsible for their own work. 00:10:07.690 --> 00:10:11.027 They should at the very least check sources, check any number, 00:10:11.027 --> 00:10:14.947 check any facts that the AI gives them and check them with credible sources. 00:10:15.031 --> 00:10:18.326 The second principle is that the AI is not a person. 00:10:18.326 --> 00:10:21.996 It's easy to imbue the AI with a personality or to feel like you're 00:10:21.996 --> 00:10:25.958 talking to a person, but it's not a person and it doesn't know you. 00:10:26.042 --> 00:10:29.378 The third principle is really to give it a lot of context. 00:10:29.378 --> 00:10:30.921 The AI doesn't know you. 00:10:30.921 --> 00:10:34.342 It doesn't know your context or your experience or your expertise. 00:10:34.425 --> 00:10:38.012 The more context you give it, the more useful it'll be for you. 00:10:38.054 --> 00:10:40.806 And the fourth principle is that you're in charge. 00:10:40.806 --> 00:10:42.391 Not only should you evaluate 00:10:42.391 --> 00:10:46.312 and interrogate its output, but if it's leading you in a conversation 00:10:46.395 --> 00:10:49.732 that is no longer useful to you, or if it's stuck in a loop, 00:10:49.815 --> 00:10:52.693 or if you'd like to change the direction of the conversation, 00:10:52.693 --> 00:10:55.196 you should absolutely feel free to take charge. 00:11:01.786 --> 00:11:04.413 So when we talk about AI 00:11:04.413 --> 00:11:07.333 and these generative AI solutions, 00:11:07.333 --> 00:11:09.335 we tend to talk about large language models. 00:11:09.335 --> 00:11:10.461 And there's actually only 00:11:10.461 --> 00:11:13.923 a few large scale general purpose, large language models. 00:11:13.923 --> 00:11:18.928 There is the models created by OpenAI, which are GPT 3.5 or GPT 4 00:11:19.136 --> 00:11:23.057 GPT 3.5 is the free version that you get through 00:11:23.140 --> 00:11:26.352 through Chat GPT and GPT 4 is either 00:11:26.352 --> 00:11:29.647 through the paid Chat GPT or through Microsoft Bing in creative mode. 00:11:29.855 --> 00:11:34.527 And when we talk about specialized apps, almost all of them are using 00:11:34.527 --> 00:11:38.739 one of these models and providing prompts and other information on top of it. 00:11:38.823 --> 00:11:42.368 I generally think instructors should get familiar with the models 00:11:42.368 --> 00:11:46.664 themselves because those are the models that are actually producing the answers 00:11:46.831 --> 00:11:49.375 and you can manipulate them directly that way and learn how they work. 00:11:49.375 --> 00:11:51.377 So if you're trying to buy an off the shelf solution, 00:11:51.377 --> 00:11:53.796 they're almost certainly using one of these existing models 00:11:53.796 --> 00:11:57.258 and then providing some sort of wrapper or other information on top of it, 00:11:57.466 --> 00:12:00.761 and it's often cheaper and more effective and gives you more control 00:12:00.886 --> 00:12:02.680 to use the foundation models yourself. 00:12:02.680 --> 00:12:04.223 But that's a choice you get to make. 00:12:04.223 --> 00:12:09.311 So when developing the prompt, we really and for all of our prompts, 00:12:09.520 --> 00:12:11.814 we really look at the science of learning 00:12:11.814 --> 00:12:14.817 and try to combine that with the power of the AI. 00:12:14.900 --> 00:12:19.071 So for instance, a good tutor pushes you for information. 00:12:19.155 --> 00:12:21.657 It doesn't just hand to you, a good tutor 00:12:21.657 --> 00:12:26.287 finds out what you know and builds on that prior knowledge. 00:12:26.370 --> 00:12:29.623 A good tutor will also find out a little bit about you. 00:12:29.623 --> 00:12:34.128 A good tutor also knows that you need lots and varied kinds of examples 00:12:34.211 --> 00:12:35.838 and analogies, 00:12:35.838 --> 00:12:40.509 and a good tutor knows that the way to you show evidence of mastery 00:12:40.509 --> 00:12:43.512 is by being able to explain something in your own words 00:12:43.512 --> 00:12:48.392 to someone else and give an example of it, which is exactly these are exactly 00:12:48.392 --> 00:12:52.146 the steps in the kinds of questions that we use in the tutor prompt. 00:12:52.396 --> 00:12:54.398 But you'll notice when you look at our prompts 00:12:54.398 --> 00:12:58.569 that they do things like provide context the AI as life has discussed already, 00:12:58.569 --> 00:13:02.656 the idea that it asks you who you are, and we tell the AI who it is. 00:13:02.656 --> 00:13:04.867 It's an instructor with this kind of setting, 00:13:04.867 --> 00:13:07.703 you'll notice that it also tells it exactly 00:13:07.703 --> 00:13:11.415 the scientific framework to use this idea of context matters. 00:13:11.499 --> 00:13:12.666 We provide controls. 00:13:12.666 --> 00:13:16.629 We ask it to go step by step through sets of questions to ask, sometimes 00:13:16.629 --> 00:13:20.049 not in these prompts, we provide examples of good output, and then we tested a lot. 00:13:20.257 --> 00:13:23.803 You can't do prompting without testing, and that's one of the great things 00:13:23.803 --> 00:13:25.387 about testing your error expertise. 00:13:25.387 --> 00:13:26.680 It's cheap to do 00:13:26.680 --> 00:13:30.142 and so you get to experiment a lot and that makes for good prompts. 00:13:30.309 --> 00:13:34.563 And we should also say we test it not just on one model but on several models. 00:13:34.563 --> 00:13:37.608 So for instance, these two prompts we just worked 00:13:37.608 --> 00:13:40.778 with ChatGPT 4 for, they also work with Bing. 00:13:40.903 --> 00:13:44.865 Bing will react a little bit differently and it will 00:13:44.949 --> 00:13:46.408 because it's connected to the Internet. 00:13:46.408 --> 00:13:48.369 It will also look up citations sometimes. 00:13:48.369 --> 00:13:50.246 So the right citation, sometimes they're not. 00:13:50.246 --> 00:13:51.997 But that is available. 00:13:51.997 --> 00:13:54.416 It may or may not work with some of the other models. 00:13:54.416 --> 00:13:55.626 So you really have to test it, 00:13:55.626 --> 00:13:58.629 I think, as an instructor before you give it to your students 00:13:58.838 --> 00:14:02.383 in the context of the topic that you're teaching to see how it works. 00:14:02.466 --> 00:14:05.386 So this is all very theoretical, but I think important. 00:14:05.386 --> 00:14:06.512 So let's let's get practical. 00:14:06.512 --> 00:14:08.806 Let's talk about some examples of what AI can do. 00:14:08.806 --> 00:14:13.227 And again, this isn't a monolithic thing, a has many possible uses. 00:14:13.227 --> 00:14:14.812 As we said, it was transformative earlier. 00:14:14.812 --> 00:14:17.523 So we're going to show you a couple of prompts that we've created 00:14:17.523 --> 00:14:21.193 and those will be available to you as well to work with 00:14:21.277 --> 00:14:25.489 and these are just examples of the ways AI classroom use can work. 00:14:25.489 --> 00:14:29.577 So the first one we want to show you is a prompt that I believe you created 00:14:29.660 --> 00:14:33.122 that talks about feedback that gives proper feedback. 00:14:33.205 --> 00:14:36.750 And one of the really interesting things about the AI side of things 00:14:36.750 --> 00:14:38.168 is a more sophisticated prompt 00:14:38.168 --> 00:14:39.712 that takes into account some of the principles 00:14:39.712 --> 00:14:41.922 we were talking about earlier will result in better outcomes. 00:14:41.922 --> 00:14:44.425 So students will often ask for writing advice from an AI 00:14:44.425 --> 00:14:45.968 even if you tell them not to do it. 00:14:45.968 --> 00:14:48.220 But they're in ask for it in a way that's fairly unsophisticated 00:14:48.220 --> 00:14:51.849 and is going to give them fairly generic sounding work and possibly more mistakes. 00:14:51.932 --> 00:14:54.810 If you give a more elaborate prompt, you can get more elaborate answers. 00:14:54.810 --> 00:14:58.814 So in this case, could you explain what this prompt does, the feedback prompt. Yes. 00:14:58.814 --> 00:15:03.068 So we combine the principles of good feedback, 00:15:03.068 --> 00:15:06.363 which is feedback that takes into account your prior knowledge 00:15:06.363 --> 00:15:10.075 or what you already know from the student perspective takes into account 00:15:10.075 --> 00:15:13.537 who you are, your learning level, what grade you're in, 00:15:13.621 --> 00:15:16.624 whether you're in college or you're in a professional. 00:15:16.665 --> 00:15:21.337 And it also takes into account the idea that you want to respond to this feedback. 00:15:21.337 --> 00:15:24.548 So it is going to be actionable, it's going to be balanced, 00:15:24.548 --> 00:15:28.260 it's going to tell you what's wrong and what you can improve on 00:15:28.344 --> 00:15:31.180 and what you're doing well, and it's going to keep working with you. 00:15:31.180 --> 00:15:35.351 But like any good tutor or coach, it won't actually give you the answer. 00:15:35.517 --> 00:15:37.102 It'll push you in that direction, 00:15:37.102 --> 00:15:40.773 ask you to explain, ask you to construct your own knowledge. 00:15:40.856 --> 00:15:44.234 And so you can see the prompt hopefully on the screen here 00:15:44.318 --> 00:15:46.904 and as a place to work from. 00:15:46.904 --> 00:15:49.573 You don't need to take this as an absolute answer. 00:15:49.573 --> 00:15:52.076 This is something you can play with, but let's see it in action. 00:15:52.076 --> 00:15:53.953 So let's get started using this prompt. 00:15:53.953 --> 00:15:57.206 It says that it's a teaching assistant because that the instructions we gave it 00:15:57.373 --> 00:16:00.793 and it asks us for our grade level and subject we're studying, 00:16:00.793 --> 00:16:01.961 what should we say? 00:16:01.961 --> 00:16:05.255 So I think we're studying Macbeth and we're in 12th grade. 00:16:05.339 --> 00:16:11.387 In 12th grade. Okay, great. 00:16:11.470 --> 00:16:12.262 Okay. 00:16:12.262 --> 00:16:14.974 And so we've told the AI this information it’s feeding it 00:16:14.974 --> 00:16:17.977 into the logic that it's using here. 00:16:18.060 --> 00:16:20.771 And it's asking us about a specific assignment 00:16:20.771 --> 00:16:23.440 and it's asking if we have a rubric or other information to work with or what 00:16:23.440 --> 00:16:26.860 we're hoping to achieve with with as much information as possible. 00:16:26.860 --> 00:16:30.948 I don't have a huge amount here, so I'll say I have to write 00:16:31.031 --> 00:16:35.411 an analysis of Macbeth. 00:16:35.494 --> 00:16:38.664 It is graded 00:16:38.747 --> 00:16:40.916 based on 00:16:40.916 --> 00:16:45.921 rating style and depth of content 00:16:46.005 --> 00:16:48.799 and you'll see what it's going is it's asking ask questions 00:16:48.799 --> 00:16:50.259 and soliciting information from us, 00:16:50.259 --> 00:16:53.262 which makes it kind of a good prompt that you might hand a student 00:16:53.303 --> 00:16:56.974 better than one that is just they're just developing themselves and it's asking 00:16:56.974 --> 00:17:00.352 about specific instructions and ask us to share the assignment. 00:17:00.436 --> 00:17:05.190 Here is what I have written so far, 00:17:05.274 --> 00:17:05.566 and I 00:17:05.566 --> 00:17:08.777 have asked the I to generate a Macbeth essay. 00:17:08.777 --> 00:17:10.696 So here we go. 00:17:10.696 --> 00:17:13.323 I'm just pasting that in 00:17:13.323 --> 00:17:14.283 and we'll see what it says here. 00:17:14.283 --> 00:17:17.453 And you'll notice it's it's working on the information. 00:17:17.453 --> 00:17:19.455 It's saying it's taking time to carefully read through it. 00:17:19.455 --> 00:17:20.706 That's a bit of illusion. 00:17:20.706 --> 00:17:24.043 It's obviously not taking extra time, but it's responding in this method 00:17:24.043 --> 00:17:26.378 and you'll see it's giving a set of strengths and weaknesses. 00:17:26.378 --> 00:17:27.629 What's great about, again, 00:17:27.629 --> 00:17:31.008 using a tutor that you've built or a mentor that you built is that it 00:17:31.008 --> 00:17:33.677 can give you the kind of feedback that's educationally valuable 00:17:33.677 --> 00:17:37.598 that ties into pedagogy rather than just students asking Make this essay better. 00:17:37.806 --> 00:17:42.394 An example of a working in your favor as an educator 00:17:42.394 --> 00:17:46.148 and not necessarily working against you and undermining the point you're making. 00:17:46.231 --> 00:17:49.359 So you'll notice at the end, by the way, it gives a question 00:17:49.359 --> 00:17:51.695 that for the students to answer. 00:17:51.695 --> 00:17:54.490 So how do you plan to revise your analysis? 00:17:54.490 --> 00:17:56.825 Give me a plan and specific changes are going again. 00:17:56.825 --> 00:17:58.452 The kind of thing we would do as an instructor 00:17:58.452 --> 00:18:01.538 in a classroom soliciting changes or differences. 00:18:01.538 --> 00:18:05.751 So I think you can start to see why a 00:18:05.834 --> 00:18:06.543 tool like 00:18:06.543 --> 00:18:09.546 this can be really useful when properly applied. 00:18:09.713 --> 00:18:13.509 Now, let's also talk about one other potential use for AI, AI as a tutor. 00:18:13.509 --> 00:18:16.178 What are some of the advantages and disadvantages of that approach? 00:18:16.178 --> 00:18:17.471 So an advantage of this approach 00:18:17.471 --> 00:18:21.016 is that you're getting students to actually pay attention to the material. 00:18:21.225 --> 00:18:24.728 You're getting them to read over the rubric, to read over 00:18:24.853 --> 00:18:29.108 the purpose of the essay and the audience, and to really think through it. 00:18:29.191 --> 00:18:31.860 A disadvantage is that you 00:18:31.860 --> 00:18:36.448 you certainly can ask the AI to do it for you, but if you work with it 00:18:36.448 --> 00:18:40.452 and if you're given guidelines to work with it, it's one way to get feedback. 00:18:40.452 --> 00:18:42.746 But you would then have to evaluate 00:18:42.746 --> 00:18:46.375 something else that a teacher could do is to ask for the interactions 00:18:46.375 --> 00:18:48.585 and ask for a reflection about the interactions. 00:18:48.585 --> 00:18:49.211 What about this 00:18:49.211 --> 00:18:50.129 feedback was good? 00:18:50.129 --> 00:18:52.297 What about this feedback was not as good? 00:18:52.297 --> 00:18:56.385 And again, it's a higher order level thinking about your essay 00:18:56.468 --> 00:18:59.346 and your process. Excellent. 00:18:59.346 --> 00:19:04.143 And so why don't you do one other example where the AI acts as a direct instructor 00:19:04.226 --> 00:19:06.019 and we have a prompt for that as well. 00:19:06.019 --> 00:19:09.648 There are risks associated with asking the AI to be a direct instructor, 00:19:09.648 --> 00:19:11.441 which is that hallucination risk. 00:19:11.441 --> 00:19:14.695 It doesn't know your pedagogy, your your, your perspective. 00:19:14.945 --> 00:19:17.948 But I find in my classrooms that students are increasingly 00:19:17.948 --> 00:19:22.369 using the AI as a method of learning so they don't raise their hands as much. 00:19:22.369 --> 00:19:23.620 So when I ask them why, they're like, Well, 00:19:23.620 --> 00:19:25.205 I'd rather not show my ignorance in class, 00:19:25.205 --> 00:19:27.916 I could ask the AI to explain like I'm five. 00:19:27.916 --> 00:19:29.626 So they're already engaged in this behavior. 00:19:29.626 --> 00:19:32.880 So something like a tutor both does a useful thing 00:19:32.880 --> 00:19:36.216 of showing you what the future of AI education might look like. 00:19:36.300 --> 00:19:39.636 Like the way Khan Academy is building AI interactive tutors to work. 00:19:39.803 --> 00:19:43.432 And it also might be a tool your students can use to achieve more in class, 00:19:43.682 --> 00:19:45.475 but you should caveat that with the knowledge 00:19:45.475 --> 00:19:48.353 that AI tutors are not 100% there yet. 00:19:48.353 --> 00:19:49.771 But let's let's use an example here. 00:19:49.771 --> 00:19:52.441 So this tutor is again trying to take the right kind of format. 00:19:52.441 --> 00:19:55.444 It says, Hello there, I'm your AI tutor and I'm excited to work with you today. 00:19:55.694 --> 00:19:57.237 What do we want to learn about today? 00:19:57.237 --> 00:19:59.114 Let's learn about opportunity cost. 00:19:59.114 --> 00:20:02.117 Opportunity cost. A concept from economics. 00:20:02.201 --> 00:20:04.286 Let's see what happens. 00:20:04.328 --> 00:20:06.622 Okay, so we're telling the AI or the opportunity cost 00:20:06.622 --> 00:20:09.625 that saying it is a key concept economics 00:20:09.666 --> 00:20:12.461 it’s even throwing in a little emoji here, which is cute. 00:20:12.461 --> 00:20:14.129 Can you ask us about our learning level? 00:20:14.129 --> 00:20:15.547 What level are we at here? 00:20:15.547 --> 00:20:18.342 11th grade. 11th grade. 00:20:18.342 --> 00:20:21.428 Now, I wouldn't get too tied up on the individual grade. 00:20:21.428 --> 00:20:22.471 It's not amazing 00:20:22.471 --> 00:20:24.556 at differentiating a 10th grader from an 11th grader, 00:20:24.556 --> 00:20:26.308 but this is part of the context in which it's working in. 00:20:26.308 --> 00:20:31.104 So that pulling from some sort of universal standards here and it says, 00:20:31.146 --> 00:20:33.899 what do we know about opportunity cost? 00:20:33.899 --> 00:20:37.527 So we know that it has to do with alternative choices, 00:20:37.611 --> 00:20:41.865 has to do with alternative choices. 00:20:41.949 --> 00:20:43.158 That is it. 00:20:43.158 --> 00:20:44.826 And of course, one of the advantages of AI 00:20:44.826 --> 00:20:46.078 is this kind of freeform text 00:20:46.078 --> 00:20:49.665 and interaction is the real power of education 00:20:49.748 --> 00:20:51.917 and it's something the AI can fake reasonably well. 00:20:51.917 --> 00:20:54.878 Again, not as well as a real human instructor yet. 00:20:54.878 --> 00:20:57.881 And you'll notice it's giving us examples 00:20:58.006 --> 00:21:02.386 and explaining things in different ways, which is a powerful thing that AI can do. 00:21:02.386 --> 00:21:04.721 It's very good at breaking things down in different ways, 00:21:04.721 --> 00:21:07.182 but you'll notice that it's they're starting to ask questions. 00:21:07.182 --> 00:21:09.184 It's asking us to make choices. 00:21:09.184 --> 00:21:11.603 So one of things we know from the research on tutoring is 00:21:11.603 --> 00:21:13.855 you can't just declaim things to people. 00:21:13.855 --> 00:21:16.441 The value of tutoring comes from soliciting information, 00:21:16.441 --> 00:21:17.276 making connections. 00:21:17.276 --> 00:21:18.860 And you can see the AI starting to do this 00:21:18.860 --> 00:21:21.071 and asking us for connections in our own life. 00:21:21.071 --> 00:21:24.366 The other thing to mention too, about the tutor prompt is that it 00:21:24.366 --> 00:21:29.830 is not assuming that the student can judge their own learning. Very often 00:21:29.955 --> 00:21:32.708 you'll see in a tutor prompt that is very simple. 00:21:32.708 --> 00:21:34.251 Like explain to me like on ten, 00:21:34.251 --> 00:21:36.837 it'll ask you if you'll understand something instead. 00:21:36.837 --> 00:21:37.546 here 00:21:37.546 --> 00:21:39.131 It's not asking you to make a judgment 00:21:39.131 --> 00:21:41.800 about your own learning, which we know is inherently flawed. 00:21:41.800 --> 00:21:45.387 Instead, it's soliciting, as Ethan said, soliciting information from you 00:21:45.554 --> 00:21:49.182 to find out what you know and to help you build on your knowledge. 00:21:49.391 --> 00:21:53.478 And these sorts of subtle differences are what separates 00:21:53.562 --> 00:21:57.399 using AI in sort of an expert way in a classroom where we know 00:21:57.482 --> 00:22:00.986 what we want to have happen from just the naive use people are doing, 00:22:05.949 --> 00:22:06.742 I think that 00:22:06.742 --> 00:22:10.537 there is an advantage to taking charge of your students AI usage 00:22:10.537 --> 00:22:12.122 because they're going to be using it anyway 00:22:12.122 --> 00:22:15.417 and thinking about directing it either by giving them prompts, having these discussions 00:22:15.625 --> 00:22:17.044 and it's a really powerful tool 00:22:17.044 --> 00:22:20.047 that in the future will greatly boost classroom learning 00:22:20.047 --> 00:22:23.383 and is not a replacement or threat for teachers. 00:22:23.383 --> 00:22:27.095 It is something that we can use to improve the outputs of our work, 00:22:27.220 --> 00:22:31.558 improve student learning, make our lives easier while making students lives better 00:22:31.683 --> 00:22:34.686 and I think that that's a very powerful view of the future. 00:22:34.686 --> 00:22:37.981 And I hope that you at least embrace and experiment with AI before deciding 00:22:37.981 --> 00:22:40.359 whether you want to use it or ban it in your classrooms. 00:22:41.234 --> 00:22:46.156 Wow. There really are so many ways to enhance student learning using AI. 00:22:46.239 --> 00:22:46.865 With AI 00:22:46.865 --> 00:22:51.286 technology advancing rapidly, there will be more and more tools available. 00:22:51.370 --> 00:22:55.415 As with any new tool, educators have a responsibility to ensure 00:22:55.415 --> 00:23:00.170 they're using age appropriate tools, protecting student privacy 00:23:00.253 --> 00:23:03.590 and creating spaces for students to critically evaluate 00:23:03.673 --> 00:23:07.177 the potential pitfalls of the technology they are using. 00:23:07.260 --> 00:23:11.223 Join us in session 4 Ensuring a Responsible Approach to AI 00:23:11.390 --> 00:23:13.392 as we explore these topics. 00:23:13.392 --> 00:23:14.434 Thanks for joining us. 00:23:14.434 --> 00:23:16.103 See you again In session four. 00:23:16.103 --> 00:23:21.900 Visit the AI 101 for Teachers website at Code.org/ai101 00:23:22.109 --> 00:23:23.735 to sign up for early access 00:23:23.860 --> 00:23:27.072 and to explore additional resources from Code.org. 00:23:27.155 --> 00:23:29.950 ETS, ISTE and Khan Academy. 00:23:29.950 --> 00:23:30.909 Thanks for joining us.