WEBVTT 00:00:00.170 --> 00:00:02.280 [Music] 00:00:02.280 --> 00:00:05.280 foreign 00:00:07.140 --> 00:00:10.080 hi welcome to session three of the AI 00:00:10.080 --> 00:00:12.420 101 for teachers professional learning 00:00:12.420 --> 00:00:13.440 series 00:00:13.440 --> 00:00:15.480 in this session we are traveling to the 00:00:15.480 --> 00:00:17.400 Wharton School at the University of 00:00:17.400 --> 00:00:19.800 Pennsylvania to chat with Dr Ethan 00:00:19.800 --> 00:00:21.900 Mullick a professor who teaches 00:00:21.900 --> 00:00:24.480 Innovation and Entrepreneurship and Dr 00:00:24.480 --> 00:00:26.760 lilac Mullick who works on interactive 00:00:26.760 --> 00:00:29.340 pedagogy and AI research 00:00:29.340 --> 00:00:32.340 they will help us explore how AI can be 00:00:32.340 --> 00:00:34.620 combined with pedagogy to enhance 00:00:34.620 --> 00:00:36.180 student learning 00:00:36.180 --> 00:00:39.380 let's go meet the Molex 00:00:43.680 --> 00:00:45.600 hi I'm Ethan Malek a professor at 00:00:45.600 --> 00:00:48.180 Wharton who has been working on how we 00:00:48.180 --> 00:00:50.160 democratize access to education through 00:00:50.160 --> 00:00:52.739 tools like games and interactive tools 00:00:52.739 --> 00:00:54.360 and AI 00:00:54.360 --> 00:00:56.520 Nami and I'm director of pedagogy of 00:00:56.520 --> 00:00:57.960 Wharton interactive and I've been 00:00:57.960 --> 00:00:59.879 working at the intersection of AI and 00:00:59.879 --> 00:01:02.160 education helping to democratize 00:01:02.160 --> 00:01:04.799 education for everyone through effective 00:01:04.799 --> 00:01:07.740 pedagogically sound use of AI and we 00:01:07.740 --> 00:01:09.900 have been working together on the future 00:01:09.900 --> 00:01:11.760 of education for a while thinking about 00:01:11.760 --> 00:01:14.159 how to make education more interactive 00:01:14.159 --> 00:01:16.500 to work at scale and with the Advent of 00:01:16.500 --> 00:01:18.540 generative AI we've had a powerful new 00:01:18.540 --> 00:01:20.640 tool that can really help in the 00:01:20.640 --> 00:01:22.439 classroom but also carries some risks 00:01:22.439 --> 00:01:23.880 today we'd like to talk a little bit 00:01:23.880 --> 00:01:26.520 about the classroom use of AI upside to 00:01:26.520 --> 00:01:27.659 the downside to give you some examples 00:01:27.659 --> 00:01:30.119 to work with the first we like to start 00:01:30.119 --> 00:01:32.220 with our three guiding principles the 00:01:32.220 --> 00:01:34.619 first is that AI is undetectable there 00:01:34.619 --> 00:01:37.380 are AI tools detection tools but they 00:01:37.380 --> 00:01:39.600 are not effective the second principle 00:01:39.600 --> 00:01:43.280 is that AI is ubiquitous it's everywhere 00:01:43.280 --> 00:01:46.200 169 countries have access to Bing chat 00:01:46.200 --> 00:01:48.240 and you and your students have access to 00:01:48.240 --> 00:01:50.700 the most powerful AI available 00:01:50.700 --> 00:01:53.100 the third principle is that AI is 00:01:53.100 --> 00:01:55.680 transformative it will transform how we 00:01:55.680 --> 00:01:58.259 live how we work and how we teach and 00:01:58.259 --> 00:02:00.439 learn 00:02:01.380 --> 00:02:02.550 foreign 00:02:02.550 --> 00:02:04.979 [Music] 00:02:04.979 --> 00:02:09.539 not going away but this is probably the 00:02:09.539 --> 00:02:11.160 worst AI you're ever going to use so if 00:02:11.160 --> 00:02:13.080 this feels disruptive now I kind of have 00:02:13.080 --> 00:02:15.180 bad news which is that there isn't a 00:02:15.180 --> 00:02:16.680 reason to suspect that AI development 00:02:16.680 --> 00:02:17.700 will not continue and I think people 00:02:17.700 --> 00:02:19.920 worry a lot about like the far future or 00:02:19.920 --> 00:02:22.140 who knows how far it is where AI is 00:02:22.140 --> 00:02:23.400 smarter than humans but even over the 00:02:23.400 --> 00:02:25.680 next couple years even of the fact of 00:02:25.680 --> 00:02:27.959 this Academic Year I would expect AI to 00:02:27.959 --> 00:02:30.120 continue to improve five times ten times 00:02:30.120 --> 00:02:32.640 we have no idea but if you're not 00:02:32.640 --> 00:02:33.840 already thinking about these systems 00:02:33.840 --> 00:02:35.160 what they mean for Education what they 00:02:35.160 --> 00:02:36.300 mean for you what they mean for your 00:02:36.300 --> 00:02:38.280 students careers I think we have to 00:02:38.280 --> 00:02:39.780 think about it because these systems are 00:02:39.780 --> 00:02:41.459 not going to disappear let me make the 00:02:41.459 --> 00:02:43.620 very pragmatic case for why you might 00:02:43.620 --> 00:02:45.540 want to do this for the first part of 00:02:45.540 --> 00:02:47.340 the pragmatic case is your students will 00:02:47.340 --> 00:02:49.260 be using this anyway so you have to come 00:02:49.260 --> 00:02:50.879 to speed I don't think everybody wants 00:02:50.879 --> 00:02:52.140 to be dragged along this technology no 00:02:52.140 --> 00:02:53.940 one asked for education to be massively 00:02:53.940 --> 00:02:55.920 disrupted but it is and unfortunately 00:02:55.920 --> 00:02:57.120 we've got to figure out a way to get 00:02:57.120 --> 00:02:58.080 around that all your homework 00:02:58.080 --> 00:02:59.940 assignments can be done by AI now so you 00:02:59.940 --> 00:03:01.080 have to think about that and then I 00:03:01.080 --> 00:03:02.940 think the second thing is the pragmatic 00:03:02.940 --> 00:03:05.519 argument about how AI can make your life 00:03:05.519 --> 00:03:06.840 easier as a teacher if you put the hours 00:03:06.840 --> 00:03:08.640 down you get them back later and if 00:03:08.640 --> 00:03:09.780 you've worked on a number of prompts to 00:03:09.780 --> 00:03:11.640 help make lives easier for teachers 00:03:11.640 --> 00:03:14.040 yeah so one thing you can do prompts 00:03:14.040 --> 00:03:16.200 like give me a lesson hook prompts like 00:03:16.200 --> 00:03:18.180 create a lesson plan 00:03:18.180 --> 00:03:20.940 um or Create a quiz for me so starting 00:03:20.940 --> 00:03:23.159 to work with your material and the 00:03:23.159 --> 00:03:24.780 different models can get you give you a 00:03:24.780 --> 00:03:26.819 really good sense of how the AI Works 00:03:26.819 --> 00:03:28.920 what is good at what it what it's not 00:03:28.920 --> 00:03:31.860 good at and so and save you time in the 00:03:31.860 --> 00:03:33.540 end right so that's where I would be my 00:03:33.540 --> 00:03:35.879 push to teachers is a you have to and B 00:03:35.879 --> 00:03:37.500 you're going to want to 00:03:37.500 --> 00:03:39.180 I think the other important thing is 00:03:39.180 --> 00:03:42.000 just to try it they're very simple to 00:03:42.000 --> 00:03:43.560 use they're very intuitive because 00:03:43.560 --> 00:03:46.140 they're conversational uh you can 00:03:46.140 --> 00:03:48.900 continue a conversation it feels fairly 00:03:48.900 --> 00:03:51.180 natural and I think the the key really 00:03:51.180 --> 00:03:54.120 is experimentation see how it works with 00:03:54.120 --> 00:03:56.159 you see how it works within your context 00:03:56.159 --> 00:03:59.099 within your topic that you teach our 00:03:59.099 --> 00:04:00.659 rule of thumb is you need about 10 hours 00:04:00.659 --> 00:04:02.760 playing with AI to get what it's good 00:04:02.760 --> 00:04:04.680 and what its limitations are so I would 00:04:04.680 --> 00:04:06.840 actually start by suggesting that this 00:04:06.840 --> 00:04:08.280 that the teacher throw their own 00:04:08.280 --> 00:04:09.659 assignments into the AI and see what 00:04:09.659 --> 00:04:11.459 kind of results they get back I would 00:04:11.459 --> 00:04:14.159 think about asking them uh to ask their 00:04:14.159 --> 00:04:16.560 students to create an assignment using 00:04:16.560 --> 00:04:18.180 Ai and then critique that assignment 00:04:18.180 --> 00:04:20.820 potentially even in class to see for the 00:04:20.820 --> 00:04:22.260 students can get a sense of what the 00:04:22.260 --> 00:04:24.840 gaps and abilities of AI are I have a 00:04:24.840 --> 00:04:26.220 little bit of Freedom as an instructor 00:04:26.220 --> 00:04:28.620 because I'm teaching college and MBA 00:04:28.620 --> 00:04:30.419 students entrepreneurship so I have a 00:04:30.419 --> 00:04:32.100 lot of I have points I want them to make 00:04:32.100 --> 00:04:33.419 but they also are building things and 00:04:33.419 --> 00:04:35.460 doing things and ai's absolutely 00:04:35.460 --> 00:04:38.220 transformed how that works so 00:04:38.220 --> 00:04:39.840 um my assignments now literally call for 00:04:39.840 --> 00:04:41.340 students to do at least one impossible 00:04:41.340 --> 00:04:43.199 thing in class if you can't code you 00:04:43.199 --> 00:04:44.940 have to write working programs if you 00:04:44.940 --> 00:04:46.680 have never if you can't do design work 00:04:46.680 --> 00:04:48.060 you have to create a full graphic design 00:04:48.060 --> 00:04:50.460 working prototypes that's literally that 00:04:50.460 --> 00:04:51.840 part of the class so where it used to be 00:04:51.840 --> 00:04:53.220 right up write a little bit of an essay 00:04:53.220 --> 00:04:55.380 do a prototype on paper now you have to 00:04:55.380 --> 00:04:57.180 create a full working product every 00:04:57.180 --> 00:04:58.979 assignment that they turn in has to be 00:04:58.979 --> 00:05:00.300 critiqued by at least five famous 00:05:00.300 --> 00:05:01.740 entrepreneurs through history and they 00:05:01.740 --> 00:05:03.479 use AI to invoke those there's a 00:05:03.479 --> 00:05:04.979 pedagogical reason too which is that 00:05:04.979 --> 00:05:06.720 entrepreneurs tend to be overconfident 00:05:06.720 --> 00:05:08.100 so you want feedback from different 00:05:08.100 --> 00:05:11.460 sources so to me it is let me teach 10 00:05:11.460 --> 00:05:12.900 times more than I did I used to teach 00:05:12.900 --> 00:05:14.639 Advanced intermediate entrepreneurship 00:05:14.639 --> 00:05:16.680 course I can now in the intermediate or 00:05:16.680 --> 00:05:18.720 basic course get all the way past the 00:05:18.720 --> 00:05:20.759 advanced material and further so I think 00:05:20.759 --> 00:05:21.960 we're going to see that shake out more 00:05:21.960 --> 00:05:23.160 in the future but some of this is about 00:05:23.160 --> 00:05:24.960 powering a past what we could do before 00:05:24.960 --> 00:05:27.060 and I think that's exciting as well 00:05:27.060 --> 00:05:29.580 apart from student tutors as AI 00:05:29.580 --> 00:05:31.699 assignments teachers can certainly use 00:05:31.699 --> 00:05:35.160 AI coaches and AI assistants to help 00:05:35.160 --> 00:05:37.440 students prepare for discussions help 00:05:37.440 --> 00:05:39.539 students outline help students do 00:05:39.539 --> 00:05:41.820 research help students get feedback on 00:05:41.820 --> 00:05:43.620 assignments and just help students 00:05:43.620 --> 00:05:46.259 develop explanations I think there are 00:05:46.259 --> 00:05:48.180 Myriad of approaches that are 00:05:48.180 --> 00:05:49.880 pedagogically sound 00:05:49.880 --> 00:05:52.979 that teachers can assign to students and 00:05:52.979 --> 00:05:55.680 watch their work and ask for the back 00:05:55.680 --> 00:05:57.479 and forth interaction to really see that 00:05:57.479 --> 00:05:58.860 students are paying attention to and 00:05:58.860 --> 00:06:01.880 focusing on the material 00:06:02.759 --> 00:06:03.950 foreign 00:06:03.950 --> 00:06:07.089 [Music] 00:06:07.280 --> 00:06:10.080 teacher's perspective so because of the 00:06:10.080 --> 00:06:12.300 ubiquity of AI you've got some choices 00:06:12.300 --> 00:06:14.759 to make in terms of your AI policies in 00:06:14.759 --> 00:06:16.560 your class so do you want to permit AI 00:06:16.560 --> 00:06:18.720 do you want to forbid AI how are you 00:06:18.720 --> 00:06:19.860 going to enforce these sets of things 00:06:19.860 --> 00:06:21.419 we're going to assume that you want to 00:06:21.419 --> 00:06:23.940 use AI to some extent and we'll dive 00:06:23.940 --> 00:06:25.620 into a little bit of the details here so 00:06:25.620 --> 00:06:27.600 as a instructor you should know a few 00:06:27.600 --> 00:06:29.880 things one is there's obviously ongoing 00:06:29.880 --> 00:06:31.620 ethical debates about Ai and those are 00:06:31.620 --> 00:06:33.240 complicated debates there are debates 00:06:33.240 --> 00:06:35.400 over whether or not the AI is trained on 00:06:35.400 --> 00:06:37.080 the right kind of data about the biases 00:06:37.080 --> 00:06:40.860 AI might have about the use of AI and 00:06:40.860 --> 00:06:42.660 the outcomes for student learning and 00:06:42.660 --> 00:06:44.100 it's worth acknowledging these sets of 00:06:44.100 --> 00:06:46.380 things but this tool is out there and it 00:06:46.380 --> 00:06:47.520 is worth thinking about how you want to 00:06:47.520 --> 00:06:49.800 use it if you decide that that is okay 00:06:49.800 --> 00:06:51.479 and how you want to communicate that 00:06:51.479 --> 00:06:53.819 information beyond the initial ethical 00:06:53.819 --> 00:06:56.100 concerns there's also concerns about how 00:06:56.100 --> 00:06:59.100 AI actually works so the large language 00:06:59.100 --> 00:07:01.680 models that power today's AI don't 00:07:01.680 --> 00:07:03.419 actually have knowledge of the world 00:07:03.419 --> 00:07:05.039 they're predicting the next word they're 00:07:05.039 --> 00:07:06.900 predicting the right kinds of sentences 00:07:06.900 --> 00:07:08.819 or information to give out and as a 00:07:08.819 --> 00:07:11.100 result they make stuff up they would 00:07:11.100 --> 00:07:12.840 call hallucinate so there are often 00:07:12.840 --> 00:07:14.460 errors or mistakes now it's not always 00:07:14.460 --> 00:07:15.840 clear those errors or mistakes are worse 00:07:15.840 --> 00:07:17.100 than the errors and mistakes humans 00:07:17.100 --> 00:07:18.900 would make but you need to be aware that 00:07:18.900 --> 00:07:20.099 there's going to be those kind of errors 00:07:20.099 --> 00:07:22.139 and mistakes and then finally you need 00:07:22.139 --> 00:07:24.599 to think about as a instructor how 00:07:24.599 --> 00:07:26.819 you're going to be using AI to Aid 00:07:26.819 --> 00:07:28.380 learning which means being really clear 00:07:28.380 --> 00:07:30.300 about what you want to accomplish with 00:07:30.300 --> 00:07:32.819 an AI tool they can be boost for student 00:07:32.819 --> 00:07:35.160 learning but ai's many possible uses in 00:07:35.160 --> 00:07:36.900 classroom so do you want to use them to 00:07:36.900 --> 00:07:38.400 have students generate ideas which I do 00:07:38.400 --> 00:07:40.139 in my classes and we get better project 00:07:40.139 --> 00:07:42.180 ideas as a result do you want them to 00:07:42.180 --> 00:07:44.819 use them as tutors to explain Concepts 00:07:44.819 --> 00:07:46.380 to them they don't understand do you 00:07:46.380 --> 00:07:48.240 want these students to get feedback from 00:07:48.240 --> 00:07:51.180 AI by asking it for questions about work 00:07:51.180 --> 00:07:52.740 that they're doing do you want to be a 00:07:52.740 --> 00:07:54.479 writing companion do you wanted to 00:07:54.479 --> 00:07:56.400 explain why quiz answers might be right 00:07:56.400 --> 00:07:57.960 or wrong and then once you've decided 00:07:57.960 --> 00:07:59.340 what to do as an instructor you need to 00:07:59.340 --> 00:08:01.620 decide what you're going to tell your 00:08:01.620 --> 00:08:03.979 students 00:08:06.120 --> 00:08:07.560 [Music] 00:08:07.560 --> 00:08:10.259 AI detectors don't work they just don't 00:08:10.259 --> 00:08:11.819 work you shouldn't use them and it's 00:08:11.819 --> 00:08:13.919 worse than them not working because they 00:08:13.919 --> 00:08:15.660 have a high false positive rate that 00:08:15.660 --> 00:08:17.639 means they flag things as AI written 00:08:17.639 --> 00:08:19.680 that aren't AI written and that 00:08:19.680 --> 00:08:21.360 disproportionately falls on people whose 00:08:21.360 --> 00:08:23.699 English is a second language this is 00:08:23.699 --> 00:08:25.199 just not something that we can do and I 00:08:25.199 --> 00:08:27.419 think trying to close the barn door here 00:08:27.419 --> 00:08:29.520 after it's been opened and try and 00:08:29.520 --> 00:08:31.740 detect AI is not the future for 00:08:31.740 --> 00:08:34.080 responsibility in classrooms the the 00:08:34.080 --> 00:08:36.620 other thing to note too is that students 00:08:36.620 --> 00:08:39.360 were using shortcuts in the past it's 00:08:39.360 --> 00:08:40.919 not that they weren't using Google it's 00:08:40.919 --> 00:08:42.120 so that they weren't using you know 00:08:42.120 --> 00:08:43.860 other students essays this was happening 00:08:43.860 --> 00:08:45.600 in the past but this is a major 00:08:45.600 --> 00:08:47.940 disruption and I think it does call for 00:08:47.940 --> 00:08:50.820 a rethinking of how we do essays so 00:08:50.820 --> 00:08:52.800 thinking a little bit more about the 00:08:52.800 --> 00:08:54.600 learning goal for an essay or the 00:08:54.600 --> 00:08:58.160 learning goal for any assignments 00:09:00.340 --> 00:09:02.399 [Music] 00:09:02.399 --> 00:09:04.500 one of the things that we're noticing as 00:09:04.500 --> 00:09:06.300 we watch teachers do this is they all 00:09:06.300 --> 00:09:09.120 feel an obligation to talk about Ai and 00:09:09.120 --> 00:09:10.800 dive deep into the ethical implications 00:09:10.800 --> 00:09:13.380 of AI and so on I think that's important 00:09:13.380 --> 00:09:15.120 but I don't think it needs to be the 00:09:15.120 --> 00:09:16.620 theme of every class I don't think every 00:09:16.620 --> 00:09:18.600 class needs to be a discussion about AI 00:09:18.600 --> 00:09:20.519 just like every class that uses 00:09:20.519 --> 00:09:21.540 computers doesn't need to be a 00:09:21.540 --> 00:09:23.279 discussion about computers I think it's 00:09:23.279 --> 00:09:24.600 important to have that conversation and 00:09:24.600 --> 00:09:26.820 right now we're all just reacting so 00:09:26.820 --> 00:09:28.200 it's not clear who's supposed to have 00:09:28.200 --> 00:09:30.180 that so I totally get teachers wanting 00:09:30.180 --> 00:09:31.980 to have ai discussions but it's even 00:09:31.980 --> 00:09:34.560 harder to come up to speed not just on 00:09:34.560 --> 00:09:37.080 the use of AI but how it works it's you 00:09:37.080 --> 00:09:39.240 know standards it's ethical implications 00:09:39.240 --> 00:09:41.640 so I think teachers should feel a little 00:09:41.640 --> 00:09:44.279 bit of um okayness with experimenting 00:09:44.279 --> 00:09:45.899 with AI without having to make it the 00:09:45.899 --> 00:09:48.779 subject of class 00:09:48.779 --> 00:09:50.959 foreign 00:09:51.220 --> 00:09:53.339 [Music] 00:09:53.339 --> 00:09:55.860 first is as Ethan mentioned that AI can 00:09:55.860 --> 00:09:57.839 fabricate that means that any output 00:09:57.839 --> 00:10:00.300 that the AI gives a student may be made 00:10:00.300 --> 00:10:03.480 up it may be mistaken it may be very 00:10:03.480 --> 00:10:05.459 subtly mistaken and so students should 00:10:05.459 --> 00:10:08.100 be responsible for their own work they 00:10:08.100 --> 00:10:09.959 should at the very least check sources 00:10:09.959 --> 00:10:12.300 check any number check any facts that 00:10:12.300 --> 00:10:13.740 the AI gives them and check them with 00:10:13.740 --> 00:10:16.560 credible sources the second principle is 00:10:16.560 --> 00:10:19.260 that the AI is not a person it's easy to 00:10:19.260 --> 00:10:21.660 imbue the AI with a personality or to 00:10:21.660 --> 00:10:23.700 feel like you're talking to a person but 00:10:23.700 --> 00:10:25.440 it's not a person and it doesn't know 00:10:25.440 --> 00:10:28.080 you the third principle is really to 00:10:28.080 --> 00:10:30.660 give it a lot of context the AI doesn't 00:10:30.660 --> 00:10:32.459 know you it doesn't know your context or 00:10:32.459 --> 00:10:34.980 your experience or your expertise the 00:10:34.980 --> 00:10:37.200 more context you give it the more useful 00:10:37.200 --> 00:10:38.940 it'll be for you and the fourth 00:10:38.940 --> 00:10:41.399 principle is that you're in charge not 00:10:41.399 --> 00:10:43.380 only should you evaluate and interrogate 00:10:43.380 --> 00:10:45.540 its output but if it's leading you in a 00:10:45.540 --> 00:10:48.060 conversation that is no longer useful to 00:10:48.060 --> 00:10:50.459 to you or if it's stuck in a loop or if 00:10:50.459 --> 00:10:51.899 you'd like to change the direction of 00:10:51.899 --> 00:10:54.000 the conversation you should ask feel 00:10:54.000 --> 00:10:56.839 free to take charge 00:10:57.120 --> 00:10:58.370 thank you 00:10:58.370 --> 00:11:01.860 [Music] 00:11:01.860 --> 00:11:05.940 so when we talk about Ai and these 00:11:05.940 --> 00:11:08.220 generative AI Solutions we tend to talk 00:11:08.220 --> 00:11:10.019 about large language models and there's 00:11:10.019 --> 00:11:12.720 actually only a few large-scale general 00:11:12.720 --> 00:11:14.640 purpose large language models there is 00:11:14.640 --> 00:11:17.160 the models created by openai which are 00:11:17.160 --> 00:11:20.519 GPT 3.5 or gbt4 gbd 3.5 is the free 00:11:20.519 --> 00:11:23.579 version that you get through uh through 00:11:23.579 --> 00:11:26.940 chat CBT and gbt4 is either through the 00:11:26.940 --> 00:11:29.100 page chat gbt or through Microsoft Bing 00:11:29.100 --> 00:11:32.160 in creative mode and when we talk about 00:11:32.160 --> 00:11:34.320 specialized apps almost all of them are 00:11:34.320 --> 00:11:36.360 using one of these models and providing 00:11:36.360 --> 00:11:38.459 prompts and other information on top of 00:11:38.459 --> 00:11:41.459 it I generally think instructors should 00:11:41.459 --> 00:11:42.959 get familiar with the models themselves 00:11:42.959 --> 00:11:45.420 because those are the models that are 00:11:45.420 --> 00:11:47.399 actually producing the answers and you 00:11:47.399 --> 00:11:48.779 can manipulate them directly that way 00:11:48.779 --> 00:11:50.220 and learn how they work so if you're 00:11:50.220 --> 00:11:51.480 trying to buy an off-the-shelf solution 00:11:51.480 --> 00:11:53.040 they're almost only using one of these 00:11:53.040 --> 00:11:55.320 existing models and then providing some 00:11:55.320 --> 00:11:57.000 sort of wrapper or other information on 00:11:57.000 --> 00:11:58.980 top of it and it's often cheaper and 00:11:58.980 --> 00:12:00.480 more effective and gives you more 00:12:00.480 --> 00:12:02.399 control to use the foundation models 00:12:02.399 --> 00:12:04.140 yourself but that's a choice you get to 00:12:04.140 --> 00:12:06.660 make so when develop helping the prompt 00:12:06.660 --> 00:12:09.959 we really and for all of our prompts we 00:12:09.959 --> 00:12:11.579 really look at the science of learning 00:12:11.579 --> 00:12:14.100 and try to combine that with the power 00:12:14.100 --> 00:12:16.860 of the AI so for instance a good tutor 00:12:16.860 --> 00:12:19.560 pushes you for information uh it doesn't 00:12:19.560 --> 00:12:22.380 just hand it to you a good tutor finds 00:12:22.380 --> 00:12:24.480 out what you know and builds on that 00:12:24.480 --> 00:12:25.820 prior knowledge 00:12:25.820 --> 00:12:29.160 a good tutor will also find out a little 00:12:29.160 --> 00:12:31.440 bit about you a good tutor also knows 00:12:31.440 --> 00:12:33.839 that you need lots and varied kinds of 00:12:33.839 --> 00:12:37.440 examples and analogies and a good tutor 00:12:37.440 --> 00:12:39.540 knows that the way that you show 00:12:39.540 --> 00:12:41.579 evidence of Mastery is by being able to 00:12:41.579 --> 00:12:44.040 explain something in your own words to 00:12:44.040 --> 00:12:46.100 someone else and give an example of it 00:12:46.100 --> 00:12:48.779 which is exactly these are exactly the 00:12:48.779 --> 00:12:51.000 steps and the kinds of questions that we 00:12:51.000 --> 00:12:53.100 use in the tutor prompt but you'll 00:12:53.100 --> 00:12:54.720 notice when you look at our prompts that 00:12:54.720 --> 00:12:56.940 they do things like provide context to 00:12:56.940 --> 00:12:58.980 the AI as Leo has discussed already the 00:12:58.980 --> 00:13:01.200 idea that it it asks you who you are and 00:13:01.200 --> 00:13:03.240 we tell the AI who it is it's an 00:13:03.240 --> 00:13:04.680 instructor with this kind of setting 00:13:04.680 --> 00:13:07.139 you'll notice that it also tells it 00:13:07.139 --> 00:13:09.480 exactly the scientific framework to use 00:13:09.480 --> 00:13:12.240 this idea of context matters we provide 00:13:12.240 --> 00:13:14.279 controls we ask it to go step by step 00:13:14.279 --> 00:13:16.019 through sets of questions to ask 00:13:16.019 --> 00:13:17.579 sometimes not in these prompts we 00:13:17.579 --> 00:13:19.260 provide examples of good output and then 00:13:19.260 --> 00:13:22.079 we test it a lot you can't do prompting 00:13:22.079 --> 00:13:23.579 without testing and that's one of the 00:13:23.579 --> 00:13:24.600 great things about testing your 00:13:24.600 --> 00:13:27.779 expertise it's cheap to do and so you 00:13:27.779 --> 00:13:29.399 get to experiment a lot and that makes 00:13:29.399 --> 00:13:31.680 for good prompts and and we should also 00:13:31.680 --> 00:13:33.839 say we tested not just on one model but 00:13:33.839 --> 00:13:36.180 on several models so for instance these 00:13:36.180 --> 00:13:38.160 two prompts we just worked with chat 00:13:38.160 --> 00:13:41.639 gpt4 they also work with Bing Bing will 00:13:41.639 --> 00:13:44.279 react a little bit differently and it 00:13:44.279 --> 00:13:46.019 will um because it's connected to the 00:13:46.019 --> 00:13:47.940 internet it will also look up citations 00:13:47.940 --> 00:13:49.260 sometimes they're the right citations 00:13:49.260 --> 00:13:50.880 sometimes they're not but that is 00:13:50.880 --> 00:13:53.760 available it may or may not work with 00:13:53.760 --> 00:13:54.959 some of the other models so you really 00:13:54.959 --> 00:13:56.700 have to test it I think as an instructor 00:13:56.700 --> 00:13:59.339 before you give it to your students in 00:13:59.339 --> 00:14:01.019 the context of the topic that you're 00:14:01.019 --> 00:14:03.300 teaching to see how it works so this is 00:14:03.300 --> 00:14:04.980 all very theoretical but I think 00:14:04.980 --> 00:14:06.600 important so let's let's get practical 00:14:06.600 --> 00:14:08.339 let's talk about some examples of what 00:14:08.339 --> 00:14:10.620 AI can do and again this isn't a 00:14:10.620 --> 00:14:12.720 monolithical thing a AI has many 00:14:12.720 --> 00:14:14.160 possible uses as we said it was 00:14:14.160 --> 00:14:15.779 transformative earlier so we're going to 00:14:15.779 --> 00:14:17.459 show you a couple of prompts that we've 00:14:17.459 --> 00:14:18.839 created and those will be available to 00:14:18.839 --> 00:14:22.079 you as well to work with and these are 00:14:22.079 --> 00:14:25.079 just examples of the ways AI classroom 00:14:25.079 --> 00:14:27.000 use can work so the first one we want to 00:14:27.000 --> 00:14:29.160 show you is a prompt that I believe you 00:14:29.160 --> 00:14:31.800 created that talks about uh feedback 00:14:31.800 --> 00:14:34.380 that gives proper feedback and 00:14:34.380 --> 00:14:35.579 one of the really interesting things 00:14:35.579 --> 00:14:37.620 about the AI side of things is a more 00:14:37.620 --> 00:14:38.880 sophisticated prompt that takes into 00:14:38.880 --> 00:14:40.139 account some of the principles we were 00:14:40.139 --> 00:14:41.459 talking about earlier will result in 00:14:41.459 --> 00:14:43.199 better outcomes so students will often 00:14:43.199 --> 00:14:44.820 ask for writing advice from an AI even 00:14:44.820 --> 00:14:46.260 if you tell them not to do it but 00:14:46.260 --> 00:14:47.160 they're going to ask for it in a way 00:14:47.160 --> 00:14:48.660 that's fairly unsophisticated and is 00:14:48.660 --> 00:14:49.800 going to give them fairly generic 00:14:49.800 --> 00:14:51.660 sounding work and possibly more mistakes 00:14:51.660 --> 00:14:54.060 if you give a more elaborate prompt you 00:14:54.060 --> 00:14:55.260 can get more elaborate answers so in 00:14:55.260 --> 00:14:56.880 this case could you explain what this 00:14:56.880 --> 00:14:59.399 prompt does the feedback prompt yes so 00:14:59.399 --> 00:15:02.699 we combine the principles of good 00:15:02.699 --> 00:15:04.740 feedback which is feedback that takes 00:15:04.740 --> 00:15:06.720 into account your prior knowledge or 00:15:06.720 --> 00:15:08.579 what you already know from the student 00:15:08.579 --> 00:15:10.680 perspective takes into account who you 00:15:10.680 --> 00:15:12.660 are your learning level what grade 00:15:12.660 --> 00:15:14.760 you're in uh whether you're in college 00:15:14.760 --> 00:15:17.399 or you're a professional and it also 00:15:17.399 --> 00:15:19.920 takes into account uh the idea that you 00:15:19.920 --> 00:15:22.440 want to respond to this feedback so it 00:15:22.440 --> 00:15:24.120 is going to be actionable it's going to 00:15:24.120 --> 00:15:25.500 be balanced it's going to tell you 00:15:25.500 --> 00:15:27.899 what's wrong and what you can improve on 00:15:27.899 --> 00:15:30.060 and what you're doing well and it's 00:15:30.060 --> 00:15:31.740 going to keep working with you but like 00:15:31.740 --> 00:15:34.019 any good tutor or coach it won't 00:15:34.019 --> 00:15:36.300 actually give you the answer it'll push 00:15:36.300 --> 00:15:38.459 you in that direction ask you to explain 00:15:38.459 --> 00:15:40.560 ask you to construct your own knowledge 00:15:40.560 --> 00:15:43.380 and so you can see the prompt hopefully 00:15:43.380 --> 00:15:46.019 on the screen here uh and uh as a place 00:15:46.019 --> 00:15:48.120 to work from you don't need to take this 00:15:48.120 --> 00:15:50.339 as an absolute answer this is something 00:15:50.339 --> 00:15:51.720 you can play with but let's see it in 00:15:51.720 --> 00:15:53.880 action so let's get started using this 00:15:53.880 --> 00:15:55.560 prompt it says that it's a teaching 00:15:55.560 --> 00:15:56.579 assistant because that's the 00:15:56.579 --> 00:15:58.320 instructions we gave it and it asks us 00:15:58.320 --> 00:16:00.600 for our grade level and subject we're 00:16:00.600 --> 00:16:02.760 studying what should we say so I think 00:16:02.760 --> 00:16:05.040 we're studying Macbeth and we're in 12th 00:16:05.040 --> 00:16:09.320 grade in 12th grade okay great 00:16:11.459 --> 00:16:14.160 okay and so we've told the AI as 00:16:14.160 --> 00:16:16.800 information it's feeding it into the 00:16:16.800 --> 00:16:18.540 logic that it's using here and it's 00:16:18.540 --> 00:16:20.940 asking us about a specific assignment 00:16:20.940 --> 00:16:22.320 um and it's asking if we have a rubric 00:16:22.320 --> 00:16:23.519 or other information to work with or 00:16:23.519 --> 00:16:25.680 what we're hoping to achieve with uh 00:16:25.680 --> 00:16:27.300 with as much of Rage as possible I don't 00:16:27.300 --> 00:16:29.639 have a huge amount here so I'll say I 00:16:29.639 --> 00:16:35.459 have to write an analysis of Macbeth 00:16:35.459 --> 00:16:38.720 it is graded 00:16:38.820 --> 00:16:40.980 based on 00:16:40.980 --> 00:16:46.019 writing style and depth of content 00:16:46.019 --> 00:16:47.940 and you'll see what it's going is it's 00:16:47.940 --> 00:16:49.560 asking us questions soliciting 00:16:49.560 --> 00:16:51.300 information from us which makes it a 00:16:51.300 --> 00:16:52.500 kind of a good prompt that you might 00:16:52.500 --> 00:16:54.839 hand a student better than one that is 00:16:54.839 --> 00:16:56.279 just they're just developing themselves 00:16:56.279 --> 00:16:57.540 and it's asking about Specific 00:16:57.540 --> 00:16:59.339 Instructions and as gets to share the 00:16:59.339 --> 00:17:01.800 assignment uh here is what I have 00:17:01.800 --> 00:17:04.459 written so far 00:17:05.280 --> 00:17:08.100 and I have asked the AI to generate a 00:17:08.100 --> 00:17:11.280 Macbeth essay so here we go I'm just 00:17:11.280 --> 00:17:13.199 pasting that in 00:17:13.199 --> 00:17:14.699 and we'll see what it says here and 00:17:14.699 --> 00:17:17.100 you'll notice it's it's it's working on 00:17:17.100 --> 00:17:18.299 the information it's saying it's taking 00:17:18.299 --> 00:17:19.980 time to carefully read through it that's 00:17:19.980 --> 00:17:21.480 a bit of an illusion it's obviously not 00:17:21.480 --> 00:17:23.220 taking any extra time but uh it's 00:17:23.220 --> 00:17:24.540 responding in this method and you'll see 00:17:24.540 --> 00:17:25.679 it's giving a set of strengths and 00:17:25.679 --> 00:17:27.480 weaknesses what's great about again 00:17:27.480 --> 00:17:29.820 using a tutor that you've built or a 00:17:29.820 --> 00:17:31.380 mentor that you built is that it can 00:17:31.380 --> 00:17:32.460 give you the kind of feedback that's 00:17:32.460 --> 00:17:34.500 educationally valuable that ties into 00:17:34.500 --> 00:17:36.240 pedagogy rather than just students 00:17:36.240 --> 00:17:38.340 asking make this essay better a nice 00:17:38.340 --> 00:17:41.700 example of AI working in your favor uh 00:17:41.700 --> 00:17:43.440 as an educator and not necessarily 00:17:43.440 --> 00:17:44.940 working against you and undermining the 00:17:44.940 --> 00:17:46.260 points you're making 00:17:46.260 --> 00:17:48.419 so you'll notice at the End by the way 00:17:48.419 --> 00:17:51.120 it gives a question that you for the 00:17:51.120 --> 00:17:53.760 student's answer so how do you plan to 00:17:53.760 --> 00:17:55.559 revise your analysis give me a plan on 00:17:55.559 --> 00:17:56.520 specific changes you're going to make 00:17:56.520 --> 00:17:58.140 again the kind of thing we would do as 00:17:58.140 --> 00:17:59.160 an instructor in our classroom 00:17:59.160 --> 00:18:02.400 soliciting changes or differences so I 00:18:02.400 --> 00:18:06.480 think you can start to see why a uh tool 00:18:06.480 --> 00:18:08.640 like this can be really useful when 00:18:08.640 --> 00:18:11.039 properly applied now let's also talk 00:18:11.039 --> 00:18:13.320 about one other potential use for ai ai 00:18:13.320 --> 00:18:14.400 as a tutor what are some of the 00:18:14.400 --> 00:18:15.539 advantages of disadvantage of that 00:18:15.539 --> 00:18:17.340 approach so an advantage of this 00:18:17.340 --> 00:18:19.320 approach is that you're getting students 00:18:19.320 --> 00:18:20.760 to actually pay attention to the 00:18:20.760 --> 00:18:22.919 material you're getting them to read 00:18:22.919 --> 00:18:25.500 over the rubric to read over the purpose 00:18:25.500 --> 00:18:27.480 of the essay and the audience and to 00:18:27.480 --> 00:18:30.299 really think through it a disadvantage 00:18:30.299 --> 00:18:33.600 is that you you certainly can ask the AI 00:18:33.600 --> 00:18:36.600 to do it for you but if you work with it 00:18:36.600 --> 00:18:37.980 and if you're given guidelines to work 00:18:37.980 --> 00:18:40.500 with it it's one way to get feedback 00:18:40.500 --> 00:18:42.360 that you would then have to evaluate 00:18:42.360 --> 00:18:44.280 something else that a teacher could do 00:18:44.280 --> 00:18:46.980 is to ask for the interaction and ask 00:18:46.980 --> 00:18:48.660 for a reflection about the interactions 00:18:48.660 --> 00:18:50.580 what about this feedback was good what 00:18:50.580 --> 00:18:52.799 about this feedback was not as good and 00:18:52.799 --> 00:18:54.720 again it's a higher order level thinking 00:18:54.720 --> 00:18:58.740 about your essay and and your process 00:18:58.740 --> 00:19:01.020 excellent and so I want to show you one 00:19:01.020 --> 00:19:02.880 other example where the AI acts as a 00:19:02.880 --> 00:19:05.220 direct instructor and we have a prompt 00:19:05.220 --> 00:19:07.140 for that as well there are risks 00:19:07.140 --> 00:19:09.059 associated with asking the AI to be a 00:19:09.059 --> 00:19:10.140 direct instructor which is that 00:19:10.140 --> 00:19:12.240 hallucination risk it doesn't know your 00:19:12.240 --> 00:19:15.480 pedagogy your your your perspective but 00:19:15.480 --> 00:19:17.460 I find in my classrooms that students 00:19:17.460 --> 00:19:20.340 are increasingly using the AI as a 00:19:20.340 --> 00:19:21.780 method of learning so they don't raise 00:19:21.780 --> 00:19:23.039 their hands as much I'm going to ask 00:19:23.039 --> 00:19:24.120 them why they're like well I'd rather 00:19:24.120 --> 00:19:25.679 not show my evenings in class I could 00:19:25.679 --> 00:19:28.320 ask the AI to explain like I'm five so 00:19:28.320 --> 00:19:29.760 they're already engaging this Behavior 00:19:29.760 --> 00:19:32.460 so something like a tutor both does a 00:19:32.460 --> 00:19:33.840 useful thing of showing you what the 00:19:33.840 --> 00:19:35.820 future of AI education might look like 00:19:35.820 --> 00:19:38.100 uh like the way Khan Academy is building 00:19:38.100 --> 00:19:40.380 AI interactive tutors to work and it 00:19:40.380 --> 00:19:42.059 also might be a tool your students can 00:19:42.059 --> 00:19:44.280 use to achieve more in class but you 00:19:44.280 --> 00:19:45.660 should caveat that with the knowledge 00:19:45.660 --> 00:19:48.720 that AI tutors are not 100 there yet but 00:19:48.720 --> 00:19:50.280 let's let's use an example here so this 00:19:50.280 --> 00:19:52.080 tutor is again trying to take the right 00:19:52.080 --> 00:19:53.400 kind of format it says hello there I'm 00:19:53.400 --> 00:19:54.720 your AI tutor and I'm excited to work 00:19:54.720 --> 00:19:56.580 with you today what do we want to learn 00:19:56.580 --> 00:19:59.000 about today 00:19:59.100 --> 00:20:01.200 opportunity cost the concept from 00:20:01.200 --> 00:20:04.320 economics let's see what happens 00:20:04.320 --> 00:20:06.179 okay so we're telling the AI where the 00:20:06.179 --> 00:20:08.400 opportunity cost it's saying it is a key 00:20:08.400 --> 00:20:10.320 concept of Economics that's even throw a 00:20:10.320 --> 00:20:12.780 little Emoji here which is cute uh can 00:20:12.780 --> 00:20:14.400 you ask us about our learning level what 00:20:14.400 --> 00:20:17.340 level are we at here 11th grade 11th 00:20:17.340 --> 00:20:18.360 grade 00:20:18.360 --> 00:20:20.940 now I wouldn't get too tied up on the 00:20:20.940 --> 00:20:22.740 individual grade it's not amazing at 00:20:22.740 --> 00:20:24.240 differentiating a 10th grader from 11th 00:20:24.240 --> 00:20:25.799 grader but this is part of the context 00:20:25.799 --> 00:20:27.059 in which it's working and so it's not 00:20:27.059 --> 00:20:28.679 pulling from some sort of universal 00:20:28.679 --> 00:20:31.679 standards here and it says what do we 00:20:31.679 --> 00:20:34.320 know about opportunity cost 00:20:34.320 --> 00:20:36.600 well we know that it has to do with 00:20:36.600 --> 00:20:40.380 choices has to do with alternative 00:20:40.380 --> 00:20:41.880 choices 00:20:41.880 --> 00:20:44.340 that is it and of course the advance one 00:20:44.340 --> 00:20:45.539 of the advantages of AI is this kind of 00:20:45.539 --> 00:20:48.480 freeform texted interaction is the real 00:20:48.480 --> 00:20:50.460 power of education and it's something 00:20:50.460 --> 00:20:52.260 the AI can fake reasonably well again 00:20:52.260 --> 00:20:54.120 not as well as a real human instructor 00:20:54.120 --> 00:20:56.580 yet and you'll notice it's giving us 00:20:56.580 --> 00:20:59.340 examples and explaining things in 00:20:59.340 --> 00:21:01.860 different ways which is a powerful thing 00:21:01.860 --> 00:21:03.240 that AI can do it's very good at 00:21:03.240 --> 00:21:04.679 breaking things down in different ways 00:21:04.679 --> 00:21:06.179 but you'll notice that it's now starting 00:21:06.179 --> 00:21:08.940 to ask questions it's asking us to make 00:21:08.940 --> 00:21:10.320 choices so one of the things we know 00:21:10.320 --> 00:21:12.120 from the research on tutoring is you 00:21:12.120 --> 00:21:14.160 can't just declaim things to people the 00:21:14.160 --> 00:21:15.179 advantage of tutoring comes from 00:21:15.179 --> 00:21:16.740 soliciting information making 00:21:16.740 --> 00:21:18.360 connections and you can see the AI 00:21:18.360 --> 00:21:19.679 starting to do this and asking us for 00:21:19.679 --> 00:21:21.720 Connections in our own life the other 00:21:21.720 --> 00:21:23.580 thing to mention too about the Tudor 00:21:23.580 --> 00:21:26.460 prompt is that it is not assuming that 00:21:26.460 --> 00:21:28.380 the student can judge their own learning 00:21:28.380 --> 00:21:31.320 uh very often you'll see in a tutor 00:21:31.320 --> 00:21:33.179 prompt that is very simple like explain 00:21:33.179 --> 00:21:35.280 to me like I'm 10 It'll ask you if you 00:21:35.280 --> 00:21:37.919 understand something instead here it's 00:21:37.919 --> 00:21:39.419 not asking you to make a judgment about 00:21:39.419 --> 00:21:40.919 your own learning which we know is 00:21:40.919 --> 00:21:42.600 inherently flawed instead it's 00:21:42.600 --> 00:21:44.580 soliciting as Ethan said soliciting 00:21:44.580 --> 00:21:46.799 information from you to find out what 00:21:46.799 --> 00:21:48.960 you know and to help you build on your 00:21:48.960 --> 00:21:50.760 knowledge and these sorts of subtle 00:21:50.760 --> 00:21:53.520 differences are what separates 00:21:53.520 --> 00:21:56.280 using AI in sort of an expert way in a 00:21:56.280 --> 00:21:58.320 classroom where we know what we want to 00:21:58.320 --> 00:22:00.539 have happen from just the naive use 00:22:00.539 --> 00:22:03.080 people are doing 00:22:04.830 --> 00:22:06.000 [Music] 00:22:06.000 --> 00:22:08.220 I I think that there is an advantage to 00:22:08.220 --> 00:22:10.559 taking charge of your students AI usage 00:22:10.559 --> 00:22:11.640 because they're going to be using it 00:22:11.640 --> 00:22:13.440 anyway and thinking about directing it 00:22:13.440 --> 00:22:14.760 either by giving them prompts having 00:22:14.760 --> 00:22:16.500 these discussions and it's a really 00:22:16.500 --> 00:22:18.480 powerful tool that in the future will 00:22:18.480 --> 00:22:20.760 greatly boost classroom learning and is 00:22:20.760 --> 00:22:22.740 not a replacement or threat for students 00:22:22.740 --> 00:22:24.600 for teachers it is something that we can 00:22:24.600 --> 00:22:27.240 use to improve the outputs of our work 00:22:27.240 --> 00:22:29.400 improve student learning make our lives 00:22:29.400 --> 00:22:31.380 easier while making students lives 00:22:31.380 --> 00:22:33.059 better and I think that that's a very 00:22:33.059 --> 00:22:35.340 powerful view of the future and I hope 00:22:35.340 --> 00:22:36.419 that you at least Embrace that 00:22:36.419 --> 00:22:38.159 experiment with AI before deciding 00:22:38.159 --> 00:22:39.780 whether you want to use it or ban it in 00:22:39.780 --> 00:22:41.220 your classrooms 00:22:41.220 --> 00:22:43.919 wow there really are so many ways to 00:22:43.919 --> 00:22:46.260 enhance student learning using AI 00:22:46.260 --> 00:22:48.840 with AI technology advancing rapidly 00:22:48.840 --> 00:22:50.460 there will be more and more tools 00:22:50.460 --> 00:22:53.640 available as with any new tool Educators 00:22:53.640 --> 00:22:55.799 have a responsibility to ensure they are 00:22:55.799 --> 00:22:57.960 using age-appropriate tools 00:22:57.960 --> 00:23:01.020 protecting student privacy and creating 00:23:01.020 --> 00:23:02.880 spaces for students to critically 00:23:02.880 --> 00:23:05.460 evaluate the potential of pitfalls of 00:23:05.460 --> 00:23:08.400 the technology they are using join us in 00:23:08.400 --> 00:23:10.380 session four ensuring a responsible 00:23:10.380 --> 00:23:12.659 approach to AI as we explore these 00:23:12.659 --> 00:23:15.000 topics thanks for joining us see you 00:23:15.000 --> 00:23:18.059 again in session four visit the AI 101 00:23:18.059 --> 00:23:22.380 for teachers website at co.org AI 101 to 00:23:22.380 --> 00:23:24.960 sign up for Early Access and to explore 00:23:24.960 --> 00:23:27.620 additional resources from code.org ETS 00:23:27.620 --> 00:23:30.840 IST and Khan Academy thanks for joining 00:23:30.840 --> 00:23:33.020 us