>> Vance Stevens: We're live! Hello, everybody. Somehow my video disappeared. It's there, but that's my - it's just in avatar format. Plus does that every now and then. OK, well anyway, this is Vance Stevens in Abu Dhabi..., no, sorry, in Al Ain. I'm living in Al Ain now, I forget where I'm living. Today is the 8th of December. They move me around so much, you know. And, anyway, it's the 8th of December 2013. We're talking with a good friend of mine, Phil Hubbard, from Stanford University in Palo Alto, California. And he's been doing some really neat stuff in Cal. I've known him for a long time in the Cal Intersection TESOL. >> Phil Hubbard: Since we were kids. >> Stevens: We were, it was like 20 years ago [Hubbard laughs] >> Hubbard: reaching 30 [check] [background voice] >> Stevens: Someone has a -- someone needs to have a headset on. [missed words] is muted. Errh not sure: it could be someone listening to the stream. Yeah, if you're listening to the stream -- OK. The echo has gone away [check] Someone has corrected it, that's good. All right, well, OK. Someone has announced in the stream chat that they're listening to it there. So that's good, everything seems to be working. We're doing a Hangout on Air, as we often do. We're streaming it on webheadsinaction.org/live At the moment we have six people in the hangout, there's room for four more. So if anyone is listening on the stream and would like to join us, they can. And right now we've got Claire Siskin and Jim Buckingham, Rita Zeinstejer and let's see, and also Rob, Rob is there, and me, Vance Stevens. Rob Permanus, is that correct? Correct me if I'm wrong. Permanus, Permanus - how do you pronounce your name? >> Hubbard: You have to unmute him chuckles >> Stevens: it's Perhamus -- Perhamus, OK, Good, I'll never forget that again, all right. Thank you very much, Rob. Rob is an occasional participant in our hangouts. Well Phil, take it away and anybody who wants to -- by the way, you're all muted by default when you come into the hangout. You can unmute yourself. If you're going to unmute yourself and talk, please mute yourself again, so we don't get keyboard noises and things like that. And there's Elizabeth Anne, also shown up from Grenoble in France. And Halima Ozimova in Tashkent has also joined us, I see. >> Hubbard [check] I think we're great, well, hello, everybody. It's Good Morning for me, a little early in the morning, but the sun is beginning to show through the back window here. Thank you all for being here from all over the world. What I wanted to do today is talk about largely an idea and a project that I've been working on for the last couple of years, very sporadically. Unfortunately I get interrupted easily, as I'm sure all of you do, so what started out as a -- what I hoped was going to be a much more robust collection of materials has turned out to be a little more anemic but I still think that I have enough here that I can demonstrate the idea and especially share my thoughts about how to go about dealing with this relatively new notion of curation, although in some ways, maybe it's just a label for an old notion that we've had for quite some time. So, let me give you a little bit of the background, like several of the things I've worked on in the last few years, like learner training. This is something that has emerged out of my classroom experience with an advanced listening and vocabulary class, and I see Vance is showing some of the slides now. The class is for graduate students at Stanford and it's a really nice sandbox for playing with ideas, because these are -- well, they're all in graduate school already, they're, for the most part, in the high 90's onwards to the 100s in the TOEFL iBT so they really are advanced in that sense. And many of them are taking the course because we require them to do it. So they're kind of a captive audience but it's also a small course: we have a maximum 14 students in it and it allows me to not only play around with ideas, but get a chance to talk to the students afterward, not usually with formal research, but just informally as part of our normal tutorial sessions and find out what they thought about them and what I can do to make them work a little better. So, the problem that I noticed - an important part of this class is that students do independent projects and those independent projects are supposed to be for a minimum of three hours a week. Sounds like I am getting some echo in the background, but I will keep pushing through here.. Uhh.. those projects are for three hours a week and they are responsible for doing the selection of the material with my help and with my guidance both before and after. And over the years, I have discovered that they are actually not really good at that. What they are good at is finding material that is interesting to them. But, they are not necessarily good at finding material that helps them. They discover that on their own a little bit down the road and often it doesn't become clear to both of us because I have a very slow learning curve and quickly forget things. So, I get to the end of the class and then I go "Oh, I should have provided them with a little more guidance.". So, about 2 years ago, I started doing this and it came as a juxtaposition of a couple of things. First of all, just my own general interest in the development of autonomy had been growing and as I have gone out and collected materials that I would just use in class, it was pretty clear to me that there is a huge amount of really interesting materials out there. And people have been collecting these for a while and teachers have been building lessons out of them -- sometimes pretty sophisiticated lessons -- but I needed something that students could work with on their own. And so, I wanted to find a way to help them without just my advice as to how to look for materials, to actually start collecting materials in ways that would still give them quite a bit of freedom of choice but would also make it better as a language learning experience. As part of this course, they are also required to build vocabulary. They have to identify at least 35 new words and phrases every week, from the material they are using. So, this is a bit of the backdrop. In 2011, I came across a book, kind of independently. It was just recommended to me, for some reason, by Amazon: you know how that works. And the book was called 'Curation Nation' and there is, I think, a slide there perhaps somewhere, it's like the sixth slide. There's a -- if you want to pop that up. If not, it's just a picture of the book. But it's a book it's a book by Steven Rosembaum. >>Stevens: I will. Could I -- I am supposed to be able to mute mikes, as the owner of the chat, but I am unable to mute Halima's for some reason and that is where the echo is coming from. So, Halima, could I ask if you could click on the "mute" on your mike when not speaking? And if you want to unmute, you can always speak to us. That is where our echo is coming from. And okay, I will do what Phil has asked me to do and pull up 'Curation Nation'. >> Hubbard: laughs Alright, thanks. Anyway, this is not a book about education by any stretch, but it did come up with this notion that we have so much material on-line now and we are having so much difficulty in sorting out what the good stuff is from the chaff, for any reason, for news and so on. Now we have all these feeds: You know, if you -- those of you on Twitter or any of the other networks that have lots of feeds, you get the -- even Google+ -- you get feeds from your friends, you get feeds from people that whoever runs the site thinks might be interesting to you and you are just overwhelmed with an enormous amount of material. Some of it's pretty cool. Much of it is stuff you wouldn't find on your own and that's great. But when you've got the specific target of trying to improve your language -- and of course, the group that I work with doesn't actually do a whole lot with social media because they don't have time as full-time graduate students -- I am lucky if I can squeeze a few hours out of them to do the work that they need for the course that they are taking for credit from me. So, this notion of curation is based roughly on the idea of what people do in museums and in art galleries. You get an expert, somebody who actually knows a fair amount about a particular area and you have that expert create collections, add value to them in one way or another, and then you release those collections for the consumer - whoever it might be -- to have a look at and to interact with. So, the key difference between this and what a lot of people are doing with this material -- you may have heard concepts like "digital curation", which can just mean curating digital materials but often means that computers are doing the job for you. Google news is a really good example of that: I find a lot of interesting stuff in there, I can even ask it to find particular categories, but it's still being selected without any human intervention. You compare that with something like Huffington Post, which is material that's been brought in by people who are -- in some cases, they're producing it, but in other cases they are aggregating it and trying to make sense out of it for the rest of us. So, a key point here is that curation isn't the same as aggregation, or listing, or tagging. It's okay to use that term for that but that's not the way I am using it. There is a really nice quote in my slide there that -- I think it's maybe -- two more slides down, Vance. One more. There you go. Past curation.. yeah, that one. So this is - it's maybe a little mean, but I think it's right on point that when you just get collections of things, you've just got collections of things and its not necessarily anything other than "these are things that I liked" or "these are things that I think you will like". So, I prefer the next slide: you want to go to it, Vance? This is more the way I see curation, where you collect material, you organize it, there is even the potentially a path, well, there is certainly a path through the individual material groups, and then mayble even a path through the groups, although at the moment I haven't done that last point. So, this is, you know, kind of captures the idea that I want to talk about today. Curation, importantly, is not the same as creation or recreation or adaptation or sampling, or synthesizing. It's taking the material and adding something to it, maybe just a commentary, maybe just collecting it into some logical framework or logical sequence. So, when I took that idea, which I was getting through the Curation Nation book, and thought about it with respect to the material that I was using, I decided to experiment with that and come up with some collections of materials from -- as you probably know from the title here and also the PDF, if you've had a look at it -- comes from TED Talks. And in a moment I will talk about why I think TED talks is so good for that but at the base level, these were very popular with my students. What the students were doing more-- they were having trouble coming up with good ones. They would always pick what was interesting and then often come back to me and say "Well, this was interesting, but I had trouble understanding it because my -- the accent of the speaker was not easy for me to understand." or "I had trouble understanding it because -- it was interesting because I didn't know anything about it and I didn't have the background so there was a whole bunch of new vocabulary." So t could be interesting for all sorts of reasons, but it wasn't interesting for the right reasons, for what we think is good for independent language learning. Again, this doesn't mean that all of those collections, with the help of a teacher, couldn't have been very valuable in a classroom and especially getting to the content for connecting to discussions. But that's not the same thing as letting students work on their own. So, I do want to emphasis that. My perspective here, at least initially, is getting students to be able to do these things outside of class and then just come back and report on them rather than having something we do in class or that everybody does the same homework assignment on. Alright, so that's the set-up for what I believe curation should be, or at least can be, within this framework. So, I think what I'll do here is pause for a second and see if anybody has questions. and bring it up by trying to look at some of the chat pieces here Uh -- [he hums] [reading:] "What is meant by sign..." OK, so some of these chats are to each other about the chats. So I got to go to the other window Uh -- anybody -- anybody have any questions here? If not, I'll continue on. >> Stevens: I have to admit I have trouble following all the chats. There's also a back channel here, with Google: some people could be in that one. I never see that one until I get off of -- >> Hubbard: Well, the last chat -- the last piece on the group chat said: "Yeah, we agree with you, Phil." So: that's great. I'll stop [check] there and if everybody agrees with me, I don't really need to -- >> Stevens: you need go no further >> Hubbard: [overlapping, inaudible] No [Hubbard and Stevens laugh] >> Hubbard: OK, well, so, again, that's kind of the background, this idea that I needed to start collecting things. So, I'm still kind of almost two years in the past, now, telling you the story of how I got to where I got here. So I picked TED talks and I started going into TED talks. I wasn't quite sure how I wanted to collect them but I knew there were some of the ones that I liked and I also knew some characteristics that I thought were useful for the students. I thought it was important to collect them into themes. You know, we've known for a long time that if you have related content, that it kind of feeds -- the materials feed one another and the students get probably a better and a richer experience, they get more natural repetition and key vocabulary than if you have people just kind of jumping out piecemeal with unconnected bits of material. I -- in the 1980's I was forced to teach a course with a book I don't remember the name of that. I do remember the author, but I'm not going to mention it on air. It was a reading textbook and the reading textbook had really interesting little chapters, at least most of them were interesting to me, but, you know, one chapter would be on the Olympics and the next chapter would be on sea-horses. And it's that kind of jumping around -- we typically don't do that with textbooks anymore. And yet when we turn students loose, a lot of times, that's what they decide to do. So again, even though I had been giving them guidance, saying: "Well, collect several bits of, you know, pieces of material, videos or podcasts that are related to one another in some way," they wouldn't follow that advice, because it hadn't been done for them. They were still kind of chasing around, looking for the spots that just seemed interesting. OK. I think what I'll do is tell you what the -- at a kind of the abstract level, what I came up with about what the curator's role should be. And again, this is specifically for this target audience, but I think it can be tweaked and extended to other ones. The first thing you have to do is collect the stuff: you want digital materials, you want to organize them in some way: mine are organized systematically, but you could do -- you know, you could take news stories and do them chronologically. You need to sequence them and this is where a lot of collections fall short. They're just -- they're either randomly sequenced or they're not sequenced at all. And I think it is possible, as, you know, as the resident [check] expert, the teacher, to be able to say: "Here's a way to move so that the earlier ones might be a little bit easier to follow and the later ones are better understood if you've done the earlier ones." The fourth point there that -- on the slide that Vance has -- is the hardest part of all of this, and that is trying to get this material levelled in some way. Wilfried Decoo in 2010 wrote a book, it's at the end -- the reference is at the end of the slideshow here -- on systemization. And it was kind of a return to the idea that even if you're using authentic material, and especially if you're trying to create course material yourself, that you need to have a kind of natural development of that material from, you know, easier at lower levels, to harder and he went to the point of even talking about keeping databases that were very finely tuned, so you would be able to pull out lexical items and grammatical points and so on in a scope and sequence that fit what we thought we knew about language learning. And you know his -- I think his perspective is what I think is a reasonable one to bring up again, because I think we are often not cognizant of the difference between accessible and barely accessible and inaccessible materials, especially now that students can go in and, you know, get their first-language subtitles and transcripts for a lot of these materials and then have the illusion that they are actually understanding the English, in this case, and that they're building their English proficiency, where they -- -- they may be to some extent, but probably not to the extent that they think they are. So there is the, you know, that idea of -- well, in Decoo's book of fine tuning material. That doesn't work for me because at the levels I have, first of all, I have mixed-level classes to some degree, although they are all fairly advanced. They come from different backgrounds, I don't know what they know going in. So it's a little tricky to do it in the way that he likes. But it still gave me the impetus to try and see if I could come up with something, you know, I'll show you that in a bit. So, the last part of that, then, once you can give at least some kind of level information, is to go ahead and then present your pedagogical support, whatever it might be. This is fairly open-ended, I mean teachers can get -- and often do get -- into material and they start stripping out what they think are key vocabulary, they produce, you know, pre-listening activities, they have post-listening activities, they have discussion activities. All these are great, but they're based kind of on a classroom model and even more important: they take a lot of time away from the job of collecting this material. So if you put the hours into making full lessons, you end up not having the time to even produce as much as I have, which, as I mentioned, is not as much as I'd like. OK, so that's the curator's role and then -- Vance, if you could go to the next slide. Did we lose you? >> Jim Buckingham [check]: Hi Phil, I just wanted to add to something you-- >> Hubbard: Yes, go ahead >> Buckingham: Just because of my background: I used to work in museums >> Hubbard: Oh, fantastic >> Buckingham: in education and curation >> Hubbard: A real curator! >> Buckingham: Yeah. Just one other item I would add to the list and I made a note of it in the chat section and that's the -- often without knowing it we're making assumptions about our audience. >> Hubbard: Ah! >> Buckingham: When we're selecting things, whether they be objects for display or -- like in the museums -- or objects for presentations to students, we're often unknowingly making assumptions and I think it's a really important thing to know, to challenge ourselves about the assumptions we're making in making those selections, those choices, as experts. >> Hubbard: Yeah, I mean that's a very good point and I have to -- as individuals, the students always change in my classes. As a group, you know, I get to know the group better. So I think, in this very targeted group, I can -- I can come up with at least, initially, some likely ones, but I do in fact ask them for feedback on -- Well, first of all, I give them choices and then I ask them for feedback both on, you know, what they chose and why, of the ones I selected for them, and also what else they might like to see. So it becomes a little bit od a dialog, and that could be even more of a dialog, you know, if you have -- the way my class is structured, again, because it's so small, we do a lot both within class discussion and with the individual tutorials. But if you got a larger class and you got a discussion board or a wiki or something like that where, you know, students can -- can chime in more regularly, then you could get some information. I also haven't formally surveyed them, so that would be useful too. I -- >> Buckingham: You're inviting their feedback to inform -- >> Hubbard: Very much so. Yeah. >> Buckingham: Yeah -- >> Hubbard: But not as richly as I could. So one idea I had was that, you know, like you've seen probably in museums, sometimes they have the displays but they'll also have, you know, places where people can, you know, write cards and make suggestions and say things and drop those off and I think, probably increasingly, we'll see museum displays where the, you know, the viewers' thoughts are right up there and accessible to other viewers when they go to look at the material. So I think you're making a really good point and, you know, this is the -- figuring out exactly the role of the students who are still kind of developing, you want to meet them half way but you also, in the curation model, I think, want to be careful about the difference between curation and crowdsourcing, because I've had students come up with some materials that they thought were really exciting, but when I looked at it, I could see what the problems were in terms of the -- the use of it by other students. >> Buckingham: Now I take your point: it's you acting as the filter. >> Buckingham: and finding -- >> Hubbard: Yeah, and that's -- and again that's -- and again that's the -- this is the kind of, to me, this the curation model. >> Buckingham: Yeah >> Hubbard: The crowdsourcing model is a great model too, it's just a different model and it may work better in some cases. Of course it also depends on, you know, I've been to museums that I didn't think were very well run, were very well organized or were confusing. So -- >> Buckingham: Yeah. >> Hubbard: as soon as you have the human expert coming in, they may not be as much of an expert as they think they are. That's probably true of me, in fact. >> Buckingham: Yeah, and there are lots of people [check] a lot of examples of museums, because I'm into curating things and then I'm finding out that the interpretations that they were expecting audiences to have were completely off-base. >> Hubbard: Yeah. >> Buckingham: I think that's a good example of big money going into these exhibitions and then being interpreted in a completely unexpected -- >> Hubbard: Yeah, well, the good news here is, I have no big money. I mostly have no money at all for this. So -- [he laughs] It's also, the nice thing is, you know, compared to the museum, where you have all of these Unkosten [? check] to putting the material in, once you have something, you start a web page: if it is a disaster, or if it needs to be tweaked or significantly changed, it's possible to do that just by finding a little bit of time. [Buckingham and Hubbard overlap] >> Buckingham: It's just [missed words check] There's even an opportunity, actually, in, as an expert, putting together a series of well-chosen articles and then inviting students to assemble them and put them into a -- into an order or sequence, and to try and explain the rationale that they've used, what connections they've seen in the works. It's just another angle to it I sure would -- >> Hubbard: No, it's a very good angle and in fact, you know, as I've moved through stages in probably about 15 years of teaching this course, I've tried to give students more independence but also to give them guidance in that independence and one of the -- what I hope I'm doing with the material I have, I do show them how I put it together. And I hope I'm, you know, kind of modeling curation for them as well. The idea of getting them to maybe do a little curated piece of their own, that could be an interesting final project for the course. I will be revisiting it again in Spring. I'll be away from it in Winter quarter here because we have -- we teach 10-week quarters. But that's a possibility for Spring, actually. It could also greatly enrich the collection of material that's available to other students. Again, as long as I'm there to be a kind of a filter, rather than just releasing these into the wild. Or if I do release them, you know, making sure that students know the difference between ones that are student-produced and the once that I produced and why, you know, I did mine one way. Then they can -- they can judge to some extent, you know, whether they think the rationale used by their peers, you know, was useful for them. So, that's a nice idea, I'm making a note of that. OK, shall I move on? >> [Stevens? check] Yeah. I'm aware of a podcast - there's the slide on I'm talking -- >> Hubbard: Yeah, thanks [they laugh] >> Stevens (?): I listened to a podcast where some educators had gone to Europe, probably on a junket but ostensibly [Hubbard laughs] >> Stevens: to visit museums and find out, you know, especially ones that had audience attract-- you know, the idea was that museums, people didn't have to go there, they have to attract people. So what do they do to attract the people, as opposed to schools? And then, how can we design our classroom environment so it's more like a museum? So that was actually a serious project and I'll never remember -- I'll never forget how to get it back, but maybe I will tell you in Portmont [check]. >> Hubbard: Ah OK? So that was good. Yeah, so Vance has put up the slide that I wanted to make a point of here, because there are a couple of things that are important about this slide, I think. The first is, even though these are just little bullet points, that actually took me a while to kind of figure this out, maybe because I'm slow, but -- Oop, Vance, I lost the slide. >> Stevens: it is here again? >> Hubbard: thanks. Because of all the other distractions I have and because of other elements of where I am and what the -- sort of the visibility, the first thing I have to make sure is that anything that I curate is actually legally available. And a certain amount of stuff that I had used years before, even in my own class, I wasn't quite so sure about what the legality was, I think, in the early days of the internet. Even now with YouTube I try to be careful about making sure that what I've found is something that whoever put it up either has the right to or they're reposting something that is -- that's already got a Creative Commons license or something like that. So, especially for something I'm going to put some time into here, I want to make sure that what I've got is something I can use. I also always want to make it freely available because my students have friends back in their home countries and they have even colleagues here who don't end up taking my class and I have colleagues that are interested in using some of the material I do, so everything I do in this kind of a project, I try to make sure it's freely available on the Web. Vance, we lost the slide again, or at least I did. [incomprehensible metallic voice - check] >> Hubbard: Oh wait, is this Halima saying something? Uh, you know-- >> Stevens: No, Halima is unmuting herself as soon as she comes into the chat. So I'm going to have to -- Halima, can you mute your microphone? Because it's causing feedback. And I hope you can figure that out, and meanwhile we put this back. Is it back yet [missed words check] Phil? >> Hubbard: Yeah, that's great. >> Vance: OK >> Hubbard: Yeah, so the "freely and legally available" is an important quality and you know, TED talks obviously are ideal for that. They're likely to be interesting. Again that's something -- oops, lost the slide again, but I'll just go ahead and walk through these. "Likely to be interesting", I guess that connects to a previous commentary [laughs] that we don't always know what students think are interesting, but I try to pick things that I think are, you know, have a good chance of being interesting for the students. The good technical quality: there is a lot of stuff, obviously, available on the Web that's not, that's interesting and freely and legally available, but the technical quality is such that it may be less ideal for language learning. We're getting better at that now, certainly, than in the old days, but when - when you're looking for material, if it's been overly compressed, or it was done with devices that weren't that good in the first place, it doesn't necessarily lend itself as well for language learning. Stability is a really important point, because I don't want to do this and then find out what I did is not available the next time I teach the class, or even the next week. So again, finding material that has -- either has been up for a while or that you know is going to continue to be up for a while. The 5th one is a -- you know, people have different views of this, but because I'm so tied in with vocabulary development along with comprehension, to me it's critical to have captions at least -- [coughs] excuse me, losing my voice here -- to have captions at least and ideally, to have transcripts. And one of the reasons for transcripts is to be able to try to use some material which I'll show you in a moment here some of you are probably familiar with: the vocabulary profile from lextutor. By using -- by dumping the transcript into that, you can get an idea of levelling. And if you don't have a transcript, then you have to kind of use just intuitive feels for what's the level. Then I've personally seen some pretty significant problems with that. I may mention one towards the end here when I get to some of the alternative sites I know that already exist for this. And then ideally, if you can find complem -- something that has complementary materials. Again, in the case of TED talks, you've got materials that are -- you have a brief summary of whatever the talk is, right there available, you don't have to create it as the curator, you've got the bio of the speaker, which is good background information, and in some cases you even have -- I think, what do they call it, TED Ed or something -- there are some TED talks that even have some additional material that -- that people have added to them, in the way of discussion questions and things like that. TED's not as rich as, say, you know, if you're doing a newscast for example, and you might have several written forms of the same news story that you can use for back up: it's not quite as rich as that, but it's still pretty good with giving you some of these complementary materials, besides the video itself. OK. You want to move on to the next -- >> Hubbard: Actually, it's probably the next couple of slides >> Stevens: Yeah. >> Hubbard: does someone have a question? >> Stevens [check]: Yes, Peggy George has asked questions in the text chat, the Etherpad one. Let's see, I can -- she asks: "Are your students able to share your curated content with others outside the course?" >> Hubbard: Yes. Yes, som you'll see the -- in fact I think it comes up here on the next slide or couple of slides. Actually the next slide, if you go to the next slide, let me talk briefly about that, because it does have to do with the sharing. >> Stevens: Mmm - OK >> Hubbard: So that the link there is to the advanced listening website and you'll see, you know, quite a bit of material there, not just the TED talks. The link - the specific link to the curated TED talks is a couple of slides from here but those are -- those themselves are legally and freely available. They're my websites, they are on the Stanford server: Stanford is not going away any time soon, as far as I know I'm not going away any time soon. So those are not only, you know, available on the World Wide Web, unless you happen to be from a country that is for some reason blocking access to Stanford: that has happened a few times in the past. But if not, then you can get to that material and all it does is jump out to the TED talks themselves and the TED talks again are, you know, freely available. I noticed in one of the preliminary discussions that somebody had put in some comments, before this began, on the learning2gether site, and mentioned YouTube videos, and YouTube videos are certainly a great resource, most of my students are from China and most of them, then, unless things have changed, can't freely and legally get the YouTube videos there. And so for that reason I try to -- I don't avoid YouTube but I try to limit it and I like to make the curated collections something that my students will be able to use and their friends will be able to use. OK. Any other questions? Uh, so, yeah, so they are available and when I -- just so you know -- when I redo the course every quarter, that URL there stays the same, the material is new. Well, most of it is old actually, but I do update it sometimes because I come up with other ideas and sometimes because some of my other class material disappears. But the home page of that each quarter has the link to the previous quarter's materials, so you can actually step back from quarter to quarter and go back. I never throw anything away on the Web, so it's probably got stuff from 5 years ago if you keep clicking back through the previous quarters' material. So you can see what it was like in the past ["without"? check] sort of my own Internet Archive. OK. The way that I did this material, let me move on to the -- Well, I guess on this slide, the problems that my students have, typically, fall into issues with speech rate: some of the TED talks are too fast. It doesn't mean they can't, you know, use top-down skills to understand the basic content, but that's not necessarily going to help them drive their -- either their listening proficiency, you know, their ability to process English, automatize it, or their ability to pick out the vocabulary that they don't understand or -- even more interesting is the vocabulary they sort of understand or partially understand, but they just can't get to it, they can't access it in the time with a faster speaker. And there are others in my class, actually, that do OK with some of the faster speakers, but just having knowledge of the speech rate is useful. Preliminary knowledge of the accent: just a -- since in some cases we have students that are having particular difficulties with particular accents, often of their professors, and they may actually be doing a project where they're trying to focus on that accent. And so in that case, knowing more about the accent is helpful. And others are really trying to -- I wouldn't say "master", but at least becoming -- become more proficient with the North American accent because they plan on not only doing their graduate work here, but staying a few years afterwards. It's a very common professional track for our students whether at the Master's or the Ph.D. level, to -- because so many of them are in technology, they want to hang around Silicon Valley as much as the can after they, after the graduate. OK. If you could go to the next slide, Vance? >> Stevens: OK I might [both overlap] >> Stevens: You mentioned Claude Almansi's contribution to the wiki earlier and one thing that she said -- she left this on the Google+ page as well: I post this to several pages. Let me just get rid of that slide for a second. I see I can do that by clicking off the screen share for a second, OK? Well, anyway. She does work in closed captioning, she does a lot of very interesting work relating to MOOCs [check] where she is. And one of the suggestions she made -- I didn't know this, but maybe you did already, but you can -- she said you can, if you get the MP4, if you get an MP4 of a YouTube video, you can then load it into Audacity -- I didn't know that -- and then you can adjust the rate of speech there, without causing any chipmunk effects. >> Hubbard: Mmm. >> Stevens: I thought that was kind of neat. Sounds like useful information? >> Hubbard: Yeah, that's -- again, there are lots of things you can do to go more deeply into this stuff. I -- one of the things I do with TED talks is, you can also download TED talks and you can -- even if you put them into something, well I use the VLC player, because the speech rate slider is right on the top, it's much easier to get at than it is in QuickTime or in Windows Media Player. I like the VLC player for other reasons, in fact. But, you know, once you have downloaded you can use the VLC player to -- for the most part you don't really get the chipmunk effect because it's trying to expand the time domain without changing the frequencies, it's not like the old days with LP's and cassette tapes where time and frequency were connected to one another. Digitally, you can isolate those. What we found is that if you slow somebody down to about 80%, you can get a lot more processing time and it still sounds natural as long as you have good material. If you have material that's already been compressed too much, then those compression artefacts become stronger if you try to slow it down. Occasionally, we get people that my students want to speed up but most of the time, for language learning processes, we're talking about slowing it down. So it's -- using, changing speech rate, that's a whole different talk, but it's, I think, a very underused functionality and something that students sometimes baulk from but we have some research evidence that it's helpful when the students have control over it. Anyway, I don't want to diverge too much on that, but that's a -- I do encourage everybody to read that post and see in more detail what some of the options are for doing that. In fact, one of the -- one of the problems with using the VLC player with those is, if you -- if you do try to slow down the speech rate by downloading it and putting it in the VLC player, you actually move the subtitles, because the subtitle feature -- the captioning feature in the TED website is built into the website, it's not built into the video. And so you would need to do some additional captioning if you want to do that. My -- if your goal is general comprehension and you've got decent material, then I'm a fan of using the Google beta transcription. Even with good material, it makes a lot of mistakes and with material which, you know, isn't really, really clear, either because the speaker wasn't clear, or because the signal wasn't clear, it makes a lot more mistakes. And in my case, when I'm trying to have students use it for vocabulary development, if it's got -- if it picks the wrong word, then they're going to be learning something pretty weird. And it does that all the time. If you change that and, you know, get around to Google Translate, to get first-language captions, you just accentuate the error rate. So again, it really depends on what the goal is. If the goal is letting students watch a video for cultural and general content information, maybe to trigger classroom discussions, things like that, then using the automated captions is not a bad idea and being able to slow down is not necessarily -- is, well, I think a good idea. So again, it depends on what the goals are, but you have to be careful, because the Google beta, there is a reason why they keep calling it beta, it's because it's pretty error-prone. It's getting better but it's not there yet. And if students think it's an accurate rendition that's going to be even more difficult. If you do use the automated captions then the students need to be prepared for -- you know, to be able to recognize when something doesn't make sense. It's usually -- it's a very obvious semantic issue with the words they pick. OK. One other thing, I don't remember if it was in that post or another one but something I hadn't noticed before someone mentioned that there is a slight delay in the synchronization of the captions in TED compared to the system that they were suggesting. So, uh, that's something else to take into account. You might, If that delay seems to be an issue for you or your students, then -- it's something that I plan to explore because I hadn't noticed that before. Okay, a little bit about how I finally figured out to do this, which is not the way I would recommend doing it now necessarily But this is how I started working on this. When I did it I guess it was Spring of 2011. The first thing was to...oh no, it wasn't Spring: Fall of 2011. The first thing to do is to get the TED database. It turns out you can get an excel spreadsheet that has all of the Ted talks on it. If you go to their website you can see that there's a link for that. And the nice thing about that is that you can skim that a whole lot more easily than you can skim other material and you can also look, among other things, it tells you what the length of the talk is. And most Ted talks are around 18 minutes and most students attention focus ability is less. Um, okay, the database then, when I did it myself, it was smaller for one thing, at that point. But I did sort of skim it and looked for ideas, looked for themes and searched for keywords. So creativity was one of the first ones I did, so I was just able to search for anything that had creativity either in its description or in its title. I put together a list of candidates within that. I was looking for four or five talks to make a kind of a cluster, a sort of a virtual room in the museum if you will. And for each of those, I -- well, first of all, I did listen to the accent and got that. I wanted to get at least a proxy for the speech speed and so -- the speech rate -- so I just took the transcript, dumped it into Word so that I got a word count, divided that and came up with words per minute. I was actually quite surprised at the range that I could see there. If you go to the website for cre-- the link for "creativity" on my ted1 website of the curated talks there -- the slowest speech rate is like 91 words a minute. Some of that is because there are pictures being shown in between but it still means you got a lot more time to process the language coming in than if you got somebody coming in at -- at a higher rate. Some of my students do a -- there's a website at Stanford called "Entrepreneurship corner" and they have a lot of Silicon Valley types come in and give talks on campus. They also have transcripts and subtitles for that and one of the talks that I always have the students try is Marissa Mayer who, at the time she gave the talk, was a VP for Google but is now the CEO of Yahoo!. And she talks between 220 and 237 words a minute on the one I have, so I use her as an example of where you might try to use the speech rate shift and be able to use the slider to slow her down to 80%. OK. The next thing, once I have that rough speed -- and again, it's just a rough speed, but it's better than not using technology and try just to use intuition about "This is too fast, this is too slow." The vocabulary profiler -- this is Tom Cobb's work of genius in my opinion. There are a lot of parts to that lextutor.ca site, but the one that I use for this purpose is the -- well, at the time, was the British National Corpus profiler and there is the link to it there. Basically, you dump a text, a transcript into it and it gives you as output all the words divided into 1'000 verbal frequency bands, so, you know, which words are in the first thousand words of English, the second thousand words of English, and so on, all the way up to the 20'000 level. For my students, we try to focus more on the, you know, just doing a short -- well, we -- I try to get them to focus more around the 5'000 level, so anything below that that they don't know, it means it's a word that they should learn. And when you go to my site, you can see how that's split up. I skim the transcript for unusual terms and idioms -- Oh, I meant to mention: in the last few weeks, Tom has actually added the Coca, it's a contemporary corpus of American English and blended those in, so it now goes up to the 25'000 level. And it has much more American English in it now, rather than just the British. So, for those of you who like, you know, concordancing and corpus studying, study and so on, it's got a much richer layering out than it did when I was using it for this purpose. OK. So that's the process. Now I said, you know, I would do it a little bit differently, probably. It turns out that, since the time I began this and now, TED has come up with its own curated collections. And so, if you go to the TED website, you will see a link to something called "playlists" and these are collections of material that people have put together. In some cases, it's done by TED itself, you know, whoever is in the background working there, but they also have curated collections by Bill Gates and Bono and, you know, other famous folk, or in some cases, they're people who are less famous but, you know, are very well-known within their, you know, their more restricted field. And there's some really, really good collections there. So now, instead of just going to the database, my inclination would be to go to that -- to those other curated playlists. Those have been curated just by interest and so if you have a list of maybe 10 or 12 videos on one topic, you go through those, and maybe you pick out the 4 or 5 that you think are easiest to work with. So, that's there on the "Recent changes". I did -- number 2 there where it says -- this is from my talk last July -- Well, I did have a project assistant who has collected some more material for me and basically run it through the -- the Word, you know, done some of the preliminary work for the words for minute and the vocabulary profile. Unfortunately, that came at the end of Summer, right before Fall quarter started for me and I have not had a chance to really look through her material but I do have some partially digested material that should help me create some new stuff. I guess, at this point, probably the most useful thing -- Vance, could you -- can you actually click on that link to the TED1, just so I can sort of show people what -- [Stevens and Hubbard speak together] >> Stevens: ... already. It's in the text chat. So -- >> Hubbard: Ah, OK, so people can go to it on their own? Alright, then you -- >>Stevens: I can also share it. >> Hubbard: Well, the only thing -- I think if you go down -- go to the next slide actually, there's the Creativity group. >> Stevens: OK. >> Hubbard: and this will show -- This is a more condensed version of what you would see on the page. >> Stevens: Mmh. OK: let me share it. >> Hubbard: but this is, yeah, this is the Andy Hobs-- Hobsbawm, I'm not sure how to pronounce his name. This is a nice beginning talk, I think it's the second talk in the Creativity group. You can see it's a pretty short talk, just three and a half minutes. You can see the speed is 135 words per minute. He has kind of a dramatic presentation style, so that's why it's a little bit more slow, a little bit slower, it's very articulate, it's very easy to hear, it is more of a British English rather than an American English version. The vocabulary you can see at the 5'000 level, so if you -- to read that, I mean, 95% of the words are in the first 5'000 words of English. And, you know, 98% of the first 10'000, and then Off-List. The Off-List on these are often proper nouns. So those don't necessarily cause a great deal of difficulty, especially if some of the names in it, or names of places are ones that the students already are familiar with. [stifles a sneeze] Excuse me. About to sneeze. Wasn't expecting to do that online. And then a little bit of a comment: "creativity is repeated a number of times." So, if you go to the website, you'll see it's a -- it's a little richer than that but this kind of captures the main point I want to say. So I said, I'm giving value added, as the expert, and not only am I collecting these things, but I'm using technology to give students some idea of level. Ultimately, it would be great if I could say, you know, this is level 5 of 10 levels, or this is at the B1 level of the C, CEF, or just even, you know, this is high-intermediate, or something like that. I don't have that confidence yet, so at this point I'm giving students more the raw data, but I do actually tell them, and I may highlight this in future versions, that at the 5'000 level is probably the most important pivot point for my students. If that level, you know, if that's up to 96 or 97%, that means they're going to understand that fairly well. If it's down even to 92 or 93%, then there are likely to be enough words in there that they are going to have gaps. And they'll be able to process it top-down, extract information from it, but it won't be as valuable for language learning. If you look at the, you know, the research on reading, which is much better established than on listening, anywhere between 95 and 98% is what people typically quote as material that's ideal for language learning. And below that percentage, if you don't know those words, then you're not going to be reading or listening in the same way: you're not going to be processing the language in the same way. OK. I see we're moving ahead quite a bit on time here. So I think I'll go to a -- just a final slide. Vance, could you go to the "Related examples" one? >> Stevens: Uh-uh. >> Hubbard: Just to let you -- yeah -- these are just a few places that I know of where people are trying to do something similar. The CLILSTORE Project is a very big European project and they have collected material. Some of it they've done themselves, some of it teachers have put into there, to their database, they have information on the talk itself, they've -- whoever has done it has put in a CEF level, so it starts with all the A1 material, then the A2, and then the B1 and then the B2. It's done in -- I checked -- and it's apparently done intuitively. It's not apparently done by running it through, you know, the kind of material that I was using there, lextutor and so on. But at least, it means an expert teacher has said: "Here is the level I think this is at." I will mention some issues with that myself, because I looked at it and they put a TED talk in the A1 level, somebody had it. And it's definitely not the A1 level for -- at least for comprehension purposes. It may be that the person put that in because the content was so valuable and that they thought that this is something that could be used you know, with a lot of help. But the main feature of the CLILSTORE material is that all of the words -- there is a transcript and all the words in the transcript are linked to multilingual dictionaries. So you just have to click on a word in a transcript you don't know, and you immediately get the response. Ayamel is a Brigham Young media project where they've been collecting and cataloging authentic media. All of these use authentic media, I want to emphasize there, but you could go back, you know, and curate Randall's Cyber Listening Lab or something like that as well. And then I haven't checked this one. I hope the link is still good. There is a product called Lingle that will index according to Common European Framework level, so if you put something into it, it will give its best guess as to, you know, what the level is for both reading and for listening. OK. Let's see. I see somebody else has put a CEF level (check) over in the chat -- James. So, some of you might want to take a look at some of the [missed word, check] material. As somebody else mentioned the SRA, by the way, and I don't have it in this talk, but I -- some of you may be familiar with Tom Robb's work on graded readers. Charles Brown has also been doing a lot of work in that. And graded readers are clearly examples of material that fits into this. The difference is, they're not freely available. Unless somebody has come up with a really good collection that I don't know about yet, you're always stuck with having to pay a fair amount to a publisher to get the graded readers, and they're not authentic, but I think they're useful enough, like, I don't think that authenticity is all that great a thing if you're -- if you're still at a lower level and just trying to get the language in. So, I think, something that's well-written and, you know, well-produced, to me, whether it's technically authentic or not is a secondary issue. My key here is this idea of freely available, and often it's not. OK. Let me go ahead and stop again: the, you know, the final comments and the reference list and all of that, you can get if you go to the PDF yourself but I do want to give people a chance, especially some of those who have, you know, either put things over in the chat or who have read things over in the chat that look interesting, and certainly the participants here. What -- what questions and comments do we have? >> Stevens: Claude Almansi has arrived in the chat but she's shy about coming in because she hasn't heard the whole conversation, but anyway: I guess she could if she wanted. There are seven people in the chat now, so we have -- that is, in the hangout -- so we have room for three. >> Buckingham: Hi Phil, I just wanted to add, I think that this is >>Hubbard: Uh uh >> Buckingham: This -- it lit a light bulb for me. I just thought this getting a group of people together, use this kind, this style of approach to collecting transcripts and analyzing the transcripts and the being able to put them, I don't know where, somewhere up in the cloud, I mean over time, you could have quite an assortment of these readings to be shared collectively, and then, basically, circumvent the SRA and the [missed words - check]. This would be fantastic. Just thinking aloud here. >> Hubbard: Well it's, you know, certainly part of what I'm trying to do here, because I don't have the time that I wish I did for this, you know, is to both encourage other people to be interested in it and get into, whether they're formal or informal collaborative projects for putting stuff together, and then, secondly, just for experimenting for yourself, because this is something that I've thought about a lot, I've done some reading in, and I've also tried things along the way and have ideas about how to make it better, but it's still very much in its infancy. And this is if -- if is the subtitle of Rosenbaum's book is that, you know, content is the -- curation is the future of content, then this is something we need to get better at and be thinking about that for different purposes, so, if you're curating materials, for example, for teachers to use in classrooms, that could be quite different from curating materials that you're going to have the students use independently. It could hopefully overlap quite a bit, but it won't be the same. And curating materials for cultural purposes, or for triggering discussions, is not going to be the same as the way I've curated here, you know, where I'm more concerned with level than other people might be, whereas others might be more concerned with the content itself and, you know, don't have a problem with letting people listen to it with subtitles in their native language. Or, you know, with sort of getting the gist of it without necessarily getting all the details. So there is a -- there's a very rich area here to explore in lots of different directions. What I try to do too, is keep the curation pretty light. When I've given this talk before, a couple of times, people often come up with ideas for, you know, adding discussion questions, basically making it more, into more lesson-like and adding -- in other words, having more value added. And I think that's great. I'm trying to come up with sort of an intermediate stage where I'm doing something that's helpful, but not something that -- that I wouldn't have time to do otherwise. Are there questions, comments? I see there is a lot of people talking about SRA [laughs] over in the chat. >> Stevens: [missed words - check] >> Hubbard: Yeah. >> Hubbard: So for those of you would be listening [missed words check] I'd be looking SRA Science Research Associates is a -- a set of graded readers that was very popular in the US, probably going back to the 1960's, 1970's. I remember using it in my first reading lab when I was teaching in the early 80's and it's, unless it's changed, it's designed for native speakers materials, but it's aimed at trying to lead you step by step into reading proficiency. And it's a fairly traditional approach of, you know, read, respond to comprehension questions and so on, but it does have -- it does timed readings and some other things as well, and is very much in the general graded readers approach. Oops, perhaps I'm not hearing you. >> Stevens: [missed word check], how about this one. Oops. Is that working? OK, yeah. My USB went out, now I'm on word mike [check] OK, well, anyway, I was saying that I have some perspectives on this, having seen you present some of this at the TESOL conference in the KIS group. Your focus was a little bit different at that time, it was on the videos themselves. And I wonder if you have a link to that presentation that you could put in there, because there were really nice examples of what you can do with this, as far as finding graded materials for your students, and -- because I think you had those organized in such a way that they started simple and went to more difficult. >> Hubbard: Right, the materials and I think what I did in that talk is basically walk through the -- the ted1 web page. And so if you go to that link that we've already -- that I've already put in there, the one that ends in ted1-- >> Stevens: yeah >> Hubbard: let me see for sure if I can bring it up here >> Stevens: I might be able to do that. OK, yeah, I probably got that here: here we go. Yes, I got it here. >> Hubbard: Yeah, so that gives a little bit the back -- one of the problems is this particular long scrolling web page that needs to be broken up and do a little bit of a hierarchy. But for right now, this is sort of what I have available. So it gives a little bit of background, if it's the one I think it is, let me double-check here and -- It's not up on the main screen so I'll have to -- >> Stevens: Would it be this one? Yes. We -- have I got it up there now? >> Hubbard: Errh, that's the one, yeah. [Stevens and Hubbard overlap] >> Hubbard: This is actually what the students see. So this is really for them. >> Stevens: You've got to find the window -- >> Hubbard: And so again, it explains, you know, how I have -- I've -- the material that's in there, the support material that's in there for the students And then this is the example group 1, the first one I did was on creativity and I said that one is a little easier, because sometimes the students want to have some idea of -- that was just my impression that it was easier. And so it starts with -- you can see the words per minute speed on this, since 91, the vocabulary is relatively easier,you know, 95.4 at the 5K level, it's a US standard. It does have a, you know, almost no student who takes this actually knows the word "tinker", going in, and this is about the "tinkering school". So, one of things, if I had a little more time, is that I would add under the comment, instead of just saying: "Some good vocabulary to learn," I would probably define the word "tinker" there, without spending too much more time: just looking at this, it reminds me. The second one, also fairly short, is the one I already mentioned, on the "Do the Green Thing." The third one, students find this really interesting. It's a 12-year old giving a talk on what adults can learn from kids. She speaks faster, but the vocabulary level is a little bit lower. [1:14:44] And it's kind of inspiring to them to see what 12-year olds can talk about. The next one -- the last one there, "Amy Tan on creativity," because, as I mentioned, I have so many Chinese students in my group, this is a, you know, a Chinese-American who writes about the Chinese-American experience. Interestingly, you'd kind of expect somebody from literature to maybe be using high-level language, but her vocabulary profile is actually the easiest. So, 97.5% at the 5K or below, 96% at the first 3'000 words level. So this one, in terms of accessibility for vocabulary, is by far the easiest. It is, however, also longer and because of that, its -- OK, I have to I got a -- a little thing popped up on my screen: that's David Wexler "Open Capture" smile. So, am I supposed to click "Open capture"? [Stevens giggles] >> Hubbard: Anybody know? >> Stevens: I don't know. >> Hubbard: they are saying [check] Sorry, mistake, no, sorry, mis-clicking up there [check] I'm gonna -- I'm just going to close that window because now I can't see the rest of my stuff. [laughs] I'll be happy to send a photo to you if you'd like. [Stevens laughs] >> Hubbard: even autographed. OK, so that was one group. The next group, the one on humor, you'll notice the -- I tell them the overall time on these, so this one's like about 40 minutes long It begins with this talk, which has actually very little English in it, but it is a -- this is an example of what happens in curation. The talk by itself isn't that useful for language learning, but it sets the stage very much for the first -- for the next two talks that follow it. "The TED speaker's worst nightmare", the guy starts to talk and then -- much as I experienced this morning when I was trying to get in here -- runs into some significant technical difficulty. And then, the next one on, "Gotta share" is -- is actually one of the talks, it's not technically a TED talk, but it's something posted on the TED website. It has all the -- it has the text support although it doesn't have a -- it doesn't have a transcript. So you -- but it's actually sort of a musical about social media that's-- that's quite interesting. And specially as a lot of my students actually, you know, they know about Facebook but they don't know about things like vimeo, they don't recognize the reference to MySpace, where the character says, you know -- "Is anybody else on MySpace?" and everybody says "No," you know, that one [check], FourSquare, and a whole bunch of social media points [check] So, it's a very entertaining song but it's also built into -- well, I don't want to give it away too much, in case you want to go and see it. But the third one, "The shared experience of absurdity," is actually by the guy who organized those first two. So it's more of an academic-style TED talk about, you know, why they go out and do these sorts of things. Among other things, he describes how bigger the project, where people kept getting on without their pants on at different subway stops in New York and then they filmed the crowd's response to that. He talks a whole lot faster. His vocabulary, though, is at a fairly accessible level: look at that: 98.4% at the 5'000 and below. He's talking more conversationally. It is interesting that often what we see in TED talks, and probably in other ones as well, is that the faster the speech rate becomes, the lower the vocabulary rate is. That, I think. would make a really nice Master's thesis sometime, to see if that actually happens but just impressionistically, I see a lot of that going on here. It doesn't seem to be the case with the next one, though, which is, this one is called "Sherman the" -- "Learning from Sherman the Shark" and it's actually the next -- I'm sorry, the next two -- Oops, I mixed myself up there. So, Number 4 is the one that was at 98.4%. So this is by Liza Donnelly who's a New York -- a New Yorker cartoonist, in fact actually the first woman to do -- to be a cartoonist for the New Yorker and talks about that experience. Knowing the phrases like "the glass ceiling", which shows up in that talk. And then another cartoonist, so we have two kind of related talks, finishes that one out. I may not go through all these, but you can see the last group is on the brain. I have a certain percentage of students who are either directly or indirectly interested in that topic. So I will go ahead and let you all explore that one on your own. Vance, I see we're starting to lose people, so -- >> Stevens: Yeah [missed words check] >> Hubbard: What would you like to do here? -- yeah. >> Stevens: Can you hear me? >> Hubbard: Yes. >> Stevens: Yeah, I guess when we've got to that point about brain that sort of faded [check]. [They laugh] >> Stevens: Well, I'll tell you what: It's kind of getting towards dinner time here -- >> Hubbard: Yes. >> Stevens: And maybe breakfast time for you, and I sort of -- I feel like we got two presentations in one, there. That's really nice. Two discussions, or how we want to put it. Well obviously, people are, you know, I mean you said earlier that you didn't have time yourself for all this but it's -- but you planned to see it in others, you know, but by making things open and free and accessible and by giving -- by talking about it in various venues, face to face, online, you know, you give people ideas and that's -- that will, you know. Now, go ahead. >> Hubbard: Right, I will put one challenge out there, because I'm -- I'm really interested in this myself, because of all the work I do with people outside of Stanford as well, who don't have the advantage of the level of students that I have in terms of, you know, already being at the TOFL level, high enough to get into the university here, I would like to see collections that are more targeted at, you know, the high, beginning, low-intermediate level. And I have had trouble finding clusters of material that really do that. Again, as I said, when I look at the CLILSTORE material, a lot of that, I think, linguistically at least, even though it may be tagged at A1 or A2 level, isn't really at that level: It may be usable in classes at that level, but that's different. >> Stevens: Have you -- have your students do this! You said that they don't have a lot of time, but still, you know the -- >> Hubbard: Yeah. Well, if I -- if my students were going to do it, they would be doing it at their level and it wouldn't be -- they wouldn't be looking for stuff at lower levels. And I -- it's just, you know, as we well know, you know, the number of people who are at lower levels of English, still learning, is much, much greater than those at a higher level, Then the value sort of, globally, for a curated collection would be much higher if people could find, you know, short -- short pieces that otherwise met those criteria, of being stable and freely available and having the -- the language support. I don't know to what extent, you know, the CLILSTORE will get there: I've only explored pieces of it but it doesn't seem to be quite there yet. It's great for many other reasons and I strongly recommend, if you haven't explored that one, that you take a look at it and consider donating to it as well, if you have materials. Right, so, I got more people moving on. A couple of people are still hanging in here with us. Anybody have other comments? >> Stevens: Yeah, I'm looking in the text -- in the Etherpad chat. The -- I don't know, if someone else wants to make a comment? Ah! Peggy makes - points out that there is a separate chat in the hangout, and we're not seeing that. You know, what [missed words check] tried to do is get people NOT to interact in the hangout, this doesn't seem to work but anyway, I suppose about, you know -- I was thinking about that when some of the links were posted earlier in the, you know, chat, I tried, actually tried to copy them and pasted them into the Etherpad chat. Yes there -- we sort of really -- I've been doing this in the Etherpad chat so we keep it a little bit together. That's one thing I learned from Jeff [missed words check] minute, so [Hubard laughs] >> Stevens: Yeah, sorry about that, Peggy, we'll try next time. I'll probably enforce that. Anyway -- >> Hubbard: Somebody got their finger up. >> Stevens: Ah ah. >> Hubbard [missed words check] [laughs] >> Stevens: No, he's got to unmute himself [check] and he's done that. >> Hubbard: Ah. OK. Hello! >> David [check surname]: Can I -- I can unmute myself. >> Hubbard: Hi David. >> Stevens: You just muted yourself again. You did unmute yourself, now carry on. >> Hubbard: that's right [check] >> Michael: Oh. the line stays through the microphone as though it's muted. But it changes from red to black, I guess. >> Hubbard: Yeah, I had that problem too. >> Michael: That's visuals, Google! Uh, so I've been posting a little bit enthusiastically in the Google text chat, so you guys may not have seen it, because I don't know where the Etherpad is, but who cares, at this point? But I just wanted to mention, Phil: Helen Young has been a scholar connected with Stanford who lives in Palo Alto and is the mother of my brother-in-law. I don't know if you've interacted with her, but she spent a fair amount of time, you know, close to twenty years, maybe, working at Beijing, and writing a book about the women on the Long March who she interviewed as much older women, you know, probably in the 90's and early 2000's, which is called "Women of the Revolution" -- "Choosing Revolution," which Stanford University Press published. But I did a quick search for her and although she's not really on the Stanford website, there's a link to digital recordings of her interviews, complete with translations, and I think, transcripts. I -- the -- I don't know if you can open the chat in Google, but the -- the link to it I posted there >> Stevens [check]: OK. >> Michael: A number of, you know, 20 minutes ago, maybe. But, you know, whether that's a topic of interest to any of your students, because it's, you know, interesting historical China stuff. But I had no idea that there was actually, you know, sound recordings along with the transcripts. I'm assuming -- I don't know it they're in -- I'm guessing they were in Chinese originally, the records -- >> Hubbard: That would be my guess too. Which might make this repository actually valuable to the people learning Chinese here. >> Michael: Right, and apparently it's not online but somewhere, her papers on -- her research for this project is somehow connected to Stanford. >> Hubbard: OK, Well, we have -- various libraries here have collections of papers and it may be that they are inside one of the libraries. I'll -- So it's Y-A-N-G? >> Michael: Yeah. Just Google it and it'll come up with, you know, a link to Amazon for the book. But my sister and her kids and my brother-in-law, two Summers ago, in the Summer of 2011, were on, essentially a book tour for two weeks in China, because the original English publication had been translated five years later into Chinese and my nephew, who's now a college student, two days after his high-school graduation, was, you know, flying to China for a kind of whirlwind tour, so it was an interesting cultural experience for my sister and her family. >> Hubbard: OK. Shall we call it a day, Vance? >> Stevens: Let's do and just a point of reference: David put some links in earlier, so if you scroll up, maybe you can find some of the links that he referred you to. But -- >> Hubbard: OK. >> Stevens: -- Well, this is learning together, this is typical learning together that actually goes on just as long as we want it to. I love that. And anyway, it's still December 8, 2013. We've been talking with Dr. Phil Hubbard who is a good friend and professor at Stanford and director of the English language program there, the ESL program. So, well, and he's been telling us about curation and how he applies it to a database of TED talks videos and that's really cool. So, thank you very much, I really appreciate your spending your time with us this morning. And I can see that a lot of other people have appreciated it as well. >> Hubbard: All right, well, thanks and I hope to see -- well, I'll see you for sure in Portland and I hope to see some of the rest of you either there or at some other conference sometime. All right, bye bye. >> Stevens: OK: my cat says Bye -- >> Hubbard: Aw, that's cute -- >> Stevens: -- everybody! Yeah. [Hubbard and Stevens laugh] >> Stevens: OK, well, thanks very much, I'll stop broadcasting and we'll leave the hangout on for a little bit, just in -- because I want to try and capture some of the text chat. [missed word check] I'll see if that works. >> Hubbard: All right, thanks. >> Stevens: OK, bye bye. (Hangouts On Air)