Learning2gether with Phil Hubbard, Curation in CALL and TED Talk videos

Learning2gether with Phil Hubbard, Curation in CALL and TED Talk videos

>> Vance Stevens: We're live!

Hello, everybody. Somehow my video disappeared.

It's there, but that's my - it's just in avatar format.

Plus does that every now and then.

OK, well anyway, this is Vance Stevens in Abu Dh..., no, sorry, in L.A.

I'm living in L.A. now, I forget where I'm living.

Today is the 8th of December.

They move me around so much, you know.

And, anyway, it's the 8th of December 2013.

We're talking with a good friend of mine, Phil Hubbard,

from Stanford University in Palo Alto, California.

And he's been doing some really neat stuff in Cal.

I've known him for a long time in the Cal Intersection TESOL.

>> Phil Hubbard: Since we were kids.

>> Stevens: We were, it was like 20 years ago
[Hubbard laughs]

>> Hubbard: reaching 30 [check]
[background voice]

>> Stevens: Someone has a -- someone needs to have a headset on.

[missed words] is muted.

Errh not sure: it could be someone listening to the stream.

Yeah, if you're listening to the stream -- OK.

Their call has gone away [check]

Someone has corrected it, that's good.

All right, well, OK. Someone has announced in the stream chat that they're listening to it there.

So that's good, everything seems to be working.

We're doing a Hangout on Air, as we often do.

We're streaming it on webheadsinaction.org/live

At the moment we have six people in the hangout,

there's room for four more.

So if anyone is listening on the stream and would like to join us, they can.

And right now we've got Claire Siskin and Jim Buckingham, Rita Zeinstejer and

let's see, and also Rob, Rob is there, and me, Vance Stevens. Rob Permanus, is that correct?

Correct me if I'm wrong. Permanus, Permanus - how do you pronounce your name?

>> Hubbard: You have to unmute him chuckles

>> Stevens: it's Perhamus -- Perhamus, OK, Good, I'll never forget that again, all right.

Thank you very much, Rob. Rob is an occasional participant in our hangouts.

Well Phil, take it away and anybody who wants to --

by the way, you're all muted by default when you come into the hangout.

You can unmute yourself.

If you're going to unmute yourself and talk, please mute yourself again,

so we don't get keyboard noises and things like that.

And there's Elizabeth Anne, also shown up from Grenoble in France.

And Halima [check] in Tashkent has also joined us, I see.

>> Hubbard [check] I think we're great, well, hello, everybody.

It's Good Morning for me, a little early in the morning,

but the sun is beginning to show through the back window here.

Thank you all for being here from all over the world.

What I wanted to do today is talk about largely an idea and a project that I've been working on

for the last couple of years, very sporadically.

Unfortunately I get interrupted easily, as I'm sure all of you do,

so what started out as a -- what I hoped was going to be a much more robust collection of materials

has turned out to be a little more anemic

but I still think that I have enough here that I can demonstrate the idea

and especially share my thoughts about how to go

about dealing with this relatively new notion of curation,

although in some ways, maybe it's just a label for an old notion that we've had for quite some time.

So, let me give you a little bit of the background,

like several of the things I've worked on in the last few years,

like learner training.

This is something that has emerged out of my classroom experience

with an advanced listening and vocabulary class,

and I see Vance is showing some of the slides now.

The class is for graduate students at Stanford

and it's a really nice sandbox for playing with ideas,

because these are -- well, they're all in graduate school already,

they're, for the most part, in the high 90's onwards to the 100s in the TOEFL iBT

so they really are advanced in that sense.

And many of them are taking the course because we require them to do it.

So they're kind of a captive audience

but it's also a small course: we have a maximum 14 students in it

and it allows me to not only play around with ideas, but get a chance to talk to the students afterward,

not usually with formal research, but just informally as part of our normal tutorial sessions

and find out what they thought about them and what I can do to make them work a little better.

So, the problem that I noticed - an important part of this class

is that students do independent projects

and those independent projects are supposed to be for a minimum of three hours a week.

Sounds like I am getting some echo in the background, but I will keep pushing through here..

Uhh.. those projects are for three hours a week

and they are responsible for doing the selection of the material

with my help and with my guidance both before and after.

And over the years, I have discovered that they are actually not really good at that.

What they are good at is finding material that is interesting to them.

But, they are not necessarily good at finding material that helps them.

They discover that on their own a little bit down the road

and often it doesn't become clear to both of us

because I have a very slow learning curve and quickly forget things.

So, I get to the end of the class and then I go

"Oh, I should have provided them with a little more guidance.".

So, about 2 years ago, I started doing this

and it came as a juxtaposition of a couple of things.

First of all, just my own general interest in the development of autonomy had been growing

and as I have gone out and collected materials that I would just use in class,

it was pretty clear to me that there is a huge amount of really interesting materials out there.

And people have been collecting these for a while

and teachers have been building lessons out of them

-- sometimes pretty sophisiticated lessons --

but I needed something that students could work with on their own.

And so, I wanted to find a way to help them without just my advice

as to how to look for materials, to actually start collecting materials

in ways that would still give them quite a bit of freedom of choice

but would also make it better as a language learning experience.

As part of this course, they are also required to build vocabulary.

They have to identify at least 35 new words and phrases every week,

from the material they are using.

So, this is a bit of the backdrop.

In 2011, I came across a book, kind of independently.

It was just recommended to me, for some reason, by Amazon:

you know how that works.

And the book was called 'Curation Nation'

and there is, I think, a slide there perhaps somewhere, it's like the sixth slide.

There's a -- if you want to pop that up.

If not, it's just a picture of the book.

But it's a book it's a book by Steven Rosembaum.

>>Stevens: I will. Could I --

I am supposed to be able to mute mikes, as the owner of the chat,

but I am unable to mute Halima's for some reason

and that is where the echo is coming from.

So, Halima, could I ask if you could click on the "mute" on your mike when not speaking?

And if you want to unmute, you can always speak to us.

That is where our echo is coming from.

And okay, I will do what Phil has asked me to do and pull up 'Curation Nation'.

>> Hubbard: laughs Alright, thanks.

Anyway, this is not a book about education by any stretch,

but it did come up with this notion that we have so much material on-line now

and we are having so much difficulty in sorting out

what the good stuff is from the chaff, for any reason, for news and so on.

Now we have all these feeds:

You know, if you -- those of you on Twitter or any of the other networks that have lots of feeds,

you get the -- even Google+ -- you get feeds from your friends,

you get feeds from people that whoever runs the site thinks might be interesting to you

and you are just overwhelmed with an enormous amount of material.

Some of it's pretty cool.

Much of it is stuff you wouldn't find on your own and that's great.

But when you've got the specific target of trying to improve your language

-- and of course, the group that I work with doesn't actually do a whole lot with social media

because they don't have time as full-time graduate students --

I am lucky if I can squeeze a few hours out of them to do the work

that they need for the course that they are taking for credit from me.

So, this notion of curation is based roughly

on the idea of what people do in museums and in art galleries.

You get an expert, somebody who actually knows a fair amount about a particular area

and you have that expert create collections, add value to them in one way or another,

and then you release those collections for the consumer - whoever it might be --

to have a look at and to interact with.

So, the key difference between this and what a lot of people are doing with this material

-- you may have heard concepts like "digital curation",

which can just mean curating digital materials

but often means that computers are doing the job for you.

Google news is a really good example of that:

I find a lot of interesting stuff in there, I can even ask it to find particular categories,

but it's still being selected without any human intervention.

You compare that with something like Huffington Post,

which is material that's been brought in by people who are

-- in some cases, they're producing it, but in other cases they are aggregating it

and trying to make sense out of it for the rest of us.

So, a key point here is that curation isn't the same as aggregation, or listing, or tagging.

It's okay to use that term for that but that's not the way I am using it.

There is a really nice quote in my slide there that -- I think it's maybe --

two more slides down, Vance. One more. There you go. Past curation.. yeah, that one.

So this is - it's maybe a little mean, but I think it's right on point

that when you just get collections of things, you've just got collections of things

and its not necessarily anything other than "these are things that I liked"

or "these are things that I think you will like".

So, I prefer the next slide: you want to go to it, Vance?

This is more the way I see curation,

where you collect material, you organize it,

there is even the potentially a path, well, there is certainly a path

through the individual material groups,

and then mayble even a path through the groups,

although at the moment I haven't done that last point.

So, this is, you know, kind of captures the idea that I want to talk about today.

Curation, importantly, is not the same as creation or recreation

or adaptation or sampling, or synthesizing.

It's taking the material and adding something to it, maybe just a commentary,

maybe just collecting it into some logical framework or logical sequence.

So, when I took that idea, which I was getting through the Curation Nation book,

and thought about it with respect to the material that I was using,

I decided to experiment with that and come up

with some collections of materials from -- as you probably know from the title here and also the PDF,

if you've had a look at it -- comes from TED Talks.

And in a moment I will talk about why I think TED talks is so good for that

but at the base level, these were very popular with my students.

What the students were doing more--

they were having trouble coming up with good ones.

They would always pick what was interesting

and then often come back to me and say

"Well, this was interesting, but I had trouble understanding it because my --

the accent of the speaker was not easy for me to understand."

or "I had trouble understanding it because -- it was interesting

because I didn't know anything about it and I didn't have the background

so there was a whole bunch of new vocabulary."

So t could be interesting for all sorts of reasons,

but it wasn't interesting for the right reasons,

for what we think is good for independent language learning.

Again, this doesn't mean that all of those collections, with the help of a teacher,

couldn't have been very valuable in a classroom

and especially getting to the content for connecting to discussions.

But that's not the same thing as letting students work on their own.

So, I do want to emphasis that.

My perspective here, at least initially,

is getting students to be able to do these things outside of class

and then just come back and report on them

rather than having something we do in class

or that everybody does the same homework assignment on.

Alright, so that's the set-up for what I believe curation should be,

or at least can be, within this framework.

So, I think what I'll do here is pause for a second and see if anybody has questions.

and bring it up by trying to look at some of the chat pieces here

Uh -- [he hums]

[reading:] "What is meant by sign..."

OK, so some of these chats are to each other about the chats.

So I got to go to the other window

Uh -- anybody -- anybody have any questions here?

If not, I'll continue on.

>> Stevens: I have to admit I have trouble following all the chats.

There's also a back channel here, with Google: some people could be in that one.

I never see that one until I get off of --

>> Hubbard: Well, the last chat -- the last piece on the group chat said:

"Yeah, we agree with you, Phil."

So: that's great.

I'll stop [check] there and if everybody agrees with me, I don't really need to --

>> Stevens: you need go no further
>> Hubbard: [overlapping, inaudible]

No [Hubbard and Stevens laugh]

>> Hubbard: OK, well, so, again, that's kind of the background,

this idea that I needed to start collecting things.

So, I'm still kind of almost two years in the past, now,

telling you the story of how I got to where I got here.

So I picked TED talks and I started going into TED talks.

I wasn't quite sure how I wanted to collect them

but I knew there were some of the ones that I liked

and I also knew some characteristics that I thought were useful for the students.

I thought it was important to collect them into themes.

You know, we've known for a long time that if you have related content,

that it kind of feeds -- the materials feed one another

and the students get probably a better and a richer experience,

they get more natural repetition and key vocabulary

than if you have people just kind of jumping out piecemeal

with unconnected bits of material.

I -- in the 1980's I was forced to teach a course with a book I don't remember the name of that.

I do remember the author, but I'm not going to mention it on air.

It was a reading textbook and the reading textbook had really interesting little chapters,

at least most of them were interesting to me,

but, you know, one chapter would be on the Olympics

and the next chapter would be on sea-horses.

And it's that kind of jumping around -- we typically don't do that with textbooks anymore.

And yet when we turn students loose, a lot of times, that's what they decide to do.

So again, even though I had been giving them guidance, saying:

"Well, collect several bits of, you know, pieces of material, videos or podcasts

that are related to one another in some way,"

they wouldn't follow that advice, because it hadn't been done for them.

They were still kind of chasing around, looking for the spots that just seemed interesting.

OK. I think what I'll do is tell you what the

-- at a kind of the abstract level, what I came up with

about what the curator's role should be.

And again, this is specifically for this target audience,

but I think it can be tweaked and extended to other ones.

The first thing you have to do is collect the stuff: you want digital materials,

you want to organize them in some way:

mine are organized systematically, but you could do

-- you know, you could take news stories and do them chronologically.

You need to sequence them and this is where a lot of collections fall short.

They're just -- they're either randomly sequenced

or they're not sequenced at all.

And I think it is possible, as, you know, as the resident [check] expert, the teacher,

to be able to say:

"Here's a way to move so that the earlier ones might be a little bit easier to follow

and the later ones are better understood if you've done the earlier ones."

The fourth point there that

-- on the slide that Vance has --

is the hardest part of all of this,

and that is trying to get this material levelled in some way.

Wilfried Decoo in 2010 wrote a book, it's at the end

-- the reference is at the end of the slideshow here --

on systemization.

And it was kind of a return to the idea that

even if you're using authentic material,

and especially if you're trying to create course material yourself,

that you need to have a kind of natural development of that material

from, you know, easier at lower levels, to harder

and he went to the point of even talking about keeping databases

that were very finely tuned,

so you would be able to pull out lexical items and grammatical points and so on

in a scope and sequence that fit

what we thought we knew about language learning.

And you know his -- I think his perspective is

what I think is a reasonable one to bring up again,

because I think we are often not cognizant of the difference between

accessible and barely accessible and inaccessible materials,

especially now that students can go in and, you know,

get their first-language subtitles and transcripts for a lot of these materials

and then have the illusion that they are actually understanding the English, in this case,

and that they're building their English proficiency, where they --

-- they may be to some extent, but probably not to the extent that they think they are.

So there is the, you know, that idea of --

well, in Decoo's book of fine tuning material.

That doesn't work for me because at the levels I have,

first of all, I have mixed-level classes to some degree,

although they are all fairly advanced.

They come from different backgrounds, I don't know what they know going in.

So it's a little tricky to do it in the way that he likes.

But it still gave me the impetus to try and see if I could come up with something,

you know, I'll show you that in a bit.

So, the last part of that, then, once you can give at least some kind of level information,

is to go ahead and then present your pedagogical support,

whatever it might be.

This is fairly open-ended, I mean teachers can get -- and often do get -- into material

and they start stripping out what they think are key vocabulary,

they produce, you know, pre-listening activities,

they have post-listening activities,

they have discussion activities.

All these are great, but they're based kind of on a classroom model

and even more important: they take a lot of time away

from the job of collecting this material.

So if you put the hours into making full lessons,

you end up not having the time to even produce as much as I have,

which, as I mentioned, is not as much as I'd like.

OK, so that's the curator's role and then -- Vance, if you could go to the next slide.

Did we lose you?

>> Museum curator MC [check]: Hi Phil, I just wanted to add to something you--

>> Hubbard: Yes, go ahead

>> MC: Just because of my background:

I used to work in museums
>> Hubbard: Oh, fantastic

>> MC: in education and curation
>> Hubbard: A real curator!

>> MC: Yeah. Just one other item I would add to the list

and I made a note of it in the chat section

and that's the -- often without knowing it we're making assumptions about our audience.

>> Hubbard: Ah!
>> MC: When we're selecting things,

whether they be objects for display or -- like in the museums -- or

objects for presentations to students, we're often unknowingly making assumptions

and I think it's a really important thing to know, to challenge ourselves

about the assumptions we're making in making those selections, those choices, as experts.

>> Hubbard: Yeah, I mean that's a very good point

and I have to -- as individuals, the students always change in my classes.

As a group, you know, I get to know the group better.

So I think, in this very targeted group, I can --

I can come up with at least, initially, some likely ones,

but I do in fact ask them for feedback on --

Well, first of all, I give them choices and then I ask them for feedback

both on, you know, what they chose and why, of the ones I selected for them,

and also what else they might like to see.

So it becomes a little bit od a dialog,

and that could be even more of a dialog, you know, if you have --

the way my class is structured, again, because it's so small,

we do a lot both within class discussion and with the individual tutorials.

But if you got a larger class and you got a discussion board or a wiki or something like that

where, you know, students can -- can chime in more regularly,

then you could get some information.

I also haven't formally surveyed them, so that would be useful too. I --

>> MC: You're inviting their feedback to inform --
>> Hubbard: Very much so. Yeah.

>> MC: Yeah --
>> Hubbard: But not as richly as I could.

So one idea I had was that, you know, like you've seen probably in museums,

sometimes they have the displays but they'll also have, you know,

places where people can, you know, write cards

and make suggestions and say things and drop those off

and I think, probably increasingly, we'll see museum displays

where the, you know, the viewers' thoughts are right up there and accessible to other viewers

when they go to look at the material.

So I think you're making a really good point and, you know, this is the --

figuring out exactly the role of the students who are still kind of developing,

you want to meet them half way but you also, in the curation model, I think,

want to be careful about the difference between curation and crowdsourcing,

because I've had students come up with some materials

that they thought were really exciting,

but when I looked at it, I could see what the problems were in terms of the --

the use of it by other students.

>> MC: Now I take your point: it's you acting as the filter.

>> MC: and finding --
>> Hubbard: Yeah, and that's --

and again that's -- and again that's the -- this is the kind of, to me, this the curation model.

>> MC: Yeah
>> Hubbard: The crowdsourcing model

is a great model too, it's just a different model

and it may work better in some cases.

Of course it also depends on, you know,

I've been to museums that I didn't think were very well run, were very well organized

or were confusing.

So --
>> MC: Yeah.

>> Hubbard: as soon as you have the human expert coming in,

they may not be as much of an expert as they think they are.

That's probably true of me, in fact.
>> MC: Yeah, and there are lots of people [check]

a lot of examples of museums, because I'm into curating things

and then I'm finding out that the interpretations that they were expecting audiences to have

were completely off-base.

>> Hubbard: Yeah.
>> MC: I think that's a good example

of big money going into these exhibitions and then being interpreted in a completely unexpected --

>> Hubbard: Yeah, well, the good news here is, I have no big money.

I mostly have no money at all for this. So -- [he laughs]

It's also, the nice thing is, you know, compared to the museum,

where you have all of these Unkosten [? check] to putting the material in,

once you have something, you start a web page:

if it is a disaster, or if it needs to be tweaked or significantly changed,

it's possible to do that just by finding a little bit of time.

[MC and Hubbard overlap]
>> MC It's just [missed words check]

There's even an opportunity, actually, in, as an expert,

putting together a series of well-chosen articles

and then inviting students to assemble them and put them into a -- into an order or sequence,

and to try and explain the rationale that they've used,

what connections they've seen in the works.

It's just another angle to it I sure would --

>> Hubbard: No, it's a very good angle and in fact, you know,

as I've moved through stages in probably about 15 years of teaching this course,

I've tried to give students more independence but also to give them guidance in that independence

and one of the -- what I hope I'm doing with the material I have,

I do show them how I put it together.

And I hope I'm, you know, kind of modeling curation for them as well.

The idea of getting them to maybe do a little curated piece of their own,

that could be an interesting final project for the course.

I will be revisiting it again in Spring.

I'll be away from it in Winter quarter here

because we have -- we teach 10-week quarters.

But that's a possibility for Spring, actually.

It could also greatly enrich the collection of material that's available to other students.

Again, as long as I'm there to be a kind of a filter,

rather than just releasing these into the wild.

Or if I do release them, you know, making sure that students know the difference

between ones that are student-produced and the once that I produced

and why, you know, I did mine one way.

Then they can -- they can judge to some extent, you know,

whether they think the rationale used by their peers, you know, was useful for them.

So, that's a nice idea, I'm making a note of that.

OK, shall I move on?

>> [Stevens? check] Yeah. I'm aware of a podcast - there's the slide on I'm talking --

>> Hubbard: Yeah, thanks
[they laugh]

>> Stevens (?): I listened to a podcast where some educators had gone to Europe,

probably on a junket but ostensibly
[Hubbard laughs]

>> Stevens: to visit museums and find out, you know,

especially ones that had audience attract--

you know, the idea was that museums, people didn't have to go there,

they have to attract people.

So what do they do to attract the people, as opposed to schools?

And then, how can we design our classroom environment

so it's more like a museum?

So that was actually a serious project and I'll never remember --

I'll never forget how to get it back, but maybe I will tell you in Portmont [check].

>> Hubbard: Ah OK? So that was good.

Yeah, so Vance has put up the slide that I wanted to make a point of here,

because there are a couple of things that are important about this slide, I think.

The first is, even though these are just little bullet points,

that actually took me a while to kind of figure this out, maybe because I'm slow, but --

Oop, Vance, I lost the slide.
>> Stevens: it is here again? >> Hubbard: thanks.

Because of all the other distractions I have

and because of other elements of where I am and what the -- sort of the visibility,

the first thing I have to make sure is that anything that I curate is actually legally available.

And a certain amount of stuff that I had used years before, even in my own class,

I wasn't quite so sure about what the legality was, I think, in the early days of the internet.

Even now with YouTube I try to be careful about making sure that

what I've found is something that whoever put it up either has the right to

or they're reposting something that is --

that's already got a Creative Commons license or something like that.

So, especially for something I'm going to put some time into here,

I want to make sure that what I've got is something I can use.

I also always want to make it freely available

because my students have friends back in their home countries

and they have even colleagues here who don't end up taking my class

and I have colleagues that are interested in using some of the material I do,

so everything I do in this kind of a project, I try to make sure it's freely available on the Web.

Vance, we lost the slide again, or at least I did.
[incomprehensible metallic voice - check]

>> Hubbard: Oh wait, is this Halima saying something? Uh, you know--

>> Stevens: No, Halima is unmuting herself as soon as she comes into the chat.

So I'm going to have to -- Halima, can you mute your microphone?

Because it's causing feedback.

And I hope you can figure that out, and meanwhile we put this back.

Is it back yet [missed words check] Phil?
>> Hubbard: Yeah, that's great. >> Vance: OK

>> Hubbard: Yeah, so the "freely and legally available" is an important quality

and you know, TED talks obviously are ideal for that.

They're likely to be interesting.

Again that's something -- oops, lost the slide again,

but I'll just go ahead and walk through these.

"Likely to be interesting", I guess that connects to a previous commentary [laughs]

that we don't always know what students think are interesting,

but I try to pick things that I think are,

you know, have a good chance of being interesting for the students.

The good technical quality: there is a lot of stuff, obviously,

available on the Web that's not, that's interesting and freely and legally available,

but the technical quality is such that it may be less ideal for language learning.

We're getting better at that now, certainly, than in the old days,

but when - when you're looking for material, if it's been overly compressed,

or it was done with devices that weren't that good in the first place,

it doesn't necessarily lend itself as well for language learning.

Stability is a really important point, because I don't want to do this

and then find out what I did is not available the next time I teach the class,

or even the next week.

So again, finding material that has -- either has been up for a while

or that you know is going to continue to be up for a while.

The 5th one is a -- you know, people have different views of this,

but because I'm so tied in with vocabulary development along with comprehension,

to me it's critical to have captions at least -- [coughs] excuse me, losing my voice here --

to have captions at least and ideally, to have transcripts.

And one of the reasons for transcripts is to be able to try to use some material

which I'll show you in a moment here some of you are probably familiar with:

the vocabulary profile from lextutor.

By using -- by dumping the transcript into that, you can get an idea of levelling.

And if you don't have a transcript,

then you have to kind of use just intuitive feels for what's the level.

Then I've personally seen some pretty significant problems with that.

I may mention one towards the end here

when I get to some of the alternative sites I know that already exist for this.

And then ideally, if you can find complem --

something that has complementary materials.

Again, in the case of TED talks, you've got materials that are --

you have a brief summary of whatever the talk is, right there available,

you don't have to create it as the curator,

you've got the bio of the speaker, which is good background information,

and in some cases you even have -- I think, what do they call it, TED Ed or something --

there are some TED talks that even have some additional material that --

that people have added to them, in the way of discussion questions and things like that.

TED's not as rich as, say, you know, if you're doing a newscast for example,

and you might have several written forms of the same news story

that you can use for back up:

it's not quite as rich as that,

but it's still pretty good with giving you some of these complementary materials, besides the video itself.

OK. You want to move on to the next --

>> Hubbard: Actually, it's probably the next couple of slides
>> Stevens: Yeah.

>> Hubbard: does someone have a question?

>> Stevens [check]: Yes, Peggy George has asked questions in the text chat, the Etherpad one.

Let's see, I can -- she asks:

"Are your students able to share your curated content with others outside the course?"

>> Hubbard: Yes. Yes, som you'll see the --

in fact I think it comes up here on the next slide or couple of slides.

Actually the next slide, if you go to the next slide, let me talk briefly about that,

because it does have to do with the sharing.
>> Stevens: Mmm - OK

>> Hubbard: So that the link there is to the advanced listening website

and you'll see, you know, quite a bit of material there,

not just the TED talks.

The link - the specific link to the curated TED talks is a couple of slides from here

but those are -- those themselves are legally and freely available.

They're my websites, they are on the Stanford server:

Stanford is not going away any time soon, as far as I know I'm not going away any time soon.

So those are not only, you know, available on the World Wide Web,

unless you happen to be from a country that is for some reason blocking access to Stanford:

that has happened a few times in the past.

But if not, then you can get to that material

and all it does is jump out to the TED talks themselves

and the TED talks again are, you know, freely available.

I noticed in one of the preliminary discussions

that somebody had put in some comments, before this began, on the learning2gether site,

and mentioned YouTube videos, and YouTube videos are certainly a great resource,

most of my students are from China and most of them, then, unless things have changed,

can't freely and legally get the YouTube videos there.

And so for that reason I try to -- I don't avoid YouTube

but I try to limit it and I like to make the curated collections

something that my students will be able to use and their friends will be able to use.

OK. Any other questions?

Uh, so, yeah, so they are available and when I -- just so you know --

when I redo the course every quarter, that URL there stays the same, the material is new.

Well, most of it is old actually, but I do update it

sometimes because I come up with other ideas

and sometimes because some of my other class material disappears.

But the home page of that each quarter has the link to the previous quarter's materials,

so you can actually step back from quarter to quarter and go back.

I never throw anything away on the Web, so it's probably got stuff from 5 years ago

if you keep clicking back through the previous quarters' material.

So you can see what it was like in the past ["without"? check] sort of my own Internet Archive.

OK. The way that I did this material, let me move on to the --

Well, I guess on this slide,

the problems that my students have, typically, fall into issues with speech rate:

some of the TED talks are too fast.

It doesn't mean they can't, you know, use top-down skills to understand the basic content,

but that's not necessarily going to help them drive their --

either their listening proficiency, you know, their ability to process English, automatize it,

or their ability to pick out the vocabulary that they don't understand or --

even more interesting is the vocabulary they sort of understand or partially understand,

but they just can't get to it, they can't access it in the time with a faster speaker.

And there are others in my class, actually, that do OK with some of the faster speakers,

but just having knowledge of the speech rate is useful.

Preliminary knowledge of the accent: just a -- since in some cases we have students

that are having particular difficulties with particular accents, often of their professors,

and they may actually be doing a project where they're trying to focus on that accent.

And so in that case, knowing more about the accent is helpful.

And others are really trying to -- I wouldn't say "master",

but at least becoming -- become more proficient with the North American accent

because they plan on not only doing their graduate work here, but staying a few years afterwards.

It's a very common professional track for our students whether at the Master's or the Ph.D. level, to --

because so many of them are in technology, they want to hang around Silicon Valley

as much as the can after they, after the graduate.

OK. If you could go to the next slide, Vance?

>> Stevens: OK I might
[both overlap]

>> Stevens: You mentioned Claude Almansi's contribution to the wiki earlier

and one thing that she said -- she left this on the Google+ page as well:

I post this to several pages.
Let me just get rid of that slide for a second.

I see I can do that by clicking off the screen share for a second, OK?

Well, anyway. She does work in closed captioning,

she does a lot of very interesting work relating to MOOCs [check] where she is.

And one of the suggestions she made -- I didn't know this, but maybe you did already,

but you can -- she said you can, if you get the MP4, if you get an MP4 of a YouTube video,

you can then load it into Audacity -- I didn't know that --

and then you can adjust the rate of speech there, without causing any chipmunk effects.

>> Hubbard: Mmm.
>> Stevens: I thought that was kind of neat.

Sounds like useful information?

>> Hubbard: Yeah, that's -- again, there are lots of things you can do to go more deeply into this stuff.

I -- one of the things I do with TED talks is, you can also download TED talks and you can --

even if you put them into something, well I use the VLC player,

because the speech rate slider is right on the top,

it's much easier to get at than it is in QuickTime or in Windows Media Player.

I like the VLC player for other reasons, in fact.

But, you know, once you have downloaded you can use the VLC player to --

for the most part you don't really get the chipmunk effect

because it's trying to expand the time domain without changing the frequencies,

it's not like the old days with LP's and cassette tapes

where time and frequency were connected to one another.

Digitally, you can isolate those.

What we found is that if you slow somebody down to about 80%,

you can get a lot more processing time and it still sounds natural as long as you have good material.

If you have material that's already been compressed too much,

then those compression artefacts become stronger if you try to slow it down.

Occasionally, we get people that my students want to speed up

but most of the time, for language learning processes, we're talking about slowing it down.

So it's -- using, changing speech rate, that's a whole different talk,

but it's, I think, a very underused functionality and something that students sometimes baulk from

but we have some research evidence that it's helpful when the students have control over it.

Anyway, I don't want to diverge too much on that, but that's a --

I do encourage everybody to read that post

and see in more detail what some of the options are for doing that.

In fact, one of the -- one of the problems with using the VLC player with those is,

if you -- if you do try to slow down the speech rate by downloading it and putting it in the VLC player,

you actually move the subtitles, because the subtitle feature --

the captioning feature in the TED website is built into the website, it's not built into the video.

And so you would need to do some additional captioning if you want to do that.

My -- if your goal is general comprehension and you've got decent material,

then I'm a fan of using the Google beta transcription.

Even with good material, it makes a lot of mistakes

and with material which, you know, isn't really, really clear,

either because the speaker wasn't clear, or because the signal wasn't clear,

it makes a lot more mistakes.

And in my case, when I'm trying to have students use it for vocabulary development,

if it's got -- if it picks the wrong word, then they're going to be learning something pretty weird.

And it does that all the time.

If you change that and, you know, get around to Google Translate, to get first-language captions,

you just accentuate the error rate.

So again, it really depends on what the goal is.

If the goal is letting students watch a video for cultural and general content information,

maybe to trigger classroom discussions, things like that,

then using the automated captions is not a bad idea

and being able to slow down is not necessarily -- is, well, I think a good idea.

So again, it depends on what the goals are, but you have to be careful,

because the Google beta, there is a reason why they keep calling it beta,

it's because it's pretty error-prone.

It's getting better but it's not there yet.

And if students think it's an accurate rendition

that's going to be even more difficult.

If you do use the automated captions then the students need to be prepared for --

you know, to be able to recognize when something doesn't make sense.

It's usually -- it's a very obvious semantic issue

with the words they pick.

OK. One other thing, I don't remember if it was in that post or another one but

something I hadn't noticed before someone mentioned that there is a slight delay

in the synchronization of the captions in TED

compared to the system that they were suggesting.

So, uh, that's something else to take into account.

You might, If that delay seems to be an issue for you or your students, then --

it's something that I plan to explore because I hadn't noticed that before.

Okay, a little bit about how I finally figured out to do this,

which is not the way I would recommend doing it now necessarily

But this is how I started working on this. When I did it I guess it was Spring of 2011.

The first thing was to...oh no, it wasn't Spring: Fall of 2011.

The first thing to do is to get the TED database.

It turns out you can get an excel spreadsheet that has all of the Ted talks on it.

If you go to their website you can see that there's a link for that.

And the nice thing about that is that you can skim that a whole lot more easily

than you can skim other material

and you can also look, among other things, it tells you what the length of the talk is.

And most Ted talks are around 18 minutes and most students attention focus ability is less.

Um, okay, the database then, when I did it myself,

it was smaller for one thing, at that point.

But I did sort of skim it and looked for ideas, looked for themes

and searched for keywords.

So creativity was one of the first ones I did,

so I was just able to search for anything that had creativity

either in its description or in its title.

I put together a list of candidates within that.

I was looking for four or five talks to make a kind of a cluster,

a sort of a virtual room in the museum if you will.

And for each of those, I -- well, first of all, I did listen to the accent and got that.

I wanted to get at least a proxy for the speech speed

and so -- the speech rate --

so I just took the transcript, dumped it into Word

so that I got a word count, divided that and came up with words per minute.

I was actually quite surprised at the range that I could see there.

If you go to the website for cre-- the link for "creativity"

on my ted1 website of the curated talks there --

the slowest speech rate is like 91 words a minute.

Some of that is because there are pictures being shown in between

but it still means you got a lot more time to process the language coming in

than if you got somebody coming in at -- at a higher rate.

Some of my students do a --

there's a website at Stanford called "Entrepreneurship corner"

and they have a lot of Silicon Valley types come in

and give talks on campus.

They also have transcripts and subtitles for that

and one of the talks that I always have the students try

is Marissa Mayer who, at the time she gave the talk, was a VP for Google

but is now the CEO of Yahoo!.

And she talks between 220 and 237 words a minute on the one I have,

so I use her as an example of where you might try to use the speech rate shift

and be able to use the slider to slow her down to 80%.

OK. The next thing, once I have that rough speed --

and again, it's just a rough speed, but it's better than not using technology

and try just to use intuition about "This is too fast, this is too slow."

The vocabulary profiler -- this is Tom Cobb's work of genius in my opinion.

There are a lot of parts to that lextutor.ca site,

but the one that I use for this purpose is the --

well, at the time, was the British National Corpus profiler

and there is the link to it there.

Basically, you dump a text, a transcript into it

and it gives you as output all the words divided into 1'000 verbal frequency bands,

so, you know, which words are in the first thousand words of English,

the second thousand words of English, and so on,

all the way up to the 20'000 level.

For my students, we try to focus more on the, you know, just doing a short --

well, we -- I try to get them to focus more around the 5'000 level,

so anything below that that they don't know,

it means it's a word that they should learn.

And when you go to my site, you can see how that's split up.

I skim the transcript for unusual terms and idioms --

Oh, I meant to mention: in the last few weeks,

Tom has actually added the Coca, it's a contemporary corpus of American English

and blended those in,

so it now goes up to the 25'000 level.

And it has much more American English in it now,

rather than just the British.

So, for those of you who like, you know, concordancing

and corpus studying, study and so on,

it's got a much richer layering out than it did when I was using it for this purpose.

OK. So that's the process.

Now I said, you know, I would do it a little bit differently, probably.

It turns out that, since the time I began this and now,

TED has come up with its own curated collections.

And so, if you go to the TED website,

you will see a link to something called "playlists"

and these are collections of material that people have put together.

In some cases, it's done by TED itself,

you know, whoever is in the background working there,

but they also have curated collections by Bill Gates and Bono

and, you know, other famous folk,

or in some cases, they're people who are less famous

but, you know, are very well-known within their, you know, their more restricted field.

And there's some really, really good collections there.

So now, instead of just going to the database,

my inclination would be to go to that -- to those other curated playlists.

Those have been curated just by interest

and so if you have a list of maybe 10 or 12 videos on one topic,

you go through those, and maybe you pick out the 4 or 5

that you think are easiest to work with.

So, that's there on the "Recent changes".

I did -- number 2 there where it says

-- this is from my talk last July --

Well, I did have a project assistant who has collected some more material for me

and basically run it through the -- the Word, you know,

done some of the preliminary work for the words for minute and the vocabulary profile.

Unfortunately, that came at the end of Summer,

right before Fall quarter started for me

and I have not had a chance to really look through her material

but I do have some partially digested material

that should help me create some new stuff.

I guess, at this point, probably the most useful thing --

Vance, could you -- can you actually click on that link to the TED1,

just so I can sort of show people what --

[Stevens and Hubbard speak together]
>> Stevens: ... already.

It's in the text chat. So --
>> Hubbard: Ah, OK, so people can go to it

on their own? Alright, then you --
>>Stevens: I can also share it.

>> Hubbard: Well, the only thing -- I think if you go down -- go to the next slide actually,

there's the Creativity group.
>> Stevens: OK.

>> Hubbard: and this will show --

This is a more condensed version of what you would see on the page.

>> Stevens: Mmh. OK: let me share it.

>> Hubbard: but this is, yeah, this is the Andy Hobs-- Hobsbawm,

I'm not sure how to pronounce his name.

This is a nice beginning talk, I think it's the second talk in the Creativity group.

You can see it's a pretty short talk, just three and a half minutes.

You can see the speed is 135 words per minute.

He has kind of a dramatic presentation style,

so that's why it's a little bit more slow, a little bit slower,

it's very articulate, it's very easy to hear,

it is more of a British English rather than an American English version.

The vocabulary you can see at the 5'000 level,

so if you -- to read that, I mean, 95% of the words are in the first 5'000 words of English.

And, you know, 98% of the first 10'000, and then Off-List.

The Off-List on these are often proper nouns.

So those don't necessarily cause a great deal of difficulty,

especially if some of the names in it,

or names of places are ones that the students already are familiar with.

[stifles a sneeze] Excuse me.

About to sneeze. Wasn't expecting to do that online.

And then a little bit of a comment: "creativity is repeated a number of times."

So, if you go to the website, you'll see it's a -- it's a little richer than that

but this kind of captures the main point I want to say.

So I said, I'm giving value added, as the expert,

and not only am I collecting these things,

but I'm using technology to give students some idea of level.

Ultimately, it would be great if I could say, you know,

this is level 5 of 10 levels, or this is at the B1 level of the C, CEF,

or just even, you know, this is high-intermediate, or something like that.

I don't have that confidence yet,

so at this point I'm giving students more the raw data,

but I do actually tell them, and I may highlight this in future versions,

that at the 5'000 level is probably the most important pivot point for my students.

If that level, you know, if that's up to 96 or 97%,

that means they're going to understand that fairly well.

If it's down even to 92 or 93%, then there are likely to be enough words in there

that they are going to have gaps.

And they'll be able to process it top-down, extract information from it,

but it won't be as valuable for language learning.

If you look at the, you know, the research on reading,

which is much better established than on listening,

anywhere between 95 and 98% is what people typically quote as

material that's ideal for language learning.

And below that percentage, if you don't know those words,

then you're not going to be reading or listening in the same way:

you're not going to be processing the language in the same way.

OK. I see we're moving ahead quite a bit on time here.

So I think I'll go to a -- just a final slide.

Vance, could you go to the "Related examples" one?

>> Stevens: Uh-uh.
>> Hubbard: Just to let you -- yeah --

these are just a few places that I know of

where people are trying to do something similar.

The CLILSTORE Project is a very big European project

and they have collected material.

Some of it they've done themselves,

some of it teachers have put into there, to their database,

they have information on the talk itself, they've --

whoever has done it has put in a CEF level,

so it starts with all the A1 material, then the A2, and then the B1 and then the B2.

It's done in -- I checked -- and it's apparently done intuitively.

It's not apparently done by running it through, you know,

the kind of material that I was using there, lextutor and so on.

But at least, it means an expert teacher has said:

"Here is the level I think this is at."

I will mention some issues with that myself,

because I looked at it and they put a TED talk in the A1 level, somebody had it.

And it's definitely not the A1 level for -- at least for comprehension purposes.

It may be that the person put that in

because the content was so valuable

and that they thought that this is something that could be used

you know, with a lot of help.

But the main feature of the CLILSTORE material is that all of the words --

there is a transcript

and all the words in the transcript are linked to multilingual dictionaries.

So you just have to click on a word in a transcript you don't know,

and you immediately get the response.

Ayamel is a Brigham Young media project

where they've been collecting and cataloging authentic media.

All of these use authentic media, I want to emphasize there,

but you could go back, you know,

and curate Randall's Cyber Listening Lab or something like that as well.

And then I haven't checked this one.

I hope the link is still good.

There is a product called Lingle

that will index according to Common European Framework level,

so if you put something into it, it will give its best guess

as to, you know, what the level is for both reading and for listening.

OK. Let's see.

I see somebody else has put a CEF level (check) over in the chat -- James.

So, some of you might want to take a look at some of the [missed word, check] material.

As somebody else mentioned the SRA, by the way,

and I don't have it in this talk,

but I -- some of you may be familiar with Tom Robb's work on graded readers.

Charles Brown has also been doing a lot of work in that.

And graded readers are clearly examples of material that fits into this.

The difference is, they're not freely available.

Unless somebody has come up with a really good collection that I don't know about yet,

you're always stuck with having to pay a fair amount to a publisher

to get the graded readers, and they're not authentic,

but I think they're useful enough,

like, I don't think that authenticity is all that great a thing if you're --

if you're still at a lower level and just trying to get the language in.

So, I think, something that's well-written and, you know, well-produced,

to me, whether it's technically authentic or not is a secondary issue.

My key here is this idea of freely available, and often it's not.

OK. Let me go ahead and stop again:

the, you know, the final comments and the reference list and all of that,

you can get if you go to the PDF yourself

but I do want to give people a chance, especially some of those who have, you know,

either put things over in the chat

or who have read things over in the chat that look interesting,

and certainly the participants here.

What -- what questions and comments do we have?

>> Stevens: Claude Almansi has arrived in the chat

but she's shy about coming in because she hasn't heard the whole conversation,

but anyway: I guess she could if she wanted.

There are seven people in the chat now, so we have -- that is, in the hangout --

so we have room for three.

>> MC: Hi Phil, I just wanted to add, I think that this is
>>Hubbard: Uh uh

>> MC: This -- it lit a light bulb for me.

I just thought this getting a group of people together,

use this kind, this style of approach to collecting transcripts

and analyzing the transcripts

and the being able to put them, I don't know where,

somewhere up in the cloud,

I mean over time, you could have quite an assortment of these readings

to be shared collectively, and then, basically, circumvent the SRA

and the [missed words - check].

This would be fantastic.

Just thinking aloud here.

>> Hubbard: Well it's, you know, certainly part of what I'm trying to do here,

because I don't have the time that I wish I did for this,

you know, is to both encourage other people to be interested in it

and get into, whether they're formal or informal collaborative projects

for putting stuff together,

and then, secondly, just for experimenting for yourself,

because this is something that I've thought about a lot, I've done some reading in,

and I've also tried things along the way

and have ideas about how to make it better,

but it's still very much in its infancy.

And this is if -- if is the subtitle of Rosenbaum's book is that, you know,

content is the -- curation is the future of content,

then this is something we need to get better at

and be thinking about that for different purposes,

so, if you're curating materials, for example, for teachers to use in classrooms,

that could be quite different from curating materials

that you're going to have the students use independently.

It could hopefully overlap quite a bit,

but it won't be the same.

And curating materials for cultural purposes, or for triggering discussions,

is not going to be the same as the way I've curated here,

you know, where I'm more concerned with level than other people might be,

whereas others might be more concerned with the content itself

and, you know, don't have a problem with letting people listen to it

with subtitles in their native language.

Or, you know, with sort of getting the gist of it without necessarily getting all the details.

So there is a -- there's a very rich area here to explore in lots of different directions.

What I try to do too, is keep the curation pretty light.

When I've given this talk before, a couple of times,

people often come up with ideas for, you know,

adding discussion questions, basically making it more,

into more lesson-like and adding --

in other words, having more value added.

And I think that's great.

I'm trying to come up with sort of an intermediate stage

where I'm doing something that's helpful,

but not something that -- that I wouldn't have time to do otherwise.

Are there questions, comments?

I see there is a lot of people talking about SRA [laughs] over in the chat.

>> Stevens: [missed words - check]
>> Hubbard: Yeah.

>> Hubbard: So for those of you would be listening [missed words check]

I'd be looking SRA Science Research Associates is a --

a set of graded readers that was very popular in the US,

probably going back to the 1960's, 1970's.

I remember using it in my first reading lab

when I was teaching in the early 80's and it's,

unless it's changed, it's designed for native speakers materials,

but it's aimed at trying to lead you step by step into reading proficiency.

And it's a fairly traditional approach of, you know, read,

respond to comprehension questions and so on,

but it does have -- it does timed readings and some other things as well,

and is very much in the general graded readers approach.

Oops, perhaps I'm not hearing you.

>> Stevens: [missed word check], how about this one. Oops.

Is that working? OK, yeah.

My USB went out, now I'm on word mike [check]

OK, well, anyway, I was saying that I have some perspectives on this,

having seen you present some of this at the TESOL conference in the KIS group.

Your focus was a little bit different at that time,

it was on the videos themselves.

And I wonder if you have a link to that presentation

that you could put in there,

because there were really nice examples of what you can do with this,

as far as finding graded materials for your students,

and -- because I think you had those organized in such a way that they started simple

and went to more difficult.

>> Hubbard: Right, the materials and I think what I did in that talk

is basically walk through the -- the ted1 web page.

And so if you go to that link that we've already -- that I've already put in there,

the one that ends in ted1--
>> Stevens: yeah

>> Hubbard: let me see for sure if I can bring it up (1:12:05)