YouTube Developers Live: Captioning with Amara

This week, we'll be chatting with the folks behind Amara, a captioning/subtitling platform with deep YouTube integration.

Hi, I'm Jeffrey Posnick from the

YouTube API Developers Relations team.

And I'm here with Dean Jansen from Amara

joining us live from New York City.

And via Hangout, we have Arthur.

Hello, Arthur.

(No audio)

And I'm having a little trouble hearing you, Arthur.

Hopefully, we'll be able to work

through those sound issues.

Arthur's joining us from Sao Paulo.

And he is the developer working on Amara,

and Dean is one of the co-founders?

– Yep.
– Cool.

OK, so we're talking about Amara.

What is Amara, and how did you get started?

What's some background?

Yeah, yeah.

So I guess we've been working on this piece of sofware for

about three years now.

Amara is essentially, we think, the easiest way to

create captions from scratch or subtitles for videos.

And the beginnings of the project really were around

why is it so difficult to create captions?

A lot of the software, while freely available, can be difficult to use.

You've got to download it, install it on your computer, etc.

And so that's kind of what the birth of Amara was.

And we figured, this is the Internet age.

Why isn't there an easy way to do this,

and why isnt't it collaborative?

And so the whole idea behind Amara was make it so that you

can take a video from anywhere on the web, wheter it's on

YouTube or Vimeo or hosted on your own server,

plug it into out platform, and then be up and running in seconds,

and also be able to engage friends, strangers, volunteers,

anyone on the internet,

to help you then translate video.

So a big part of it also is around collaboration and

the Wikipedia style model, yeah.

And to be sure I have my terminology right,

is there a difference between captions and subtitles?

I don't want to use one instead of the other.

That's a really good question.

And we had a lot of learning to do also,

and I think we're still learning a lot.

But captions in the US are oftentimes thought of as

something for someone who is deaf or hard of hearing,

that will have the audio information.

So if a door slams shut, then you'd get "the door slamming".

Whereas subtitle is, I think, a bit more of a generic term,

and is often used when you're talking about translated captions,

possibly without the audio information.

OK. Thank you for clearing that up.

So all right, you talked about the different video platforms,

that you integrate with, and obviously, YouTube shows.

We're very interested in hearing about your YouTube integration.

But can we talk a little bit more about who the target

audience is for your platform?

Who do you find using Amara?

Yeah. That's another good question.

Well, of course, I think it's also worth mentioning,

we're a nonprofit organization.

It's an open source project.

And the reason behind what we're doing really is to make

it so that video can become accessible, and not just in

the sense of accessibility for deaf and hard of hearing,

but also accessibility in an broader linguistics sense of

translating videos into as many different

languages as possible.

And so the audiences are varied.

Someone who's going to be a consumer of captions or

subtitles is going to be anyone who doesn't speak a language,

which I'm sure there are a couple of languages you dont speak.

I certainly don't speak a lot of languages.

– Absolutely.
– And then we have a pretty exiting

community of both deaf and hard-of-hearing people,

but also we have a caption team that's purely around

requests from deaf and hard-of-hearing users.

So they can basically say:

Hey, I'm interested in this video.
I'd like to have it captioned.

And then people caption it.

So there's a lot of different viewing audinces.

And then, of course, I think anyone who's making video is

potentially a creator audience.

Sure, totally. And obviously, it goes withour saying,

YouTube is a global platform.

I'm an English-language speaker,

but I often find myself interested in content that's

not in English, and it's a real barrier to me if there

doesn't happen to be subtitles for that.

So I certainly appreciate it even as someone who's an English speaker.

So we've talked a little bit about content creators.

And let's say I'm not somebody who would initially think of,

hey, I should be providing captions or subtitles for my videos.

What's the value proposition?

Why should I, as a content creator, really go about

captioning as much as possible?

Well, I think if you strip out a single reason,

then it can be easy to brush past it.

But when you think about the broad application of having

same-language subtitles or captions as well as translated

captions, you start out with having a basis of better SEO.

So your video is captioned,

you've got more metadata around the video content,

it's easier for people to find,

it's easier for you to go back and refer to.

Well, when we're talking about dones in that new sequence?

And we can get right there.

On top of that, you have the accessibility reasons for deaf

and hard-of-hearing folks.

On top of that, you start to potentially,

if you're engaging volunteers, you're bringing in folks

and engaging them in a real and meaningful way in your content.

It seems like a nice way of building community around a video.

Well, the TED Talks Conference,

they have a humongous volunteer community of translators.

I think there are 14,000 or 15,000 of them right now.

They're all volunteers.

And TED thinks of that community as a hugely valuable

part of the overall TED Conference organization.

And they speak very highly of the people who are putting in

their time and effort and a lot of hard work to translate

the 12,000 or 15,000 TED talks and 20,000 TEDx Talks.

Yeah. And this is obviously one of the more scalable ways of doin that.

That's a very daunting process if you have one person.

One person who happens to know dozens of languages sitting at

a keyboard, that doesn't scale very well.

Right. So to get back to your question, even someone who's a

content creator that has a popular YouTube channel,

let's say, and may not know a lot of reason to do this,

if they want to get a more global audience,

this is a way not only to get your video subtitled into different

languages, but also to have ambassadors,

poentially, in these other languages.

And this is the sort of thing that's hard

to come up with really strong metrics about,

but ceratinly, I would assume that this increases viewer engagement.

It just really raises the potential audience from just

everybody who speak one language to a much wider

global audience.

This was your trickies question, I think.

This is something that because we're not the primary viewing

platform, you can come on to Amara and watch videos in

different languages.

But I think a lot of times, people are syncing subtitles

back to YouTube, let's say.

Or on ted.com, they're hosting them.

They're caching them on their side.

So a lot of the metrics we actually don't have access to.

We would love it if YouTube started making some of this

stuff more public or even a broad summary of here's what

we're seeing across lots of videos.

Yeah. That's something I'd like to follow up with internally, definitely.

OK, so hopefully, folks are aware of the power of captions

and why going about captioning your

videos is pretty important.

Could we actually dive into a demo of how you

could use Amara to do that?

And I might have to unlock for you right now.

But if we could pull up the demo,

I think we should be good to go.

All right. So I chose a nice and non-copyrighted

Creative Commons video for us to experiment with here.

I'll jump over to the Amara website.

I guess right now, there's a lot goint on right now.

But the most important thing is there's a box right here

that says "put a URL in and click Subtitle".

So I'll go back to my YouTube tab.

I'm goning to copy the URL here of this

Creative Commons video.

I'll paste it inn, press "Subtitle",

and fingers coressed... bingo!

This is an Amara video page.

This is the home base for the video.

And if you're wanting to invite other folks in,

this where you'd point them.

OK. So again, you can put any video ID you want here.

It's not necessarily something in your own channel.

It might be something that you're just interested in,

and you want to engage the community in

crowdsourcing the captions.

– Yes
– Right

And it can be pretty much any kind of video,

any format that you want.

So I'm just going to click on the "Subtitle Me" button,

and then it's going to ask what language is the video in.

It's in English.

We're going to subtitle into English.

And there's a little tutorial video that I won't go into.

Let's see.

So this is the interface.

This is pretty much all HTML, CSS, JavaScript.

Of course the video itself, I think, may be Flash.

But it doesn't require Flash.

Any modern browser will use this.

I think we're in Chrome right now.

Hopefully, you're using the iframe player API,

which will us the HTML5 video for playback.

I haven't double-checked that,

but we can talk about the techical aspects with Arthur.

We'll ask Arthur, and we've got a new editor

that we're just about to start rolling out.

So I think that if we're not doing it now, we will be.

OK, so to start with, I think for anyone who's done captioning,

this may or may not look familiar.

But the process is oftentimes: set in point,

type out what you hear,

set out point, next...

And so we've broken it into three distinct steps

that break that apart.

The first is just transcribing everything you hear

and laying it all out.

So I'll do one line here just to show how it works,

and then I've got another window that has more work done.

So I just press play here.

I'll rewind.

– What does it mean to be human if we don't have a shared culture?

So being a speedy typer is pretty important.

So the interesting thing is that we've go a thing here

called magical autopause, that's one of the modes.

So as long as I'm typing, I'm doing a little bit of movement

of the cursor to keep it typing.

When I let off, it's going to say, OK, you're done typing

and unpause the video for me and rewind it just a little bit.

Oh, that's quite nice, OK.

There it goes and so I can just keep typing,

and it will watch the background of my typing

and pause and unpause for me.

That's a really nice integration.

So once I've got everything typed out in this other window...

Oh, let's see if this is going to work.

Successfully resumed editing...
Yeah, all right. Nice.

So in this one, I've typed out a bunch of lines.

You just go through.

You type out all of the dialogue.

Then you go to the second step.

I've got another helpful video.

The second step is modeled after a video game.

It'l like Dance Dance Revolution.

And the idea is that you're just tapping every time you

hear what's the dialogue, and it's going to time the lines out for you.

So I'll just do a copule of these lines

so we can see how it works.

And I don't know if you're able to hear it.

( VIDEO PLAYBACK )

So I'll pause that.

And the last thing that I think is worth showing quickly

is the final step, which is the review,

where you're able to just go in and fine tune things.

So if I've timed something late, it's like a nonlinear video editor.

I'm able to hop in, change everything up.

If I made a mistake, I can easily edit it, etc... etc...

It's pretty intuitive and a simple interface,

and we've seen people as young as eight years old use this thing.

That's awesome.

I've obviously interacted with YouTube captions before,

and I've done some API integration.

And I know it as having to provide this data file that

has time codes and everything like that.

So having this whole automated process for generating that,

seems like a huge win.

So that's very cool.

And it's really nice integration, too.

And once you're done, you submit your work.

You can then download the SRT files.

We provide a lot of different formats.

You can upload those onto YouTube, and now we'll talk,

I'm sure, about the YouTube syncing integration as well.

Excellent. Cool.

So thanks so much for showing that demo.

Hopefully, we have Arthur via Hangout,

and I'd like to say hello again.

Arthur, can you hear me OK?

Yeah, perfectly fine.

Ok. And I can hear you now, so that is definitely a lot better

than earlier on the show.

So tanks so much for joining us.

I know that you're one of the developers who was working on

this open source project.

I guess, first of all, it's interesting that this is all

an open source project.

So if folks are interesten in looking at some or all of the code,

how would they go about doing that?

It's on GitHub.

It's github.com/pculture/unisubs

I'll paste the link on the chat.

Yeah, and we'll add that to this video

description also, maybe do a nice annotation.

Or maybe it will show up in the captions for this video.

This definitley is a video that we should get captioned.

Cool. I saw a really wide range of different

YouTube API integrations there, Arthur.

So can you talk a little bit more about which are the APIs

you are using?

Yeah. We use YouTube API in a bunch of places, actually.

So when you're adding videos to Amara, basically our

servers are using Version 2 of the API, which is the

Atom-based one, to gather data atbout that video.

Everything from:
Is this video private?

Can we view it?

Then metadata, the title, descirption, and length,

and stuff like that.

So that's how we register that video on our website.

As soon as you add the video, we pull every caption that the

video has on YouTube already.

Not the auto-created ones, but the ones that are

user-generated on YouTube.

And that's part of the standard v2 API

that anybody could do for pulling the captions.

That's right. It's part of the captions API which is just standard,

so we pull those in.

Whatever you have already on you video on YouTube,

it's going to be there on Amara.

So it's nice because if any subtitling work had been done,

on YouTube, once you add it to Amara, the work isn't lost.

It's already there.

So that's using the regular API, captions API,

to bring them in.

That's pretty much on the back end.

When you are using our tool to subtitle the video,

or we also allow you to embed the video anywhere else,

on website, on a blog, just like [INAUDIBLE],

we are obviously interfacing heavlily with the JavaScript API.

The Player APIs.

Are you actually using the iframe player?

Do you know if you're using that or the older...

The iframe player is in beta.

We are now using, I think, the iframe player only on Android,

because we'd just rather not use Flash.

We ale slowly oving to the iframe player.

OK. We're out of the experimental mode for the iframe player now,

which we were in for an extended period of time.

So at this point, it's definitely what I'd recommend

using for both desktop web-based interfaces

and also on mobile.

So you can take a look at that.

And there is just another use of the Player API

that's interesting that we're doing,

is that you can embed the video plus the Amara widget on you website.

Sometimes, let's say you already have a blog with

dozens of YouTube videos, and you don't want to go back to

your blog and change all the embeded codes to our embeded code.

You can use something we call the widgetizer.

It's just one JavaScript you insert.

And as soon as we load, we are going to look at the page to

see if there's any YouTube video.

And you hook up into the already embedded YouTube video

and load our widget together with it.

So it can attach to an already playing YouTube video.

And that widget will display the captions out of bound?

Yeah. it's going to show those captions anyway.

Very cool.

So in terms of programming languages, there's obviously a

bunch of JavaScript on the Player side of things.

What language are you using to interface with the Data API?

At the back end, we are using Python.

I know that using Python with our older Data API is a bit of

a challenge in some regards, so thanks for

bearing with us there.

At this portion of the show, I would normally tell people,

hey, maybe you should check out our newer Version 3

of the Data API.

Unfortunately, one of the things that we have not yet

managed to port over to Version 3

of the Data API is the support for the cpations.

So that is something that very legitimately still requires

use of our older Data API.

But I'm certainly hopeful, at least, it will be added to

Version 3 before too much longer.

So stay tuned for that.

We make announcements about that sort of things on our blog

and Google+ feed and that sort of place.

So any general feedback that you have about the development

process, any gotchas that you want to point out to other

developers who might either be looking at your code and

trying to learn from it or trying to do similar things themselves?

Well, I think the Data API, the older

versions were a bit hard to work with.

They were a bit for both.

They are not very Pythonic, I think.

In general, within the Python ecosystem,

they are a bit off place there.

But in general, it works.

You just need to be careful.

It's reasonably documented, and things are pretty predictable.

I think at the scale we arok, we sometimes...

API quotas and smaller things that once you get a bit big,

they start to be a concern.

But in general, everything has been pretty smooth.

The only thing I would say, depending if you are doing

deep captions integration, YouTube uses foreign language codes PCP-47,

which is the same standard that XML:lang uses.

And while it's a very good standard,

it's also a bit complex.

So if you're doing a lot of back and forth between subtitles,

you just might want to watch out for that because

the way the standard does, locator-specific and just

language definitions is a bit wonky.

It's something to be wary of.

Ok. That's definitely very good feedback to be aware of.

At this point, I actually wantet to quickly check to see

wheter we had any questions on Google Moderator for

anybody who might be watching this show.

I'm going to pull that up quick.

And OK, so there are a couple that might be a little off topic,

but there is one here from Sparkyish saying,

"Awesome stuff.

Obviously, this setup is for on-demand videos.

Any plans to tackle live video streaming with a similar setup

and speed demon typists?"

We're broadcasting right now as part of

YouTube live platform.

Have you been in any discussions with anybody on

our end about YouTube lives or other types of streams and

captioning there?

Have not yet, but I think that's an exciting idea.

It certainly has come up before and is something that

we'd love to explore.

Absolutely, yeah.

And I don't think we are ready to yet on YouTube Live

platform necessarily, but there's always the ability to

have captions that are out of bound from the video

and that sort of things.

So yeah, definitley a very intersting topic.

OK, so thanks so much for taking time to go into the system, Arthur.

I'm sorry I cut you a little bit off there.

Hopefully, you got through all the topics that you wanted to

cover on the technical side of things.

And thanks so much, Dean, for joining us in person and

talking about Amara.

We will make sure that this video is captioned.

Sounds good.

We could go through the whole Amara system as well.

Great. Yeah, we can try and get it translated

in a couple of different languages.

Awesome.
Very cool.

One last thing, we plug at the end of most of our shows that

Google Developer Relations, which is the team that I'm on,

is hiring, as always.

So if you're interested in doing the same sort of things

that I'm doing, going on camera and helping developers

and chatting with people doing cool API integrations,

checkout developers.google.com/jobs

So that's it for this week.

Join us next week for our upcoming show.

Cheers, everybody.

Thank you guys! Later.

Thanks.