a11ynyc Plover

CAMERON: Hi, everyone. Welcome to Pivotal
Labs and the New York City Accessibility Meetup.

Thank you for coming tonight.

We're very excited to have our second meetup,
and we're happy to have all of you here.

So Pivotal Labs -- I just want to give a shoutout
for hosting us tonight.

I work at Pivotal Labs.

We're an agile development consultancy,

doing mostly web development
and mobile app development.

So if you have any needs in web development,
or even web development with accessibility,

we do that, so come talk to me.

Today I'm excited to introduce to you Mirabai Knight,

who works on Plover,
which is an open-source stenography tool.

Without further ado,
I'll let you introduce yourself, Mirabai.

MIRABAI: Hello.

Hi. My name is Mirabai Knight,
and I'm a stenographer.

I won't keep doing that,
because we have Stan to caption me,

but I just want to talk a little bit about Plover,
my open source project,

and the accessibility implications of it,

and then I'm going to hand it over
to Plover's lead developer,

Hesky Fisher, and he'll talk a little bit about

developing open source projects
that have accessibility implications

and managing the community
and stuff along those lines.

So how many people here
have actually seen a live captioner in action?

Not on television, but in the room?

That's awesome.

That's definitely what I like to see in a room
full of accessibility people.

That's, like, probably 90% of the room.

Glad to hear it.

Because we're a fairly obscure
profession, even now.

Steno machines have been around
since about 1912,

but we were only hooked up to computers
as of the late 1980s, so as a profession,

live captioning is very young, and most people,
if they have heard of it,

only think of it for television and not for live applications.

But myself, I work in universities, primarily,
for Deaf and hard of hearing college students.

I also work with professionals
for business meetings and conferences.

And it was around six years ago
that I graduated from steno school.

I'd gotten started as sort of an apprentice captioner,

and I was very frustrated
with my proprietary steno software,

which cost $4000, had really obnoxious DRM

that required me to jump through
all sorts of hoops even to use the software,

and really limited my ability to use it
the way I wanted to.

And it didn't have a number of key features
that I really needed for my captioning work,

because all commercial stenography software
is for court reporters, which...

I've never done any court reporting.

So my brother had sort of infected me with the
open source bug when I was around ten years old.

He's a big open source evangelist.

And my frustration with the software,

combined with that sort of thought
in the back of my head

that getting involved with open source
was a good thing to do,

made me think that this might be the way to go.

So originally I thought that I would actually
have to learn to program and develop it myself,

because I didn't think anyone could possibly
want to do it for me.

But by a ridiculous stroke of luck,

I put a posting in the elevator
of my coworking space,

asking for a Python tutor,

and the guy who answered it
and started off tutoring me in Python --

it turned out that he had a PhD
from the MIT Media Lab,

and was both a hardware and a software guy,
and after a few weeks it was clear

that I did not have a gift for programming,
and starting from scratch it would be forever

before I was able to develop
the software that I actually wanted.

But he got so excited about it,

he decided he was just going to take over
the development from me and do it on his own.

I paid him as much as I could,
but he worked at a steep discount.

So he developed Plover for about a year.

Then he got another job and had to give it up.

When Hesky, my savior,

contacted me out of the blue,
because his girlfriend was in steno school.

Actually, the same steno school
that I graduated from.

And he wanted to do his part to make steno
cheap and accessible.

So he's been developing it ever since.

He's amazing,
and he'll tell you all about that story later.

But basically...

I can go over the nuts and bolts
of steno if you want,

maybe in the questions,
if you're curious about the details,

but because I don't have that much time,

I think I want to focus more on the potential
of steno in various accessibility areas.

So first off, I think,
It's pretty obvious: Captioning.

This guy right here, Stan Sakai, my captioner,
actually started out on Plover.

He originally just wanted to use steno to take notes
when he was in college,

but he wound up getting so excited about it,

he taught himself, you know,
and practiced ten hours a night

for about a year, and finally realized
that he had gotten up to

about 230 words per minute,

which is the speed you really
need to be a entry-level captioner,

and I think dropped out of college
and launched his career as a captioner.

I think he's pretty happy about it.

He didn't do that with Plover the whole way.

He actually switched to proprietary software,

because Plover wasn't in
the proper shape at that point,

But I still count him as one of our success stories.

So captioning for Deaf and hard of hearing
people is incredibly important.

It's very useful for all sorts of people,

but primarily people with hearing loss
who don't know sign language

or might not even
acknowledge their hearing loss,

which is the vast majority of people
who have hearing loss

that interferes with their life moderately
to significantly in some situations,

but not at all in others.

These are people -- often they've begun to lose
their hearing in middle age,

and that carries through to, you know,
into their 60s and 70s.

They don't acknowledge their hearing loss,
they don't necessarily recognize it,

and they have no idea what they can do to compensate for it.

Hearing aids can only do so much.

Many of them are not candidates
for cochlear implants,

and they often don't know that captioning exists.

But along the way,
as this accommodation has sort of picked up speed,

more and more captioning is offered
as a matter of course,

not necessarily specifically
requested by Deaf advocates

who know their rights and are able to ask for it,
but it's just become an included accommodation,

and so this sort of invisible pool of people
who don't know that they have rights under

the Americans with Disabilities Act,
who may be fine one-on-one in a small room,

but who are totally at sea in a large auditorium,
where they can't read anyone's lips --

they're finally beginning to realize
that there's an accommodation

that works for them.

Also there are people who use sign interpreters
in some situations,

who prefer captioning in other situations.

You know, they might want to have sign interpretation for conversational, or mobile, or very interactive sessions,

but for things like lectures, where there's very
specific terminology

that might not have specific analogs in sign,
captioning might be better.

Captioning is also really useful for people
with attention deficit disorder,

it's useful for some people with dyslexia,
which might seem counterintuitive,

but having the bimodal input of getting something
both from your ears and into your eyes at the same time

can often help people to comprehend information
and process information more thoroughly,

even if they have a reading disability.

It's also extremely useful for people who are
not necessarily fluent in English,

or can read it better than they can understand it aurally,

which is true of a lot of people who are just learning English.

So captioning as Universal Design, I think,
is really important.

I probably don't have to make the case too hard for you guys,

but I just thought I'd lay out all of the ways
that captioning benefits a lot of people,

including that often-neglected pool of people
who don't self-identify as having a disability,

and don't know their rights under the ADA,
which is a very large group of people

who have been almost totally neglected
by traditional accessibility solutions.

So that's one option,
one sort of way that stenography is useful in accessibility.

Another way is for people with speech disabilities
who want to communicate,

who might use augmentative communication devices,

but if any of you guys have seen those in action,
you'll know that even the best of them are very slow,

and, to a certain degree, somewhat stilted.

If people are just using qwerty to type,
they can do maybe 100, 120 words a minute.

If people are using systems such as Minspeak,

they can sort of cluster ideas
and get the sentences out somewhat faster,

but even so, they're nowhere near
a conversational level of speech.

But with steno,
you can basically write as fast as you can talk.

And if you just hook this into a text-to-speech engine,
and you make it portable,

which is still something I'm working on,
you can make an AAC device that allows people to speak

at a conversational speed,
which is unprecedented and somewhat revolutionary.

So I think that's a really important thing
that we can look forward to in the future.

There aren't yet any really good mobile
or portable steno input devices,

but I think there's a lot of potential for that.

I'm also working on an application
that hooks Plover into Glass.

I've got a pair of Glass,
and I've got someone developing an app for it,

so I think that having that sort of feedback will also be useful.

Certainly make it more mobile and portable.

The third area, and I think this one might
be particularly of interest to you guys,

is addressing the terrible underemployment of blind
and low-vision people,

in this country and around the world.

There are incredibly well-educated,
brilliant, fantastic minds out there

that are going to waste,
because no one will employ them,

and one thing specifically that makes
stenography a really good fit for people with vision loss

is that text processing speed,
or rather speech processing speed, I think,

is the fundamental bottleneck of steno.

If you look at Stan,
or if you look at me when I'm writing,

our fingers are not moving particularly quickly.

People might think that it's a matter of dexterity,
but it's really all what happens in the brain.

To be able to comprehend English speech
very quickly and to encode it into steno,

and then send the code to your fingers,
of those three steps,

by far the hardest is comprehending English
without slowing down and seizing up

when people are speaking to you at 240,
260, 280 words a minute.

Those speeds are very fast for your typical English speaker.

They're quite slow for your typical screen reader user.

I know people who use screen readers
who listen to them at 500, 600 words per minute.

So for people who have already done the work
training their brains to process speech at that level --

I don't have any scientific evidence for this,
but I think there's a very good chance

that they've already done a lot of the really hard work,
and if they want to try to learn stenography,

I think they will have a considerable leg
up over most people, who, honestly,

find themselves very hard pressed to achieve
the speeds of 230 words per minute

that are required to be captioners,
court reporters, and CART providers like me.

There's an 85% dropout rate in steno schools nationwide,
which is pretty disgraceful,

but I think a lot of that is because people do not have
the sufficient speech processing speed going into it,

and they're not able to develop it while they're in school.

So those are my three ideas
for how stenography can impact accessibility.

And now, with Plover, which is free,

and works with hardware that's $45,
as opposed to this little number, which is about $4000,

I feel like we might be poised on the edge
of a sort of Steno Renaissance.

I'm really hoping to get that going.

So I'm going to turn it over to Hesky,
and he'll tell you all about how this goes.

HESKY: Static.

Is this working?

Excellent.

Hi, everyone.

I'm Hesky, and I'm the lead developer on Plover right now.

As Mirabai said,
my girlfriend enrolled in stenography school,

and I wanted to learn a little bit about it.

It's very hard to find information about stenography out there,

and it turns out that Mirabai's blog is, I think,
the only well-written description of it on the Internet.

I found it, and then I saw that she was
working on this project,

and I just wanted to make it useful for us,
so I started adding features that I needed,

and basically I just started working on it for fun.

I never thought of myself as an accessibility programmer,
despite the sort of obvious connection,

until I was asked to speak here.

And I started to think about what
I generally think of as accessibility programming,

and how that relates to what I do,
and I saw some parallels beyond Plover's use case.

(clearing throat)

Excuse me.

I think that accessibility programming,
like coding for Plover,

often involves an intention,

without necessarily having the skill first,
of doing something incredibly complicated.

So if you want to make some application usable,
write that app or operating system or whatever,

then you suddenly have to become an expert into it,
beyond what a normal developer would have to know,

to somehow dig into its guts,
and make it give you its text,

or change its colors or anything like that.

And that's what it's been like,
developing Plover.

From the very beginning, writing normal code
to do the logic that Plover needs to do is fairly easy.

But then, suddenly, I had to convince the operating system
to do things that it desperately did not want to do.

As you can see, it involves things
like being on top of other applications,

or, you know,
coming up, going down.

And then, of course, the community wanted
it for every operating system out there.

So that became a journey of suddenly trying
to become that type of expert

on every operating system that I could get my hands on,
and similar things like that.

For example, Josh was the original programmer on Plover
that Mirabai mentioned.

He is quite amazing, and he is working on building
an open source stenography machine.

The machine that Mirabai uses here is $4000,
and that's not unusual.

And Josh is trying to target a much, much lower pricepoint.

I don't know exactly what that's going to be yet.

And I'm helping out.

MIRABAI: $300.

HESKY: $300.

That's very good.

So that's orders of mag...
That's very good.

So I started from scratch.

Again, I had the intention -- I'd like to make
a machine that's a stenography machine.

But I don't know any of the required techniques that I need.

So once again, you know, one minor example is:

Usually the machine speaks via USB.

I had never done any USB.

I had always thought it would be a good idea to learn USB,
but like many people, I had an idea

that I wanted to learn hardware engineering,
but I never had a project I wanted to do.

Well, the problem is:
Once you get to the project to do,

then you have an intention now, but you haven't built that skill,
and it's kind of a Catch-22.

So if I can encourage anybody to start with projects earlier
and build up the skills that become necessary

as soon as you know what you actually want to do.

So the other aspect of working on Plover that's
interesting and similar to usability is --

for many people who do accessibility programming --
I'm not the user.

So it's very hard to...

I'm not a stenographer, and it's quite difficult
to guess what a stenographer actually wants,

especially when I'm making up a feature.

Even when I'm asked explicitly
for a feature, I'm interpreting it, you know,

based on my understanding of it.

And I think that probably has a lot in common
if you're doing something for a user

that's hearing-disabled or vision-disabled.

You can only put yourself in their shoes so well.

And so the most valuable tool that Plover has
is its community.

To constantly throw things out there and encourage feedback.

There's no way I could have made any progress without
the Plover community constantly giving feedback.

Some of it not so polite.

But that's still very worthwhile.

And I think that has a lot of parallels, here, too.

So, speaking of the community,
I did not realize that I would become a babysitter,

taking on this programming role.

As soon as I had a official position,
where I was the main programmer,

suddenly it kind of became my responsibility to make sure
that the community didn't self-destruct, at times.

Every mailing list that's able to be
joined openly will attract...

Different types of destructive elements.

People who post about their pet peeve
on something unrelated.

But less destructive are people who are passionate
about the project, but want it to go in their direction.

And it's really hard to deal
with that kind of thing.

Because it goes in two directions.

I want to take their feedback, and it's extremely
valid in most cases, but then, very often,

it immediately starts to conflict with, say,
my vision of where I think the project should go.

But then I have to ask myself fairly:
Is my vision the right one?

Right?

These are responses from the users.

And in Plover's case, there's actually
an interesting split between the users.

There are the people that I think of as stenographers.

People who are going to stenography school,
or tried stenography school and are now learning

on their own or out of books, but sort of classic stenography,

and they agree to be bound by the restrictions and rules
that all stenographers work by.

And there's the blue sky users.

People who show up to stenography and say,
"That's great.

Now, how can we make it a hundred times better?"

Let's add 30 more buttons,
and let's map the keyboard to everything."

And, again, I have to try to
balance this notion with...

Well, that's not what this app is for.

But maybe it is,
because these make up a certain number of users,

and maybe I'm the crazy one, right?

They've got the million dollar idea,
and I'm just saying that's stupid.

Let's not do it.

And so it's a complicated balance.

A balancing act.

To try to figure out which is the right way to go.

If I had a simple answer, this talk would be shorter,

But I'd say I just have to wait.

And it's not clear that I always make the right decision,

but what I try very hard to do
is to not make that irreversible,

in the sense that I just shoot it down.

I usually just say, "That sounds great, but I don't have the time.

But maybe if you would like to contribute that,
that would be fantastic."

And that's the nice thing about open-source projects,
is it does attract people who are passionate

and capable of contributing.

And so we do get contributions, people who write code
for us, and some of our best features come that way.

And when that started happening,
I felt like we had truly achieved a vibrant

and self-supporting community, and I think
that should be the goal for every open source program.

If I'm the only programmer, then that's a strong
single source of failure for our entire project.

So those are all the points that I wanted to touch.

It's time for Q&A.

You can ask me, or Mirabai, or both.

CAMERON: If you have any questions,
please raise your hand, and I'll give you the mic.

HESKY: Testing.

AUDIENCE MEMBER: I was wondering...

You guys both touched on Plover being
free and also open source,

and the kind of community around that.

Do you think that the freeness,
in terms of monetary cost,

was a big decision about why people use Plover?

Like, to get people into it?

MIRABAI: Huge. Huge.

Right now, your options -- discounting Plover -- your options
for stenography software are, I guess, threefold.

There's a free app -- free as in beer, not as in speech --

app on the iPad that's basically useless.

It pretends to emulate a steno machine, but without haptic feedback it's almost impossible to actually use it.

So there's that.

But that actually came around after Plover got started.

There's student software, which is around $500.

You have to have proof of enrollment in a steno school,
and it's missing a lot of key features.

Like, it doesn't allow you to save files.

It basically just restricts you so massively
that you can barely do anything with it.

And then there's the $4000 court reporting software,
which is fully featured for court reporters,

but obviously not accessible to most people.

So for hobbyists, amateurs, people who want to use steno to write novels, people who want to use it to code software,

Plover had to be free.

I mean, that was just the only way.

There's enough of a barrier to entry just in the learning curve of learning how to do stenography

that making the software completely free,
and making the hardware about $50,

which is what it is right now,
the cheapest option to interface with Plover,

that had to be in place before we could start
making the big push to get people to learn Plover.

Which, by the way, speaking of Learn Plover,

the author of our textbook,
Zachary Brown, is back there.

He's collaborating with me to write a free online textbook
to teach people stenography,

which we hope also will someday
get turned into an interactive video game,

which will hopefully make it even more accessible.

HESKY: I just wanted to add a little bit to that.

It's very clear from our users that money is an object,

and that expresses itself technically by very difficult questions
on how to get Plover running on older and older machines,

and, in fact, some people on the group have gone so far as defining "accessibility" as "working on their computers",

and I'm only half-kidding.

MIRABAI: No, it's true.

HESKY: And it could be almost heartbreaking
when they tell you that, like...

"So I've finally got all
the equipment together!"

And then I have to tell them that they have to buy some cable,
and it just breaks the bank,

because they have to connect things.

And it's just...

AUDIENCE MEMBER: I have a follow-up on that.

So if somebody wants to get Plover or contribute to Plover,
what is the website?

MIRABAI: The website is ploversteno.org.

HESKY: All right, we actually suffer from too many websites.

MIRABAI: Well, Sveta, right here, amazing usability expert,
user experience designer, and web designer

is actually helping us consolidate
our websites into one general hub,

so that people can just go to just one page
and find what they're looking for there,

instead of the terrible sort of fractured sprawl
we have right now.

Thank you, Sveta!

HESKY: Right now there is the blog, the code,
the download page, the wiki, the forum,

the mailing list, the textbook.

AUDIENCE MEMBER: Hi.

You mentioned the 85% dropout rate from steno school.

Is there any conversation either in Plover
or within the larger community about how to work on that?

MIRABAI: I could speak about this for hours.

I think there are a lot of economic components.

(mic booming)

Sorry. I think, to a large degree, that dropout rate is because:
A) steno's very difficult to do at a high level.

Meaning professional speeds.

I think it's much less difficult to do at sort of
general conversational speed,

or for text composition and text entry.

I think that's a pretty reasonable goalpost for most people.

But actual professional steno speeds,

230 to 240 words per minute,
very difficult for many people.

The other thing to consider is that a lot of people
are going into steno school who don't necessarily

have the baseline language skills in English.

It's seen as sort of a clerical field,
and a good middle class career.

But most of the schools are for-profit,
and there aren't really any admission requirements.

So they'll basically recruit on the subways;
they'll tell everyone to just come in.

They'll sell them the $1,000 student steno machine,
let them go through a theory class,

after a certain point, these students don't
get to their goal speeds,

their financial aid runs out, they're not passing tests,
they drop out,

they sell their machine back to the school,
the school sells it back again to another set of students.

So there isn't really any incentive for these
for-profit schools to improve the graduation rates.

And, honestly, I mean, even the ones that acting
in good faith and trying to graduate

as many students of possible --

the pool of people entering steno school is not necessarily the people who have the baseline skills necessary to succeed.

So my solution is just to make steno not
something you have to go to school for,

and that you don't have to buy a $1,000 machine for
and pay tuition for.

Make steno something that,
if you want to give it a try and see if you have a knack for it,

and play around with it, you know, play a video game
a couple hours a night,

and see if your speed takes off, because you're just
one of those inherently inborn natural stenographers,

like Stan --

that's something you can do without risking
a ton of time and money and effort and risk.

HESKY: Just quickly wanted to add that,

because of the structure of how you learn steno,
you enroll in school,

you have the large initial outlay for the equipment,
software, and tuition, and then,

when you realize that you're not going to make it in the profession, there's a huge frustration,

and usually a complete dumping of the thing.

You sell your old machine back, your old textbooks,
and you move on.

And it's unfortunate, because of that structure,
dropping out is considered a failure.

Because the person who drops out
could have reached 130, 180, or something like that,

and that's a useful skill in and of itself,

but that's not what they signed up for in terms of schooling.

We're hoping that, with people having open source
and free methods, that that's not a failure.

That's just reaching a very, very good typing speed.

There's one.

He already has the mic.

Somebody already has the microphone.

AUDIENCE MEMBER: I'm hearing in this talk sort of --

it sounds like there's two separate communities.

There's the steno schools and closed source software
and closed source hardware,

and all of the stuff that goes around that point of view,
I guess, and then there's Plover

and the community that's built around that.

And I'm wondering...

What are their relative sizes?

Are they really separate communities?

And is Plover changing what is steno?

Or is steno still what it always has been?

And what's the social relationship between the two communities?

Are they getting along?

Is there animosity?

HESKY: Can I take this one?

MIRABAI: Yeah, go for it.

HESKY: Mirabai will probably
have more to add to this,

but we've done at least one demo session at the local school,
for students to see, and Plover's still adding features

that some of the other software comes with.

I don't know how it's going to go in the future,
but the relationship between Plover

and the software companies and hardware companies
is between being ignored and being hostile.

AUDIENCE MEMBER: But there have been
some hardware donations, right?

HESKY: That's true.

So what has happened is that, like every world,

there are underdogs, and the underdogs
have been far friendlier than the entrenched players.

Right, I should have acknowledged that some
of the companies have been friendly.

But, you know, the hardware, for example,

comes with protocols that have to be decoded
in order to work with them,

and several companies have been very aggressive
about not letting us get to the protocol.

And it's a very small world.

I think at this point I've spoken
to all the CEOs randomly, by email.

And I've heard that one person who volunteered
to work on one aspect of Plover

actually had a salesperson, like, target him.

It was a very bizarre story.

But (inaudible)...

MIRABAI: I can speak just in terms of numbers.

I think there are around 30,000 active
professional stenographers in the country.

There's around 250 users on the Plover mailing list, which --

they're our most active and most engaged users.

I don't know exactly how many downloads there are.

So right now, the proportion of Plover users
to professional stenographers is very, very skewed.

But I think and hope that
that will change pretty drastically,

and honestly, from companies,
software and hardware vendors,

there's a certain wariness directed towards us,
but from professional stenographers,

by and large, there's been a lot of encouragement,
because a lot of people are worried that this technology

is vanishing, that the profession is dying.

You know, the average age of a professional stenographer,
I heard somewhere, was something like 55 years old,

and as more and more people reach retirement age,

and fewer and fewer younger people are graduating,
it really leaves the profession vulnerable

to being co-opted by less accurate, less useful, non-verbatim
technologies to just fill the vacuum that's left behind

if there aren't any stenographers
to keep the place open.

So most professional stenographers that I've talked to are very excited about Plover and are very encouraging.

So we'll see what happens.

CAMERON: We're going to have one more question,
and then we'll take a quick break.

AUDIENCE MEMBER: I have a comment as a Deaf person.

I think that it's great for...

I think that Plover is great for people who are trying
to help people become stenographers

and encourage that profession, but as a deaf person, myself,

I think that it's frustrating sometimes
to find a good quality captionist.

And, just to let you know, captioning is not the same
as what they do in court.

Court reporting.

Court reporting is entirely different than captioning.

So, in my experience, court reporters --

using both court reporters and captionists --

they're completely different.

The training, I think, needs to be different.

Those people who do captioning should be trained a different way for Deaf and hard of hearing people.

I also know that Plover, when you use it,
you still have to have some training,

to have some professional training.

You shouldn't just have a person who, you know,
plays around with the program

and then becomes a professional captionist.

It's the same thing with interpreters.

Just because someone knows sign language doesn't
mean that they would be a good-quality interpreter.

So I think it's important to note that to have
a good quality captionist that can work

with Deaf and hard of hearing people,
they need to be professionally trained.

MIRABAI: Can I just briefly respond,
really quickly?

Yeah, absolutely agree.

Of those 30,000 professional stenographers,

only about 300 in the country are certified captioners,

which I think is just staggering,
and it's very true that the skills are extremely different.

So Plover is actually the only software
that's specifically designed for live captioning.

It doesn't work with broadcast captioning.

It doesn't work with court reporting.

Unlike every other proprietary software out there,

which is specifically directed for court reporters.

So, as a live captioner myself,

I definitely want to sort of shepherd the potential captioning prodigies from trying steno out as an amateur,

and learning through Plover,
to get up there,

and then sort of giving them that final push
of captioning training, including ethics,

including, you know, Deaf Culture,

including all the sorts of things
that professional captioners need to know,

that you can't get just as an an amateur,
playing around with the software.

So I feel very passionately about that,
and I feel like it's really vital to preserving my own career,

to help bring up the next generation of captioners via Plover.

CAMERON: Great.

Thank you so much, Hesky and Mirabai.

Please, a round of applause.

[ Applause ]

CAMERON: Coming up, we're going to have
John Schimmel and DIYAbility crew

talking about doing your own DIY hardware accessibility.

So let's take five minutes.

Introduce yourselves, please,
chat amongst yourselves,

and then we'll come back at 8:00
and pick up with John Schimmel.

Thanks!