How Twitter needs to change

Chris Anderson:
What worries you right now?

You've been very open
about lots of issues on Twitter.

What would be your top worry

about where things are right now?

Jack Dorsey: Right now,
the health of the conversation.

So our purpose is to serve
the public conversation,

and we have seen
a number of attacks on it.

We have seen abuse, we have harassment,

we've seen manipulation,

automation, human coordination,

misinformation.

So these are all dynamics
that we were not expecting

13 years ago when we
were starting the company,

but we do now see them at scale,

and what worries me most

is just our ability to address it
in a systemic way that is scalable,

that has a rigorous understanding
of how we're taking action,

a transparent understanding
of how we're taking action,

and a rigorous appeals process
for when we're wrong,

because we will be wrong.

Whitney Pennington Rodgers:
And I'm really glad to hear

that that's something that concerns you,

because I think there's been
a lot written about people

who feel they've been abused
and harassed on Twitter,

and I think no one more so
than women and women of color

and black women

and there's been data that's come out,

Amnesty International put out
a report a few months ago

where they showed that a subset
of active black female Twitter users

were receiving on average
one in 10 of their tweets

were some form of harassment.

And so when you think about health
for the community on Twitter,

I'm interested to hear,
health for everyone,

but specifically how are you looking
to make Twitter a safe space

for that subset, for women,
for women of color and black women.

JD: Yeah.

So it's a pretty terrible situation

when you're coming to a service

that ideally you want to learn
something about the world

and you spend the majority of your time

reporting abuse, receiving abuse,

receiving harassment.

So what we're looking most deeply at
is just the incentives

that the platform naturally provides
and the service provides.

Right now the dynamic of the system
makes it super-easy to harass

and to abuse others through the service,

and unfortunately the majority
of our system in the past

worked entirely based on people
reporting harassment and abuse.

So about midway last year we decided
that we were going to apply

a lot more machine learning,
a lot more deep learning to the problem

and try to be a lot more proactive
around where abuse is happening

so that we can take the burden
off the victim completely.

And we've made some progress recently.

About 38 percent of abusive tweets

are now proactively identified
by machine learning algorithms

so that people don't actually
have to report them,

but those that are identified
are still reviewed by humans,

so we do not take down content or accounts
without a human actually reviewing it.

But that was from zero percent
just a year ago.

So that meant, at that zero percent,

every single person who received abuse
had to actually report it,

which was a lot of work for them,
a lot of work for us,

and just ultimately unfair.

The other thing that we're doing
is making sure that we, as a company,

have representation of all the
communities that we're trying to serve.

We can't build a business
that is successful

unless we have a diversity
of perspective inside of our walls

that actually feel these issues
every single day,

and that's not just with the team
that's doing the work,

it's also within our leadership as well.

So we need to continue to build empathy
for what people are experiencing

and give them better tools to act on it

and also give our customers
a much better and easier approach

to handle some of the things
that they're seeing.

So a lot of what we're doing
is around technology,

but we're also looking at
the incentives on the service.

What does Twitter incentivize you to do
when you first open it up?

And in the past, it's incentivized
a lot of mob behavior,

it's incentivized a lot
of group harassment,

and we have to look a lot deeper
at some of the fundamentals

of what the service is doing
to make the bigger shifts.

We can make a bunch of small shifts
around technology, as I just described,

but ultimately we have to look deeply
at the dynamics in the network itself,

and that's what we're doing right now.

CA: And what's your sense?

What is the kind of thing
that you might be able to change

that would actually
fundamentally shift behavior?

JD: Well, one of the things,
we started the service

with this concept of following an account,

as an example,

and I don't believe that's why
people actually come to Twitter.

I believe Twitter is best
as an interest-based network.

People come with a particular interest.

They have to do a ton of work to find
and follow the related accounts

around those interests.

What we could do instead
is allow you to follow an interest,

follow a hashtag, follow a trend,

follow a community,

which gives us the opportunity
to show all of the accounts,

all the topics, all the moments,
all the hashtags

that are associated with that
particular topic and interest,

which really opens up
the perspective that you see.

But that is a huge fundamental shift

to bias the entire network
away from just an account bias

towards a topics and interest bias.

CA: Because isn't it the case

that one reason why you have
so much content on there

is a result of putting millions
of people around the world

in this kind of gladiatorial
contest with each other

for followers, for attention?

Like, from the point of view
of people who just read Twitter,

that's not an issue, but for
the people who actually create it,

everyone's out there saying,

"You know, I wish I had
a few more likes, followers, retweets."

And so they're constantly experiment,

trying to find the path to do that,

and what we've all discovered
is that the number one path to do that

is to be some form of provocative,

obnoxious, eloquently obnoxious,

like eloquent insults
are a dream on Twitter,

where you rapidly pile up,

and it becomes this self-fueling
process of driving outrage.

How do you defuse that?

JD: Yeah, I mean, I think you're spot on,

but that goes back to the incentives.

Like, one of the choices
we made in the early days

was we had this number

that showed how many people follow you.

We decided that number
should be big and bold

and anything that's on the page
that's big and bold has importance,

and those are the things
that you want to drive.

Was that the right decision at the time?

Probably not.

If I had to start the service again,

I would not emphasize
the follower account as much.

I would not emphasize
the like count as much.

I don't think I would even
create like in the first place,

because it doesn't actually push

what we believe now
to be the most important thing,

which is health contribution
back to the network,

and conversation to the network,

participation within conversation,

learning something from the conversation.

Those are not things
that we thought of 13 years ago

and we believe are extremely
important right now.

So we have to look at
how we display the follower account,

how we display retweet count,

how we display likes,

and just ask the deep question:

is this really the number
that we want people to drive up?

Is this the thing that,
when you open Twitter,

you see, "That's the thing
I need to increase?"

And I don't believe
that's the case right now.

(Applause)

WPR: I think look at some of the tweets

that are coming in
from the audience as well.

CA: Let's see what you guys are asking.

I mean, this is generally one of
the amazing things about Twitter

is how you can use it for crowd wisdom.

You know,

that more knowledge, more questions,
more points of view than you can imagine,

and sometimes many of them really healthy.

WPR: And I think one that I saw
that passed already quick down here,

"What's Twitter's plan to combat
foreign meddling in the 2020 US election?"

And I think that's something
that's an issue we're seeing

on the internet in general,

that we have a lot of malicious
automated activity happening.

And on Twitter, for example,

we have in fact some work

that's come from our friends
at Zignal Labs, rather,

and maybe we can even see that
to give us an example

of what exactly I'm talking about,

where you have these bots, if you will,

or coordinated automated
malicious account activity,

that is being used to influence
things like elections.

And in this example we have
from Zignal which they've shared with us

using the data that
they have from Twitter,

you actually see that in this case,

white represents the humans,

human accounts,
each dot is an account.

The pinker it is,

the more automated the activity is,

and you can see how you have
a few humans interacting with bots,

and in this case it's related
to the election in Israel,

and spreading misinformation
about Benny Gantz,

and as we know in the end,
that was an election

that Netanyahu won by a slim margin,

and that may have been
in some case influenced by this,

and when you think about
that happening on Twitter,

what are the things that you
are doing specifically to ensure

that you don't have misinformation
like this spreading in this way,

influencing people in ways
that could affect democracy?

JD: Just to back up a bit,

we asked ourselves a question:

can we actually measure
the health of a conversation,

and what does that mean?

And in the same way
that you have indicators

and we have indicators as humans
in terms of are we healthy or not

such as temperature,
the flushness of your face,

we believe that we could find
the indicators of conversational health,

and we worked with a lab
called Cortico at MIT

to propose four starter indicators

that we believe that we could ultimately
measure on the system.

And the first one is
what we're calling shared attention.

It's a measure of how much
of the conversation is attentive

on the same topic versus disparate.

The second one is called shared reality,

and this is what percentage
of the conversation

shares the same facts,

not whether those facts
are truthful or not

but are we sharing
the same facts as we converse.

The third is receptivity.

How much of the conversation
is receptive or civil

or the inverse, toxic?

And then the fourth
is variety of perspective.

So are we seeing filter bubbles
or echo chambers

or are we actually getting
a variety of opinions

within the conversation?

And implicit in all four of these
is the understand that as they increase,

the conversation gets
healthier and healthier.

So our first step is to see
if we can measure these online,

which we believe we can.

We have the most momentum
around receptivity.

We have a toxicity score,
a toxicity model,

on our system that can actually measure

whether you are likely to walk away
from a conversation

that you're having on Twitter
because you feel it's toxic

with some pretty high degree.

We're working to measure the rest,

and the next step is,

as we build up solutions,

to watch how these measurements

trend over time

and continue to experiment.

And our goal is to make sure
that these are balanced,

because if you increase one,
you might decrease another.

If you increase variety of perspective,

you might actually decrease
shared reality.

CA: Just picking up on some of
the questions flooding in here.

A lot of people are puzzled why,

how hard is it to get rid
of Nazis from Twitter?

JD: So we have policies
around violent extremist groups,

and the majority of our work
and our terms of service

works on conduct, not content,

so we're actually looking for conduct,

so conduct being

using the service to repeatedly
or episodically harass someone,

using hateful imagery

that might be associated with the KKK

or the American Nazi Party.

Those are all things that we did,
that we act on immediately.

We're in a situation right now
where that term is used fairly loosely,

and we just cannot take
any one mention of that word

accusing someone else

as a factual indication that they
should be removed from the platform.

So a lot of our models are based around,

number one, is this account associated
with a violent extremist group,

and if so we can take action,

and we have done so on the KKK
and the American Nazi Party and others,

and number two, are they using imagery
or conduct that would associated them

as such as well.

CA: How many people do you have
working on content moderation

to look at this?

JD: It varies.

We want to be flexible on this,

because we want to make sure
that we're, number one,

building algorithms instead of just
hiring massive amounts of people,

because we need to make sure
that this is scalable,

and there are no amount of people
that can actually scale this.

So this is why we've done so much work

around proactive detection of abuse

that humans can then review.

So we want to have a situation

where algorithms are constantly
scouring every single tweet

and bringing the most
interesting ones to the top

so that humans can bring their judgment
to whether we should take action or not

based on our terms of service.

WPR: But there's not an amount
of people that are scalable,

but how many people do you
currently have monitoring these accounts

and how do you figure out what's enough?

JD: They're completely flexible.

Sometimes we associate folks with spam.

Sometimes we associate folks
with abuse and harassment.

We're going to make sure that
we have flexibility in our people

so that we can direct them
at what is most needed.

Sometimes, the elections,

so we've had a string of elections

in Mexico,

one coming up in India,

obviously the election last year,
the midterm election,

so we just want to be flexible
with our resources.

So when people, just as an example,

if you go to our current terms of service

and you bring the page up

and you're wondering about

abuse and harassment
that you just received

and whether it was against
our terms of service to report it,

the first thing you see
when you open that page

is around intellectual
property protection.

You scroll down and you get to
abuse, harassment,

and everything else
that you might be experiencing.

So I don't know how that happened
over the company's history,

but we put that above
the thing that people want

the most information on
and to actually act on.

And just our ordering, like, shows
the world what we believed was important.

So we're changing all that.

We're ordering it the right way,
but we're also simplifying the rules

so that they are human readable

so that people can actually
understand themselves

when something is against our terms
and when something is not,

and then we're making, again,

like, our big focus is on removing
the burden of work from the victims.

So that means push more towards technology

rather than humans doing the work.

That means the humans receiving the abuse

and also the humans
having to review that work.

So we want to make sure

that we're not just encouraging more work

around something
that's super, super-negative

and we want to have a good balance
between the technology

and where humans can actually be creative,

which is the judgment of the rules

and not just all the mechanical stuff
of finding them and reporting them.

So that's how we think about it.

CA: I'm curious to dig in
more about what you said.

I mean, I loved that you said
you are looking for ways

to re-tweak the fundamental
design of the system

to discourage some of
the reactive behavior,

and perhaps,

to use Tristan Harris-type language,

engage people's more reflective thinking.

How far advanced is that?

What would alternatives
to that "like" button be?

JD: Well, first and foremost,

my personal goal with the service

is that I believe fundamentally
that public conversation is critical.

There are existential problems
facing the world

that are facing the entire world,
not any one particular nation-state,

that global public conversation benefits,

and that is one of the unique
dynamics of Twitter

is it is completely open,

it is completely public,

it is completely fluid,

and anyone can see any other conversation
and participate in it.

So there are conversation
like climate change.

There are conversations
like the displacement in the work

through artificial intelligence.

There are conversations
like economic disparity.

No matter what any one nation-state does,

they will not be able to solve
the problem alone.

It takes coordination around the world,

and that's where I think
Twitter can play a part.

The second thing is that

Twitter right now, when you go to it,

you don't necessarily walk away
feeling like you learned something.

Some people do.

Some people have
a very, very rich network,

a very rich community
that they learn from every single day.

But it takes a lot of work
and a lot of time to build up to that.

So we want to get people
to those topics and those interests

much, much faster

and make sure that
they're finding something that,

no matter how much time
they spend on Twitter,

and I don't want to maximize
the time on Twitter,

I want to maximize

what they actually take away from
and what they learn from it --

CA: Well, do you, though,

because that's the core question
that a lot of people want to know,

is surely, Jack, you're constrained
to a huge extent by the fact

that you're a public company,
you've got investors pressing on you,

the number one way you make your money

is from advertising.

That depends on user engagement.

Are you willing to sacrifice
user time if need be

to go for a more reflective conversation?

JD: Yeah. More relevance
means less time on the service,

and that's perfectly fine,

because we want to make sure
that, like, you're coming to Twitter

and you see something immediately
that you learn from and that you push.

We can still serve an ad against that.

That doesn't mean you need to spend
any more time to see more.

The second thing that we're looking at --

CA: But just, on that goal,

daily active usage,

if you're measuring that,
that doesn't necessarily mean things

that people value every day.

It may well mean things that people

are drawn to like a moth
to the flame every day.

We are addicted because we see
something that pisses us off,

so we go in and add fuel to the fire,

and the daily active usage goes up,

and there's more ad revenue there,

but we all get angrier with each other.

How do you define -- ?

Daily active usage seems like a really
dangerous term to be optimizing.

(Applause)

JD: Taken alone, it is,

but let me finish the other metric,

which is, we're watching for conversations

and conversation chains.

So we want to incentivize
health contribution back to the network,

and what we believe that is
is actually participating in conversation

that is healthy,

as defined by those four indicators
I articulated earlier.

So you can't just optimize
around one metric.

You have to balance and look constantly

at what is actually going to create
a healthy contribution to the network

and a healthy experience for people.

Ultimately, we want to get to a metric

where people can tell us that,
hey, I learned something from Twitter

and I'm walking away
with something valuable.

That is our goal ultimately over time,

but that's going to take some time.

CA: