WEBVTT
00:00:00.000 --> 00:00:18.680
35C3 preroll music
00:00:18.680 --> 00:00:22.800
Herald Angel: Alright. Then it's my great
pleasure to introduce Toni to you
00:00:22.800 --> 00:00:28.290
She's going to talk about "the Social Credit
System," which is, kind of, feels to me
00:00:28.290 --> 00:00:36.010
like a Black Mirror episode coming to
life. So, slightly nervous and really
00:00:36.010 --> 00:00:41.115
curious what we're going to learn today.
So please give a huge, warm round of
00:00:41.115 --> 00:00:45.606
applause and welcome Toni!
Applause
00:00:51.918 --> 00:00:55.469
Toni: Good morning, everyone! Before I'm
going to be talking I'm going into my talk
00:00:55.469 --> 00:01:00.399
I'm just going to be presenting the
Chinese translation streams for everyone
NOTE Paragraph
00:01:00.399 --> 00:01:02.709
who doesn't speak English.
00:01:02.709 --> 00:01:17.330
speaks chinese
00:01:17.330 --> 00:01:23.254
Applause
00:01:23.254 --> 00:01:26.536
So because today's talk is about China we
figured it would be good to have it
00:01:26.536 --> 00:01:31.780
in Chinese as well. And, I'm going to be
talking today about
00:01:31.780 --> 00:01:35.439
the Social Credit system in China, where
"the" Social Credit system that you
NOTE Paragraph
00:01:35.439 --> 00:01:38.970
always hear about in Western media
actually doesn't really exist
00:01:38.970 --> 00:01:43.471
and most of my talk will actually be
talking about what all we don't know.
00:01:43.471 --> 00:01:47.030
Which could fill an entire hour or even
00:01:47.030 --> 00:01:51.470
more. But I'm just going to be focusing on
some of the most interesting things for
00:01:51.470 --> 00:01:57.560
me. First of all, a little bit about me.
I'm an economist, but I'm not I'm not only
00:01:57.560 --> 00:02:03.050
concerned with money. I'm kind of looking
at economy, at economics as the study of
00:02:03.050 --> 00:02:08.310
incentives, which means that what I'm
really interested in is how humans respond
00:02:08.310 --> 00:02:12.480
to different kind of incentives. I don't
believe that humans are completely
00:02:12.480 --> 00:02:18.250
rational. But I do believe that humans do
try to maximize what they think is their
00:02:18.250 --> 00:02:23.830
best interest. Now, some words about me: I
studied math, economics and political
00:02:23.830 --> 00:02:29.190
science in a couple of different cities
all around the world. I spent overall 19
00:02:29.190 --> 00:02:35.140
months in China. Most recently I was there
in July on a government scholarship, which
00:02:35.140 --> 00:02:39.640
was really, really interesting, because
while there I read all of these Western
00:02:39.640 --> 00:02:43.710
newspaper articles about the Chinese
Social Credit system, and I went to a
00:02:43.710 --> 00:02:48.200
pretty good university and I asked them:
So what do you think about this system?
00:02:48.200 --> 00:02:52.360
And most of them basically looked at me
blankly, and were like: What system, I
00:02:52.360 --> 00:02:56.730
haven't even heard of this! So that was
kind of an interesting experience to me
00:02:56.730 --> 00:03:01.670
because in the West it's like this huge,
all-encompassing system. And in China,
00:03:01.670 --> 00:03:07.140
most people that aren't directly -- that
aren't directly in touch with it actually
00:03:07.140 --> 00:03:11.320
don't know anything about this. I'm
broadly interested in the impact of
00:03:11.320 --> 00:03:17.170
technology on society, life, and the
economy, obviously, and in my free time I
00:03:17.170 --> 00:03:21.400
do a lot of data science and machine
learning with Python and R. So, I thought
00:03:21.400 --> 00:03:27.180
it was quite interesting to look at the
Social Credit system, also from this point
00:03:27.180 --> 00:03:31.730
of view because you always heard that it's
like this big data initiative, and then
00:03:31.730 --> 00:03:35.200
when coming down to it, what you actually
see is that, they don't actually use
00:03:35.200 --> 00:03:40.020
machine learning all that much. They have,
basically, a rule based catalog where, if
00:03:40.020 --> 00:03:44.459
you do this you get 50 points, if you do
this you get 50 points, and then they
00:03:44.459 --> 00:03:48.959
actually have a lot of people that are
reporting on other people's behavior. I'm
00:03:48.959 --> 00:03:53.310
going to be talking about how exactly it
looks, later on but I was very, very
00:03:53.310 --> 00:03:56.870
surprised after reading a lot of the
Western newspaper articles that were
00:03:56.870 --> 00:04:02.300
basically "Oh, this is this big dystopia,
Orwellian, with big data working." And
00:04:02.300 --> 00:04:08.160
then, you read what's actually happening
and they have huge lists of "if you
00:04:08.160 --> 00:04:14.700
jaywalk, you get 10 points detracted from
you," this kind of thing. If you want to
00:04:14.700 --> 00:04:19.000
get in touch with me you can use Twitter
but you can also use different e-mails
00:04:19.000 --> 00:04:23.099
either my professional e-mail or my
personal e-mail address, that you can both
00:04:23.099 --> 00:04:27.870
see there. If you have any thoughts on
that or are interested in this a little
00:04:27.870 --> 00:04:31.759
more I can give you more resources as
well, because obviously today's talk will
00:04:31.759 --> 00:04:38.420
only be scratching on the surface. So,
perceptions of the Social Credit System.
00:04:38.420 --> 00:04:43.310
One of the interesting things that I've
talked about before was how, in the West
00:04:43.310 --> 00:04:48.410
and in China, the perception is completely
different. So in the West, which is from
00:04:48.410 --> 00:04:54.550
financialtimes.com, you see this huge
overwhelming guy, and he basically puts
00:04:54.550 --> 00:04:59.280
every Chinese person under a microscope.
They're all kind of hunched over, and
00:04:59.280 --> 00:05:03.750
everyone has this score attached to them,
and they seem pretty sad and, like, very,
00:05:03.750 --> 00:05:08.700
very Orwellian concept. Whereas, in China,
this is actually from a Chinese state
00:05:08.700 --> 00:05:13.810
media, and what it says is, well, we can
all live in harmony with this new system
00:05:13.810 --> 00:05:19.139
and all trust each other. And
interestingly Chinese people actually
00:05:19.139 --> 00:05:24.310
believe that, to some degree. They believe
that technology will fix all this current
00:05:24.310 --> 00:05:29.900
problems in society, especially because,
in China currently, trust is a rare
00:05:29.900 --> 00:05:39.400
commodity. And this new system will lead
to more efficiency and trust, and a better
00:05:39.400 --> 00:05:43.650
life. And I have a really, really
interesting quote from a Western scholar,
00:05:43.650 --> 00:05:47.800
that really summarizes the Western
perspective: "What China is doing here is
00:05:47.800 --> 00:05:52.949
selectively breeding its population to
select against the trait of critical,
00:05:52.949 --> 00:05:57.490
independent thinking. This may not be the
purpose, indeed I doubt it's the primary
00:05:57.490 --> 00:06:02.330
purpose, but it's nevertheless the effect
of giving only obedient people the social
00:06:02.330 --> 00:06:07.290
ability to have children, not to mention
successful children." This, basically,
00:06:07.290 --> 00:06:11.960
plays with the idea that if you have a low
score, currently, in the cities that are
00:06:11.960 --> 00:06:17.320
already testing this system, what happens
is, your children can't attend good
00:06:17.320 --> 00:06:23.190
schools. What happens is, you cannot take
trains, you cannot take planes. You cannot
00:06:23.190 --> 00:06:31.140
book good hotels. Your life is just very,
very inconvenient. And this is by design.
00:06:31.140 --> 00:06:38.620
This is kind of the plan. The Chinese
government, they say it's a little
00:06:38.620 --> 00:06:42.340
different, the idea is about changing
people's conduct by ensuring they are
00:06:42.340 --> 00:06:48.199
closely associated with it. One of the
main things about this system is, there
00:06:48.199 --> 00:06:53.330
isn't very much new data being generated
for the system. Instead, what's happening
00:06:53.330 --> 00:07:00.010
is, all the existing data that is already
collected about you is, basically,
00:07:00.010 --> 00:07:06.370
combined into one big database for each
and every person by your ID number. So, in
00:07:06.370 --> 00:07:10.190
China, once you're born, you get an ID
number, which is similar to a Social
00:07:10.190 --> 00:07:13.960
Security number in the U.S. We don't
really have a similar concept in Germany,
00:07:13.960 --> 00:07:19.310
and it used to be that your ID number was
only necessary for public -- like for
00:07:19.310 --> 00:07:25.810
government stuff, but now you need your ID
number for getting a bank account, you
00:07:25.810 --> 00:07:29.580
need your ID number for buying a cell
phone, even if it's a prepaid cell phone,
00:07:29.580 --> 00:07:34.640
you still need your ID number. So all your
online activity that happens with your
00:07:34.640 --> 00:07:38.430
cell phone is associated with your ID
number, which means you can't really do
00:07:38.430 --> 00:07:44.210
anything anonymously, because it's all
going back to your ID number. There's a
00:07:44.210 --> 00:07:49.450
couple of predecessors, some of them going
actually back to the 1990s, that are
00:07:49.450 --> 00:07:54.880
supposed to be integrated into the new
system. One of them, or like two of them
00:07:54.880 --> 00:08:00.800
are blacklists. One of them is a court
blacklist. So in China, courts work a
00:08:00.800 --> 00:08:06.560
little bit differently. They tend to like
giving you fines, as they do in other
00:08:06.560 --> 00:08:11.521
countries, but they also like giving you
"apologies to do." So one of the things,
00:08:11.521 --> 00:08:17.470
if you do something, for example you're a
company, your food safety wasn't up to par
00:08:17.470 --> 00:08:21.800
– you have to pay a fine. But in addition
to this fine you also have to write a
00:08:21.800 --> 00:08:27.150
public apology letter in the newspaper,
how you are very sorry that this happened
00:08:27.150 --> 00:08:31.560
and it won't happen again, and it was a
moral failing on your part, and it won't
00:08:31.560 --> 00:08:37.460
happen again. And if you don't do that,
you go on this blacklist. Similarly, if
00:08:37.460 --> 00:08:42.698
you take out a line of credit and don't
pay it back within three months, or like
00:08:42.698 --> 00:08:47.930
don't don't do any payments for three
months, you go on this debtors blacklist.
00:08:47.930 --> 00:08:52.009
If you're on this blacklist, which again
is associated with your shēnfènzhèng, so
00:08:52.009 --> 00:08:57.709
your ID number – what happens is you
cannot take trains you cannot take planes.
00:08:57.709 --> 00:09:02.309
Your life basically becomes very very
inconvenient, your children can't go to
00:09:02.309 --> 00:09:06.749
good public schools, your children can't
go to private schools, your children can't
00:09:06.749 --> 00:09:14.589
go to universities, all of these issues
are suddenly coming up. There is also a
00:09:14.589 --> 00:09:19.829
company database that's called Credit
China which is basically similar to the
00:09:19.829 --> 00:09:24.959
public debtors blacklist but it's
basically a credit system a credit score
00:09:24.959 --> 00:09:28.860
for companies. And then there's the credit
reference center of the People's Bank of
00:09:28.860 --> 00:09:34.569
China which is a credit score. It was
supposed to be like Schufa or like the
00:09:34.569 --> 00:09:42.629
U.S. FICO for individuals. But one of the
big problems in China is that there are a
00:09:42.629 --> 00:09:46.769
lot of people that aren't part of the
formal economy. A lot of people are
00:09:46.769 --> 00:09:52.529
migrant workers. They get their money in
cash. They do not have bank accounts. They
00:09:52.529 --> 00:10:00.410
do not have anything… they do not have rent
or utilities or anything like this because
00:10:00.410 --> 00:10:04.610
they live in the country. So they own
their own home which they built themselves
00:10:04.610 --> 00:10:10.920
so they didn't even finance it and their
home isn't officially theirs because in
00:10:10.920 --> 00:10:16.069
China you can't actually own property.
Instead the government leases it to you.
00:10:16.069 --> 00:10:20.839
So there were a lot of people that were
not covered in this system, and I think
00:10:20.839 --> 00:10:26.739
the last data that I had was that less
than 10 percent of Chinese adult citizens
00:10:26.739 --> 00:10:32.699
were actually in the system and had any
sort of exposure to banks, which is very,
00:10:32.699 --> 00:10:38.089
very little. And that meant that people
couldn't get credit because banks would
00:10:38.089 --> 00:10:42.220
only give credit to people that were in
the system or people where they had
00:10:42.220 --> 00:10:46.649
some sort of handling on whether they
would be paid back. Now, the
00:10:46.649 --> 00:10:51.839
implementation details of the new system
are very very scarce, but the basic idea
00:10:51.839 --> 00:10:56.079
is that Chinese citizens are divided into
trustworthy individuals and what the
00:10:56.079 --> 00:11:02.309
Chinese call "trust breakers". Sometimes
you have five different groups, sometimes
00:11:02.309 --> 00:11:05.459
you have two different groups, but in
general there's sort of this cut-off:
00:11:05.459 --> 00:11:13.619
above this line it's good and beyond this
line it's bad. This is one graphic from
00:11:13.619 --> 00:11:20.409
the Wall Street Journal that just shows
some of the inputs that go into the
00:11:20.409 --> 00:11:25.440
system. And one of the things that we see
is that the inputs are _crazy_ crazy
00:11:25.440 --> 00:11:35.007
varied. So it is: do you pay income taxes?
Do you pay your utility bills on time?
00:11:35.007 --> 00:11:40.201
Do you respect your parents? However they
measure that. Do you have a criminal
00:11:40.201 --> 00:11:46.680
record? Do you pay for public
transportation or have you been caught
00:11:46.680 --> 00:11:50.479
not paying? What about your friends?
00:11:50.479 --> 00:11:58.439
Do you retweet or use WeChat
to distribute sort of information against
00:11:58.439 --> 00:12:04.300
the party, which they call reliability. In
actuality it's not about whether it's
00:12:04.300 --> 00:12:08.459
factual, it's about whether it's against
the party or not. Where do you buy and
00:12:08.459 --> 00:12:12.019
what do you buy, apparently if you buy
diapers it's better than if you buy
00:12:12.019 --> 00:12:16.679
videogames. For your score. Because you
know if you buy videogames obviously
00:12:16.679 --> 00:12:20.089
you're not very responsible. And if you
buy diapers you have a kid, you are sort
00:12:20.089 --> 00:12:29.019
of conforming to the societal ideal. And
then your score is supposed to go into all
00:12:29.019 --> 00:12:33.679
these different categories, you're
supposed to have better access to social
00:12:33.679 --> 00:12:38.759
services if your score is good. You're
supposed to have better access to internet
00:12:38.759 --> 00:12:43.309
services. So in theory the idea is that at
one point if your score is too bad, you're
00:12:43.309 --> 00:12:47.890
not allowed to use WeChat anymore. You're
not allowed to use Alibaba anymore. You
00:12:47.890 --> 00:12:57.110
can't become a government worker. You can
not take planes and high speed trains. You
00:12:57.110 --> 00:13:04.519
can not get a passport. And your insurance
premiums will go up. So it's supposed to
00:13:04.519 --> 00:13:12.540
be this really really big, overwhelming
system. But in actuality what they say
00:13:12.540 --> 00:13:19.459
their stated goals are, is "it's a
shorthand for a broad range of efforts to
00:13:19.459 --> 00:13:24.790
improve market security and public safety
by increasing integrity and mutual trust
00:13:24.790 --> 00:13:31.899
in society." So one idea is to allocate
resources more efficiently. Resource
00:13:31.899 --> 00:13:39.339
allocation in China is a pretty big
problem, because people grow up with:
00:13:39.339 --> 00:13:43.940
There's 1.3 billion people. So there's -
it's always going to be scarce. And a lot
00:13:43.940 --> 00:13:50.160
of stuff is – people grow up with this
idea that it's just very very scarce, and
00:13:50.160 --> 00:13:54.639
current distribution strategies, which are
mostly financially based but also often
00:13:54.639 --> 00:13:59.360
guanxi-based, don't really seem fair. For
example, public transport in China is
00:13:59.360 --> 00:14:04.769
highly subsidized, which means that the
price does not reflect whether – does not
00:14:04.769 --> 00:14:10.249
reflect true scarcity. So currently the
way it works is in theory it's first come
00:14:10.249 --> 00:14:15.019
first serve, in practice there's people
that are buying up all the tickets for,
00:14:15.019 --> 00:14:19.319
for example, the high speed train from
Shanghai to Beijing and then selling it at
00:14:19.319 --> 00:14:23.120
a profit, or selling it to certain
companies that have good ties to the
00:14:23.120 --> 00:14:28.040
government. That seems very unfair. So the
new system is supposed to distribute them
00:14:28.040 --> 00:14:33.670
more fairly and more efficiently. The
other thing is restoring trust in people.
00:14:33.670 --> 00:14:38.209
Perceived inter-personal trust and trust
in institutions is extremely low in China.
00:14:38.209 --> 00:14:43.119
If you're from Germany, you might have
heard that there is Chinese gangs
00:14:43.119 --> 00:14:48.089
basically buying up German milk powder and
selling it in China. This is actually
00:14:48.089 --> 00:14:54.559
happening, because in 2008 there was a big
scandal with laced milk powder. And ever
00:14:54.559 --> 00:14:59.380
since then, anyone who can afford it does
not use Chinese milk powder, because they
00:14:59.380 --> 00:15:03.730
don't trust the government, or the
regulations, the firms, enough to buy
00:15:03.730 --> 00:15:09.929
Chinese milk powder so they are actually
importing this. And the big irony is:
00:15:09.929 --> 00:15:15.519
sometimes this milk powder is produced in
China, exported to Germany, and then
00:15:15.519 --> 00:15:22.249
exported back to China. The Social Credit
system is then supposed to identify those
00:15:22.249 --> 00:15:27.449
that deserve the trust. And the third
point is sort of a reeducation of people.
00:15:27.449 --> 00:15:32.959
The idea is: they want to make people in
the image that the Communist Party thinks
00:15:32.959 --> 00:15:40.529
people should be. And one additional way
to the punishments and rewards this could
00:15:40.529 --> 00:15:45.420
work, is the feeling of being surveyed.
Because you can't do anything anonymously,
00:15:45.420 --> 00:15:49.470
you will automatically adapt your behavior
because you know someone is watching you
00:15:49.470 --> 00:15:55.500
all the time, and this is how a lot of the
Chinese firewall actually works, because
00:15:55.500 --> 00:16:00.920
most people I know that are sort of more–
more educated, they know ways to
00:16:00.920 --> 00:16:05.019
circumvent the Chinese firewall, but they
also know that they're always being
00:16:05.019 --> 00:16:09.399
watched, so they don't do that because,
you know, they're being watched, so they
00:16:09.399 --> 00:16:18.449
self– they censorship– they censor
themselves. As I said before, allocation
00:16:18.449 --> 00:16:23.619
of scarce resources so far is mainly
through financial guanxi channels. guanxi
00:16:23.619 --> 00:16:29.290
is basically an all permeating network of
relationships with a clear status
00:16:29.290 --> 00:16:34.399
hierarchy. So if I attend a school,
everyone who also attended this school
00:16:34.399 --> 00:16:40.319
will be sort of in my guanxi network. And
there's this idea that we will have a
00:16:40.319 --> 00:16:45.679
system where we are all in-group, and in-
group we trust each other and we do favors
00:16:45.679 --> 00:16:51.120
for each other, and everyone who's outside
of my immediate group I don't trust and I
00:16:51.120 --> 00:16:56.350
don't do favors for. And in some ways the
guanxi system right now is a substitute
00:16:56.350 --> 00:17:02.639
for formal institutions in China. For
example if you want a passport right now.
00:17:02.639 --> 00:17:07.359
You can of course apply for passports
through regular channels, which might take
00:17:07.359 --> 00:17:12.390
months and months. Or you can apply for a
passport through knowing someone and
00:17:12.390 --> 00:17:16.750
knowing someone, which might take only two
days. Whereas in Germany you have these
00:17:16.750 --> 00:17:22.510
very regular, formal institutions, in
China they still use guanxi. But,
00:17:22.510 --> 00:17:26.849
increasingly especially young people find
that guanxi are very unfair, because a lot
00:17:26.849 --> 00:17:31.560
of these are: where you went to school,
which is determined by where you're born,
00:17:31.560 --> 00:17:37.200
who your parents are, and all these
things. Another thing that's important to
00:17:37.200 --> 00:17:42.170
understand because: the system works
through public shaming. And in a lot of
00:17:42.170 --> 00:17:48.430
western society we can't really imagine
that, like, I wouldn't really care if my
00:17:48.430 --> 00:17:54.700
name was in a newspaper of someone who
jaywalked for example. It would be: oh
00:17:54.700 --> 00:17:59.631
well, that's okay. But in China this is
actually a very very serious thing. So
00:17:59.631 --> 00:18:05.030
saving face is very very important in
China. And when I went to school there I
00:18:05.030 --> 00:18:13.080
actually – we had this dormitory, and it
was an all foreigners dormitory, where the
00:18:13.080 --> 00:18:17.680
staff that were responsible for the
dormitory felt that foreigners were not
00:18:17.680 --> 00:18:24.660
behaving in the way they should. So their
idea was to put the names, the pictures,
00:18:24.660 --> 00:18:29.250
and the offenses of the foreigners in the
elevator to shame them publicly. So for
00:18:29.250 --> 00:18:33.690
example if you brought a person of the
opposite sex to your room, they would put
00:18:33.690 --> 00:18:39.910
your name, your offense and your room
number in the elevator. And of course this
00:18:39.910 --> 00:18:44.630
didn't work because for a lot of western
people it was basically like: "oh well I'm
00:18:44.630 --> 00:18:49.140
going to try to be there as often as
possible because this is like a badge of
00:18:49.140 --> 00:18:53.670
honor for me" and the Chinese people they
figured "well this is really really shame
00:18:53.670 --> 00:18:58.080
and I'm losing my face". She brought
alcohol. So this didn't really work at
00:18:58.080 --> 00:19:06.390
all. But this is kind of the mindset that
is behind a lot of these initiatives. As I
00:19:06.390 --> 00:19:12.230
said there's a lot of problems with – we
don't really know what's going to happen.
00:19:12.230 --> 00:19:17.000
And one of the ways that we can see what
might happen is actually to look at pilot
00:19:17.000 --> 00:19:24.000
systems. China has – or like ever since
the Communist Party took hold – the
00:19:24.000 --> 00:19:28.630
Chinese government has tried a lot of
policy experimentation. So whenever they
00:19:28.630 --> 00:19:33.850
try a new policy, they don't roll it out
all over, but they choose different pilot
00:19:33.850 --> 00:19:38.890
cities or pilot districts, and then they
choose "oh well this is the district where
00:19:38.890 --> 00:19:42.900
I'm going to be trying this system and I'm
going to be trying another system in
00:19:42.900 --> 00:19:48.360
another district or city". And this is
also what they did for the, or what
00:19:48.360 --> 00:19:52.520
they're doing for the Social Credit
system. Now I have three systems that I
00:19:52.520 --> 00:19:58.300
looked at intensively for this
presentation, overall there's about 70
00:19:58.300 --> 00:20:07.030
that I know of - the Suining system,
Suining is a city in China, the Rongcheng
00:20:07.030 --> 00:20:11.410
system, another city in China and Sesame
Credit. Sesame Credit is a commercial
00:20:11.410 --> 00:20:16.430
system from Alibaba - I assume everyone
knows Alibaba, the're basically the
00:20:16.430 --> 00:20:22.770
Chinese Amazon, except they're bigger and
have more users and make more money,
00:20:22.770 --> 00:20:27.400
actually. And they have their own little
system. One of the problems with this kind
00:20:27.400 --> 00:20:31.620
of system that I found when I tried
modeling it, was that it's a very very
00:20:31.620 --> 00:20:38.930
complex system and small changes in input
actually changed the output significantly.
00:20:38.930 --> 00:20:43.550
So when they try– usually when they try
this pilot system they basically have a
00:20:43.550 --> 00:20:47.850
couple of pilots, then they choose the
pilot that is best and they roll it out
00:20:47.850 --> 00:20:52.560
all over. But for this kind of thing,
where you have a lot of complex issues, it
00:20:52.560 --> 00:20:59.410
might not be the best way to do that. The
Suining system is actually considered the
00:20:59.410 --> 00:21:05.980
predecessor of all current systems. It had
a focus on punishment, and it was quite
00:21:05.980 --> 00:21:12.000
interesting. At the beginning of the trial
period they published a catalogue of
00:21:12.000 --> 00:21:17.260
scores and consequences. Here is an
example. This is basically taken from this
00:21:17.260 --> 00:21:22.640
catalog. So if you took out bank loans and
didn't repay them, you got deducted 50
00:21:22.640 --> 00:21:26.910
points. Everyone started with 1000
points for this system. If you didn't pay
00:21:26.910 --> 00:21:32.690
back your credit cards you also got
deducted 50 points. If you evaded taxes,
00:21:32.690 --> 00:21:43.890
also 50 points. If you sold fake goods, 35
points were deducted. And actually the
00:21:43.890 --> 00:21:50.450
system was abolished I think in 2015,
2016, because all the Chinese state media
00:21:50.450 --> 00:21:56.490
and also a lot of Internet citizens talked
about how it's an Orwellian system and how
00:21:56.490 --> 00:22:03.020
it's not a good system, because it's all
very centralized and everything that you
00:22:03.020 --> 00:22:08.880
do is basically recorded centrally. But
Creemers writes: "Nonetheless, the Suining
00:22:08.880 --> 00:22:13.090
system already contained the embryonic
forms of several elements of subsequent
00:22:13.090 --> 00:22:17.290
social credit initiatives: The notion of
disproportional disincentives against rule
00:22:17.290 --> 00:22:22.500
breaking, public naming and shaming of
wrongdoers, and most importantly, the
00:22:22.500 --> 00:22:26.900
expansion of the credit mechanism outside
of the market economic context, also
00:22:26.900 --> 00:22:29.760
encompassing compliance with
administrative regulations and urban
00:22:29.760 --> 00:22:34.300
management rules." So one of the things
that is difficult for especially German
00:22:34.300 --> 00:22:40.191
speakers is that credit in Chinese,
xìnyòng, means credit as in "loan", but
00:22:40.191 --> 00:22:46.350
also means credit as in "trust". So the
Social Credit System is one way of trying
00:22:46.350 --> 00:22:52.240
to conflate those two – the economic
credit and the trust credit – into one big
00:22:52.240 --> 00:23:01.670
system. But the Suining system basically
failed. So, they adapted the system and
00:23:01.670 --> 00:23:07.220
are now practicing a new kind of system,
the Rongcheng system. Whenever you read a
00:23:07.220 --> 00:23:11.310
newspaper article on the social credit
system in the west, most people went to
00:23:11.310 --> 00:23:15.060
Rongcheng because they just received a
couple of awards from the Chinese
00:23:15.060 --> 00:23:22.230
government for being so advanced at this
social credit thing. But it's very
00:23:22.230 --> 00:23:25.990
difficult to call this "one system"
because there's actually many many
00:23:25.990 --> 00:23:31.020
intertwined systems. There is one city
level system, where city level offenses
00:23:31.020 --> 00:23:38.520
are recorded. For example tax evasion, and
there's a couple of rules. If you evade
00:23:38.520 --> 00:23:43.730
taxes your score goes down 50. But then if
you live in one neighborhood your score
00:23:43.730 --> 00:23:47.620
might go up for volunteering with the
elderly. If you live in another
00:23:47.620 --> 00:23:54.200
neighborhood your score might go up for,
for example, planting some trees in
00:23:54.200 --> 00:23:59.180
your garden or backyard. So depending on
your neighborhood, your score might be
00:23:59.180 --> 00:24:05.590
different. If you work for a– if you work
for a taxi cab company, for example, they
00:24:05.590 --> 00:24:10.780
also have their own little score system
and your score might go up if you get good
00:24:10.780 --> 00:24:16.570
reviews from your drive…, from your
passengers. Your score might go down if
00:24:16.570 --> 00:24:25.780
you don't follow traffic rules, these
kinds of things. There are designated
00:24:25.780 --> 00:24:31.550
scorekeepers at each level. So, each
district chooses a couple of people who
00:24:31.550 --> 00:24:38.330
are responsible for passing on the
information to the next higher level,
00:24:38.330 --> 00:24:42.870
about who did what. There is supposed to
be an official appeals procedure, so
00:24:42.870 --> 00:24:46.940
whenever you score changes you're supposed
to be notified, but apparently that's not
00:24:46.940 --> 00:24:54.490
happening at this point for most people.
Again, it's a system of data sharing, and
00:24:54.490 --> 00:24:58.520
one thing that they haven't really
disclosed yet is what kind of data is
00:24:58.520 --> 00:25:04.760
shared. Are they only sharing the points,
so if I'm in a district and I plant some
00:25:04.760 --> 00:25:11.480
trees, does the central system get the
information "person A planted some trees,"
00:25:11.480 --> 00:25:16.300
or does the central system get the
information "person A got 5 points?" We
00:25:16.300 --> 00:25:21.570
don't know at this point. And it would
mean something very different for how the
00:25:21.570 --> 00:25:27.030
system could be used. But still the end
result, at this point, is that there's one
00:25:27.030 --> 00:25:31.360
score. So you have one central score and
it's kind of– there's all these different
00:25:31.360 --> 00:25:35.780
smaller systems that go into this score.
But at the end, everyone has one central
00:25:35.780 --> 00:25:44.530
score, and currently about 85 percent of
people are between 950 and 1050. So you
00:25:44.530 --> 00:25:49.990
start off with a thousand – and those are
basically the normal people – and then
00:25:49.990 --> 00:25:58.940
anyone above a 1050 is considered a
trustworthy person, and anyone below 1050
00:25:58.940 --> 00:26:05.340
is considered a trust-breaker. And, as
I've said before, with the naming and
00:26:05.340 --> 00:26:12.310
shaming and all these things, what you can
actually see here is a billboard with the
00:26:12.310 --> 00:26:17.520
best trustworthy families in Rongcheng. So
these are the families that have the
00:26:17.520 --> 00:26:23.590
highest scores, for example. Sesame Credit
is a little different. It's the only
00:26:23.590 --> 00:26:27.990
system that actually uses machine learning
and artificial intelligence to determine
00:26:27.990 --> 00:26:32.980
the outputs. In Rongcheng, for example,
they have artificial intelligence, they
00:26:32.980 --> 00:26:37.120
have computer vision, for the most part,
and the computer vision cameras they
00:26:37.120 --> 00:26:42.220
decide– they try to recognize you when you
jaywalk. And then when they recognize you
00:26:42.220 --> 00:26:48.250
when jaywalking, you get a small SMS;
"well, we just saw you jaywalking, your
00:26:48.250 --> 00:26:56.810
score is now dropping." But how the score
develops, depending on your jaywalking,
00:26:56.810 --> 00:27:01.120
isn't really determined by machine
learning or artificial intelligence.
00:27:01.120 --> 00:27:06.840
Instead, it's determined by rules. You
know: one time jaywalking deducts five
00:27:06.840 --> 00:27:12.460
points, and this is stated somewhere.
Sesame Credit doesn't work like that.
00:27:12.460 --> 00:27:19.810
Instead it uses a secret algorithm, and
the way– I talked to some people that work
00:27:19.810 --> 00:27:25.330
for Sesame Credit or for Alibaba, and the
way they described it was; they basically
00:27:25.330 --> 00:27:33.260
clustered people based on behavior, then
gave scores to these clusters, and
00:27:33.260 --> 00:27:40.700
then afterwards, did basically reverse
engineered their own score, using machine
00:27:40.700 --> 00:27:45.950
learning, so that whenever something new
happens, you can move to a different
00:27:45.950 --> 00:27:54.330
cluster. This Sesame Credit was actually
refused accreditation as a credit score in
00:27:54.330 --> 00:28:03.400
2017, so banks are not allowed to use the
Sesame Credit score for your– to use the
00:28:03.400 --> 00:28:09.190
Sesame Credit score to determine whether
they give you loans or not. Because Sesame
00:28:09.190 --> 00:28:13.770
Credit is quite ingenious – obviously
Alibaba wants to keep you within their
00:28:13.770 --> 00:28:19.920
platform – so if you buy using Alibaba and
using Alipay, your score goes up. If you
00:28:19.920 --> 00:28:29.090
buy using Weechatpay, which is a competing
platform, your score goes down. This uses
00:28:29.090 --> 00:28:34.000
many of the same rewards mechanisms of the
official government systems, and this is
00:28:34.000 --> 00:28:38.390
just an illustration of what kind of
scores you can have, apparently your
00:28:38.390 --> 00:28:46.070
scores can go between 350 and 850, and in
Chinese there's basically five different
00:28:46.070 --> 00:28:55.809
levels. So 385 is a "trust-breaker" or
"missing trust". And then 731 is "trust is
00:28:55.809 --> 00:29:06.170
exceedingly high". So one way I tried to
approach this issue was through agent-
00:29:06.170 --> 00:29:10.100
based modeling. Social Credit System is
individual level, but what we're really
00:29:10.100 --> 00:29:13.400
interested in, or what I'm really
interested in, is actually societal-level
00:29:13.400 --> 00:29:19.040
consequences. So if everyone gets this
score, what does that mean for society?
00:29:19.040 --> 00:29:24.050
And agent-based modeling works quite well
for that, because it allows us to imbue
00:29:24.050 --> 00:29:28.960
agents with some sort of rationality, but
with a bounded rationality. What does
00:29:28.960 --> 00:29:32.800
bounded rationality mean? Usually in
economics people assume agents are
00:29:32.800 --> 00:29:38.070
completely rational, so they are profit
maximizers, they have all the information.
00:29:38.070 --> 00:29:45.380
But in reality, agents don't have all the
information, they have a lot of issues
00:29:45.380 --> 00:29:51.100
with keeping stuff in their mind. So a lot
of the time, they won't choose the best
00:29:51.100 --> 00:29:57.760
thing in the world, but they choose the
best thing that they see. And bounded
00:29:57.760 --> 00:30:02.040
rationality allows us to account for this
thing. It allows us to account for
00:30:02.040 --> 00:30:08.380
heuristics and these things. And what I
did is I took the propensity for specific
00:30:08.380 --> 00:30:12.320
behavior from current state of the art
research, mostly from behavioral
00:30:12.320 --> 00:30:17.900
economics. For example, I looked at tax
evasion, and I looked at who is likely to
00:30:17.900 --> 00:30:24.630
evade taxes in a system, and then
obviously there was some stochastic –
00:30:24.630 --> 00:30:30.809
some chance element. But the
distribution that I chose is related to
00:30:30.809 --> 00:30:37.680
the current research. And I also checked
that my model has similar results to the
00:30:37.680 --> 00:30:45.400
Rongcheng model, which I modeled at at the
beginning. So on average 87% of my users
00:30:45.400 --> 00:30:49.240
have a score of within 10 percent of the
original score, which is also the data
00:30:49.240 --> 00:31:00.740
that Rongcheng city actually publishes.
Now, for the most part, I compared design
00:31:00.740 --> 00:31:05.220
choices in two axes. One of them was a
centralized system versus a multi-level
00:31:05.220 --> 00:31:10.710
system, and a rule-based system versus a
machine learning system. The centralized
00:31:10.710 --> 00:31:20.650
system is basically: you have a central –
all the information is kept centrally, and
00:31:20.650 --> 00:31:27.190
everyone in China, or wherever, in
Rongcheng has the exact same scoring
00:31:27.190 --> 00:31:35.520
opportunities. Now, if you have a
centralized system the clear expectations
00:31:35.520 --> 00:31:39.600
were pretty good. But, at the same time,
the acceptance from the population was
00:31:39.600 --> 00:31:46.590
really, really low, which they found
during the Suining experiment. And
00:31:46.590 --> 00:31:50.540
there's also the problem of a single point
of failure. Who decides the central
00:31:50.540 --> 00:31:59.080
catalog, and, depending on who, sort of,
has the power, it kind of, just,
00:31:59.080 --> 00:32:05.100
reproduces power structures. So because
you have this central catalog, the same
00:32:05.100 --> 00:32:10.760
people that are in power centrally, they
are basically deciding some sort of score
00:32:10.760 --> 00:32:15.340
mechanism that works for them very well,
so that they and their family will have
00:32:15.340 --> 00:32:23.280
high scores. And multi-level system has
the advantage that local adaptation kind
00:32:23.280 --> 00:32:29.080
of works, and there's sort of many points
of failure. But in my model, when I
00:32:29.080 --> 00:32:36.860
allowed locals to basically set their own
rules, what happened was that they
00:32:36.860 --> 00:32:42.520
competed. So, it started out being this
district of Rongcheng, for example, and
00:32:42.520 --> 00:32:46.120
this district of Rongcheng, they compete
for the best people that they want to
00:32:46.120 --> 00:32:52.000
attract, and suddenly you have this kind
of race to the bottom, where people want
00:32:52.000 --> 00:32:58.090
to move where they wouldn't be prosecuted,
so they move to places where there's less
00:32:58.090 --> 00:33:04.000
cameras, for example. At the same time,
there's many points of failure, especially
00:33:04.000 --> 00:33:13.500
the way it's currently set up, with people
reporting data to the next high level.
00:33:13.500 --> 00:33:19.450
And, a lot of the time, what we have
actually seen in Rongcheng, was that they
00:33:19.450 --> 00:33:24.360
reported data on people they didn't like
more than data on people they did like.
00:33:24.360 --> 00:33:29.390
Or, their families got better scores than
people they didn't know. So it also kind
00:33:29.390 --> 00:33:41.320
of reproduced these biases. The rule based
system has the advantage that people were
00:33:41.320 --> 00:33:46.530
more prone to adapt their behavior,
because they actually knew what they
00:33:46.530 --> 00:33:50.630
needed to do in order to adapt their
behavior. But the score didn't really
00:33:50.630 --> 00:33:54.050
correlate with the important
characteristics that they actually cared
00:33:54.050 --> 00:34:01.610
about. And, as opposed to in this machine
learning system, you know how in Germany
00:34:01.610 --> 00:34:06.910
we don't really know the Schufa algorithm.
And I, for example, don't exactly know
00:34:06.910 --> 00:34:12.139
what I could do in order to improve my
Schufa score. And this is a similar system
00:34:12.139 --> 00:34:16.969
in China with the Sesame Credit score. A
lot of people don't really – they say,
00:34:16.969 --> 00:34:21.800
"well I really want to adapt my behavior
to the score, to improve my score, but
00:34:21.800 --> 00:34:29.119
when I tried doing that my score actually
got worse." And you can have different
00:34:29.119 --> 00:34:36.440
biases, that I'm going to be talking about
in a little bit. There's also this big
00:34:36.440 --> 00:34:42.579
problem of incentive mismatch. So, the
decentralized, rules-based systems like
00:34:42.579 --> 00:34:47.299
Rongcheng, which is the system that I
analyzed the most. Why, because I believe
00:34:47.299 --> 00:34:51.530
this is the system that we're moving
towards right now. Because Rongcheng won a
00:34:51.530 --> 00:34:57.018
lot of awards. So the Chinese government,
the way they usually work is, they try
00:34:57.018 --> 00:35:01.710
pilots, then they choose the best couple
of systems, they give them awards, and
00:35:01.710 --> 00:35:06.890
then they roll out the system nationwide.
So I assume that the system that's going
00:35:06.890 --> 00:35:12.990
to be – the system in the end will be
similar to the Rongcheng system. Now, one
00:35:12.990 --> 00:35:19.670
problem that I actually saw in my
simulation was that you could have this
00:35:19.670 --> 00:35:24.960
possible race to the bottom. There's also
this conflict of interest in those that
00:35:24.960 --> 00:35:29.710
set the rules, because a lot of the time,
the way it works is, you have your
00:35:29.710 --> 00:35:35.940
company, and your company, you, in
combination with your party leaders,
00:35:35.940 --> 00:35:43.760
actually decide on the rules for the score
system. But the scores of all your
00:35:43.760 --> 00:35:48.319
employees actually determines your
company's score. If you employ a lot of
00:35:48.319 --> 00:35:52.589
people with high scores you get a better
score. So you will have this incentive to
00:35:52.589 --> 00:35:57.400
give out high scores and to make sure that
everyone gets high scores. But at the same
00:35:57.400 --> 00:36:04.720
time the government has an incentive for
scores to be comparable. So there's a lot
00:36:04.720 --> 00:36:10.020
of incentives mismatch. The government
also has the incentive to keep false
00:36:10.020 --> 00:36:15.539
negatives down, but they actually, the way
the Chinese system currently works is,
00:36:15.539 --> 00:36:22.589
they emphasize catching trust-breakers
more than rewarding trust-follow... or
00:36:22.589 --> 00:36:28.750
trustworthy people. So, false positives,
for them, are less important, but false
00:36:28.750 --> 00:36:34.359
positives erode the trust in the system,
and they lead to a lot less behavioral
00:36:34.359 --> 00:36:40.740
adaptation. I was actually able to show
this using some nudging research that
00:36:40.740 --> 00:36:47.789
showed that as soon as you introduce an
error probability and you can be caught
00:36:47.789 --> 00:36:54.530
for something that you didn't do, your
probability of changing your behavior
00:36:54.530 --> 00:37:02.119
based on this score is actually lower. And
in Rongcheng, one of the perverse things
00:37:02.119 --> 00:37:09.880
that they're doing is, you can donate
money to the party or to, like, party
00:37:09.880 --> 00:37:17.339
affiliated social services, and this will
give you points, which is kind of an
00:37:17.339 --> 00:37:24.269
indulgence system. Which is quite
interesting, especially because a lot of
00:37:24.269 --> 00:37:32.690
these donation systems work in a way that
you can donate 50000 renminbi and you get
00:37:32.690 --> 00:37:36.740
50 points, and then you donate another
50000 renminbi and you get another 50
00:37:36.740 --> 00:37:44.400
points. So you can basically donate a lot
of money and then behave however you want,
00:37:44.400 --> 00:37:56.220
and still get a good score. And the trust
in other people can actually go down even
00:37:56.220 --> 00:38:00.559
more in this system, because suddenly you
only trust them because of their scores,
00:38:00.559 --> 00:38:05.460
and the current system is set up so that
you can actually look up scores of
00:38:05.460 --> 00:38:09.730
everyone that you want to work with, and
if they don't have a score high enough
00:38:09.730 --> 00:38:15.140
then suddenly you don't want to work with
them. The trust in the legal system can
00:38:15.140 --> 00:38:22.259
also decrease, actually. Why? Because trust
in the legal system in China is already
00:38:22.259 --> 00:38:26.039
low, and a lot of the things, like
jaywalking, they're already illegal in
00:38:26.039 --> 00:38:30.380
China, as they are here, but no one cares.
And suddenly, you have this parallel
00:38:30.380 --> 00:38:37.269
system that punishes you for whatever.
But, why don't you just try to fix the
00:38:37.269 --> 00:38:45.190
legal system, which would be my approach.
Suddenly, illegal activity could happen
00:38:45.190 --> 00:38:51.720
more offline, and this is one of those
things that is quite interesting. In
00:38:51.720 --> 00:38:58.059
countries that we've seen that have moved
towards mobile payments, and away
00:38:58.059 --> 00:39:05.069
from cash, you see less robberies but you
don't actually see less crime. Instead you
00:39:05.069 --> 00:39:12.430
see more new types of crime. So, you see
more credit card fraud, you see more phone
00:39:12.430 --> 00:39:19.390
robberies, these kinds of things. And this
is also where things could move in the
00:39:19.390 --> 00:39:29.710
Chinese case. One major problem is also
that this new system – I've talked a
00:39:29.710 --> 00:39:34.499
little bit about this one, but – it can
introduce a lot of new bias, and reproduce
00:39:34.499 --> 00:39:44.710
the bias even more. So, for example, China
is a country of 55 minorities. The Han are
00:39:44.710 --> 00:39:50.249
a big majority, they have about 94 percent
of the population. So any computer vision
00:39:50.249 --> 00:39:58.020
task, we've shown, that they are really,
really bad at discriminating between
00:39:58.020 --> 00:40:04.930
individuals in smaller ethnic groups. In
the U.S., most computer vision tasks
00:40:04.930 --> 00:40:09.920
perform worse for African-Americans, they
perform worse for women, because all of
00:40:09.920 --> 00:40:16.930
the training sets are male and white, and
maybe Asian. In China, all of these tasks
00:40:16.930 --> 00:40:26.609
are actually performing worse for ethnic
minorities, for the Uyghurs, for example.
00:40:26.609 --> 00:40:32.460
And one way that they could try to abuse
the system is to basically just – what
00:40:32.460 --> 00:40:38.210
they're also doing already in Xinjiang is
– to basically just identify, "oh this is
00:40:38.210 --> 00:40:45.979
a person of the minority, well I'm just
going to go and check him or her more
00:40:45.979 --> 00:40:50.210
thoroughly." This is actually what happens
in Xinjiang. If you're in Xinjiang and you
00:40:50.210 --> 00:40:59.250
look like a Turkish person, or like from
Turkmenistan, from a Turkish people, you
00:40:59.250 --> 00:41:04.210
are a lot more likely to be questioned.
You're a lot more likely to be stopped and
00:41:04.210 --> 00:41:12.579
they ask you or require you to download
spyware on your phone. And this is
00:41:12.579 --> 00:41:17.660
currently what happens and this new kind
of system can actually help you with that.
00:41:17.660 --> 00:41:24.710
I've said that it can reproduce these kind
of power structures, and now obviously we
00:41:24.710 --> 00:41:29.890
all know neutral technology doesn't really
exist, but in the Chinese case, in the
00:41:29.890 --> 00:41:33.529
social credit case, they don't even
pretend – they always say "well, this
00:41:33.529 --> 00:41:37.349
is neutral technology and it's all a lot
better," but actually it's the people
00:41:37.349 --> 00:41:43.970
currently in power, they decide on what
gives you point and what deducts points
00:41:43.970 --> 00:41:50.029
for you. Another problem, currently the
entire system is set up in a way that it
00:41:50.029 --> 00:41:54.619
all goes together with your shēnfènzhèng,
with your I.D. card. What if you don't
00:41:54.619 --> 00:41:59.410
have an I.D. card? That's foreigners for
one. But it's also people in China that
00:41:59.410 --> 00:42:05.479
were born during the one child policy and
were not registered. There's quite a lot
00:42:05.479 --> 00:42:09.200
of them, actually. They're not registered
anywhere and suddenly they can't do
00:42:09.200 --> 00:42:13.700
anything, because they don't have a score,
they can't get a phone, they can't do
00:42:13.700 --> 00:42:20.880
anything, really. And part of the push
with this social credit system is to go
00:42:20.880 --> 00:42:26.779
away from cash, actually. So if you need
to use your phone to pay, but for your
00:42:26.779 --> 00:42:29.519
phone you need your shēnfènzhèng.
If you don't have a shēnfènzhèng,
00:42:29.519 --> 00:42:32.569
well, tough luck for you.
00:42:32.569 --> 00:42:38.680
And currently the system in Rongcheng
is set up in a way that you can check
00:42:38.680 --> 00:42:44.900
other people's scores and you can also see
what they lose points for. So you can
00:42:44.900 --> 00:42:49.779
actually, sort of, choose to discriminate
against people that are gay, for example,
00:42:49.779 --> 00:42:53.049
because they might have lost points for
going to a gay bar, which you can lose
00:42:53.049 --> 00:43:02.130
points for. Another big issue, currently,
is data privacy and security. Personal
00:43:02.130 --> 00:43:07.250
data is grossly undervalued in China. If
you ask a Chinese person, "what do you
00:43:07.250 --> 00:43:14.690
think, how much is your data worth?," they
say "what data? I don't have data." And,
00:43:14.690 --> 00:43:19.130
currently, the way it works is, if you
have someone's ID number, which is quite
00:43:19.130 --> 00:43:24.890
easy to find out, you can actually buy
access to a lot of personal information
00:43:24.890 --> 00:43:31.359
for a small fee. So you pay about 100
euros and you get all hotel bookings of
00:43:31.359 --> 00:43:35.960
the last year, you get information of who
booked these hotels with them, you get
00:43:35.960 --> 00:43:40.829
information of where they stay, you get
train bookings, you get access to all of
00:43:40.829 --> 00:43:47.200
the official databases for this one
person. And for another 700 renminbi you
00:43:47.200 --> 00:43:52.829
can actually get live location data, so
you can get the data of where this person
00:43:52.829 --> 00:43:56.400
is right now, or where his or her phone is
right now, but if you've ever been to
00:43:56.400 --> 00:44:03.440
China you know that where the phone is,
usually, the people aren't far. Supchina
00:44:03.440 --> 00:44:08.500
actually did an experiment where a couple
of journalists tried buying that, because
00:44:08.500 --> 00:44:14.190
it's actually these kind of services are
offered on weechat, pretty publicly. And
00:44:14.190 --> 00:44:26.359
you can just buy them, quite easily. So
one additional thing that I looked at is,
00:44:26.359 --> 00:44:30.579
because one of the things that is quite
interesting is, you have this idea of
00:44:30.579 --> 00:44:39.309
credit as twofold. Credit is trust credit
but credit is also loan credit, and what
00:44:39.309 --> 00:44:43.730
if credit institutions actually use this
unified credit score to determine credit
00:44:43.730 --> 00:44:49.059
distribution? The idea is that it's
supposed to lead to reduced information
00:44:49.059 --> 00:44:55.471
asymmetry, obviously, so fewer defaults
and overall more credit creation. New
00:44:55.471 --> 00:44:59.549
people are supposed to get access to
credit, and there's supposed to be less
00:44:59.549 --> 00:45:04.589
shadow banking. But what actually happens?
I'm not going to be talking about how I
00:45:04.589 --> 00:45:08.619
set up the model but just about my
results. If you have this kind of score
00:45:08.619 --> 00:45:14.369
that includes credit information but also
includes morally good – or measures of
00:45:14.369 --> 00:45:18.780
being morally good – what you have is, in
the beginning, about 30 percent more
00:45:18.780 --> 00:45:23.960
agents get access to credit, and
especially people that previously have not
00:45:23.960 --> 00:45:29.710
gotten credit access suddenly have credit
access. But the problem is that this
00:45:29.710 --> 00:45:36.170
social credit score that correlates all of
these different issues, it correlates only
00:45:36.170 --> 00:45:41.809
very, very weakly with repayment ability
or repayment wishes, and thus suddenly you
00:45:41.809 --> 00:45:47.880
have all of these non-performing loans.
You have – and what we see is sort of
00:45:47.880 --> 00:45:51.960
like – we have non-performing loans.
Banks give out less loans because they
00:45:51.960 --> 00:45:59.400
have so many non-performing loans, and
then the non-performing loans are written
00:45:59.400 --> 00:46:03.999
off, and suddenly banks give out more
loans. But you have this oscillating
00:46:03.999 --> 00:46:09.239
financial system, where you give out a lot
of loans, a lot of them are non-
00:46:09.239 --> 00:46:13.019
performing, then you give out a lot of
loans again. And this is very, very
00:46:13.019 --> 00:46:19.150
vulnerable to crisis. If you have a real
economic crisis during the time where non-
00:46:19.150 --> 00:46:24.569
performing loans are high, then a lot of
banks will actually default, which is
00:46:24.569 --> 00:46:29.920
very, very dangerous for a financial
system as nationed as the Chinese one.
00:46:29.920 --> 00:46:36.799
Now, what are some possible corrections?
You could create a score that basically is
00:46:36.799 --> 00:46:41.259
the same as the Schufa score. So that it
looks only at credit decisions, but
00:46:41.259 --> 00:46:45.190
suddenly, you lose a lot of incentives for
the social credit score, if the social
00:46:45.190 --> 00:46:48.430
credit score doesn't matter for credit
distribution anymore.
00:46:48.430 --> 00:46:52.040
Another thing, and this is, I
think, the more likely one,
00:46:52.040 --> 00:46:55.959
is that you have a blacklist for people
that have not repaid a loan
00:46:55.959 --> 00:46:59.400
in the past. So you basically
get one freebie, and afterwards
00:46:59.400 --> 00:47:04.150
if you didn't repay your loan in the past
then you will not get a loan in the
00:47:04.150 --> 00:47:08.400
future. You will still be part of the
social credit system, and your social
00:47:08.400 --> 00:47:12.359
credit score will still be important for
all of these other access issues, but it
00:47:12.359 --> 00:47:15.849
won't be important for access to loans
anymore, once you've been on this
00:47:15.849 --> 00:47:21.670
blacklist. Which is probably something
that the Chinese government could go
00:47:21.670 --> 00:47:30.180
behind, but it's also more effort to take
care of it; then you have to think about,
00:47:30.180 --> 00:47:33.819
"well, you can't leave them on the
blacklist forever, so how long do you
00:47:33.819 --> 00:47:37.599
leave them on the black list? Do they have
to pay back the loan and then they get off
00:47:37.599 --> 00:47:45.670
the blacklist? Or do they have to pay back
the loan and then stay not in default
00:47:45.670 --> 00:47:52.859
for a year, or for five years?" There's a
lot of small decisions that, in my
00:47:52.859 --> 00:47:57.549
opinion, the Chinese government hasn't
really thought about, up until now,
00:47:57.549 --> 00:48:01.170
because they're basically doing all these
pilot studies, and all of these regional
00:48:01.170 --> 00:48:05.160
governments are thinking of all these
small things, but they're not documenting
00:48:05.160 --> 00:48:10.349
everything that they're doing. So, once
they – they want to roll it out by 2020,
00:48:10.349 --> 00:48:15.200
by the way, nationwide – once they've
rolled it out there's a pretty big chance,
00:48:15.200 --> 00:48:18.630
in my opinion, that they'll have a lot of
unintended consequences. A lot of things
00:48:18.630 --> 00:48:28.519
that they haven't thought about, and that
they will then have to look at. So, I
00:48:28.519 --> 00:48:33.269
believe that some sort of system is likely
to come, just in terms of how much energy
00:48:33.269 --> 00:48:37.289
they've expended into this one, and for
the Chinese government at this point, for
00:48:37.289 --> 00:48:41.900
the party, it would be losing face if they
did not include any such system, because
00:48:41.900 --> 00:48:45.969
they've been talking about this for a
while. But most likely, it would be a kind
00:48:45.969 --> 00:48:53.019
of decentralized data sharing system. And
when I ran my simulation... By the way I
00:48:53.019 --> 00:48:59.700
will make public my code, I still need
some, basically, I used some proprietary
00:48:59.700 --> 00:49:06.460
data for my model, and I still need the
permission to publish this. Once I publish
00:49:06.460 --> 00:49:11.289
this one I will also tweet it, and we'll
put it on GitHub for everyone to play
00:49:11.289 --> 00:49:16.200
around with, if you want to. And some of
these implementation details that were
00:49:16.200 --> 00:49:20.450
very important in determining model
outcomes where "do we have a relative or
00:49:20.450 --> 00:49:25.289
absolute ranking?" So far, all of the
systems I looked at had absolute rankings,
00:49:25.289 --> 00:49:30.700
but there's a point to be made for
relative rankings. Do we have one score,
00:49:30.700 --> 00:49:35.089
where, basically, if you're a Chinese
person you get one score? Or do we have
00:49:35.089 --> 00:49:40.880
different sub-scores in different fields?
Do we have people reporting behavior, or
00:49:40.880 --> 00:49:46.369
do we have automatic behavior recording?
How do you access other people's scores?
00:49:46.369 --> 00:49:50.339
How much information can you get from
other people's scores? Currently, if
00:49:50.339 --> 00:49:55.529
someone is on a blacklist, for example, if
you have their ID number, again, you can
00:49:55.529 --> 00:49:59.650
put it into this blacklist, and then they
will say "oh, this person is on this
00:49:59.650 --> 00:50:04.630
blacklist for not following this judge's
order," and then it says what kind of
00:50:04.630 --> 00:50:10.660
judge's order it was. So, most likely, it
will be something like this. The idea is
00:50:10.660 --> 00:50:16.049
that the Social Credit system isn't only
for individuals, but also for firms and
00:50:16.049 --> 00:50:22.219
for NGOs. So, what kind of roles will
firms play in the system? I haven't looked
00:50:22.219 --> 00:50:28.319
at that, in detail, at this point, but it
would be very interesting. Another idea
00:50:28.319 --> 00:50:34.390
that western people often talk about is,
do people also rank each other? Currently,
00:50:34.390 --> 00:50:39.390
that's not part of the system in China,
but it might be at one point. And lastly,
00:50:39.390 --> 00:50:44.839
where does the aggregation happen? So I've
said that a lot of it is actually data
00:50:44.839 --> 00:50:53.749
sharing in China. So what kind of data is
shared? Is the raw data shared? "Person A
00:50:53.749 --> 00:51:03.900
did something." Or is the aggregated data
shared? "Person A got this score." At this
00:51:03.900 --> 00:51:07.890
point, most of the time, it is actually
the raw data that is shared, but that also
00:51:07.890 --> 00:51:12.809
has sort of these data privacy issues, of
course, that I've talked about. OK,
00:51:12.809 --> 00:51:18.950
perfect! No there's 10 more minutes. Thank
you for your attention! If you have
00:51:18.950 --> 00:51:24.270
questions, remarks you can ask them now or
you can catch me up later. You can tweet
00:51:24.270 --> 00:51:29.829
to me or send me an e-mail, whatever
you're interested in. Thank you very much!
00:51:29.829 --> 00:51:37.329
applause
00:51:37.329 --> 00:51:42.039
Herald Angel: Hello! As Toni said, we have
10 minutes left for questions. If you have
00:51:42.039 --> 00:51:46.981
a question in the room, please go crouch in
front of our five microphones. If you're
00:51:46.981 --> 00:51:49.701
watching the stream, please ask your
questions through IRC or Twitter, and
00:51:49.701 --> 00:51:53.847
we'll also try to make sure to get to
those. Let's just go ahead and start with
00:51:53.847 --> 00:51:56.339
mic one.
Question: Good! Thank you very much for
00:51:56.339 --> 00:52:02.809
this beautiful talk. I was wondering how
did the Chinese government, companies, and
00:52:02.809 --> 00:52:07.470
most of all, the citizens themselves,
respond to you doing this research, or,
00:52:07.470 --> 00:52:11.519
let's put it differently, if you would
have been in the system yourself,
00:52:11.519 --> 00:52:14.440
how would your research affect your
social credit score?
00:52:14.440 --> 00:52:17.270
laughter
00:52:17.270 --> 00:52:25.639
Answer: So, um... There's actually two
different responses that I've seen. When I
00:52:25.639 --> 00:52:31.380
talk to the government themselves, because
I was there on a government scholarship,
00:52:31.380 --> 00:52:34.869
and mentioned that I'm really interested
in this, they basically said oh well this
00:52:34.869 --> 00:52:38.930
is just a technical system. You don't
really need to be concerned with this. It
00:52:38.930 --> 00:52:43.439
is not very important. Just, you know,
it's just a technicality. It's just for us
00:52:43.439 --> 00:52:49.299
to make life more efficient and better for
everyone. So I assume my score would
00:52:49.299 --> 00:52:54.660
actually go down from doing this research,
actually. But when I talk to a lot of
00:52:54.660 --> 00:53:00.890
people at universities, they were also
very – they were very interested in my
00:53:00.890 --> 00:53:05.230
research, and a lot of them mentioned that
they didn't even know that the system
00:53:05.230 --> 00:53:10.200
existed!
Herald: Before we go to a question from
00:53:10.200 --> 00:53:14.729
our signal angel, a request for all the
people leaving the room, please do so as
00:53:14.729 --> 00:53:20.500
quietly as possible, so we can continue
this Q and A. The signal angel, please!
00:53:20.500 --> 00:53:26.249
Signal Angel: Jaenix wants to know, is
this score actually influenced by
00:53:26.249 --> 00:53:31.549
association with people with a low score.
Meaning that, is there any peer pressure
00:53:31.549 --> 00:53:36.200
to stay away from people with bad scores?
Answer: The Sesame credit score definitely
00:53:36.200 --> 00:53:42.849
is influenced by your friends' scores, the
Rongcheng score, so far, apparently, is
00:53:42.849 --> 00:53:47.619
not influenced, but it is definitely in
the cards, and it is planned that it will
00:53:47.619 --> 00:53:53.800
be part of this. I think WeChat, which is
the main platform – it's sort of like
00:53:53.800 --> 00:53:59.880
WhatsApp, except it can do a lot a lot
more – WeChat is still not connected to
00:53:59.880 --> 00:54:05.039
the Social Credit Score in Rongcheng. Once
they do that, it will most likely also
00:54:05.039 --> 00:54:09.849
reflect your score.
Herald: All right, let's continue with
00:54:09.849 --> 00:54:15.430
mic 3.
Q: I have a question about your models.
00:54:15.430 --> 00:54:19.789
I'm wondering, what kind of interactions
are you modeling? Or actions, like, what
00:54:19.789 --> 00:54:24.910
can the agents actually do? You mentioned
moving somewhere else. And, what else?
00:54:24.910 --> 00:54:31.190
A: Okay so the way I set up my model was,
I set up a multilevel model. So I looked
00:54:31.190 --> 00:54:38.339
at different kinds of levels. I started
out with, basically, they can evade taxes,
00:54:38.339 --> 00:54:46.890
they can get loans and repay loans, they
can choose where to live, and they can
00:54:46.890 --> 00:54:54.479
follow traffic rules or not follow traffic
rules. And because these were, sort of,
00:54:54.479 --> 00:54:58.660
four big issues that were mentioned in all
of the different systems, so I started out
00:54:58.660 --> 00:55:04.810
with these issues, and looked at, what
kind of behavior do I see? I used some
00:55:04.810 --> 00:55:11.029
research that – some friends of mine
actually sent out surveys to people and
00:55:11.029 --> 00:55:16.299
asked them "well, you're now part of the
system. Did your behavior change, and how
00:55:16.299 --> 00:55:23.109
did it change depending on your responses,
depending on your score, and depending on
00:55:23.109 --> 00:55:27.729
the score system that exists?" And I,
basically, used that, and some other
00:55:27.729 --> 00:55:34.430
research on nudging and on behavioral
adaptation, to look at how likely is it
00:55:34.430 --> 00:55:39.160
that someone would change their behavior
based on the score.
00:55:39.160 --> 00:55:42.109
Herald: All right let's do another
question from the interwebs.
00:55:42.109 --> 00:55:48.489
Q: Yeah, it's actually two questions in
one. How does this system work for Chinese
00:55:48.489 --> 00:55:53.859
people living abroad, or for noncitizens
that do business in China?
00:55:53.859 --> 00:56:00.529
A: Currently the system does not work for
noncitizens that do business in China,
00:56:00.529 --> 00:56:04.700
because it works through the shēnfènzhèng.
You only get a shēnfènzhèng if you're a
00:56:04.700 --> 00:56:10.789
Chinese citizen or you live in China for
10 or more years. So everyone who is not
00:56:10.789 --> 00:56:16.029
Chinese is currently excluded. Chinese
people not living in China, if they have a
00:56:16.029 --> 00:56:21.099
shēnfènzhèng, are on this system, but
there's not a lot of information.
00:56:21.099 --> 00:56:28.859
Herald: All right, mic 4.
Q: Well, we've come a long way since the
00:56:28.859 --> 00:56:34.029
Volkszählungsurteil. Can you
tell us anything about the dynamic in the
00:56:34.029 --> 00:56:43.700
time dimension? How quickly can I regain
credit that was lost? Do you have any
00:56:43.700 --> 00:56:47.490
observations there?
A: So in the Suining system what they
00:56:47.490 --> 00:56:53.519
actually did was they had a very, very
strict period. So if you evaded taxes your
00:56:53.519 --> 00:56:58.599
score would be down for two years and then
it would rebounce. In the Rongcheng
00:56:58.599 --> 00:57:03.359
system, they did not publish this kind of
period. So, my assumption is that it's
00:57:03.359 --> 00:57:08.760
going to be more on a case by case basis.
Because, I looked at the Chinese data, I
00:57:08.760 --> 00:57:13.849
looked at the Chinese policy documents,
and they didn't really, for most of the
00:57:13.849 --> 00:57:20.019
stuff, they didn't say how long it would
count. For the blacklists, which was kind
00:57:20.019 --> 00:57:24.630
of the predecessor that we look at
currently, the way it works is you stay on
00:57:24.630 --> 00:57:29.309
there until whatever the blacklist –
until whatever the reason for the
00:57:29.309 --> 00:57:34.510
blacklist is has been resolved. So, you
stay on there until you send off this
00:57:34.510 --> 00:57:40.039
apology that the judge ordered you to. And
then, usually, you still needed to apply to
00:57:40.039 --> 00:57:44.979
get off. So it doesn't – for blacklists,
it does not work that you automatically
00:57:44.979 --> 00:57:49.410
get off. You need to apply, you need to
show that you've done what they've asked
00:57:49.410 --> 00:57:53.089
you to do, and then you can get off this
blacklist. And I assume it will be a
00:57:53.089 --> 00:57:57.019
similar sort of appeals procedure for the
system.
00:57:57.019 --> 00:58:04.069
Herald: All right. Let's go to mic 2.
Q: Thank you. I just wanted to if looking
00:58:04.069 --> 00:58:08.910
up someone else's data in details, like
position et cetera, does affect your own
00:58:08.910 --> 00:58:12.049
score?
A: Currently, it apparently does not, or
00:58:12.049 --> 00:58:17.369
at least they haven't published that it
does. It might in the future, but most
00:58:17.369 --> 00:58:22.549
likely it's actually behavior that they
want. So they want you to look up other
00:58:22.549 --> 00:58:26.480
people's scores before doing business with
them. They want you to, basically, use
00:58:26.480 --> 00:58:30.069
this to decide who you're going to
associate with.
00:58:30.069 --> 00:58:33.200
Q: Thank you!
Herald: All right, do we have another
00:58:33.200 --> 00:58:38.749
question from the Internet, maybe?
Signal: Yes, I do! Standby... The question
00:58:38.749 --> 00:58:47.569
is, how is this actually implemented for
the offline rural population in China?
00:58:47.569 --> 00:58:53.119
A: Quite easily; not at all at this point.
The idea is, by 2020, that they will
00:58:53.119 --> 00:58:59.089
actually have all of this is implemented.
But even for the offline – or let's say
00:58:59.089 --> 00:59:05.869
offline rural population in China is
getting smaller and smaller. Even in rural
00:59:05.869 --> 00:59:13.039
villages you have about 50-60% of people
that are online. And most of them are
00:59:13.039 --> 00:59:16.049
online via smartphone, and their
smartphone is connected to the
00:59:16.049 --> 00:59:21.309
shēnfènzhèng. So it's not very complicated
to do that for everyone who is online. For
00:59:21.309 --> 00:59:25.989
everyone who's offline, off course, this
is more problematic, but I think the end
00:59:25.989 --> 00:59:31.799
goal is to not have people offline at all.
Herald: All right. Let's jump right back
00:59:31.799 --> 00:59:38.059
to microphone 2, please.
Q: Thank you for the very good and
00:59:38.059 --> 00:59:44.940
frightening talk, so far. At first I have
to correct you in one point. In Germany we
00:59:44.940 --> 00:59:50.279
have a similar system because we have this
tax I.D., which is set from birth on and
00:59:50.279 --> 00:59:58.769
rests 30 years after a person's dead.
Yeah. So we have a lifelong I.D.
00:59:58.769 --> 01:00:02.249
A: You're right. I just... I don't know
mine, so I figured… dismissive sound.
01:00:02.249 --> 01:00:08.130
Q: No problem! But, at least we could
establish a similar system, if we have a
01:00:08.130 --> 01:00:16.150
government which would want it. A question
for you: you mentioned this "guanxi." Is
01:00:16.150 --> 01:00:20.789
it a kind of a social network? I didn't
understand it, really.
01:00:20.789 --> 01:00:26.000
A: Yes, it is a kind of social network,
but one that is a lot more based on
01:00:26.000 --> 01:00:31.930
hierarchies than it is in the West. So you
have people that are above you and people
01:00:31.930 --> 01:00:36.519
that are below you. And the expectation is
that, while it's a quid pro quo, people
01:00:36.519 --> 01:00:41.539
that are above you in the hierarchy will
give you less than you will give to them.
01:00:41.539 --> 01:00:46.529
Q: Aha, okay.
Herald: OK, all right. Unfortunately, we
01:00:46.529 --> 01:00:52.099
are out of time, so, please give another
huge applause for Toni!
01:00:52.099 --> 01:00:54.394
applause
01:00:54.394 --> 01:00:57.842
postroll music
01:00:57.842 --> 01:01:17.000
subtitles created by c3subtitles.de
in the year 2019. Join, and help us!