0:00:00.000,0:00:18.680
35C3 preroll music
0:00:18.680,0:00:22.800
Herald Angel: Alright. Then it's my great[br]pleasure to introduce Toni to you
0:00:22.800,0:00:28.290
She's going to talk about "the Social Credit[br]System," which is, kind of, feels to me
0:00:28.290,0:00:36.010
like a Black Mirror episode coming to[br]life. So, slightly nervous and really
0:00:36.010,0:00:41.115
curious what we're going to learn today.[br]So please give a huge, warm round of
0:00:41.115,0:00:45.606
applause and welcome Toni![br]Applause
0:00:51.918,0:00:55.469
Toni: Good morning, everyone! Before I'm[br]going to be talking I'm going into my talk
0:00:55.469,0:01:00.399
I'm just going to be presenting the[br]Chinese translation streams for everyone
0:01:00.399,0:01:02.709
who doesn't speak English.
0:01:02.709,0:01:17.330
speaks chinese
0:01:17.330,0:01:23.254
Applause
0:01:23.254,0:01:26.536
So because today's talk is about China we[br]figured it would be good to have it
0:01:26.536,0:01:31.780
in Chinese as well. And, I'm going to be[br]talking today about
0:01:31.780,0:01:35.439
the Social Credit system in China, where[br]"the" Social Credit system that you
0:01:35.439,0:01:38.970
always hear about in Western media[br]actually doesn't really exist
0:01:38.970,0:01:43.471
and most of my talk will actually be[br]talking about what all we don't know.
0:01:43.471,0:01:47.030
Which could fill an entire hour or even
0:01:47.030,0:01:51.470
more. But I'm just going to be focusing on[br]some of the most interesting things for
0:01:51.470,0:01:57.560
me. First of all, a little bit about me.[br]I'm an economist, but I'm not I'm not only
0:01:57.560,0:02:03.050
concerned with money. I'm kind of looking[br]at economy, at economics as the study of
0:02:03.050,0:02:08.310
incentives, which means that what I'm[br]really interested in is how humans respond
0:02:08.310,0:02:12.480
to different kind of incentives. I don't[br]believe that humans are completely
0:02:12.480,0:02:18.250
rational. But I do believe that humans do[br]try to maximize what they think is their
0:02:18.250,0:02:23.830
best interest. Now, some words about me: I[br]studied math, economics and political
0:02:23.830,0:02:29.190
science in a couple of different cities[br]all around the world. I spent overall 19
0:02:29.190,0:02:35.140
months in China. Most recently I was there[br]in July on a government scholarship, which
0:02:35.140,0:02:39.640
was really, really interesting, because[br]while there I read all of these Western
0:02:39.640,0:02:43.710
newspaper articles about the Chinese[br]Social Credit system, and I went to a
0:02:43.710,0:02:48.200
pretty good university and I asked them:[br]So what do you think about this system?
0:02:48.200,0:02:52.360
And most of them basically looked at me[br]blankly, and were like: What system, I
0:02:52.360,0:02:56.730
haven't even heard of this! So that was[br]kind of an interesting experience to me
0:02:56.730,0:03:01.670
because in the West it's like this huge,[br]all-encompassing system. And in China,
0:03:01.670,0:03:07.140
most people that aren't directly -- that[br]aren't directly in touch with it actually
0:03:07.140,0:03:11.320
don't know anything about this. I'm[br]broadly interested in the impact of
0:03:11.320,0:03:17.170
technology on society, life, and the[br]economy, obviously, and in my free time I
0:03:17.170,0:03:21.400
do a lot of data science and machine[br]learning with Python and R. So, I thought
0:03:21.400,0:03:27.180
it was quite interesting to look at the[br]Social Credit system, also from this point
0:03:27.180,0:03:31.730
of view because you always heard that it's[br]like this big data initiative, and then
0:03:31.730,0:03:35.200
when coming down to it, what you actually[br]see is that, they don't actually use
0:03:35.200,0:03:40.020
machine learning all that much. They have,[br]basically, a rule based catalog where, if
0:03:40.020,0:03:44.459
you do this you get 50 points, if you do[br]this you get 50 points, and then they
0:03:44.459,0:03:48.959
actually have a lot of people that are[br]reporting on other people's behavior. I'm
0:03:48.959,0:03:53.310
going to be talking about how exactly it[br]looks, later on but I was very, very
0:03:53.310,0:03:56.870
surprised after reading a lot of the[br]Western newspaper articles that were
0:03:56.870,0:04:02.300
basically "Oh, this is this big dystopia,[br]Orwellian, with big data working." And
0:04:02.300,0:04:08.160
then, you read what's actually happening[br]and they have huge lists of "if you
0:04:08.160,0:04:14.700
jaywalk, you get 10 points detracted from[br]you," this kind of thing. If you want to
0:04:14.700,0:04:19.000
get in touch with me you can use Twitter[br]but you can also use different e-mails
0:04:19.000,0:04:23.099
either my professional e-mail or my[br]personal e-mail address, that you can both
0:04:23.099,0:04:27.870
see there. If you have any thoughts on[br]that or are interested in this a little
0:04:27.870,0:04:31.759
more I can give you more resources as[br]well, because obviously today's talk will
0:04:31.759,0:04:38.420
only be scratching on the surface. So,[br]perceptions of the Social Credit System.
0:04:38.420,0:04:43.310
One of the interesting things that I've[br]talked about before was how, in the West
0:04:43.310,0:04:48.410
and in China, the perception is completely[br]different. So in the West, which is from
0:04:48.410,0:04:54.550
financialtimes.com, you see this huge[br]overwhelming guy, and he basically puts
0:04:54.550,0:04:59.280
every Chinese person under a microscope.[br]They're all kind of hunched over, and
0:04:59.280,0:05:03.750
everyone has this score attached to them,[br]and they seem pretty sad and, like, very,
0:05:03.750,0:05:08.700
very Orwellian concept. Whereas, in China,[br]this is actually from a Chinese state
0:05:08.700,0:05:13.810
media, and what it says is, well, we can[br]all live in harmony with this new system
0:05:13.810,0:05:19.139
and all trust each other. And[br]interestingly Chinese people actually
0:05:19.139,0:05:24.310
believe that, to some degree. They believe[br]that technology will fix all this current
0:05:24.310,0:05:29.900
problems in society, especially because,[br]in China currently, trust is a rare
0:05:29.900,0:05:39.400
commodity. And this new system will lead[br]to more efficiency and trust, and a better
0:05:39.400,0:05:43.650
life. And I have a really, really[br]interesting quote from a Western scholar,
0:05:43.650,0:05:47.800
that really summarizes the Western[br]perspective: "What China is doing here is
0:05:47.800,0:05:52.949
selectively breeding its population to[br]select against the trait of critical,
0:05:52.949,0:05:57.490
independent thinking. This may not be the[br]purpose, indeed I doubt it's the primary
0:05:57.490,0:06:02.330
purpose, but it's nevertheless the effect[br]of giving only obedient people the social
0:06:02.330,0:06:07.290
ability to have children, not to mention[br]successful children." This, basically,
0:06:07.290,0:06:11.960
plays with the idea that if you have a low[br]score, currently, in the cities that are
0:06:11.960,0:06:17.320
already testing this system, what happens[br]is, your children can't attend good
0:06:17.320,0:06:23.190
schools. What happens is, you cannot take[br]trains, you cannot take planes. You cannot
0:06:23.190,0:06:31.140
book good hotels. Your life is just very,[br]very inconvenient. And this is by design.
0:06:31.140,0:06:38.620
This is kind of the plan. The Chinese[br]government, they say it's a little
0:06:38.620,0:06:42.340
different, the idea is about changing[br]people's conduct by ensuring they are
0:06:42.340,0:06:48.199
closely associated with it. One of the[br]main things about this system is, there
0:06:48.199,0:06:53.330
isn't very much new data being generated[br]for the system. Instead, what's happening
0:06:53.330,0:07:00.010
is, all the existing data that is already[br]collected about you is, basically,
0:07:00.010,0:07:06.370
combined into one big database for each[br]and every person by your ID number. So, in
0:07:06.370,0:07:10.190
China, once you're born, you get an ID[br]number, which is similar to a Social
0:07:10.190,0:07:13.960
Security number in the U.S. We don't[br]really have a similar concept in Germany,
0:07:13.960,0:07:19.310
and it used to be that your ID number was[br]only necessary for public -- like for
0:07:19.310,0:07:25.810
government stuff, but now you need your ID[br]number for getting a bank account, you
0:07:25.810,0:07:29.580
need your ID number for buying a cell[br]phone, even if it's a prepaid cell phone,
0:07:29.580,0:07:34.640
you still need your ID number. So all your[br]online activity that happens with your
0:07:34.640,0:07:38.430
cell phone is associated with your ID[br]number, which means you can't really do
0:07:38.430,0:07:44.210
anything anonymously, because it's all[br]going back to your ID number. There's a
0:07:44.210,0:07:49.450
couple of predecessors, some of them going[br]actually back to the 1990s, that are
0:07:49.450,0:07:54.880
supposed to be integrated into the new[br]system. One of them, or like two of them
0:07:54.880,0:08:00.800
are blacklists. One of them is a court[br]blacklist. So in China, courts work a
0:08:00.800,0:08:06.560
little bit differently. They tend to like[br]giving you fines, as they do in other
0:08:06.560,0:08:11.521
countries, but they also like giving you[br]"apologies to do." So one of the things,
0:08:11.521,0:08:17.470
if you do something, for example you're a[br]company, your food safety wasn't up to par
0:08:17.470,0:08:21.800
– you have to pay a fine. But in addition[br]to this fine you also have to write a
0:08:21.800,0:08:27.150
public apology letter in the newspaper,[br]how you are very sorry that this happened
0:08:27.150,0:08:31.560
and it won't happen again, and it was a[br]moral failing on your part, and it won't
0:08:31.560,0:08:37.460
happen again. And if you don't do that,[br]you go on this blacklist. Similarly, if
0:08:37.460,0:08:42.698
you take out a line of credit and don't[br]pay it back within three months, or like
0:08:42.698,0:08:47.930
don't don't do any payments for three[br]months, you go on this debtors blacklist.
0:08:47.930,0:08:52.009
If you're on this blacklist, which again[br]is associated with your shēnfènzhèng, so
0:08:52.009,0:08:57.709
your ID number – what happens is you[br]cannot take trains you cannot take planes.
0:08:57.709,0:09:02.309
Your life basically becomes very very[br]inconvenient, your children can't go to
0:09:02.309,0:09:06.749
good public schools, your children can't[br]go to private schools, your children can't
0:09:06.749,0:09:14.589
go to universities, all of these issues[br]are suddenly coming up. There is also a
0:09:14.589,0:09:19.829
company database that's called Credit[br]China which is basically similar to the
0:09:19.829,0:09:24.959
public debtors blacklist but it's[br]basically a credit system a credit score
0:09:24.959,0:09:28.860
for companies. And then there's the credit[br]reference center of the People's Bank of
0:09:28.860,0:09:34.569
China which is a credit score. It was[br]supposed to be like Schufa or like the
0:09:34.569,0:09:42.629
U.S. FICO for individuals. But one of the[br]big problems in China is that there are a
0:09:42.629,0:09:46.769
lot of people that aren't part of the[br]formal economy. A lot of people are
0:09:46.769,0:09:52.529
migrant workers. They get their money in[br]cash. They do not have bank accounts. They
0:09:52.529,0:10:00.410
do not have anything… they do not have rent[br]or utilities or anything like this because
0:10:00.410,0:10:04.610
they live in the country. So they own[br]their own home which they built themselves
0:10:04.610,0:10:10.920
so they didn't even finance it and their[br]home isn't officially theirs because in
0:10:10.920,0:10:16.069
China you can't actually own property.[br]Instead the government leases it to you.
0:10:16.069,0:10:20.839
So there were a lot of people that were[br]not covered in this system, and I think
0:10:20.839,0:10:26.739
the last data that I had was that less[br]than 10 percent of Chinese adult citizens
0:10:26.739,0:10:32.699
were actually in the system and had any[br]sort of exposure to banks, which is very,
0:10:32.699,0:10:38.089
very little. And that meant that people[br]couldn't get credit because banks would
0:10:38.089,0:10:42.220
only give credit to people that were in[br]the system or people where they had
0:10:42.220,0:10:46.649
some sort of handling on whether they[br]would be paid back. Now, the
0:10:46.649,0:10:51.839
implementation details of the new system[br]are very very scarce, but the basic idea
0:10:51.839,0:10:56.079
is that Chinese citizens are divided into[br]trustworthy individuals and what the
0:10:56.079,0:11:02.309
Chinese call "trust breakers". Sometimes[br]you have five different groups, sometimes
0:11:02.309,0:11:05.459
you have two different groups, but in[br]general there's sort of this cut-off:
0:11:05.459,0:11:13.619
above this line it's good and beyond this[br]line it's bad. This is one graphic from
0:11:13.619,0:11:20.409
the Wall Street Journal that just shows[br]some of the inputs that go into the
0:11:20.409,0:11:25.440
system. And one of the things that we see[br]is that the inputs are _crazy_ crazy
0:11:25.440,0:11:35.007
varied. So it is: do you pay income taxes?[br]Do you pay your utility bills on time?
0:11:35.007,0:11:40.201
Do you respect your parents? However they[br]measure that. Do you have a criminal
0:11:40.201,0:11:46.680
record? Do you pay for public[br]transportation or have you been caught
0:11:46.680,0:11:50.479
not paying? What about your friends?
0:11:50.479,0:11:58.439
Do you retweet or use WeChat[br]to distribute sort of information against
0:11:58.439,0:12:04.300
the party, which they call reliability. In[br]actuality it's not about whether it's
0:12:04.300,0:12:08.459
factual, it's about whether it's against[br]the party or not. Where do you buy and
0:12:08.459,0:12:12.019
what do you buy, apparently if you buy[br]diapers it's better than if you buy
0:12:12.019,0:12:16.679
videogames. For your score. Because you[br]know if you buy videogames obviously
0:12:16.679,0:12:20.089
you're not very responsible. And if you[br]buy diapers you have a kid, you are sort
0:12:20.089,0:12:29.019
of conforming to the societal ideal. And[br]then your score is supposed to go into all
0:12:29.019,0:12:33.679
these different categories, you're[br]supposed to have better access to social
0:12:33.679,0:12:38.759
services if your score is good. You're[br]supposed to have better access to internet
0:12:38.759,0:12:43.309
services. So in theory the idea is that at[br]one point if your score is too bad, you're
0:12:43.309,0:12:47.890
not allowed to use WeChat anymore. You're[br]not allowed to use Alibaba anymore. You
0:12:47.890,0:12:57.110
can't become a government worker. You can[br]not take planes and high speed trains. You
0:12:57.110,0:13:04.519
can not get a passport. And your insurance[br]premiums will go up. So it's supposed to
0:13:04.519,0:13:12.540
be this really really big, overwhelming[br]system. But in actuality what they say
0:13:12.540,0:13:19.459
their stated goals are, is "it's a[br]shorthand for a broad range of efforts to
0:13:19.459,0:13:24.790
improve market security and public safety[br]by increasing integrity and mutual trust
0:13:24.790,0:13:31.899
in society." So one idea is to allocate[br]resources more efficiently. Resource
0:13:31.899,0:13:39.339
allocation in China is a pretty big[br]problem, because people grow up with:
0:13:39.339,0:13:43.940
There's 1.3 billion people. So there's -[br]it's always going to be scarce. And a lot
0:13:43.940,0:13:50.160
of stuff is – people grow up with this[br]idea that it's just very very scarce, and
0:13:50.160,0:13:54.639
current distribution strategies, which are[br]mostly financially based but also often
0:13:54.639,0:13:59.360
guanxi-based, don't really seem fair. For[br]example, public transport in China is
0:13:59.360,0:14:04.769
highly subsidized, which means that the[br]price does not reflect whether – does not
0:14:04.769,0:14:10.249
reflect true scarcity. So currently the[br]way it works is in theory it's first come
0:14:10.249,0:14:15.019
first serve, in practice there's people[br]that are buying up all the tickets for,
0:14:15.019,0:14:19.319
for example, the high speed train from[br]Shanghai to Beijing and then selling it at
0:14:19.319,0:14:23.120
a profit, or selling it to certain[br]companies that have good ties to the
0:14:23.120,0:14:28.040
government. That seems very unfair. So the[br]new system is supposed to distribute them
0:14:28.040,0:14:33.670
more fairly and more efficiently. The[br]other thing is restoring trust in people.
0:14:33.670,0:14:38.209
Perceived inter-personal trust and trust[br]in institutions is extremely low in China.
0:14:38.209,0:14:43.119
If you're from Germany, you might have[br]heard that there is Chinese gangs
0:14:43.119,0:14:48.089
basically buying up German milk powder and[br]selling it in China. This is actually
0:14:48.089,0:14:54.559
happening, because in 2008 there was a big[br]scandal with laced milk powder. And ever
0:14:54.559,0:14:59.380
since then, anyone who can afford it does[br]not use Chinese milk powder, because they
0:14:59.380,0:15:03.730
don't trust the government, or the[br]regulations, the firms, enough to buy
0:15:03.730,0:15:09.929
Chinese milk powder so they are actually[br]importing this. And the big irony is:
0:15:09.929,0:15:15.519
sometimes this milk powder is produced in[br]China, exported to Germany, and then
0:15:15.519,0:15:22.249
exported back to China. The Social Credit[br]system is then supposed to identify those
0:15:22.249,0:15:27.449
that deserve the trust. And the third[br]point is sort of a reeducation of people.
0:15:27.449,0:15:32.959
The idea is: they want to make people in[br]the image that the Communist Party thinks
0:15:32.959,0:15:40.529
people should be. And one additional way[br]to the punishments and rewards this could
0:15:40.529,0:15:45.420
work, is the feeling of being surveyed.[br]Because you can't do anything anonymously,
0:15:45.420,0:15:49.470
you will automatically adapt your behavior[br]because you know someone is watching you
0:15:49.470,0:15:55.500
all the time, and this is how a lot of the[br]Chinese firewall actually works, because
0:15:55.500,0:16:00.920
most people I know that are sort of more–[br]more educated, they know ways to
0:16:00.920,0:16:05.019
circumvent the Chinese firewall, but they[br]also know that they're always being
0:16:05.019,0:16:09.399
watched, so they don't do that because,[br]you know, they're being watched, so they
0:16:09.399,0:16:18.449
self– they censorship– they censor[br]themselves. As I said before, allocation
0:16:18.449,0:16:23.619
of scarce resources so far is mainly[br]through financial guanxi channels. guanxi
0:16:23.619,0:16:29.290
is basically an all permeating network of[br]relationships with a clear status
0:16:29.290,0:16:34.399
hierarchy. So if I attend a school,[br]everyone who also attended this school
0:16:34.399,0:16:40.319
will be sort of in my guanxi network. And[br]there's this idea that we will have a
0:16:40.319,0:16:45.679
system where we are all in-group, and in-[br]group we trust each other and we do favors
0:16:45.679,0:16:51.120
for each other, and everyone who's outside[br]of my immediate group I don't trust and I
0:16:51.120,0:16:56.350
don't do favors for. And in some ways the[br]guanxi system right now is a substitute
0:16:56.350,0:17:02.639
for formal institutions in China. For[br]example if you want a passport right now.
0:17:02.639,0:17:07.359
You can of course apply for passports[br]through regular channels, which might take
0:17:07.359,0:17:12.390
months and months. Or you can apply for a[br]passport through knowing someone and
0:17:12.390,0:17:16.750
knowing someone, which might take only two[br]days. Whereas in Germany you have these
0:17:16.750,0:17:22.510
very regular, formal institutions, in[br]China they still use guanxi. But,
0:17:22.510,0:17:26.849
increasingly especially young people find[br]that guanxi are very unfair, because a lot
0:17:26.849,0:17:31.560
of these are: where you went to school,[br]which is determined by where you're born,
0:17:31.560,0:17:37.200
who your parents are, and all these[br]things. Another thing that's important to
0:17:37.200,0:17:42.170
understand because: the system works[br]through public shaming. And in a lot of
0:17:42.170,0:17:48.430
western society we can't really imagine[br]that, like, I wouldn't really care if my
0:17:48.430,0:17:54.700
name was in a newspaper of someone who[br]jaywalked for example. It would be: oh
0:17:54.700,0:17:59.631
well, that's okay. But in China this is[br]actually a very very serious thing. So
0:17:59.631,0:18:05.030
saving face is very very important in[br]China. And when I went to school there I
0:18:05.030,0:18:13.080
actually – we had this dormitory, and it[br]was an all foreigners dormitory, where the
0:18:13.080,0:18:17.680
staff that were responsible for the[br]dormitory felt that foreigners were not
0:18:17.680,0:18:24.660
behaving in the way they should. So their[br]idea was to put the names, the pictures,
0:18:24.660,0:18:29.250
and the offenses of the foreigners in the[br]elevator to shame them publicly. So for
0:18:29.250,0:18:33.690
example if you brought a person of the[br]opposite sex to your room, they would put
0:18:33.690,0:18:39.910
your name, your offense and your room[br]number in the elevator. And of course this
0:18:39.910,0:18:44.630
didn't work because for a lot of western[br]people it was basically like: "oh well I'm
0:18:44.630,0:18:49.140
going to try to be there as often as[br]possible because this is like a badge of
0:18:49.140,0:18:53.670
honor for me" and the Chinese people they[br]figured "well this is really really shame
0:18:53.670,0:18:58.080
and I'm losing my face". She brought[br]alcohol. So this didn't really work at
0:18:58.080,0:19:06.390
all. But this is kind of the mindset that[br]is behind a lot of these initiatives. As I
0:19:06.390,0:19:12.230
said there's a lot of problems with – we[br]don't really know what's going to happen.
0:19:12.230,0:19:17.000
And one of the ways that we can see what[br]might happen is actually to look at pilot
0:19:17.000,0:19:24.000
systems. China has – or like ever since[br]the Communist Party took hold – the
0:19:24.000,0:19:28.630
Chinese government has tried a lot of[br]policy experimentation. So whenever they
0:19:28.630,0:19:33.850
try a new policy, they don't roll it out[br]all over, but they choose different pilot
0:19:33.850,0:19:38.890
cities or pilot districts, and then they[br]choose "oh well this is the district where
0:19:38.890,0:19:42.900
I'm going to be trying this system and I'm[br]going to be trying another system in
0:19:42.900,0:19:48.360
another district or city". And this is[br]also what they did for the, or what
0:19:48.360,0:19:52.520
they're doing for the Social Credit[br]system. Now I have three systems that I
0:19:52.520,0:19:58.300
looked at intensively for this[br]presentation, overall there's about 70
0:19:58.300,0:20:07.030
that I know of - the Suining system,[br]Suining is a city in China, the Rongcheng
0:20:07.030,0:20:11.410
system, another city in China and Sesame[br]Credit. Sesame Credit is a commercial
0:20:11.410,0:20:16.430
system from Alibaba - I assume everyone[br]knows Alibaba, the're basically the
0:20:16.430,0:20:22.770
Chinese Amazon, except they're bigger and[br]have more users and make more money,
0:20:22.770,0:20:27.400
actually. And they have their own little[br]system. One of the problems with this kind
0:20:27.400,0:20:31.620
of system that I found when I tried[br]modeling it, was that it's a very very
0:20:31.620,0:20:38.930
complex system and small changes in input[br]actually changed the output significantly.
0:20:38.930,0:20:43.550
So when they try– usually when they try[br]this pilot system they basically have a
0:20:43.550,0:20:47.850
couple of pilots, then they choose the[br]pilot that is best and they roll it out
0:20:47.850,0:20:52.560
all over. But for this kind of thing,[br]where you have a lot of complex issues, it
0:20:52.560,0:20:59.410
might not be the best way to do that. The[br]Suining system is actually considered the
0:20:59.410,0:21:05.980
predecessor of all current systems. It had[br]a focus on punishment, and it was quite
0:21:05.980,0:21:12.000
interesting. At the beginning of the trial[br]period they published a catalogue of
0:21:12.000,0:21:17.260
scores and consequences. Here is an[br]example. This is basically taken from this
0:21:17.260,0:21:22.640
catalog. So if you took out bank loans and[br]didn't repay them, you got deducted 50
0:21:22.640,0:21:26.910
points. Everyone started with 1000[br]points for this system. If you didn't pay
0:21:26.910,0:21:32.690
back your credit cards you also got[br]deducted 50 points. If you evaded taxes,
0:21:32.690,0:21:43.890
also 50 points. If you sold fake goods, 35[br]points were deducted. And actually the
0:21:43.890,0:21:50.450
system was abolished I think in 2015,[br]2016, because all the Chinese state media
0:21:50.450,0:21:56.490
and also a lot of Internet citizens talked[br]about how it's an Orwellian system and how
0:21:56.490,0:22:03.020
it's not a good system, because it's all[br]very centralized and everything that you
0:22:03.020,0:22:08.880
do is basically recorded centrally. But[br]Creemers writes: "Nonetheless, the Suining
0:22:08.880,0:22:13.090
system already contained the embryonic[br]forms of several elements of subsequent
0:22:13.090,0:22:17.290
social credit initiatives: The notion of[br]disproportional disincentives against rule
0:22:17.290,0:22:22.500
breaking, public naming and shaming of[br]wrongdoers, and most importantly, the
0:22:22.500,0:22:26.900
expansion of the credit mechanism outside[br]of the market economic context, also
0:22:26.900,0:22:29.760
encompassing compliance with[br]administrative regulations and urban
0:22:29.760,0:22:34.300
management rules." So one of the things[br]that is difficult for especially German
0:22:34.300,0:22:40.191
speakers is that credit in Chinese,[br]xìnyòng, means credit as in "loan", but
0:22:40.191,0:22:46.350
also means credit as in "trust". So the[br]Social Credit System is one way of trying
0:22:46.350,0:22:52.240
to conflate those two – the economic[br]credit and the trust credit – into one big
0:22:52.240,0:23:01.670
system. But the Suining system basically[br]failed. So, they adapted the system and
0:23:01.670,0:23:07.220
are now practicing a new kind of system,[br]the Rongcheng system. Whenever you read a
0:23:07.220,0:23:11.310
newspaper article on the social credit[br]system in the west, most people went to
0:23:11.310,0:23:15.060
Rongcheng because they just received a[br]couple of awards from the Chinese
0:23:15.060,0:23:22.230
government for being so advanced at this[br]social credit thing. But it's very
0:23:22.230,0:23:25.990
difficult to call this "one system"[br]because there's actually many many
0:23:25.990,0:23:31.020
intertwined systems. There is one city[br]level system, where city level offenses
0:23:31.020,0:23:38.520
are recorded. For example tax evasion, and[br]there's a couple of rules. If you evade
0:23:38.520,0:23:43.730
taxes your score goes down 50. But then if[br]you live in one neighborhood your score
0:23:43.730,0:23:47.620
might go up for volunteering with the[br]elderly. If you live in another
0:23:47.620,0:23:54.200
neighborhood your score might go up for,[br]for example, planting some trees in
0:23:54.200,0:23:59.180
your garden or backyard. So depending on[br]your neighborhood, your score might be
0:23:59.180,0:24:05.590
different. If you work for a– if you work[br]for a taxi cab company, for example, they
0:24:05.590,0:24:10.780
also have their own little score system[br]and your score might go up if you get good
0:24:10.780,0:24:16.570
reviews from your drive…, from your[br]passengers. Your score might go down if
0:24:16.570,0:24:25.780
you don't follow traffic rules, these[br]kinds of things. There are designated
0:24:25.780,0:24:31.550
scorekeepers at each level. So, each[br]district chooses a couple of people who
0:24:31.550,0:24:38.330
are responsible for passing on the[br]information to the next higher level,
0:24:38.330,0:24:42.870
about who did what. There is supposed to[br]be an official appeals procedure, so
0:24:42.870,0:24:46.940
whenever you score changes you're supposed[br]to be notified, but apparently that's not
0:24:46.940,0:24:54.490
happening at this point for most people.[br]Again, it's a system of data sharing, and
0:24:54.490,0:24:58.520
one thing that they haven't really[br]disclosed yet is what kind of data is
0:24:58.520,0:25:04.760
shared. Are they only sharing the points,[br]so if I'm in a district and I plant some
0:25:04.760,0:25:11.480
trees, does the central system get the[br]information "person A planted some trees,"
0:25:11.480,0:25:16.300
or does the central system get the[br]information "person A got 5 points?" We
0:25:16.300,0:25:21.570
don't know at this point. And it would[br]mean something very different for how the
0:25:21.570,0:25:27.030
system could be used. But still the end[br]result, at this point, is that there's one
0:25:27.030,0:25:31.360
score. So you have one central score and[br]it's kind of– there's all these different
0:25:31.360,0:25:35.780
smaller systems that go into this score.[br]But at the end, everyone has one central
0:25:35.780,0:25:44.530
score, and currently about 85 percent of[br]people are between 950 and 1050. So you
0:25:44.530,0:25:49.990
start off with a thousand – and those are[br]basically the normal people – and then
0:25:49.990,0:25:58.940
anyone above a 1050 is considered a[br]trustworthy person, and anyone below 1050
0:25:58.940,0:26:05.340
is considered a trust-breaker. And, as[br]I've said before, with the naming and
0:26:05.340,0:26:12.310
shaming and all these things, what you can[br]actually see here is a billboard with the
0:26:12.310,0:26:17.520
best trustworthy families in Rongcheng. So[br]these are the families that have the
0:26:17.520,0:26:23.590
highest scores, for example. Sesame Credit[br]is a little different. It's the only
0:26:23.590,0:26:27.990
system that actually uses machine learning[br]and artificial intelligence to determine
0:26:27.990,0:26:32.980
the outputs. In Rongcheng, for example,[br]they have artificial intelligence, they
0:26:32.980,0:26:37.120
have computer vision, for the most part,[br]and the computer vision cameras they
0:26:37.120,0:26:42.220
decide– they try to recognize you when you[br]jaywalk. And then when they recognize you
0:26:42.220,0:26:48.250
when jaywalking, you get a small SMS;[br]"well, we just saw you jaywalking, your
0:26:48.250,0:26:56.810
score is now dropping." But how the score[br]develops, depending on your jaywalking,
0:26:56.810,0:27:01.120
isn't really determined by machine[br]learning or artificial intelligence.
0:27:01.120,0:27:06.840
Instead, it's determined by rules. You[br]know: one time jaywalking deducts five
0:27:06.840,0:27:12.460
points, and this is stated somewhere.[br]Sesame Credit doesn't work like that.
0:27:12.460,0:27:19.810
Instead it uses a secret algorithm, and[br]the way– I talked to some people that work
0:27:19.810,0:27:25.330
for Sesame Credit or for Alibaba, and the[br]way they described it was; they basically
0:27:25.330,0:27:33.260
clustered people based on behavior, then[br]gave scores to these clusters, and
0:27:33.260,0:27:40.700
then afterwards, did basically reverse[br]engineered their own score, using machine
0:27:40.700,0:27:45.950
learning, so that whenever something new[br]happens, you can move to a different
0:27:45.950,0:27:54.330
cluster. This Sesame Credit was actually[br]refused accreditation as a credit score in
0:27:54.330,0:28:03.400
2017, so banks are not allowed to use the[br]Sesame Credit score for your– to use the
0:28:03.400,0:28:09.190
Sesame Credit score to determine whether[br]they give you loans or not. Because Sesame
0:28:09.190,0:28:13.770
Credit is quite ingenious – obviously[br]Alibaba wants to keep you within their
0:28:13.770,0:28:19.920
platform – so if you buy using Alibaba and[br]using Alipay, your score goes up. If you
0:28:19.920,0:28:29.090
buy using Weechatpay, which is a competing[br]platform, your score goes down. This uses
0:28:29.090,0:28:34.000
many of the same rewards mechanisms of the[br]official government systems, and this is
0:28:34.000,0:28:38.390
just an illustration of what kind of[br]scores you can have, apparently your
0:28:38.390,0:28:46.070
scores can go between 350 and 850, and in[br]Chinese there's basically five different
0:28:46.070,0:28:55.809
levels. So 385 is a "trust-breaker" or[br]"missing trust". And then 731 is "trust is
0:28:55.809,0:29:06.170
exceedingly high". So one way I tried to[br]approach this issue was through agent-
0:29:06.170,0:29:10.100
based modeling. Social Credit System is[br]individual level, but what we're really
0:29:10.100,0:29:13.400
interested in, or what I'm really[br]interested in, is actually societal-level
0:29:13.400,0:29:19.040
consequences. So if everyone gets this[br]score, what does that mean for society?
0:29:19.040,0:29:24.050
And agent-based modeling works quite well[br]for that, because it allows us to imbue
0:29:24.050,0:29:28.960
agents with some sort of rationality, but[br]with a bounded rationality. What does
0:29:28.960,0:29:32.800
bounded rationality mean? Usually in[br]economics people assume agents are
0:29:32.800,0:29:38.070
completely rational, so they are profit[br]maximizers, they have all the information.
0:29:38.070,0:29:45.380
But in reality, agents don't have all the[br]information, they have a lot of issues
0:29:45.380,0:29:51.100
with keeping stuff in their mind. So a lot[br]of the time, they won't choose the best
0:29:51.100,0:29:57.760
thing in the world, but they choose the[br]best thing that they see. And bounded
0:29:57.760,0:30:02.040
rationality allows us to account for this[br]thing. It allows us to account for
0:30:02.040,0:30:08.380
heuristics and these things. And what I[br]did is I took the propensity for specific
0:30:08.380,0:30:12.320
behavior from current state of the art[br]research, mostly from behavioral
0:30:12.320,0:30:17.900
economics. For example, I looked at tax[br]evasion, and I looked at who is likely to
0:30:17.900,0:30:24.630
evade taxes in a system, and then[br]obviously there was some stochastic –
0:30:24.630,0:30:30.809
some chance element. But the[br]distribution that I chose is related to
0:30:30.809,0:30:37.680
the current research. And I also checked[br]that my model has similar results to the
0:30:37.680,0:30:45.400
Rongcheng model, which I modeled at at the[br]beginning. So on average 87% of my users
0:30:45.400,0:30:49.240
have a score of within 10 percent of the[br]original score, which is also the data
0:30:49.240,0:31:00.740
that Rongcheng city actually publishes.[br]Now, for the most part, I compared design
0:31:00.740,0:31:05.220
choices in two axes. One of them was a[br]centralized system versus a multi-level
0:31:05.220,0:31:10.710
system, and a rule-based system versus a[br]machine learning system. The centralized
0:31:10.710,0:31:20.650
system is basically: you have a central –[br]all the information is kept centrally, and
0:31:20.650,0:31:27.190
everyone in China, or wherever, in[br]Rongcheng has the exact same scoring
0:31:27.190,0:31:35.520
opportunities. Now, if you have a[br]centralized system the clear expectations
0:31:35.520,0:31:39.600
were pretty good. But, at the same time,[br]the acceptance from the population was
0:31:39.600,0:31:46.590
really, really low, which they found[br]during the Suining experiment. And
0:31:46.590,0:31:50.540
there's also the problem of a single point[br]of failure. Who decides the central
0:31:50.540,0:31:59.080
catalog, and, depending on who, sort of,[br]has the power, it kind of, just,
0:31:59.080,0:32:05.100
reproduces power structures. So because[br]you have this central catalog, the same
0:32:05.100,0:32:10.760
people that are in power centrally, they[br]are basically deciding some sort of score
0:32:10.760,0:32:15.340
mechanism that works for them very well,[br]so that they and their family will have
0:32:15.340,0:32:23.280
high scores. And multi-level system has[br]the advantage that local adaptation kind
0:32:23.280,0:32:29.080
of works, and there's sort of many points[br]of failure. But in my model, when I
0:32:29.080,0:32:36.860
allowed locals to basically set their own[br]rules, what happened was that they
0:32:36.860,0:32:42.520
competed. So, it started out being this[br]district of Rongcheng, for example, and
0:32:42.520,0:32:46.120
this district of Rongcheng, they compete[br]for the best people that they want to
0:32:46.120,0:32:52.000
attract, and suddenly you have this kind[br]of race to the bottom, where people want
0:32:52.000,0:32:58.090
to move where they wouldn't be prosecuted,[br]so they move to places where there's less
0:32:58.090,0:33:04.000
cameras, for example. At the same time,[br]there's many points of failure, especially
0:33:04.000,0:33:13.500
the way it's currently set up, with people[br]reporting data to the next high level.
0:33:13.500,0:33:19.450
And, a lot of the time, what we have[br]actually seen in Rongcheng, was that they
0:33:19.450,0:33:24.360
reported data on people they didn't like[br]more than data on people they did like.
0:33:24.360,0:33:29.390
Or, their families got better scores than[br]people they didn't know. So it also kind
0:33:29.390,0:33:41.320
of reproduced these biases. The rule based[br]system has the advantage that people were
0:33:41.320,0:33:46.530
more prone to adapt their behavior,[br]because they actually knew what they
0:33:46.530,0:33:50.630
needed to do in order to adapt their[br]behavior. But the score didn't really
0:33:50.630,0:33:54.050
correlate with the important[br]characteristics that they actually cared
0:33:54.050,0:34:01.610
about. And, as opposed to in this machine[br]learning system, you know how in Germany
0:34:01.610,0:34:06.910
we don't really know the Schufa algorithm.[br]And I, for example, don't exactly know
0:34:06.910,0:34:12.139
what I could do in order to improve my[br]Schufa score. And this is a similar system
0:34:12.139,0:34:16.969
in China with the Sesame Credit score. A[br]lot of people don't really – they say,
0:34:16.969,0:34:21.800
"well I really want to adapt my behavior[br]to the score, to improve my score, but
0:34:21.800,0:34:29.119
when I tried doing that my score actually[br]got worse." And you can have different
0:34:29.119,0:34:36.440
biases, that I'm going to be talking about[br]in a little bit. There's also this big
0:34:36.440,0:34:42.579
problem of incentive mismatch. So, the[br]decentralized, rules-based systems like
0:34:42.579,0:34:47.299
Rongcheng, which is the system that I[br]analyzed the most. Why, because I believe
0:34:47.299,0:34:51.530
this is the system that we're moving[br]towards right now. Because Rongcheng won a
0:34:51.530,0:34:57.018
lot of awards. So the Chinese government,[br]the way they usually work is, they try
0:34:57.018,0:35:01.710
pilots, then they choose the best couple[br]of systems, they give them awards, and
0:35:01.710,0:35:06.890
then they roll out the system nationwide.[br]So I assume that the system that's going
0:35:06.890,0:35:12.990
to be – the system in the end will be[br]similar to the Rongcheng system. Now, one
0:35:12.990,0:35:19.670
problem that I actually saw in my[br]simulation was that you could have this
0:35:19.670,0:35:24.960
possible race to the bottom. There's also[br]this conflict of interest in those that
0:35:24.960,0:35:29.710
set the rules, because a lot of the time,[br]the way it works is, you have your
0:35:29.710,0:35:35.940
company, and your company, you, in[br]combination with your party leaders,
0:35:35.940,0:35:43.760
actually decide on the rules for the score[br]system. But the scores of all your
0:35:43.760,0:35:48.319
employees actually determines your[br]company's score. If you employ a lot of
0:35:48.319,0:35:52.589
people with high scores you get a better[br]score. So you will have this incentive to
0:35:52.589,0:35:57.400
give out high scores and to make sure that[br]everyone gets high scores. But at the same
0:35:57.400,0:36:04.720
time the government has an incentive for[br]scores to be comparable. So there's a lot
0:36:04.720,0:36:10.020
of incentives mismatch. The government[br]also has the incentive to keep false
0:36:10.020,0:36:15.539
negatives down, but they actually, the way[br]the Chinese system currently works is,
0:36:15.539,0:36:22.589
they emphasize catching trust-breakers[br]more than rewarding trust-follow... or
0:36:22.589,0:36:28.750
trustworthy people. So, false positives,[br]for them, are less important, but false
0:36:28.750,0:36:34.359
positives erode the trust in the system,[br]and they lead to a lot less behavioral
0:36:34.359,0:36:40.740
adaptation. I was actually able to show[br]this using some nudging research that
0:36:40.740,0:36:47.789
showed that as soon as you introduce an[br]error probability and you can be caught
0:36:47.789,0:36:54.530
for something that you didn't do, your[br]probability of changing your behavior
0:36:54.530,0:37:02.119
based on this score is actually lower. And[br]in Rongcheng, one of the perverse things
0:37:02.119,0:37:09.880
that they're doing is, you can donate[br]money to the party or to, like, party
0:37:09.880,0:37:17.339
affiliated social services, and this will[br]give you points, which is kind of an
0:37:17.339,0:37:24.269
indulgence system. Which is quite[br]interesting, especially because a lot of
0:37:24.269,0:37:32.690
these donation systems work in a way that[br]you can donate 50000 renminbi and you get
0:37:32.690,0:37:36.740
50 points, and then you donate another[br]50000 renminbi and you get another 50
0:37:36.740,0:37:44.400
points. So you can basically donate a lot[br]of money and then behave however you want,
0:37:44.400,0:37:56.220
and still get a good score. And the trust[br]in other people can actually go down even
0:37:56.220,0:38:00.559
more in this system, because suddenly you[br]only trust them because of their scores,
0:38:00.559,0:38:05.460
and the current system is set up so that[br]you can actually look up scores of
0:38:05.460,0:38:09.730
everyone that you want to work with, and[br]if they don't have a score high enough
0:38:09.730,0:38:15.140
then suddenly you don't want to work with[br]them. The trust in the legal system can
0:38:15.140,0:38:22.259
also decrease, actually. Why? Because trust[br]in the legal system in China is already
0:38:22.259,0:38:26.039
low, and a lot of the things, like[br]jaywalking, they're already illegal in
0:38:26.039,0:38:30.380
China, as they are here, but no one cares.[br]And suddenly, you have this parallel
0:38:30.380,0:38:37.269
system that punishes you for whatever.[br]But, why don't you just try to fix the
0:38:37.269,0:38:45.190
legal system, which would be my approach.[br]Suddenly, illegal activity could happen
0:38:45.190,0:38:51.720
more offline, and this is one of those[br]things that is quite interesting. In
0:38:51.720,0:38:58.059
countries that we've seen that have moved[br]towards mobile payments, and away
0:38:58.059,0:39:05.069
from cash, you see less robberies but you[br]don't actually see less crime. Instead you
0:39:05.069,0:39:12.430
see more new types of crime. So, you see[br]more credit card fraud, you see more phone
0:39:12.430,0:39:19.390
robberies, these kinds of things. And this[br]is also where things could move in the
0:39:19.390,0:39:29.710
Chinese case. One major problem is also[br]that this new system – I've talked a
0:39:29.710,0:39:34.499
little bit about this one, but – it can[br]introduce a lot of new bias, and reproduce
0:39:34.499,0:39:44.710
the bias even more. So, for example, China[br]is a country of 55 minorities. The Han are
0:39:44.710,0:39:50.249
a big majority, they have about 94 percent[br]of the population. So any computer vision
0:39:50.249,0:39:58.020
task, we've shown, that they are really,[br]really bad at discriminating between
0:39:58.020,0:40:04.930
individuals in smaller ethnic groups. In[br]the U.S., most computer vision tasks
0:40:04.930,0:40:09.920
perform worse for African-Americans, they[br]perform worse for women, because all of
0:40:09.920,0:40:16.930
the training sets are male and white, and[br]maybe Asian. In China, all of these tasks
0:40:16.930,0:40:26.609
are actually performing worse for ethnic[br]minorities, for the Uyghurs, for example.
0:40:26.609,0:40:32.460
And one way that they could try to abuse[br]the system is to basically just – what
0:40:32.460,0:40:38.210
they're also doing already in Xinjiang is[br]– to basically just identify, "oh this is
0:40:38.210,0:40:45.979
a person of the minority, well I'm just[br]going to go and check him or her more
0:40:45.979,0:40:50.210
thoroughly." This is actually what happens[br]in Xinjiang. If you're in Xinjiang and you
0:40:50.210,0:40:59.250
look like a Turkish person, or like from[br]Turkmenistan, from a Turkish people, you
0:40:59.250,0:41:04.210
are a lot more likely to be questioned.[br]You're a lot more likely to be stopped and
0:41:04.210,0:41:12.579
they ask you or require you to download[br]spyware on your phone. And this is
0:41:12.579,0:41:17.660
currently what happens and this new kind[br]of system can actually help you with that.
0:41:17.660,0:41:24.710
I've said that it can reproduce these kind[br]of power structures, and now obviously we
0:41:24.710,0:41:29.890
all know neutral technology doesn't really[br]exist, but in the Chinese case, in the
0:41:29.890,0:41:33.529
social credit case, they don't even[br]pretend – they always say "well, this
0:41:33.529,0:41:37.349
is neutral technology and it's all a lot[br]better," but actually it's the people
0:41:37.349,0:41:43.970
currently in power, they decide on what[br]gives you point and what deducts points
0:41:43.970,0:41:50.029
for you. Another problem, currently the[br]entire system is set up in a way that it
0:41:50.029,0:41:54.619
all goes together with your shēnfènzhèng,[br]with your I.D. card. What if you don't
0:41:54.619,0:41:59.410
have an I.D. card? That's foreigners for[br]one. But it's also people in China that
0:41:59.410,0:42:05.479
were born during the one child policy and[br]were not registered. There's quite a lot
0:42:05.479,0:42:09.200
of them, actually. They're not registered[br]anywhere and suddenly they can't do
0:42:09.200,0:42:13.700
anything, because they don't have a score,[br]they can't get a phone, they can't do
0:42:13.700,0:42:20.880
anything, really. And part of the push[br]with this social credit system is to go
0:42:20.880,0:42:26.779
away from cash, actually. So if you need[br]to use your phone to pay, but for your
0:42:26.779,0:42:29.519
phone you need your shēnfènzhèng.[br]If you don't have a shēnfènzhèng,
0:42:29.519,0:42:32.569
well, tough luck for you.
0:42:32.569,0:42:38.680
And currently the system in Rongcheng[br]is set up in a way that you can check
0:42:38.680,0:42:44.900
other people's scores and you can also see[br]what they lose points for. So you can
0:42:44.900,0:42:49.779
actually, sort of, choose to discriminate[br]against people that are gay, for example,
0:42:49.779,0:42:53.049
because they might have lost points for[br]going to a gay bar, which you can lose
0:42:53.049,0:43:02.130
points for. Another big issue, currently,[br]is data privacy and security. Personal
0:43:02.130,0:43:07.250
data is grossly undervalued in China. If[br]you ask a Chinese person, "what do you
0:43:07.250,0:43:14.690
think, how much is your data worth?," they[br]say "what data? I don't have data." And,
0:43:14.690,0:43:19.130
currently, the way it works is, if you[br]have someone's ID number, which is quite
0:43:19.130,0:43:24.890
easy to find out, you can actually buy[br]access to a lot of personal information
0:43:24.890,0:43:31.359
for a small fee. So you pay about 100[br]euros and you get all hotel bookings of
0:43:31.359,0:43:35.960
the last year, you get information of who[br]booked these hotels with them, you get
0:43:35.960,0:43:40.829
information of where they stay, you get[br]train bookings, you get access to all of
0:43:40.829,0:43:47.200
the official databases for this one[br]person. And for another 700 renminbi you
0:43:47.200,0:43:52.829
can actually get live location data, so[br]you can get the data of where this person
0:43:52.829,0:43:56.400
is right now, or where his or her phone is[br]right now, but if you've ever been to
0:43:56.400,0:44:03.440
China you know that where the phone is,[br]usually, the people aren't far. Supchina
0:44:03.440,0:44:08.500
actually did an experiment where a couple[br]of journalists tried buying that, because
0:44:08.500,0:44:14.190
it's actually these kind of services are[br]offered on weechat, pretty publicly. And
0:44:14.190,0:44:26.359
you can just buy them, quite easily. So[br]one additional thing that I looked at is,
0:44:26.359,0:44:30.579
because one of the things that is quite[br]interesting is, you have this idea of
0:44:30.579,0:44:39.309
credit as twofold. Credit is trust credit[br]but credit is also loan credit, and what
0:44:39.309,0:44:43.730
if credit institutions actually use this[br]unified credit score to determine credit
0:44:43.730,0:44:49.059
distribution? The idea is that it's[br]supposed to lead to reduced information
0:44:49.059,0:44:55.471
asymmetry, obviously, so fewer defaults[br]and overall more credit creation. New
0:44:55.471,0:44:59.549
people are supposed to get access to[br]credit, and there's supposed to be less
0:44:59.549,0:45:04.589
shadow banking. But what actually happens?[br]I'm not going to be talking about how I
0:45:04.589,0:45:08.619
set up the model but just about my[br]results. If you have this kind of score
0:45:08.619,0:45:14.369
that includes credit information but also[br]includes morally good – or measures of
0:45:14.369,0:45:18.780
being morally good – what you have is, in[br]the beginning, about 30 percent more
0:45:18.780,0:45:23.960
agents get access to credit, and[br]especially people that previously have not
0:45:23.960,0:45:29.710
gotten credit access suddenly have credit[br]access. But the problem is that this
0:45:29.710,0:45:36.170
social credit score that correlates all of[br]these different issues, it correlates only
0:45:36.170,0:45:41.809
very, very weakly with repayment ability[br]or repayment wishes, and thus suddenly you
0:45:41.809,0:45:47.880
have all of these non-performing loans.[br]You have – and what we see is sort of
0:45:47.880,0:45:51.960
like – we have non-performing loans.[br]Banks give out less loans because they
0:45:51.960,0:45:59.400
have so many non-performing loans, and[br]then the non-performing loans are written
0:45:59.400,0:46:03.999
off, and suddenly banks give out more[br]loans. But you have this oscillating
0:46:03.999,0:46:09.239
financial system, where you give out a lot[br]of loans, a lot of them are non-
0:46:09.239,0:46:13.019
performing, then you give out a lot of[br]loans again. And this is very, very
0:46:13.019,0:46:19.150
vulnerable to crisis. If you have a real[br]economic crisis during the time where non-
0:46:19.150,0:46:24.569
performing loans are high, then a lot of[br]banks will actually default, which is
0:46:24.569,0:46:29.920
very, very dangerous for a financial[br]system as nationed as the Chinese one.
0:46:29.920,0:46:36.799
Now, what are some possible corrections?[br]You could create a score that basically is
0:46:36.799,0:46:41.259
the same as the Schufa score. So that it[br]looks only at credit decisions, but
0:46:41.259,0:46:45.190
suddenly, you lose a lot of incentives for[br]the social credit score, if the social
0:46:45.190,0:46:48.430
credit score doesn't matter for credit[br]distribution anymore.
0:46:48.430,0:46:52.040
Another thing, and this is, I[br]think, the more likely one,
0:46:52.040,0:46:55.959
is that you have a blacklist for people[br]that have not repaid a loan
0:46:55.959,0:46:59.400
in the past. So you basically[br]get one freebie, and afterwards
0:46:59.400,0:47:04.150
if you didn't repay your loan in the past[br]then you will not get a loan in the
0:47:04.150,0:47:08.400
future. You will still be part of the[br]social credit system, and your social
0:47:08.400,0:47:12.359
credit score will still be important for[br]all of these other access issues, but it
0:47:12.359,0:47:15.849
won't be important for access to loans[br]anymore, once you've been on this
0:47:15.849,0:47:21.670
blacklist. Which is probably something[br]that the Chinese government could go
0:47:21.670,0:47:30.180
behind, but it's also more effort to take[br]care of it; then you have to think about,
0:47:30.180,0:47:33.819
"well, you can't leave them on the[br]blacklist forever, so how long do you
0:47:33.819,0:47:37.599
leave them on the black list? Do they have[br]to pay back the loan and then they get off
0:47:37.599,0:47:45.670
the blacklist? Or do they have to pay back[br]the loan and then stay not in default
0:47:45.670,0:47:52.859
for a year, or for five years?" There's a[br]lot of small decisions that, in my
0:47:52.859,0:47:57.549
opinion, the Chinese government hasn't[br]really thought about, up until now,
0:47:57.549,0:48:01.170
because they're basically doing all these[br]pilot studies, and all of these regional
0:48:01.170,0:48:05.160
governments are thinking of all these[br]small things, but they're not documenting
0:48:05.160,0:48:10.349
everything that they're doing. So, once[br]they – they want to roll it out by 2020,
0:48:10.349,0:48:15.200
by the way, nationwide – once they've[br]rolled it out there's a pretty big chance,
0:48:15.200,0:48:18.630
in my opinion, that they'll have a lot of[br]unintended consequences. A lot of things
0:48:18.630,0:48:28.519
that they haven't thought about, and that[br]they will then have to look at. So, I
0:48:28.519,0:48:33.269
believe that some sort of system is likely[br]to come, just in terms of how much energy
0:48:33.269,0:48:37.289
they've expended into this one, and for[br]the Chinese government at this point, for
0:48:37.289,0:48:41.900
the party, it would be losing face if they[br]did not include any such system, because
0:48:41.900,0:48:45.969
they've been talking about this for a[br]while. But most likely, it would be a kind
0:48:45.969,0:48:53.019
of decentralized data sharing system. And[br]when I ran my simulation... By the way I
0:48:53.019,0:48:59.700
will make public my code, I still need[br]some, basically, I used some proprietary
0:48:59.700,0:49:06.460
data for my model, and I still need the[br]permission to publish this. Once I publish
0:49:06.460,0:49:11.289
this one I will also tweet it, and we'll[br]put it on GitHub for everyone to play
0:49:11.289,0:49:16.200
around with, if you want to. And some of[br]these implementation details that were
0:49:16.200,0:49:20.450
very important in determining model[br]outcomes where "do we have a relative or
0:49:20.450,0:49:25.289
absolute ranking?" So far, all of the[br]systems I looked at had absolute rankings,
0:49:25.289,0:49:30.700
but there's a point to be made for[br]relative rankings. Do we have one score,
0:49:30.700,0:49:35.089
where, basically, if you're a Chinese[br]person you get one score? Or do we have
0:49:35.089,0:49:40.880
different sub-scores in different fields?[br]Do we have people reporting behavior, or
0:49:40.880,0:49:46.369
do we have automatic behavior recording?[br]How do you access other people's scores?
0:49:46.369,0:49:50.339
How much information can you get from[br]other people's scores? Currently, if
0:49:50.339,0:49:55.529
someone is on a blacklist, for example, if[br]you have their ID number, again, you can
0:49:55.529,0:49:59.650
put it into this blacklist, and then they[br]will say "oh, this person is on this
0:49:59.650,0:50:04.630
blacklist for not following this judge's[br]order," and then it says what kind of
0:50:04.630,0:50:10.660
judge's order it was. So, most likely, it[br]will be something like this. The idea is
0:50:10.660,0:50:16.049
that the Social Credit system isn't only[br]for individuals, but also for firms and
0:50:16.049,0:50:22.219
for NGOs. So, what kind of roles will[br]firms play in the system? I haven't looked
0:50:22.219,0:50:28.319
at that, in detail, at this point, but it[br]would be very interesting. Another idea
0:50:28.319,0:50:34.390
that western people often talk about is,[br]do people also rank each other? Currently,
0:50:34.390,0:50:39.390
that's not part of the system in China,[br]but it might be at one point. And lastly,
0:50:39.390,0:50:44.839
where does the aggregation happen? So I've[br]said that a lot of it is actually data
0:50:44.839,0:50:53.749
sharing in China. So what kind of data is[br]shared? Is the raw data shared? "Person A
0:50:53.749,0:51:03.900
did something." Or is the aggregated data[br]shared? "Person A got this score." At this
0:51:03.900,0:51:07.890
point, most of the time, it is actually[br]the raw data that is shared, but that also
0:51:07.890,0:51:12.809
has sort of these data privacy issues, of[br]course, that I've talked about. OK,
0:51:12.809,0:51:18.950
perfect! No there's 10 more minutes. Thank[br]you for your attention! If you have
0:51:18.950,0:51:24.270
questions, remarks you can ask them now or[br]you can catch me up later. You can tweet
0:51:24.270,0:51:29.829
to me or send me an e-mail, whatever[br]you're interested in. Thank you very much!
0:51:29.829,0:51:37.329
applause
0:51:37.329,0:51:42.039
Herald Angel: Hello! As Toni said, we have[br]10 minutes left for questions. If you have
0:51:42.039,0:51:46.981
a question in the room, please go crouch in[br]front of our five microphones. If you're
0:51:46.981,0:51:49.701
watching the stream, please ask your[br]questions through IRC or Twitter, and
0:51:49.701,0:51:53.847
we'll also try to make sure to get to[br]those. Let's just go ahead and start with
0:51:53.847,0:51:56.339
mic one.[br]Question: Good! Thank you very much for
0:51:56.339,0:52:02.809
this beautiful talk. I was wondering how[br]did the Chinese government, companies, and
0:52:02.809,0:52:07.470
most of all, the citizens themselves,[br]respond to you doing this research, or,
0:52:07.470,0:52:11.519
let's put it differently, if you would[br]have been in the system yourself,
0:52:11.519,0:52:14.440
how would your research affect your[br]social credit score?
0:52:14.440,0:52:17.270
laughter
0:52:17.270,0:52:25.639
Answer: So, um... There's actually two[br]different responses that I've seen. When I
0:52:25.639,0:52:31.380
talk to the government themselves, because[br]I was there on a government scholarship,
0:52:31.380,0:52:34.869
and mentioned that I'm really interested[br]in this, they basically said oh well this
0:52:34.869,0:52:38.930
is just a technical system. You don't[br]really need to be concerned with this. It
0:52:38.930,0:52:43.439
is not very important. Just, you know,[br]it's just a technicality. It's just for us
0:52:43.439,0:52:49.299
to make life more efficient and better for[br]everyone. So I assume my score would
0:52:49.299,0:52:54.660
actually go down from doing this research,[br]actually. But when I talk to a lot of
0:52:54.660,0:53:00.890
people at universities, they were also[br]very – they were very interested in my
0:53:00.890,0:53:05.230
research, and a lot of them mentioned that[br]they didn't even know that the system
0:53:05.230,0:53:10.200
existed![br]Herald: Before we go to a question from
0:53:10.200,0:53:14.729
our signal angel, a request for all the[br]people leaving the room, please do so as
0:53:14.729,0:53:20.500
quietly as possible, so we can continue[br]this Q and A. The signal angel, please!
0:53:20.500,0:53:26.249
Signal Angel: Jaenix wants to know, is[br]this score actually influenced by
0:53:26.249,0:53:31.549
association with people with a low score.[br]Meaning that, is there any peer pressure
0:53:31.549,0:53:36.200
to stay away from people with bad scores?[br]Answer: The Sesame credit score definitely
0:53:36.200,0:53:42.849
is influenced by your friends' scores, the[br]Rongcheng score, so far, apparently, is
0:53:42.849,0:53:47.619
not influenced, but it is definitely in[br]the cards, and it is planned that it will
0:53:47.619,0:53:53.800
be part of this. I think WeChat, which is[br]the main platform – it's sort of like
0:53:53.800,0:53:59.880
WhatsApp, except it can do a lot a lot[br]more – WeChat is still not connected to
0:53:59.880,0:54:05.039
the Social Credit Score in Rongcheng. Once[br]they do that, it will most likely also
0:54:05.039,0:54:09.849
reflect your score.[br]Herald: All right, let's continue with
0:54:09.849,0:54:15.430
mic 3.[br]Q: I have a question about your models.
0:54:15.430,0:54:19.789
I'm wondering, what kind of interactions[br]are you modeling? Or actions, like, what
0:54:19.789,0:54:24.910
can the agents actually do? You mentioned[br]moving somewhere else. And, what else?
0:54:24.910,0:54:31.190
A: Okay so the way I set up my model was,[br]I set up a multilevel model. So I looked
0:54:31.190,0:54:38.339
at different kinds of levels. I started[br]out with, basically, they can evade taxes,
0:54:38.339,0:54:46.890
they can get loans and repay loans, they[br]can choose where to live, and they can
0:54:46.890,0:54:54.479
follow traffic rules or not follow traffic[br]rules. And because these were, sort of,
0:54:54.479,0:54:58.660
four big issues that were mentioned in all[br]of the different systems, so I started out
0:54:58.660,0:55:04.810
with these issues, and looked at, what[br]kind of behavior do I see? I used some
0:55:04.810,0:55:11.029
research that – some friends of mine[br]actually sent out surveys to people and
0:55:11.029,0:55:16.299
asked them "well, you're now part of the[br]system. Did your behavior change, and how
0:55:16.299,0:55:23.109
did it change depending on your responses,[br]depending on your score, and depending on
0:55:23.109,0:55:27.729
the score system that exists?" And I,[br]basically, used that, and some other
0:55:27.729,0:55:34.430
research on nudging and on behavioral[br]adaptation, to look at how likely is it
0:55:34.430,0:55:39.160
that someone would change their behavior[br]based on the score.
0:55:39.160,0:55:42.109
Herald: All right let's do another[br]question from the interwebs.
0:55:42.109,0:55:48.489
Q: Yeah, it's actually two questions in[br]one. How does this system work for Chinese
0:55:48.489,0:55:53.859
people living abroad, or for noncitizens[br]that do business in China?
0:55:53.859,0:56:00.529
A: Currently the system does not work for[br]noncitizens that do business in China,
0:56:00.529,0:56:04.700
because it works through the shēnfènzhèng.[br]You only get a shēnfènzhèng if you're a
0:56:04.700,0:56:10.789
Chinese citizen or you live in China for[br]10 or more years. So everyone who is not
0:56:10.789,0:56:16.029
Chinese is currently excluded. Chinese[br]people not living in China, if they have a
0:56:16.029,0:56:21.099
shēnfènzhèng, are on this system, but[br]there's not a lot of information.
0:56:21.099,0:56:28.859
Herald: All right, mic 4.[br]Q: Well, we've come a long way since the
0:56:28.859,0:56:34.029
Volkszählungsurteil. Can you[br]tell us anything about the dynamic in the
0:56:34.029,0:56:43.700
time dimension? How quickly can I regain[br]credit that was lost? Do you have any
0:56:43.700,0:56:47.490
observations there?[br]A: So in the Suining system what they
0:56:47.490,0:56:53.519
actually did was they had a very, very[br]strict period. So if you evaded taxes your
0:56:53.519,0:56:58.599
score would be down for two years and then[br]it would rebounce. In the Rongcheng
0:56:58.599,0:57:03.359
system, they did not publish this kind of[br]period. So, my assumption is that it's
0:57:03.359,0:57:08.760
going to be more on a case by case basis.[br]Because, I looked at the Chinese data, I
0:57:08.760,0:57:13.849
looked at the Chinese policy documents,[br]and they didn't really, for most of the
0:57:13.849,0:57:20.019
stuff, they didn't say how long it would[br]count. For the blacklists, which was kind
0:57:20.019,0:57:24.630
of the predecessor that we look at[br]currently, the way it works is you stay on
0:57:24.630,0:57:29.309
there until whatever the blacklist –[br]until whatever the reason for the
0:57:29.309,0:57:34.510
blacklist is has been resolved. So, you[br]stay on there until you send off this
0:57:34.510,0:57:40.039
apology that the judge ordered you to. And[br]then, usually, you still needed to apply to
0:57:40.039,0:57:44.979
get off. So it doesn't – for blacklists,[br]it does not work that you automatically
0:57:44.979,0:57:49.410
get off. You need to apply, you need to[br]show that you've done what they've asked
0:57:49.410,0:57:53.089
you to do, and then you can get off this[br]blacklist. And I assume it will be a
0:57:53.089,0:57:57.019
similar sort of appeals procedure for the[br]system.
0:57:57.019,0:58:04.069
Herald: All right. Let's go to mic 2.[br]Q: Thank you. I just wanted to if looking
0:58:04.069,0:58:08.910
up someone else's data in details, like[br]position et cetera, does affect your own
0:58:08.910,0:58:12.049
score?[br]A: Currently, it apparently does not, or
0:58:12.049,0:58:17.369
at least they haven't published that it[br]does. It might in the future, but most
0:58:17.369,0:58:22.549
likely it's actually behavior that they[br]want. So they want you to look up other
0:58:22.549,0:58:26.480
people's scores before doing business with[br]them. They want you to, basically, use
0:58:26.480,0:58:30.069
this to decide who you're going to[br]associate with.
0:58:30.069,0:58:33.200
Q: Thank you![br]Herald: All right, do we have another
0:58:33.200,0:58:38.749
question from the Internet, maybe?[br]Signal: Yes, I do! Standby... The question
0:58:38.749,0:58:47.569
is, how is this actually implemented for[br]the offline rural population in China?
0:58:47.569,0:58:53.119
A: Quite easily; not at all at this point.[br]The idea is, by 2020, that they will
0:58:53.119,0:58:59.089
actually have all of this is implemented.[br]But even for the offline – or let's say
0:58:59.089,0:59:05.869
offline rural population in China is[br]getting smaller and smaller. Even in rural
0:59:05.869,0:59:13.039
villages you have about 50-60% of people[br]that are online. And most of them are
0:59:13.039,0:59:16.049
online via smartphone, and their[br]smartphone is connected to the
0:59:16.049,0:59:21.309
shēnfènzhèng. So it's not very complicated[br]to do that for everyone who is online. For
0:59:21.309,0:59:25.989
everyone who's offline, off course, this[br]is more problematic, but I think the end
0:59:25.989,0:59:31.799
goal is to not have people offline at all.[br]Herald: All right. Let's jump right back
0:59:31.799,0:59:38.059
to microphone 2, please.[br]Q: Thank you for the very good and
0:59:38.059,0:59:44.940
frightening talk, so far. At first I have[br]to correct you in one point. In Germany we
0:59:44.940,0:59:50.279
have a similar system because we have this[br]tax I.D., which is set from birth on and
0:59:50.279,0:59:58.769
rests 30 years after a person's dead.[br]Yeah. So we have a lifelong I.D.
0:59:58.769,1:00:02.249
A: You're right. I just... I don't know[br]mine, so I figured… dismissive sound.
1:00:02.249,1:00:08.130
Q: No problem! But, at least we could[br]establish a similar system, if we have a
1:00:08.130,1:00:16.150
government which would want it. A question[br]for you: you mentioned this "guanxi." Is
1:00:16.150,1:00:20.789
it a kind of a social network? I didn't[br]understand it, really.
1:00:20.789,1:00:26.000
A: Yes, it is a kind of social network,[br]but one that is a lot more based on
1:00:26.000,1:00:31.930
hierarchies than it is in the West. So you[br]have people that are above you and people
1:00:31.930,1:00:36.519
that are below you. And the expectation is[br]that, while it's a quid pro quo, people
1:00:36.519,1:00:41.539
that are above you in the hierarchy will[br]give you less than you will give to them.
1:00:41.539,1:00:46.529
Q: Aha, okay.[br]Herald: OK, all right. Unfortunately, we
1:00:46.529,1:00:52.099
are out of time, so, please give another[br]huge applause for Toni!
1:00:52.099,1:00:54.394
applause
1:00:54.394,1:00:57.842
postroll music
1:00:57.842,1:01:17.000
subtitles created by c3subtitles.de[br]in the year 2019. Join, and help us!