Return to Video

34C3 - Gamified Control?

  • 0:00 - 0:18
    34c3 intro
    Herald: Okay, so ladies and gentlemen
  • 0:18 - 0:27
    Katika Kühnreich. So did you all
    thank the angels today? No, yes, maybe?
  • 0:27 - 0:37
    [Applause]
    Herald: Did you all took your prosĂm two
  • 0:37 - 0:45
    today? No? Okay, see, Katika will tell you
    about the social credit system from China,
  • 0:45 - 0:55
    and will tell you a little bit more about
    how a human being can be valued. She is
  • 0:55 - 1:01
    the magista in political science and
    modern and ancient Chinese science. She
  • 1:01 - 1:05
    not only has her theoretical knowledge
    from books, but she always also combines
  • 1:05 - 1:10
    it with the street study. So please
    welcome, with a very warm hand of
  • 1:10 - 1:22
    applause, Katika!
    [Applause]
  • 1:23 - 1:30
    Katika: Hey guys, I'm happy to be here,
    and first I want to thank all the people
  • 1:30 - 1:36
    who made that possible. This conference
    and this talk; so thanks to all you, and a
  • 1:36 - 1:40
    special thanks to all the people
    translating.
  • 1:40 - 1:54
    [Applause]
    Katika: Before we start, I like to say a
  • 1:54 - 2:01
    few points. The first was the thank you,
    we already have. The second is I have to
  • 2:01 - 2:07
    apologize: English is not my mother tongue
    so you will have some funny mistakes that
  • 2:07 - 2:14
    I make, so the Internet has a bit more
    fun. Because the world can be (nice). The
  • 2:14 - 2:21
    second thing, is I will take you on a
    journey through China's social credit
  • 2:21 - 2:27
    system - its current state. And because we
    only have half an hour, more or less,
  • 2:27 - 2:33
    because I'd really like to spend a lot of
    time with you guys on talks, on discussion
  • 2:33 - 2:40
    and QA, we have to leave a lot of
    interesting sites out. So there were some
  • 2:40 - 2:45
    excellent talks on the CCC about
    technological aspects of Internet and
  • 2:45 - 2:51
    Internet usage in China. I will
    concentrate really hard on this social
  • 2:51 - 2:57
    credit system. But if you have questions,
    please ask in the Q&A. The Q&A will not be
  • 2:57 - 3:03
    recorded, so you can feel free. And if you
    have more questions, just contact me after
  • 3:03 - 3:12
    the talk. And, it's a China topic and
    often when you talk about China, people
  • 3:12 - 3:19
    say "oh yeah, oh yeah: that's China." This
    is no China bashing session - I'm sorry
  • 3:19 - 3:28
    for anyone who expected that. China has a
    government, and a government wants to
  • 3:28 - 3:35
    control something. So this talk is about
    social credit systems in China but it is
  • 3:35 - 3:40
    about a tendency I see internationally.
    It's the tendency to answer social
  • 3:40 - 3:47
    problems with technological solutions, or
    at least it's the try. So please keep in
  • 3:47 - 3:53
    mind: it's happening in China, but it's
    not that we are safe in the West or cozy
  • 3:53 - 4:00
    or anything else. So it's meant as an
    example of an international tendency. And
  • 4:00 - 4:05
    I prepared some cards, so I will stick to
    the topic. I have to look at them from
  • 4:05 - 4:11
    time to time. This talk presents my
    current state of research; it's not the
  • 4:11 - 4:18
    truth. And this talked sadly represents my
    current state of my cold as well, so I'm
  • 4:18 - 4:27
    not totally fed please apologized that as
    well. For a little journey I prepared a
  • 4:27 - 4:35
    couple of stops. The first one will be a
    video from 2015 made by Extra Credit on
  • 4:35 - 4:41
    social credit systems in China. So if you
    don't like the talk, you have seven and a
  • 4:41 - 4:48
    half minutes to just vanish - nobody will
    notice. The next point will be "Updates
  • 4:48 - 4:54
    and Corrections" - it's a video from
    2015, and there's some faulty information
  • 4:54 - 5:00
    in it. And to do that, I will give you a
    quick look at the Chinese Internet
  • 5:00 - 5:09
    landscape. The next point will be the
    current state of social credit systems. If
  • 5:09 - 5:17
    I say SCS: it's the same - I'm just lazy.
    And there's one question often not
  • 5:17 - 5:23
    answered when talking about the Chinese
    credit system - it's why is it happening.
  • 5:23 - 5:28
    And again it's not because it's an evil
    government. It's a government. So we'll
  • 5:28 - 5:35
    dive in some ideological aspects of
    Chinese politics today. And in the end,
  • 5:35 - 5:38
    I'm really looking forward to that point
    because I can shut a bit up and listen to
  • 5:38 - 5:48
    your thoughts and we're really interested
    in that. So, everyone ready? Cozy? That's
  • 5:48 - 5:55
    wonderful, thanks! Then we can start with
    the video; that isn't it shown right now.
  • 5:55 - 6:11
    Now we need some technical support. It's
    much more fun when you see the pictures.
  • 6:11 - 6:15
    But we have time, so.
    Herald: Do we have the (tv guys) here?
  • 6:15 - 6:20
    Katika: We got free water!
    Herald: Gentlemen, I think we have a
  • 6:20 - 6:27
    technical problem here, so while the VOC
    and my stage manager is trying to get some
  • 6:27 - 6:33
    people from there. Yes he will do this -
    he's a good guy. He's the best guy!
  • 6:33 - 6:39
    Gentlemen, ladies, applause for the VOC!
    [Applause]
  • 6:39 - 6:49
    Herald: So while we're waiting to have the
    video; and trust me it looked really good
  • 6:49 - 6:55
    from here. Let's play a little game -
    later we will have a Q&A session but
  • 6:55 - 7:01
    unfortunately we are not allowed or we we
    can't record the Q&A session, and we will
  • 7:01 - 7:09
    also not video stream the Q&A session. So
    I would like you to have a quick hand of
  • 7:09 - 7:15
    applause for all of you, because you will
    have a great discussion with this nice
  • 7:15 - 7:22
    lady and therefore we try to make applause
    - we test the applause.
  • 7:22 - 7:26
    Katika: Okay
    [Applause]
  • 7:26 - 7:33
    Herald: No no no, that should be louder -
    come on!
  • 7:33 - 7:44
    [Applause]
    Herald: Okay we still have no video, but I
  • 7:44 - 7:49
    think over there I did not hear any
    applause. Could you, could you please try
  • 7:49 - 7:52
    it again?
    [Applause]
  • 7:52 - 7:58
    Herald: Yeah, better, better. I think
    these guys are way better. Are they? Come
  • 7:58 - 8:05
    on!
    [Applause]
  • 8:05 - 8:09
    Katika: Otherwise I can use the time to
    give you some information I'd present you
  • 8:09 - 8:17
    after the video. Thank you. The
    information you get in the video is based
  • 8:17 - 8:26
    on a press release of the American Civil
    Liberties Union, and that contains some
  • 8:26 - 8:32
    mistakes. One thing - when you want to
    know something about China, the government
  • 8:32 - 8:37
    is really open in publishing what they're
    doing, and what they're planning to do.
  • 8:37 - 8:47
    And it's seems to be much more comfortable
    to just copy, but maybe have a look into
  • 8:47 - 8:51
    what the government says because it's a
    good way to find out what they are doing
  • 8:51 - 8:59
    and they're scaring the open. It works
    with a lot of governments. And I realized
  • 8:59 - 9:04
    that because this video had a lot of
    impact, and there was a lot of news report
  • 9:04 - 9:09
    on that scary system in China but when you
    looked at the news report and you had the
  • 9:09 - 9:17
    video before, you found out that it was
    more or less the same, the very same, as
  • 9:17 - 9:22
    in the video even years later. So please
    if you want to find something out - check
  • 9:22 - 9:30
    for facts. They might be online too. So
    that's the picture, and that's my break.
  • 9:30 - 9:34
    Video: Okay, so one you recently brought
    this to our attention, and when we first
  • 9:34 - 9:37
    started looking into it, we couldn't
    believe it. Even now looking at this I
  • 9:37 - 9:40
    keep thinking that there's got to be
    something about this that I just don't
  • 9:40 - 9:45
    understand, like, I must be reading this
    wrong. It's just so clearly something out
  • 9:45 - 9:49
    of dystopian science fiction. And yet,
    here it is in our world, and people are
  • 9:49 - 9:54
    actually embracing it. Apparently China
    has gamified being an obedient citizen.
  • 9:54 - 10:01
    [Music]
    Video: Going under the innocuous name of
  • 10:01 - 10:06
    Sesame Credit, China has created a score
    for how good a citizen you are, and that's
  • 10:06 - 10:10
    one of the scariest things I've heard in
    quite a while. It's jointly run by
  • 10:10 - 10:14
    Tencent, yes that Tencent - the one that
    owns Riot Games and has a significant
  • 10:14 - 10:19
    share in Epic and Activision-Blizzard, and
    also the ascendant Chinese competitor to
  • 10:19 - 10:23
    Amazon, Alibaba, hence the name Sesame. So
    the owners of China's largest social
  • 10:23 - 10:26
    networks have partnered with the
    governments to create something akin to
  • 10:26 - 10:30
    the US credit score. But instead of
    measuring how regularly you pay your
  • 10:30 - 10:35
    bills, it measures how obediently you
    follow the party line. They dredge data
  • 10:35 - 10:38
    from your social network, so if you post
    pictures of Tiananmen Square, or share a
  • 10:38 - 10:42
    link about the recent stock market
    collapse, your Sesame Credit score goes
  • 10:42 - 10:46
    down. Share a link from the state-
    sponsored news agency about how good the
  • 10:46 - 10:50
    economy is doing and your score goes up.
    The Alibaba and Tencent are also the
  • 10:50 - 10:54
    largest online retailers in China so
    Sesame Credit is also able to pull data
  • 10:54 - 10:58
    from your purchases. If you're making
    purchases the state deems valuable like
  • 10:58 - 11:02
    work shoes or local agricultural products,
    your score goes up. If you import anime
  • 11:02 - 11:06
    from Japan, though, down the score goes.
    And this score has real-world
  • 11:06 - 11:10
    consequences. Like many games, Sesame
    Credit has tiers and levels, and having a
  • 11:10 - 11:14
    high score gives you special benefits -
    like making it easier to get the paperwork
  • 11:14 - 11:18
    you need to travel. Or making it easier to
    get a loan. Now currently, there are no
  • 11:18 - 11:22
    consequences for having a low score but
    there's been talk about implementing
  • 11:22 - 11:27
    penalties once the system becomes
    mandatory in 2020. Penalties like slower
  • 11:27 - 11:31
    Internet speeds for low scoring citizens.
    Or even restricting the jobs that a low
  • 11:31 - 11:34
    scoring person's allowed to hold. But
    there's one more layer to Sesame Credit,
  • 11:34 - 11:38
    and here's where this goes from being
    repulsive to downright insidious. Because
  • 11:38 - 11:42
    this is all part of a social network it
    also scans your friends -
  • 11:42 - 11:46
    so you will lose points for having friends
    with low obedience scores. And it tells
  • 11:46 - 11:51
    you this. At any point, anybody can check
    anyone elses score. And when you check
  • 11:51 - 11:55
    your own score, Sesame Credit provides a
    handy map of your friends to show you who
  • 11:55 - 11:58
    is dragging your score down. Have you ever
    had that thing where you play a game with
  • 11:58 - 12:02
    somebody who wasn't doing very well and
    you've tried to change their behavior to
  • 12:02 - 12:05
    make them do better? Or maybe after a
    while you just sort of stopped playing
  • 12:05 - 12:09
    with the people who are holding you back.
    That's at the heart of how this system
  • 12:09 - 12:13
    works, and it's also what makes this one
    of the most terrifying tools of
  • 12:13 - 12:17
    authoritarian oppression I've ever read
    about. Because like mass censorship, jail
  • 12:17 - 12:21
    time, assassinations - those are all big
    messy implements for keeping a population
  • 12:21 - 12:26
    in line. That messiness and severity
    foster resentment - eventually rebellion.
  • 12:26 - 12:30
    They're expensive, they're unwieldy - in
    the end, those tools are impossible to
  • 12:30 - 12:34
    maintain. But social pressure?
    Ostracization? Those things are free -
  • 12:34 - 12:38
    they happen on their own. And as a
    government tool they don't have nearly the
  • 12:38 - 12:43
    same potential for going embarrassingly
    disastrously wrong. With a system like
  • 12:43 - 12:46
    this in place, a government doesn't even
    have to tell neighbor to spy on neighbor;
  • 12:46 - 12:51
    to rat each other out. Because that's all
    built into a seemingly innocuous game
  • 12:51 - 12:55
    system. The government need not step in.
    Re-education will be handled for them by
  • 12:55 - 12:59
    friends, classmates, and relatives who
    want to maintain a high score. And if that
  • 12:59 - 13:03
    doesn't work then potentially dangerous
    ideas still end up quarantined by the
  • 13:03 - 13:08
    social isolation this game system causes.
    Express or help to spread to many radical
  • 13:08 - 13:12
    ideas, and people will stop associating
    with you. And not because some jackbooted
  • 13:12 - 13:16
    thugs showed up at the door with threats,
    but simply because associating with
  • 13:16 - 13:19
    somebody with those ideas will lose them
    all the privileges they've worked so hard
  • 13:19 - 13:24
    to obtain. It recontextualizes obedience
    to an authoritarian regime.
  • 13:24 - 13:28
    In the past, you obeyed such powers
    because you were afraid. Fear kept you
  • 13:28 - 13:32
    motivated. But fear is negative. It
    fosters resentment. The world we're
  • 13:32 - 13:36
    stepping into instead uses positive
    reinforcement to promote being subservient
  • 13:36 - 13:40
    to the will of the regime. It's Big
    Brother's kinder, gentler hand. And the
  • 13:40 - 13:45
    things that make this scary is that we've
    seen the efficacy of this only too well in
  • 13:45 - 13:48
    games. You may not actually know this, but
    when World of Warcraft was in its early
  • 13:48 - 13:53
    stages of development, it had an unrested
    penalty mechanic that started limiting
  • 13:53 - 13:57
    experienced gains for players who had
    played too much. And players hated it -
  • 13:57 - 14:01
    they resented it they complained about it
    every day. So after brainstorming on this
  • 14:01 - 14:04
    for a while, Blizzard had the idea to
    simply change how the mechanic was
  • 14:04 - 14:09
    presented. Without changing any of the
    existing numbers or systems, they started
  • 14:09 - 14:15
    referring to the unrested experience state
    as normal, and made it the default. And
  • 14:15 - 14:18
    they started calling the original normal
    experience gained state as rested. That's
  • 14:18 - 14:23
    all they changed, and everybody loved it.
    People would log on every day just to get
  • 14:23 - 14:28
    that bonus. Positive reinforcement works
    wonders. But unlike World of Warcraft,
  • 14:28 - 14:32
    which built a system just to get people to
    embrace meaningless grinding, Sesame
  • 14:32 - 14:36
    Credit has built a system to get people to
    enjoy falling into line. Now the system's
  • 14:36 - 14:41
    not mandatory yet - it's opt-in right now.
    But it's going to be mandatory in 2020,
  • 14:41 - 14:46
    and there's a terrible brilliance to
    phasing that in. The early adopters are
  • 14:46 - 14:49
    going to be people excited about this
    system. People who are already patriotic
  • 14:49 - 14:53
    and are eager for anything that'll help
    display that patriotism to the world. And,
  • 14:53 - 14:57
    as early adopters they're gonna talk it
    up. They're gonna give it an air of being
  • 14:57 - 15:00
    positive and fun.
    Then it'll be foisted on the society as a
  • 15:00 - 15:04
    whole. More than that though, the early
    adopters are gonna compete. Already you
  • 15:04 - 15:08
    can see hundreds of thousands of tweets
    displaying people's high scores or showing
  • 15:08 - 15:12
    off the new milestones they've hit. Giving
    a hard numbers to their patriotism and
  • 15:12 - 15:16
    giving them bragging rights for being the
    most patriotic, most right-thinking person
  • 15:16 - 15:20
    they know. And that's gonna set the tone
    for how Sesame Credit is intended to be
  • 15:20 - 15:25
    used - as a competition to see who can
    agree with the government the most. We've
  • 15:25 - 15:28
    talked about propaganda games on this
    channel before but for all the time we've
  • 15:28 - 15:32
    spent examining and deconstructing
    terrible games that espouse hate and for
  • 15:32 - 15:36
    all the studies on propaganda games James
    has done, this is the use of game systems
  • 15:36 - 15:40
    that frightens me the most. Because to
    most people Sesame Credit will seem
  • 15:40 - 15:44
    benign. Perhaps even fun - it's a
    conversation starter, something to share
  • 15:44 - 15:48
    with your friends. But it's making heavy
    use of all the psychological motivators
  • 15:48 - 15:52
    that we Gamemakers deploy in scoring
    systems, and ladders and levels. Systems
  • 15:52 - 15:56
    that we built to shape play habits and to
    keep people coming back. Like I said, I'm
  • 15:56 - 16:00
    still kind of in a state of disbelief
    looking at this. If any of you are
  • 16:00 - 16:03
    watching this from China, please tell me
    if I'm misunderstanding this thing,
  • 16:03 - 16:07
    because I would love to be wrong about it.
    If not, well, I hope this episode can do
  • 16:07 - 16:11
    some small amount to help the fight to
    keep such a system from becoming
  • 16:11 - 16:15
    mandatory. For everyone in the rest of the
    world, I hope this helps remind us all how
  • 16:15 - 16:19
    important it is to be aware and to be
    vigilant. All of these gamification
  • 16:19 - 16:22
    techniques we've learned through making
    games offer incredible opportunities for
  • 16:22 - 16:27
    making this world a better more engaging
    place. But every great tool carries with
  • 16:27 - 16:31
    it the potential for misuse. And it's on
    us as a community who understands this
  • 16:31 - 16:36
    amazing new medium, to do what we can to
    stop that. We'll see you next week.
  • 16:38 - 16:52
    [Applause]
    Katika: So, we had the shedule, and thank
  • 16:52 - 16:59
    you to Extra Credit for this video, and
    thank you for concentrating on
  • 16:59 - 17:06
    gamification. Because a lot of time, when
    I talk to social scientists, without a lot
  • 17:06 - 17:12
    of technological knowledge, it was like
    okay - it's a new system. And the whole
  • 17:12 - 17:20
    gamification thing triggered me, and I
    wondered about a lot of people who didn't
  • 17:20 - 17:27
    seem too interested in it. Because I said
    that the video contains some faulty
  • 17:27 - 17:33
    information. First, I'd like to
    concentrate on key aspects they told us in
  • 17:33 - 17:40
    with this video. There is a social rating
    system in China now. The system will
  • 17:40 - 17:49
    become mandatory in 2020. That's all
    right. It works on Big Data algorithms,
  • 17:49 - 17:59
    artificial intelligence, and your online
    and offline data. And it uses technologies
  • 17:59 - 18:04
    like gamification and nudging. And if you
    hear it's an Orwellian system - no it's
  • 18:04 - 18:19
    not. 1984 was about force. Gamification is
    about feeling good and cozy. So, in the
  • 18:19 - 18:23
    updates and corrections, I promise you a
    quick look at the end Chinese internet
  • 18:23 - 18:30
    landscape. What most people already know.
    From the beginning the Chinese government
  • 18:30 - 18:39
    started to control the internet as much as
    it could and its users. So some
  • 18:39 - 18:46
    vocabularies you might have heard are the
    Great Firewall, or its new extra the
  • 18:46 - 18:51
    Golden Shield, the Great Cannon, which is
    a DDoS weapon. It would be really
  • 18:51 - 18:57
    fascinating to dive in there, but we have
    to watch from far. Another point that's
  • 18:57 - 19:02
    really interesting for us looking Social
    Credit System is true name or real name
  • 19:02 - 19:07
    registration. Which means you can't use a
    lot of services if you don't give them
  • 19:07 - 19:15
    your ID number. And the mandatory system
    in 2020 will work on ID numbers as far as
  • 19:15 - 19:21
    we know. So, we watched the other side:
    the users, their millions of users - it's
  • 19:21 - 19:31
    one more than 1.4 billion people in China.
    And ICTs are really popular. Much more
  • 19:31 - 19:37
    than in Germany, for example. And another
    thing that please keep in mind - cash is
  • 19:37 - 19:41
    vanishing in China.
    A couple of years ago, for example, when I
  • 19:41 - 19:49
    lived in China 2004: I paid my rent in
    cash half a year in advance. When I paid
  • 19:49 - 19:56
    for anything I paid cash. You could
    couldn't find a lot of places accepting
  • 19:56 - 20:02
    cards. But now China didn't have this
    development of cards - they just jumped
  • 20:02 - 20:09
    right through paying with mobile apps. And
    you can buy noodles or snack or magazine
  • 20:09 - 20:15
    with your mobile phone. And the Chinese
    users found multiple ways to deal with
  • 20:15 - 20:22
    censorship, and to trespass it. It's a
    wonderful thing to look up.
  • 20:22 - 20:28
    And now, the big players of the Chinese
    Internet, because we learned some names in
  • 20:28 - 20:34
    the film. I made something very Chinese -
    I made something colorful. Okay, you have
  • 20:34 - 20:42
    three big players: its Alibaba and Tencent
    - we heard about them, and Baidu is the
  • 20:42 - 20:49
    third biggest player. Each millions of
    users, or billions. So to explain this, I
  • 20:49 - 20:58
    will use some American companies names to
    help understanding. And Baidu is known for
  • 20:58 - 21:05
    that; was the wrong; yeah, Baidu is known
    for Baidu Barker - this is like the
  • 21:05 - 21:12
    Chinese Wikipedia, it's just a bit much
    controlled. And a have Baidu Pay, and
  • 21:12 - 21:18
    Baidu is huge in artificial intelligence.
    So the next one is Tencent - we learned
  • 21:18 - 21:25
    about the gaming thing. And developed QQ -
    this is a messenger than some of you might
  • 21:25 - 21:34
    remember: ICQ. Yeah, it's just a bit
    bigger. Another app a lot of people don't
  • 21:34 - 21:39
    think it's Chinese, it is Chinese, it's
    called WeChat. It's I can say they even
  • 21:39 - 21:47
    mmm WhatsApp, right? And guess what WeChat
    developed - WeChat Pay. And there's
  • 21:47 - 21:52
    another little company Tencent holds some
    parts in its SnapChat - it's twelve
  • 21:52 - 22:00
    percent owned by Tencent. Alibaba has
    Alipay which is one of the biggest payment
  • 22:00 - 22:08
    apps. And you can even use it in Germany
    now I think, in some shops. It's running
  • 22:08 - 22:16
    Taobao Wang, which is like eBay just a bit
    bigger. And Sina Weibo. Weibo mains micro
  • 22:16 - 22:26
    blogging or twittering. And Sina is the
    biggest company in China, and they hold
  • 22:26 - 22:32
    more or less a third. Now we come to
    something really interesting - it's Ant
  • 22:32 - 22:40
    Financials. Because Ant Financials created
    something called Sesame Credit. Sesame
  • 22:40 - 22:45
    Credit is Ali Baba's child, not Tencent,
    and Ali Baba's. Tencent developed their
  • 22:45 - 22:56
    own system. So, updates and corrections
    without colors. 2014 was the year the
  • 22:56 - 23:01
    Chinese state council released a planning
    outline on the construction of social
  • 23:01 - 23:09
    credit systems in China. And it gave us a
    lot of information but only very few
  • 23:09 - 23:19
    specific information. So, they started,
    and in 2020 it will be managed mandatory.
  • 23:19 - 23:23
    But they worked on for as long as at least
    2005.
  • 23:25 - 23:32
    It allows governmental and private SCS. So
    you have both. And eight companies are
  • 23:32 - 23:40
    allowed to form their own private social
    credit systems. We learned Tencent and
  • 23:40 - 23:45
    Alibaba are two of them. In different
    regions, Hainan for example, it's an
  • 23:45 - 23:52
    island - you have social credit systems
    run by the government. So you see you have
  • 23:52 - 23:58
    a lot of different systems running at the
    same time in different regions and
  • 23:58 - 24:05
    sometimes you can choose between different
    social credit systems at the same place.
  • 24:05 - 24:15
    And Sesame Credit we have learned is the
    biggest and run by Alibaba. And Sesame
  • 24:15 - 24:21
    Credit is not the one who will be
    mandatory. I have one other point that I
  • 24:21 - 24:34
    forgot it might come back later. So, to
    give you a bit more information, let's
  • 24:34 - 24:38
    have a quick look at recent developments
    that I thought of would be really
  • 24:38 - 24:47
    interesting for you when you have that
    scenario of a society that is formed by a
  • 24:47 - 24:55
    writing system. In 2016 there were some
    new guidelines on the SCS. New
  • 24:55 - 25:00
    cybersecurity laws and you might expect it
    they are getting a bit harder and harder
  • 25:00 - 25:05
    every time. And there was a police robot -
    I don't know if you heard about it I was
  • 25:05 - 25:08
    going to write an article but then we had
    a really nice conference that made us a
  • 25:08 - 25:14
    lot of work and my article died in the
    way. So you might have watched Wall-E -
  • 25:14 - 25:18
    what was the movie couple of years ago.
    You might remember Eve - there was a
  • 25:18 - 25:24
    little cute white robot that had a
    specialty - it could taser. Someone in
  • 25:24 - 25:30
    China must have watched that movie too,
    because guess what - it looks like Eve and
  • 25:30 - 25:38
    it can tazer. You find pictures and
    clips online. And the People's Bank of
  • 25:38 - 25:44
    China announced its own digital currency
    to come soon. They are working on it for
  • 25:44 - 25:51
    years. And the real name registrations for
    online comments. So no real name
  • 25:51 - 25:59
    registrations, no online comments. And in
    2020 - the mandatory Social Credit System.
  • 26:00 - 26:09
    The next question is usually: how does the
    social criticism work? And we take a quick
  • 26:09 - 26:16
    look at Sesame Credit because they added
    more and more services that Extra Credit
  • 26:16 - 26:23
    guys could know about. So we learned a lot
    of data mining they do.
  • 26:23 - 26:28
    For example, when you use AliPay it goes
    to your scores. So your score gets higher
  • 26:28 - 26:36
    and higher the more you shop with AliPay.
    Then other Alibaba services - we know
  • 26:36 - 26:41
    there are different and millions of
    services. And now it's getting special
  • 26:41 - 26:47
    because it draws information from
    officials as well. So we have information
  • 26:47 - 26:55
    from courts, debtors registries, and
    because love always makes everything
  • 26:55 - 27:01
    better, you get your data from the biggest
    dating app in China - it's Baihe. So you
  • 27:01 - 27:08
    can see if the person you want to fall in
    love with have a good score. And of
  • 27:08 - 27:17
    course, they're including more and more
    services. And there are penalties now. If
  • 27:17 - 27:23
    you forgot to pay your child support for a
    couple of month or years or; I have heard
  • 27:23 - 27:30
    people can forget that for quite a long
    time. There are penalties: for example,
  • 27:30 - 27:36
    you can't take high-speed trains or first-
    class flight tickets. Which is a very
  • 27:36 - 27:44
    intelligent way. Again, it's a very
    intelligent way to add penalties because
  • 27:44 - 27:52
    if you start with people nobody likes,
    nobody likes openly, it's clever. And of
  • 27:52 - 28:00
    course, the companies do not tell you a
    lot about the technologies like they use
  • 28:00 - 28:08
    in the systems. So and then people want to
    know about the mandatory system, what it
  • 28:08 - 28:14
    can do, how will it look and so on, and
    there's a very simple answer - we don't
  • 28:14 - 28:20
    know. The Chinese government doesn't give
    you a lot of information on how it will
  • 28:20 - 28:28
    look. But we can guess. In having a look
    on how the government worked in the last
  • 28:28 - 28:38
    20-30 years. So, we can have a look how
    they prepare the mandatory system. Like I
  • 28:38 - 28:44
    said, different systems at the same time,
    and they will learn from this. And my
  • 28:44 - 28:50
    guess is they will have like a little
    medley of the best features of the
  • 28:50 - 28:58
    existing systems and get rid of the faulty
    ones. So, by best, I mean best from the
  • 28:58 - 29:05
    point of view of the government of course.
    So now we are bit later because of our
  • 29:05 - 29:12
    technological difficulties, thanks again
    for solving. Why do you have a Social
  • 29:12 - 29:17
    Credit System?
    Let's dive a bit back in history. Some
  • 29:17 - 29:22
    people might remember the year 1989.
    Before you had a lot of countries that
  • 29:22 - 29:28
    called themselves socialists. But after,
    you had only a few and China is one of
  • 29:28 - 29:36
    them. And how did they make it? In the 80s
    they had something called the era of
  • 29:36 - 29:41
    reform and opening-up, where they opened
    up to the West mostly, and to capitalism.
  • 29:41 - 29:49
    And you might know what that means.
    China's called the transformation society.
  • 29:49 - 29:53
    And with some people getting rich quickly,
    and a lot of people getting poorer and
  • 29:53 - 30:00
    poorer and watching the rich - you have a
    lot of social tension. And remember, China
  • 30:00 - 30:06
    is a huge country. A vast amount on
    people, and know people are really good in
  • 30:06 - 30:13
    getting on each other's nerves. So you
    have problems. I give you one number that
  • 30:13 - 30:20
    I really like - in 2014, the official
    count of mass incidents everyday was
  • 30:20 - 30:29
    nearly 250. And the number is rising, so
    it will be much more today. And a mass
  • 30:29 - 30:35
    incident can be anything from pensioners
    holding up banners demanding more money
  • 30:35 - 30:40
    because they worked their whole lives, I'm
    sorry, I really have to concentrate on not
  • 30:40 - 30:48
    swearing, to like, riots, full-grown
    riots, petrol bombs, burning buildings.
  • 30:48 - 30:54
    For example, G20 in Hamburg, you might
    have heard. That's a mass incident. That's
  • 30:54 - 31:00
    nothing to get crazy about, that's just a
    little thing that happened. Hamburg is
  • 31:00 - 31:07
    still standing. And, thank you, hello
    Hamburg.
  • 31:07 - 31:12
    [Applause]
    And keep in mind organized societies are
  • 31:12 - 31:19
    controlled on different levels. That's not
    only China. And social control is usually
  • 31:19 - 31:25
    done by groups, so it's much easier if we
    control each other. And it has the long
  • 31:25 - 31:31
    history in China too, and a lot of
    socialist countries develop new forms of
  • 31:31 - 31:36
    social control. So in China, you have a
    Hukou - that means you can only get social
  • 31:36 - 31:43
    services where you're permitted to live.
    I just give you a really really short
  • 31:43 - 31:48
    version of everything it's not it's a bit
    more complex as you know. The Tang on
  • 31:48 - 31:53
    which you can say it's an analog version a
    lot of your stuff what you've done goes in
  • 31:53 - 31:58
    there. You needed a marriage permit if you
    wanted to get married and it wasn't easy
  • 31:58 - 32:05
    to live without a married permit and as
    China became more and more digitized you
  • 32:05 - 32:14
    might know what comes next. Oh too anyone
    control became digitized as well and as
  • 32:14 - 32:20
    people are using ICTs more and more they
    create a huge amount of data. And the
  • 32:20 - 32:29
    bloody thing with data is you don't feel,
    smell or see anything vanishing, do you?
  • 32:29 - 32:33
    And people it's helpful as humans to
  • 32:33 - 32:43
    develop something towards something we
    can't see smell or fear. And more and more
  • 32:43 - 32:50
    technologies are used in civilians and
    social control. But that still leaves the
  • 32:50 - 32:59
    question: why is China doing it? Let's
    go to another point I'd really interested
  • 32:59 - 33:06
    in: ideological influences the way we see
    the world and the way we think about it
  • 33:06 - 33:15
    is formed by theories, by ideologies and
    what's the Chinese flag ? You see it
  • 33:15 - 33:20
    again, maybe you remember, some of you,
    it's red with yellow stars - that usually
  • 33:20 - 33:33
    is a code for socialists and this is Mao
    Zedong. He was kind of big in China. And
  • 33:33 - 33:40
    so you have - here you have three
    socialist symbols. It really helps to
  • 33:40 - 33:48
    know a bit about Marxism in researching
    China and especially in Mao Zedong
  • 33:48 - 33:52
    thought it's not called maoism it's
    called more Zedong thought it's about what
  • 33:52 - 33:59
    he thought and in this reform of opening
    up - I'm trying to feed you a bit of
  • 33:59 - 34:06
    Chinese history - the Chinese government
    opened up to more. For example they opened
  • 34:06 - 34:11
    up to nationalism. And if you know a bit
    about socialism, usually you wouldn't
  • 34:11 - 34:16
    put socialism and nationalism in one room,
    because they didn't like each other. The
  • 34:16 - 34:25
    Chinese government found a way they like
    each other. They got really interested
  • 34:25 - 34:32
    in PR and they really research other
    countries how governments there stood in
  • 34:32 - 34:41
    power. And some are not so well known to
    most of us this is Confucius.
  • 34:41 - 34:46
    Confucianism shaped and molded the
    Chinese societies for more than 2,000
  • 34:46 - 34:53
    years. And if you have that big
    influence, you can't just get rid of it.
  • 34:53 - 34:59
    It's in people's memories. It's in the
    whole society, and Confusianism came back
  • 34:59 - 35:07
    as an ideology, where some parts of the
    Socialists ideology vanished. Confucianism
  • 35:07 - 35:14
    came back and Confucianism is about a
    society where everyone is on her or his
  • 35:14 - 35:23
    place and her place is always below. And
    with all these new ideological
  • 35:23 - 35:29
    influences, the way of communication
    changed as well.
  • 35:29 - 35:38
    Before it was the masses and now it's the
    public. All campaigns and programs had
  • 35:38 - 35:43
    revolutionary names or poetic forms like
    hundred flowers yet the Great Leap
  • 35:43 - 35:50
    Forward, and all these campaigns molded
    our picture of China, or the picture
  • 35:50 - 35:58
    most people have in their hearts people in
    bloom, red flags. But now it's changed you
  • 35:58 - 36:03
    have new campaigns and programs and they
    have other names as well. You have
  • 36:03 - 36:10
    socialist harmonious society as a goal for
    the society and a couple of years ago
  • 36:10 - 36:16
    before, you couldn't put socialism and
    harmony in one room either. And you have
  • 36:16 - 36:26
    the Chinese dream. Yes, it sounds a bit
    like American dream, yeah. And we learned
  • 36:26 - 36:31
    that all governments are interested in
    control and feedback because you can't
  • 36:31 - 36:36
    control something, if you don't pay
    attention and can't blow up anyway. So
  • 36:36 - 36:43
    like we have an autonomous nervous system
    that tells every part what's going on, you
  • 36:43 - 36:48
    have one electronic autonomous nervous
    system now in many societies not only
  • 36:48 - 36:54
    China.
    People using ICTs and a social credit
  • 36:54 - 37:01
    system just creates a wonderful
    opportunity of feedback and on the same
  • 37:01 - 37:19
    time on influence. So, we come to the end
    of this talk and to the summary. So, again
  • 37:19 - 37:25
    this I think is really important: it's an
    international tendency to solve social
  • 37:25 - 37:33
    problems with technological solutions. And
    as far as I am concerned this is not the
  • 37:33 - 37:42
    best way maybe. And the Chinese government
    - they do not count and think in years,
  • 37:42 - 37:49
    they think in decades at least. And they
    looked at the internet and found out
  • 37:49 - 37:58
    that this is a good tool and they're using
    it. Successfully. But we know at least
  • 37:58 - 38:03
    since Edward Snowden, that other
    governments are capable of using the
  • 38:03 - 38:09
    Internet as a tool as well. A social
    credit system exists of the use of big
  • 38:09 - 38:21
    data, algorithm, ICTs and gamification.
    And it will become mandatory. And this
  • 38:21 - 38:28
    game, when it becomes mandatory, will
    define your life. And not only your life,
  • 38:28 - 38:34
    it will define the life of the people
    surrounding you as well. The schools your
  • 38:34 - 38:44
    children for example will go to will be
    chosen on your rate. And there's the
  • 38:44 - 38:53
    opportunity to put this together with a
    national cryptocurrency. So, thank you
  • 38:53 - 38:57
    all for listening that what's my current
    state of research and my current state of
  • 38:57 - 39:03
    my code, we both presented to you this
    talk and I'm really looking forward to QA
  • 39:03 - 39:13
    and discussing. Thanks.
    34c3 outro
  • 39:13 - 39:30
    subtitles created by c3subtitles.de
    in the year 2018. Join, and help us!
Title:
34C3 - Gamified Control?
Description:

more » « less
Video Language:
English
Duration:
39:31

English subtitles

Revisions Compare revisions