1
00:00:00,000 --> 00:00:09,240
31C3 Title, no sound

2
00:00:09,240 --> 00:00:12,031
Alright, welcome! So, welcome again
from me. It's great to be here!

3
00:00:12,031 --> 00:00:15,520
So many people, even to this late hour.
I've been told, this is the prime time.

4
00:00:15,520 --> 00:00:21,940
That is awesome, at 11 p.m. I'm David,
I'm a Computer Scientist from Bonn.

5
00:00:21,940 --> 00:00:24,470
And we just can start with the things
that happened so far at the congress.

6
00:00:24,470 --> 00:00:28,449
If you happened to be here at the congress

7
00:00:28,449 --> 00:00:31,420
or watched sessions on stream - 
welcome again

8
00:00:31,420 --> 00:00:35,570
to the colleagues on the internet - then 
there will be always devices that one

9
00:00:35,570 --> 00:00:39,509
does not like so much to use.
[Laughter]

10
00:00:39,509 --> 00:00:42,930
Who participated in the sessions of Tobias
Engel and Karsten Nohl, does indeed use

11
00:00:42,930 --> 00:00:47,600
his mobile phone less confident.
And who was with starbug afterwards, will

12
00:00:47,600 --> 00:00:51,899
not like to use iris scanners or finger 
print scanners anymore and may use gloves


13
00:00:51,899 --> 00:00:56,000
more frequently now.
So here a little disclaimer:

14
00:00:56,000 --> 00:01:01,940
If someone has an intimate relation to
his photocopier

15
00:01:01,940 --> 00:01:07,810
and tends to keep it like that, should
refrain from participating this session.

16
00:01:07,810 --> 00:01:12,340
We will do three things during this session.
First of all we will

17
00:01:12,340 --> 00:01:17,260
get to know one of most prevalent and
dangerous bugs of the last years.

18
00:01:17,260 --> 00:01:20,920
Secondly, we will comprehend the bug. 
That is in a manner

19
00:01:20,920 --> 00:01:25,390
nerds and muggels will understand. 
And last but not least, for the activists

20
00:01:25,390 --> 00:01:30,060
among us - may be some present here - 
we will deduct some rules

21
00:01:30,060 --> 00:01:34,210
that may apply to a single person that will 
handle a powerful opponent,

22
00:01:34,210 --> 00:01:38,860
just like a global player. 
But in your case

23
00:01:38,860 --> 00:01:42,690
it can be something completely different.
That's why I will describe precisely

24
00:01:42,690 --> 00:01:45,950
how this dispute evolved over time and

25
00:01:45,950 --> 00:01:49,770
what kind of mistakes I made.
The talk's kind of structured

26
00:01:49,770 --> 00:01:53,500
like a novel. First, there's a prologue, 
for the conspiracy theorists

27
00:01:53,500 --> 00:02:00,620
among you. The year is 2008.

28
00:02:00,620 --> 00:02:05,600
In summer 2008 the US were
having the primaries for presidential
election.

29
00:02:05,600 --> 00:02:09,009
Barack Obama was in the running against
Hillary Clinton. In the US, like here,

30
00:02:09,009 --> 00:02:14,220
there's lots of intrigue in politics.
So there were a few anonymous emails,

31
00:02:14,220 --> 00:02:18,450
that should benefit Mrs. Clinton. Those
mails claimed, among other things,

32
00:02:18,450 --> 00:02:23,260
that Obama had been born in Kenia als a
Kenian citizen. That would make him fomally

33
00:02:23,260 --> 00:02:28,060
unfit to be president. To become president
of the US, you have to be

34
00:02:28,060 --> 00:02:33,550
'natural born citizen' of the US. What
exactly a 'natural born citizen' is

35
00:02:33,550 --> 00:02:39,060
the Americans themselves even don't
really fully know. But there's a whole

36
00:02:39,060 --> 00:02:43,780
Wiki article about the controversy,
where you can read all about it.

37
00:02:43,780 --> 00:02:48,720
Two things generally acknowleged:
First, one's to be American.

38
00:02:48,720 --> 00:02:51,560
Second, one's to also be that at time of
birth. So when I come to the US,

39
00:02:51,560 --> 00:02:55,740
newly naturalized, that doesn't work.
That Obama's second name

40
00:02:55,740 --> 00:02:59,120
is Hussein was somewhat
suboptimal too in that context.

41
00:02:59,120 --> 00:03:04,640
(laughs).
Obama obviously had an interest in

42
00:03:04,640 --> 00:03:07,850
ending that 'argument' as quickly as
possible. So he made his birth certificate

43
00:03:07,850 --> 00:03:12,269
publicly available. I say 'short birth
certificate' because,

44
00:03:12,269 --> 00:03:15,580
when he was born, a short and a long one
were made. The short one ist shown here on

45
00:03:15,580 --> 00:03:20,369
the left, you see it behind me. And I
in front of me.

46
00:03:20,369 --> 00:03:24,140
But good conspiracy theorists aren't
distracted by facts.

47
00:03:24,140 --> 00:03:34,279
(laughter and applause)

48
00:03:34,279 --> 00:03:37,420
Immediatly, there are accusations
the birth certificate's faked.

49
00:03:37,420 --> 00:03:40,489
Supposedly, there was a stamp missing, and
... and ... and. Whatever you can

50
00:03:40,489 --> 00:03:45,030
come up with. You all can come up with it.

51
00:03:45,030 --> 00:03:47,910
On the right, you see a few car stickers
by Obama's enemies.

52
00:03:47,910 --> 00:03:51,760
The lowermost explicitly calls for the
birth certificate. The theory that Obama

53
00:03:51,760 --> 00:03:56,590
shouldn't be allowed to be president,
is rather wide-spread in the US.

54
00:03:56,590 --> 00:04:00,040
Obama won the primaries, and the following
election, but the dispute

55
00:04:00,040 --> 00:04:05,799
simmered on. There was a whole scene of
birthers

56
00:04:05,799 --> 00:04:13,739
that wanted to prove Obama's actually not
American.

57
00:04:13,739 --> 00:04:17,500
After the whole thing hadn't calmed down
fo two and a half years - Obama already 
being

58
00:04:17,500 --> 00:04:24,300
president for some time - in 2011 he had 
all of it. He published the scan of

59
00:04:24,300 --> 00:04:28,000
the long version of the birth certificate,
on the right in the picture. You can
already see

60
00:04:28,000 --> 00:04:32,500
there's much more information in it, and
you could think: They'll leave him alone
now.

61
00:04:32,500 --> 00:04:37,900
But far from it.
Shortly after the release
there were accusations

62
00:04:37,900 --> 00:04:43,010
the birth certificate was 
a clumly forgery.
Let's take a closer look.

63
00:04:43,010 --> 00:04:47,490
The left picture is a strong enhancemt
of the red box in the right picture.

64
00:04:47,490 --> 00:04:52,240
The numbers six and four are visible. 
These numbers have sharp,
pixel-perfect edges.

65
00:04:52,240 --> 00:04:57,450
Yes, it's even visible on the projector.
And the numers are uniformly colored.

66
00:04:57,450 --> 00:05:00,380
On their right side the number one
is blurred and colored unevenly.

67
00:05:00,380 --> 00:05:04,710
The one is as you would expect a scan
in reality. Why is there such


68
00:05:04,710 --> 00:05:09,670
a difference between two numbers in 
one and the same row of numbers?

69
00:05:09,670 --> 00:05:13,889
A few more examples. 
Again one can see numbers 
with sharp edges

70
00:05:13,889 --> 00:05:18,930
or these ticking boxes in contrast to
normal, slightly blurred numbers

71
00:05:18,930 --> 00:05:23,720
and boxes. I drew some red boxes
the ticking boxes

72
00:05:23,720 --> 00:05:27,121
and the 'and'. 
There one can see a kind of shift.
And it does really look

73
00:05:27,121 --> 00:05:31,389
as though somebody drew this using Paint.
Meaning the ancient one, 
I am sure you remeber

74
00:05:31,389 --> 00:05:35,440
from your childhood. MS Paint on
Windows 3.11. 
I used to sit at my father's workplace

75
00:05:35,440 --> 00:05:41,099
at work and stole his working hours.
Or this one,

76
00:05:41,099 --> 00:05:44,540
particulary beautiful. 
This section of the frame 
is from the stamp at the bottom.

77
00:05:44,540 --> 00:05:49,430
There's a typo, in the stamp. Yeah sure,
makes sense. We have heard that one before,

78
00:05:49,430 --> 00:05:53,460
typo in the stamp. I mean of course one 
would think it's a fraud

79
00:05:53,460 --> 00:05:56,240
the way it looks. And at the same time
think that the intern

80
00:05:56,240 --> 00:05:58,969
at the White House is too stupid
to use Photoshop.

81
00:05:58,969 --> 00:06:02,020
Laughter

82
00:06:02,020 --> 00:06:06,979
Concerning PR this was a massive failure of course.
According to a Gallup poll

83
00:06:06,979 --> 00:06:12,651
in 2011, 
5% of Americans believed, Obama was

84
00:06:12,651 --> 00:06:17,440
definitely not born in the US. And a 
further 8% thought, that he 
was 'probably not'

85
00:06:17,440 --> 00:06:22,420
born in the US. Well that didn't work out.
The White House had to

86
00:06:22,420 --> 00:06:27,769
back up pretty badly. To this day they get
requests because of this. This was the prologue.

87
00:06:27,769 --> 00:06:38,450
We will now move on to the main trial
and jump in time to 2013.

88
00:06:38,450 --> 00:06:44,139
On the 24th of June 2013 
a company, I was friends with, called me

89
00:06:44,139 --> 00:06:48,950
The had two big Xerocs Workcentres.
Xerocs Workcentres are

90
00:06:48,950 --> 00:06:53,550
those giant buisness copiers, that stand 
everywhere nowadays. They are connected via WIFI,

91
00:06:53,550 --> 00:06:58,050
can scan, print, copy, mail and 
cost as much as a small car.

92
00:06:58,050 --> 00:07:01,590
These printers aren't the ones your 
grandma uses, but have

93
00:07:01,590 --> 00:07:05,550
a few hundred users per device,
maybe more. In this picture

94
00:07:05,550 --> 00:07:10,530
you can see a construction plan. 
The black areas aren't original, I just

95
00:07:10,530 --> 00:07:14,900
cencored those afterwards,
since I would not have been allowed

96
00:07:14,900 --> 00:07:20,230
to use it. I marked three spots 
in yellow on the plans.

97
00:07:20,230 --> 00:07:24,560
These spots are standardized blocks
containing the squarefootage

98
00:07:24,560 --> 00:07:27,960
of the room. These spots will become more
important soon. The company

99
00:07:27,960 --> 00:07:32,050
told me: "Hey David, 
when we scan a construction plan

100
00:07:32,050 --> 00:07:35,260
the numbers change. 
Could you take a look at it?"

101
00:07:35,260 --> 00:07:40,250
Laughter

102
00:07:40,250 --> 00:07:44,890
On the left side, that's me.
Laughter

103
00:07:44,890 --> 00:07:52,160
Applause

104
00:07:52,160 --> 00:07:55,220
At this point I have to add, that the relationship
with them is really good. I worked my way

105
00:07:55,220 --> 00:07:57,990
through my computer sience degree.
Of course my parents also

106
00:07:57,990 --> 00:08:02,759
contributed, I won't deny that. But I
did IT-Service for the company and

107
00:08:02,759 --> 00:08:05,520
they were really nice all the time 
and of course I thought they were screwing with me.

108
00:08:05,520 --> 00:08:12,870
For sure. Copier changes numbers?? 
Of course, makes sense. We've heard that before.

109
00:08:12,870 --> 00:08:15,639
They said: "Yes, come over 
and take a look at it.

110
00:08:15,639 --> 00:08:19,449
We need the device,
it has to work."

111
00:08:19,449 --> 00:08:23,421
So I drove over there and took a look.
Still being a bit

112
00:08:23,421 --> 00:08:31,229
on the watch for the joke. 
They have a Xerox Workcentre 7535.

113
00:08:31,229 --> 00:08:34,000
Here are the three marked spots 
in the original, before scanning.

114
00:08:34,000 --> 00:08:37,760
I am not sure how good you can read it,
so I will read it out loud.

115
00:08:37,760 --> 00:08:43,850
On the top it says 14.13 sqm (square meter)
in the middle it's 21.11 sqm,

116
00:08:43,850 --> 00:08:49,380
and at the bottom 17.42 sqm. 
So I put the plans in the Workcentre

117
00:08:49,380 --> 00:08:55,510
and scanned it. And here are the 
same spots after the scan.

118
00:08:55,510 --> 00:09:02,870
Laughter and Applause

119
00:09:02,870 --> 00:09:07,529
Interesting. Suddenly all rooms 
are 14.13 sqm big.

120
00:09:07,529 --> 00:09:11,120
I thought this can't be right. 
Completely impossible. This isn't happening.

121
00:09:11,120 --> 00:09:15,720
I was still thinking they are 
screwing with me. (laughs)

122
00:09:15,720 --> 00:09:19,440
While scanning the - to clear
that out from the beginning, since I

123
00:09:19,440 --> 00:09:23,360
got that question a dozen times 
in the internet- While scanning the text 
detection

124
00:09:23,360 --> 00:09:28,040
was turned of. The number substitution 
takes place in the raw pixel data.

125
00:09:28,040 --> 00:09:33,730
The company also had a second 
Workcentre, the 7556.

126
00:09:33,730 --> 00:09:37,690
Thats bigger and faster.
Aside from these two kinds of Workcentres,

127
00:09:37,690 --> 00:09:41,290
that I mention here in the beginning,
there are a lot more. It is

128
00:09:41,290 --> 00:09:45,342
a gigantic family of devices.
In contrast to the smaller device

129
00:09:45,342 --> 00:09:52,000
which spat out the same numbers every time,...
(laughs)

130
00:09:52,000 --> 00:09:58,279
the larger one gave out different
ones every time. (Laughter)

131
00:09:58,279 --> 00:10:01,550
It is bigger and has more CPU power.

132
00:10:01,550 --> 00:10:03,730
(Laughter)

133
00:10:03,730 --> 00:10:07,010
Look at those rows and how 
the values change. At "Stelle 2",

134
00:10:07,010 --> 00:10:11,700
that is the middle row, 
first and last it's 14.13 sqm.

135
00:10:11,700 --> 00:10:16,320
And in the middle 21.11, once.
That would have been the correct value btw.

136
00:10:16,320 --> 00:10:19,490
There is a chance to get it right.
(Laughter)

137
00:10:19,490 --> 00:10:23,060
In the other rows it looks similar.

138
00:10:23,060 --> 00:10:26,900
In case one of you needs one of 
those NSA random generators....

139
00:10:26,900 --> 00:10:29,360
(laughs)

140
00:10:29,360 --> 00:10:34,930
Applause

141
00:10:34,930 --> 00:10:37,700
Keep in mind, that actually this 
is no...

142
00:10:37,700 --> 00:10:39,950
I am laughing as well, but it is no
laughing matter.

143
00:10:39,950 --> 00:10:43,030
Note that the numbers are set 
into the layout perfectly. The error

144
00:10:43,030 --> 00:10:46,880
was only noticed, because an
obviously bigger room had

145
00:10:46,880 --> 00:10:50,130
a smaller square footage than 
a smaller one next to it.

146
00:10:50,130 --> 00:10:55,529
There's a broom cupboard with
100 sqm and next to it a ball room

147
00:10:55,529 --> 00:10:58,730
with 4 sqm. 
(Laughter)

148
00:10:58,730 --> 00:11:02,060
It hardly gets any meaner.
The layout looks perfect.

149
00:11:02,060 --> 00:11:05,280
I do realise that the writing is 
really small. Don't you

150
00:11:05,280 --> 00:11:08,720
thinks this is some mean corner case
and I was working on

151
00:11:08,720 --> 00:11:13,520
for three month, just to finally
stick it up to Xerox.

152
00:11:13,520 --> 00:11:16,329
We will look at other examples.
This is the original case

153
00:11:16,329 --> 00:11:20,120
in which the bug was originally noticed,
and I didn't want to keep it from you.

154
00:11:20,120 --> 00:11:24,420
Here's the next one.
This is an expense register.

155
00:11:24,420 --> 00:11:27,880
(Laughter)

156
00:11:27,880 --> 00:11:31,340
Two sixes became eights.

157
00:11:31,340 --> 00:11:33,420
It's funny, I released the picture
it on my website,

158
00:11:33,420 --> 00:11:36,430
and I said: " Here a six became an eight."

159
00:11:36,430 --> 00:11:38,830
Then I get an e-mail:
"No, on the top there's another."

160
00:11:38,830 --> 00:11:47,260
(loud laughing and applause)

161
00:11:47,260 --> 00:11:52,200
Again perfectly set.
Why was it noticed this time?

162
00:11:52,200 --> 00:11:55,860
Because the numbers are supposed to
be sorted by size.

163
00:11:55,860 --> 00:11:58,280
What I want to say is

164
00:11:58,280 --> 00:12:00,800
it is impossible to notice. If I give
you some columns of numbers

165
00:12:00,800 --> 00:12:03,750
that don't make any noticable sense.
Then you could obviously

166
00:12:03,750 --> 00:12:07,889
not see, that there's wrong numbers.
It's always around there being

167
00:12:07,889 --> 00:12:12,390
semantic criteria, to make it
noticable. To make it

168
00:12:12,390 --> 00:12:16,199
obviously implausible. Otherwise
you have no chance to notice.

169
00:12:16,199 --> 00:12:18,420
Slowly I became a little worried.

170
00:12:18,420 --> 00:12:23,870
The neck length increases. To not let
this be some random events, I started

171
00:12:23,870 --> 00:12:28,990
working to reproduce the error on 
purpose. IT guy style

172
00:12:28,990 --> 00:12:33,460
invested a night and generated 
number columns in different

173
00:12:33,460 --> 00:12:36,890
sizes and fonts. I scanned those and 
experimented for

174
00:12:36,890 --> 00:12:42,570
a few hours. And, indeed, 
the error accurs again.

175
00:12:42,570 --> 00:12:45,670
These are my random numbers.
We will be able to work with those

176
00:12:45,670 --> 00:12:48,491
some more.
The eights marked in yellow

177
00:12:48,491 --> 00:12:54,100
should be sixes and do not 
belong there. Let's stay ourselfes shortly.

178
00:12:54,100 --> 00:12:58,300
I promised you in the introduction, that 
I would

179
00:12:58,300 --> 00:13:02,740
lay out the entire interaction with Xerox, 
that would follow, over time

180
00:13:02,740 --> 00:13:07,590
and tell you, how I felt at the corresponding
times and emphasize the things

181
00:13:07,590 --> 00:13:12,120
that according to my experience are 
extremely important

182
00:13:12,120 --> 00:13:15,450
when confronting a giant opponent.
And I will keep that promise.

183
00:13:15,450 --> 00:13:19,060
I will tell you why at all times. 
But now I will

184
00:13:19,060 --> 00:13:22,150
say one thing up front. This thing
I will discuss in different ways 
through the entire presentation.

185
00:13:22,150 --> 00:13:29,680
What never helps in my point of view
is unfriendly twittering and hating.

186
00:13:29,680 --> 00:13:34,800
(self-concious applause)

187
00:13:34,800 --> 00:13:39,490
It's really nice that you are applauding,
I wasn't sure that would happen.

188
00:13:39,490 --> 00:13:40,490
(laughter)

189
00:13:40,490 --> 00:13:42,860
I have nothing against twitter as such.
Nothing at all.

190
00:13:42,860 --> 00:13:45,220
But if you want to achieve something,
you make yourself vulnerable

191
00:13:45,220 --> 00:13:48,410
with such behaviour. And above 
all you won't be taken seriously.

192
00:13:48,410 --> 00:13:52,470
You can always be accused of
not wanting a proper discussion.

193
00:13:52,470 --> 00:13:55,380
That won't fit in 140 letters,
no matter what any of you say.

194
00:13:55,380 --> 00:14:01,580
(applause)

195
00:14:01,580 --> 00:14:04,829
Secondly you can always be accused
of seeking attention

196
00:14:04,829 --> 00:14:08,269
for yourself. Because almost 
everything is public on twitter.

197
00:14:08,269 --> 00:14:11,470
At the most twitter is useful for 
establishing first contact, when you

198
00:14:11,470 --> 00:14:15,010
ask for an e-mail adress or a phone number.
If I don't recommend twitter,

199
00:14:15,010 --> 00:14:19,649
what do I recommend? 
Much more serious and straight foreward

200
00:14:19,649 --> 00:14:23,610
is erverything, that is not public.
That way one shows willingness to work

201
00:14:23,610 --> 00:14:27,490
rationaly and not urge to scream around.
That's mail or phone calls.

202
00:14:27,490 --> 00:14:34,580
So we called the Xerox support.
Several times ...

203
00:14:34,580 --> 00:14:39,889
Often ... We phoned uo all the levels
up to the top level

204
00:14:39,889 --> 00:14:45,639
in Dublin - nobody knew 
anything.

205
00:14:45,639 --> 00:14:49,420
We also sought personal contact.
Staff from the local Xerox retailer

206
00:14:49,420 --> 00:14:54,730
came over. That's not Xerox themselves,
but a retail and support company.

207
00:14:54,730 --> 00:14:58,660
Thay were shocked - of course, right?
And then they tried to reproduce it

208
00:14:58,660 --> 00:15:03,200
themselves.
Zack! They reproduced it...

209
00:15:03,200 --> 00:15:11,320
(laughter and applause)

210
00:15:11,320 --> 00:15:15,360
That was .. we are laughing now.
They were standing there

211
00:15:15,360 --> 00:15:18,320
heads hanging low. You are standing 
there selling these things

212
00:15:18,320 --> 00:15:20,889
and suddenly you question your existence.

213
00:15:20,889 --> 00:15:25,510
That's not cool at all. At Xerox
- not the support company,

214
00:15:25,510 --> 00:15:29,779
but the entire, big Xerox, 140.000
employees,

215
00:15:29,779 --> 00:15:34,649
there was surprise, but no efforts
were made

216
00:15:34,649 --> 00:15:40,949
to help us or the retail company.
Meaning they were cautious of the problem.

217
00:15:40,949 --> 00:15:44,759
(laughs)
(laughter)

218
00:15:44,759 --> 00:15:47,829
So there were no signs at all 
of greater interest

219
00:15:47,829 --> 00:15:50,381
and no advice, as for solving 
the problem. Then one guy came

220
00:15:50,381 --> 00:15:55,310
from Xerox Central, who updated the 
software, we had an acient one

221
00:15:55,310 --> 00:15:58,250
installed. He installed the new software,
problem was still there.

222
00:15:58,250 --> 00:16:01,120
I thought: "Great, now we know
the problem existed in the fimware

223
00:16:01,120 --> 00:16:05,680
three years ago until today." Hmmm.

224
00:16:05,680 --> 00:16:08,459
When for more than a week nothing 
happened on Xerox's side

225
00:16:08,459 --> 00:16:11,330
that promised hope, I thought:
"Now you have been accommodating enough!"

226
00:16:11,330 --> 00:16:16,540
So I wrote a blog article in German and English

227
00:16:16,540 --> 00:16:21,399
about what I just told you about.
In this article I offered

228
00:16:21,399 --> 00:16:25,889
test documents to download. The readers can
print, scan and check whether

229
00:16:25,889 --> 00:16:30,699
they are affected or not. With that 
the spread of the story started.

230
00:16:30,699 --> 00:16:34,420
I have to add, my blog is not really huge
, really not. It has around

231
00:16:34,420 --> 00:16:38,990
500-1000 readers per day. That's 
not a huge amount, but also not nothing

232
00:16:38,990 --> 00:16:42,160
and the most readers are computer 
scientists of some form, I know that from the e-mails

233
00:16:42,160 --> 00:16:48,339
I get. On the bottom of my slides from now
on you can see a line.

234
00:16:48,339 --> 00:16:51,462
This line will continuously move 
further to the right. Thats a

235
00:16:51,462 --> 00:16:56,389
plot of the klicks. It's not meant 
to show off with clicks, but

236
00:16:56,389 --> 00:16:59,620
in context it's great to see, at what
time one gets attention in what way

237
00:16:59,620 --> 00:17:04,529
and also to see how fast it fades.
We will show that immediately.

238
00:17:04,529 --> 00:17:08,369
This small bump - yes, it's visible. 
The line

239
00:17:08,369 --> 00:17:12,709
moved to the right and there's a 
peak of 3000 hits/hour.

240
00:17:12,709 --> 00:17:15,400
Those numbers are from Google Analytics,
I have been told, one has

241
00:17:15,400 --> 00:17:18,459
to multiply them by two, but for order of
magnitude it's enough.

242
00:17:18,459 --> 00:17:21,989
On the 2nd and 3rd of August the story
hit on several tech-blogs.

243
00:17:21,989 --> 00:17:26,029
At this point I declare the long-known
fefe as tech-blog.

244
00:17:26,029 --> 00:17:28,810
(laughter)

245
00:17:28,810 --> 00:17:32,110
I know, I know, there's the first protest.
But I will agree on the fact,

246
00:17:32,110 --> 00:17:36,860
that fefe is read by a lot of IT-poeple.
Alright, I am

247
00:17:36,860 --> 00:17:41,300
not hearing any more protest. The peak
you see here is because of blog.fefe.de .

248
00:17:41,300 --> 00:17:44,770
The message spreads, and I get 
more and more mails from readers

249
00:17:44,770 --> 00:17:49,520
that are affected. The most concerning is
that I get e-mails with confirmations

250
00:17:49,520 --> 00:17:52,980
for a lot of Xerox-Workcentres
that I don't even know.

251
00:17:52,980 --> 00:17:54,260
(laughter)

252
00:17:54,260 --> 00:17:57,570
I told you before these things are 
one giant family of products. Very slowly

253
00:17:57,570 --> 00:18:02,429
I realise, that this could turn into
something bigger eventually.

254
00:18:02,429 --> 00:18:06,830
Lesson learned: It was good to 
release the test-documents online

255
00:18:06,830 --> 00:18:10,120
with the article. Would the users not 
have been able to check for themselves

256
00:18:10,120 --> 00:18:16,020
using the test-documents, the story would 
never have had an impact like it would soon have.

257
00:18:16,020 --> 00:18:19,450
On the 4th of August the story arrived in 
tech-portals around the world.

258
00:18:19,450 --> 00:18:23,070
In the slide is Hacker News by 
Y-Combinator, that's one of the biggest

259
00:18:23,070 --> 00:18:28,540
of this kind, you probably know it.
From now on I get hundreds of technically

260
00:18:28,540 --> 00:18:32,991
versed e-mails a day. I say "technically versed",
because there were also others

261
00:18:32,991 --> 00:18:38,250
that were less technical.
Over the entire time I

262
00:18:38,250 --> 00:18:41,350
spend days to channel and sort
the news

263
00:18:41,350 --> 00:18:45,590
I get. This enabled me to 
continue the reporting

264
00:18:45,590 --> 00:18:50,500
in a professionaly and to get to the
roots of the bug with professional help.

265
00:18:50,500 --> 00:18:53,720
The whole thing becomes an avalanche
and I am not allowed to sleep any more.

266
00:18:53,720 --> 00:18:57,350
Cause the US press is on the phone constantly.
You must not think that US- journalists

267
00:18:57,350 --> 00:19:01,500
ever realise, that there's a thing
called time zones ....

268
00:19:01,500 --> 00:19:10,450
(laughter and applause)

269
00:19:10,450 --> 00:19:13,630
Here's another anecdote. One would
think the US media journalists are

270
00:19:13,630 --> 00:19:17,230
competitors. Meaning if one had a special 
information he would not pass

271
00:19:17,230 --> 00:19:20,850
it on to the others, right? As soon as the 
colleague from ABC had my phone number

272
00:19:20,850 --> 00:19:24,860
ALL of them had it. I tell you, it's 
incredible! (laughs)

273
00:19:24,860 --> 00:19:30,510
Lesson learned: Write these things in
multiple languages! Important are English

274
00:19:30,510 --> 00:19:35,010
for the international space. Also the language 
of the home market of the company,

275
00:19:35,010 --> 00:19:39,410
you are confronting. In my case thats the 
USA, so English, again

276
00:19:39,410 --> 00:19:43,400
two birds with one stone.
By the way: in the US Xerox is so strong

277
00:19:43,400 --> 00:19:48,280
that "to copy" is called "to xerox" there.
They really say that

278
00:19:48,280 --> 00:19:51,019
in everyday conversation. The same way
we say: "Hand me a Tempo! (cotton tissue)",

279
00:19:51,019 --> 00:19:55,160
just to give you an impression of how much
repute the company and the brand

280
00:19:55,160 --> 00:19:59,410
has there. And when in the world 
of technology something like this goes around

281
00:19:59,410 --> 00:20:06,240
what's next? Mass media
(some laughing)

282
00:20:06,240 --> 00:20:10,970
And there you get the whole package.
We'll just click through here to

283
00:20:10,970 --> 00:20:14,440
illustrate it. This list is in no way
complete, there were thousands of

284
00:20:14,440 --> 00:20:17,720
articles suddenly, all over the world.
And if I show an article, then

285
00:20:17,720 --> 00:20:20,750
- just as a disclaimer - it doesn't make
a statement about the date of publishing

286
00:20:20,750 --> 00:20:23,490
statement about the date of publishing,
I just make it in a way that's good for the show

287
00:20:23,490 --> 00:20:25,350
(some laughter)

288
00:20:25,350 --> 00:20:29,840
Browsing, here is Heise, of course
that joys me as a computer scientist,

289
00:20:29,840 --> 00:20:33,530
they covered the whole story in five
articles or so.

290
00:20:33,530 --> 00:20:37,720
ZDF Hyperland, yes? I'm demonstrating
the german press a bit here.

291
00:20:37,720 --> 00:20:40,440
The german press was very
reserved. The most articles

292
00:20:40,440 --> 00:20:42,910
were in fact from abroad. 
Therefore the comment

293
00:20:42,910 --> 00:20:47,320
about the "home market". But here
a small anecdote about the german press.

294
00:20:47,320 --> 00:20:51,480
A journalist told me that he wanted to
bring the story to the "Tagesschau".

295
00:20:51,480 --> 00:20:56,549
They told him "Yeah, hmm, it's alright.
But for this we want it to happen

296
00:20:56,549 --> 00:21:00,200
during real copying, and not just
during scanning!"

297
00:21:00,200 --> 00:21:09,030
(laughter and applause)

298
00:21:09,030 --> 00:21:14,000
If anyone from the "Tagesschau" is 
watching, this applause is for you!

299
00:21:14,000 --> 00:21:15,080
(laughter)

300
00:21:15,080 --> 00:21:18,851
So I think: You geniuses!
Pro Tip: If you print a scan,

301
00:21:18,851 --> 00:21:21,940
then you have a copy!
(laughter)

302
00:21:21,940 --> 00:21:24,470
With the difference, that such a 
saved scan can cause

303
00:21:24,470 --> 00:21:28,610
harm even years later.
But please! So I thought,

304
00:21:28,610 --> 00:21:32,920
no "Tagesschau" story, it's going
around the world already anyways,

305
00:21:32,920 --> 00:21:36,790
not my problem if they are the only
ones not covering it.

306
00:21:36,790 --> 00:21:41,789
Lesson learned: Stay professional and
sovereign. Don't just bloat things

307
00:21:41,789 --> 00:21:45,530
out of thirst for attention.
Every one of you can probably name

308
00:21:45,530 --> 00:21:48,350
some affaire, that went 
rather well

309
00:21:48,350 --> 00:21:50,630
for whoever made it public,
and then in the

310
00:21:50,630 --> 00:21:54,140
decisive moment he tasted
blood and made something up.

311
00:21:54,140 --> 00:21:59,730
That's bad of course. Oh well.
The Economists, that's really

312
00:21:59,730 --> 00:22:03,530
vintage, I liked this title:
"Lies, damned lies and scans"

313
00:22:03,530 --> 00:22:09,950
That comes from Tom Sawyer:
"Lies, damned lies and statistics"

314
00:22:09,950 --> 00:22:12,380
Now PR wise, we're at a point
where it's expensive.

315
00:22:12,380 --> 00:22:15,980
The Economists has influence.
ABC News - even more expensive.

316
00:22:15,980 --> 00:22:18,500
There are the colleagues with
their phones.

317
00:22:18,500 --> 00:22:23,620
BBC, CNBC.
Suddenly, it was everywhere.

318
00:22:23,620 --> 00:22:26,480
My powerpoint is lagging, here
it is again. Business Week,

319
00:22:26,480 --> 00:22:32,890
that is a popular economy 
magazine. I'll recall here,

320
00:22:32,890 --> 00:22:37,930
until now, no reaction from Xerox.
Yes, three days in business,

321
00:22:37,930 --> 00:22:42,590
worldwide. No reaction! And when you
take that long, the tone gets

322
00:22:42,590 --> 00:22:47,539
really rough. I quote: "On the scale
of things, that are too terrible

323
00:22:47,539 --> 00:22:50,370
to imagine, document altering
scanners are somewhere

324
00:22:50,370 --> 00:22:52,500
up there with meat 
eating bacteria."

325
00:22:52,500 --> 00:23:02,990
(laughter)

326
00:23:02,990 --> 00:23:07,760
They are actually writing this in the
Business Week! (laughs)

327
00:23:07,760 --> 00:23:10,020
So I was called my a friend of 
mine, listen you have to

328
00:23:10,020 --> 00:23:14,640
read this. Great! Imagine,
there's Peter Coy, he's editor there,

329
00:23:14,640 --> 00:23:18,530
that we will see again a few more 
times over the course of this talk.

330
00:23:18,530 --> 00:23:23,910
So, my blog article is now at
about 100.000 visitors per day.

331
00:23:23,910 --> 00:23:28,250
And still, no feedback from 
Xerox. In the meantime

332
00:23:28,250 --> 00:23:31,780
I was able to explain, with the help 
of many reader-mails,

333
00:23:31,780 --> 00:23:35,870
what's happening at all.
And that's what I am telling you now,

334
00:23:35,870 --> 00:23:39,559
so we make a small excourse
about image compression.

335
00:23:39,559 --> 00:23:43,380
Here we have a test image,
that I made. It's a

336
00:23:43,380 --> 00:23:47,890
sundew, with a fly on it, that's
a plant. The fly as well as the

337
00:23:47,890 --> 00:23:52,009
text belong to this test image.
For us to have a nice variety of pictures.

338
00:23:52,009 --> 00:23:57,769
Data transfer costs time, money
and storage. Image consist,

339
00:23:57,769 --> 00:24:01,559
compared to text, of a great amount
of data. And to send and save pictures

340
00:24:01,559 --> 00:24:06,260
completely uncompressed would
be really expensive.

341
00:24:06,260 --> 00:24:10,310
And images are sent everywhere, yes?
The use is there for every one

342
00:24:10,310 --> 00:24:13,880
of us. I tell you, it goes to the highest
possible scenarios.

343
00:24:13,880 --> 00:24:16,630
Just recently there was a giant
coverage, and even an

344
00:24:16,630 --> 00:24:20,430
investigation by the government,
just because a former member of

345
00:24:20,430 --> 00:24:24,550
the parliament transferred pictures.
(laughter)

346
00:24:24,550 --> 00:24:28,680
(laughs)
So now, this member of the parliament

347
00:24:28,680 --> 00:24:33,530
can't wait for his pictures forever, 
so we have to compress the image data.

348
00:24:33,530 --> 00:24:34,710
(laughs again)

349
00:24:34,710 --> 00:24:38,680
Listen here!
(laughs stupidly)

350
00:24:38,680 --> 00:24:45,000
(applause)

351
00:24:45,000 --> 00:24:48,990
Now we have two parts of my test
image. One image part

352
00:24:48,990 --> 00:24:53,240
and one text part. And I enhanced
it so much you can see individual

353
00:24:53,240 --> 00:24:57,179
pixels. This is so we can see what 
go wrong with different compression

354
00:24:57,179 --> 00:25:01,890
methods. There is lossless 
compression. Here the

355
00:25:01,890 --> 00:25:04,720
image data stays as is, it is
just somehow stored more

356
00:25:04,720 --> 00:25:09,300
efficient. Or we accept losses,
so, changes in the image data,

357
00:25:09,300 --> 00:25:15,789
to "squish" the data and make it
even smaller.

358
00:25:15,789 --> 00:25:20,550
Here are the popular
GIF-images.

359
00:25:20,550 --> 00:25:26,540
Can I have a small hand sign, who
thinks that GIF has lossy compression?

360
00:25:26,540 --> 00:25:29,540
Wow, that's a lot! Almost everyone.

361
00:25:29,540 --> 00:25:32,600
GIF is a lossless compression
method.

362
00:25:32,600 --> 00:25:36,090
The downside is, it only supports
256 colours.

363
00:25:36,090 --> 00:25:39,220
The here shown lower quality stems not 
from the image being saved

364
00:25:39,220 --> 00:25:42,440
as a GIF, but from the colour
reduction.

365
00:25:42,440 --> 00:25:45,640
To be able to see it better, I 
reduced the colour amount to 16.

366
00:25:45,640 --> 00:25:48,730
Here you see it nicely, uiuiui. So.

367
00:25:48,730 --> 00:25:53,029
The finished image is saved pixel
for pixel, and then LZW compressed.

368
00:25:53,029 --> 00:25:57,000
LZW is an old compression algorithm,
similar to ZIP.

369
00:25:57,000 --> 00:26:01,130
GIF is very suited for graphics with few
colours. And because pixels are still

370
00:26:01,130 --> 00:26:04,580
saved completely one by one,
sharp edges are well

371
00:26:04,580 --> 00:26:09,030
represented. You can see, the 
text looks pretty good. It's less good

372
00:26:09,030 --> 00:26:14,490
in photographs, as you can see. Most
widespread are JPEG images. And JPEG

373
00:26:14,490 --> 00:26:19,920
is lossy. The original image doesn't get
saved pixel for pixel anymore,

374
00:26:19,920 --> 00:26:25,480
but instead gets split into 8x8
pixel blocks. And every block then

375
00:26:25,480 --> 00:26:29,080
gets approximated with cosinus-waves.
How exactly this works mathematically,

376
00:26:29,080 --> 00:26:32,299
we can spare ourselves from here.
But it is good to know, that this

377
00:26:32,299 --> 00:26:36,240
kind of compression, it's good for
pictures, but bad for sharp edges,

378
00:26:36,240 --> 00:26:40,730
as you can see in the letters, yes,
you can see artifacts, you can see

379
00:26:40,730 --> 00:26:44,000
some stains around it. But usually
this would be full of artifacts,

380
00:26:44,000 --> 00:26:48,160
the image. I can hold up
my notebook or so.

381
00:26:48,160 --> 00:26:51,530
Long story short. Depending on the
type of image, certain compression

382
00:26:51,530 --> 00:26:56,059
methods are good, and
others aren't.

383
00:26:56,059 --> 00:27:00,491
That's why there is the JBig2-fomat.
This is one of the special words, that I

384
00:27:00,491 --> 00:27:04,670
wrote down in three variants for the
translators.

385
00:27:04,670 --> 00:27:08,970
Here you can dissect one image in
multiple sub images. The red

386
00:27:08,970 --> 00:27:12,789
circled here as an example. These are
sub images. These sub images we call

387
00:27:12,789 --> 00:27:17,800
"patches", english for "Flicken".
As we see, there are parts of the image,

388
00:27:17,800 --> 00:27:22,039
that don't belong to any patch.
That's pretty cool, because

389
00:27:22,039 --> 00:27:24,700
the data for these won't 
need to be saved at all.

390
00:27:24,700 --> 00:27:29,890
You just say, background white. The joke
here is, these seperate patches, you can

391
00:27:29,890 --> 00:27:34,520
compress these with multiple
compression methods.

392
00:27:34,520 --> 00:27:38,970
The text patches, for example with GIF, 
I'll show it just very roughly here.

393
00:27:38,970 --> 00:27:44,680
You probably can't use GIF in JBig2.
But the principle stays.

394
00:27:44,680 --> 00:27:50,540
And the photo patch for example with JPEG.
Every patch its suited compression method.

395
00:27:50,540 --> 00:27:54,160
That's a real advancement. I probably
won't have to explain anyone here,

396
00:27:54,160 --> 00:27:58,669
that with this you will know, which patch
contains what, get a good

397
00:27:58,669 --> 00:28:04,050
quality, and probably a
smaller file size. So,

398
00:28:04,050 --> 00:28:08,140
if you dissect the image into patches
anyway, you might as well use a

399
00:28:08,140 --> 00:28:13,039
completely new high tech compression
method. You can dissect the original image

400
00:28:13,039 --> 00:28:17,990
much finer, and have every individual
letter as its own patch.

401
00:28:17,990 --> 00:28:20,669
That's a lot of patches.
A whole lot of patches.

402
00:28:20,669 --> 00:28:23,690
And you can do this with text
pages and books. And its used,

403
00:28:23,690 --> 00:28:26,910
I didn't just make that up now.

404
00:28:26,910 --> 00:28:32,240
So next we see, which patches 
are similar to each other.

405
00:28:32,240 --> 00:28:36,440
This step is called "pattern matching".
I have marked four patches with arrows

406
00:28:36,440 --> 00:28:40,720
here. These patches are very similar.
No wonder, you will say.

407
00:28:40,720 --> 00:28:45,770
All of them are small "e"s. They are
only different by a few pixels.

408
00:28:45,770 --> 00:28:50,110
Through this pattern matching, you get 
a group of similar symbols.

409
00:28:50,110 --> 00:28:54,620
For this group, you only really save one
of those symbols, and that is

410
00:28:54,620 --> 00:28:58,240
used over and over in the
compressed image.

411
00:28:58,240 --> 00:29:02,820
Instead of his brothers. From these four
marked "e"s, only one would be

412
00:29:02,820 --> 00:29:06,850
really saved, and then replaced all
the other ones. This way you can really

413
00:29:06,850 --> 00:29:10,500
save a lot of data, with minimal
quality loss.

414
00:29:10,500 --> 00:29:14,159
Here is the final product. Looks still
good, doesn't it? No artifacts

415
00:29:14,159 --> 00:29:19,870
visible. Takes a lot less data than
without pattern matching.

416
00:29:19,870 --> 00:29:24,910
Did you see that? The pattern matching
thinks the I is similar to the small L,

417
00:29:24,910 --> 00:29:28,980
so you can replace that with it.
This happens, when pattern matching

418
00:29:28,980 --> 00:29:39,519
works inaccurate. 
Did you see this too?

419
00:29:39,519 --> 00:29:43,590
These are incredibly dangerous
mistakes.

420
00:29:43,590 --> 00:29:46,520
Usual compression errors are not
so bad. Then one letter is

421
00:29:46,520 --> 00:29:52,470
unreadable. You see it, and you know that
something went wrong, "scan again please".

422
00:29:52,470 --> 00:29:56,740
But here you have actual wrong data, that
looks flawless. And they get layoutet in

423
00:29:56,740 --> 00:30:01,780
perfectly because of the similarities.
You have to actually read this, to

424
00:30:01,780 --> 00:30:05,659
notice the mistake. And even then,
you can only see the mistake,

425
00:30:05,659 --> 00:30:09,000
when the document becomes obviously
implausible, like in the blueprint.

426
00:30:09,000 --> 00:30:12,920
I don't know about you guys. But I don't
read through all of my scans,

427
00:30:12,920 --> 00:30:18,890
that I take, just to see if it has
any mistakes.

428
00:30:18,890 --> 00:30:22,059
But my friends, a politician that would
have to gloss over this,

429
00:30:22,059 --> 00:30:26,010
he would say: "Scan a medicine 
dosing with a Xerox-device

430
00:30:26,010 --> 00:30:29,130
in a retirement home, and there is
a high chance that in no time

431
00:30:29,130 --> 00:30:31,620
you'll relieve the pension funds."
(laughter)

432
00:30:31,620 --> 00:30:39,890
(applause)

433
00:30:39,890 --> 00:30:43,679
Now it is clear, that this also related to
security. Until now, you could have

434
00:30:43,679 --> 00:30:47,490
asked, why does David hold a speech about
copying machines on the congress?

435
00:30:47,490 --> 00:30:50,340
But this is actually about a severe
failure of a company,

436
00:30:50,340 --> 00:30:55,270
that is a serious security issue.
Is anyone here from Berlin?

437
00:30:55,270 --> 00:30:58,080
Maybe a hand sign?

438
00:30:58,080 --> 00:31:00,779
What did the blueprints for the
airport get scanned with?

439
00:31:00,779 --> 00:31:10,240
(laughter and applause)

440
00:31:10,240 --> 00:31:14,590
But you know what? Airports,
medicine, rockets, airplanes...

441
00:31:14,590 --> 00:31:19,220
As big as this is, that's all trivial.
It gets interesting at the question,

442
00:31:19,220 --> 00:31:22,510
where those scans got used in court
as evidence, that

443
00:31:22,510 --> 00:31:27,429
can be reexamined now.
Or the other way around,

444
00:31:27,429 --> 00:31:30,820
if one of you sues me with a 
Xerox-scan, from now on I'll just

445
00:31:30,820 --> 00:31:34,800
tell you: "Ah, you know what,
it's faulty!" (laughs)

446
00:31:34,800 --> 00:31:37,610
Now you can look for the
original first, to prove me

447
00:31:37,610 --> 00:31:41,890
otherwise. I can't prove anymore,
that that part of the scan also

448
00:31:41,890 --> 00:31:45,969
comes from the part of the paper
that you expect it to be from.

449
00:31:45,969 --> 00:31:50,500
The legal value is zero! There's hundreds
of thousands of industrial copiers

450
00:31:50,500 --> 00:31:54,610
worldwide. Those are business devices, 
every machine has many users, even more

451
00:31:54,610 --> 00:31:59,190
documents that were made by it, that were
distributed whereever. And so you can

452
00:31:59,190 --> 00:32:03,200
have an idea, a large company called
me, their letter processing works so,

453
00:32:03,200 --> 00:32:07,500
that incoming leters just get scanned
immideately by machines,

454
00:32:07,500 --> 00:32:10,479
and from there on they only exist
electronically. Have fun, if

455
00:32:10,479 --> 00:32:14,470
those contain errors. So, we come
back to the implications later again.

456
00:32:14,470 --> 00:32:20,610
But for now, back to the story. It's the
5th of August. We are three days after

457
00:32:20,610 --> 00:32:25,700
the first impact, and on the third day god
created, finally yes, a life sign

458
00:32:25,700 --> 00:32:30,189
by Xerox. Now, they are
watching after all man! (laughs)

459
00:32:30,189 --> 00:32:34,209
(applause)

460
00:32:34,209 --> 00:32:36,740
Thank you (laughs)

461
00:32:36,740 --> 00:32:40,020
The PR of Xerox Germany calls me.
The talk is very unproductive.

462
00:32:40,020 --> 00:32:42,799
They can't do anything without the 
americans. At first,

463
00:32:42,799 --> 00:32:45,750
they though it was a joke. I say,
it's not. And then

464
00:32:45,750 --> 00:32:48,540
we said, we will stay in contact.
(laughs)

465
00:32:48,540 --> 00:32:52,679
(laughter and applause)

466
00:32:52,679 --> 00:32:58,309
And so, the day after, 6th of August,
for the first time it really had a punch.

467
00:32:58,309 --> 00:33:00,950
In the morning, I get a screenshot
by a reader, from

468
00:33:00,950 --> 00:33:05,170
one of the details from the admin panel
of his Xerox-copiers. There they talk

469
00:33:05,170 --> 00:33:11,399
about letter replacement. Aha! For the
record, now. We can all learn this

470
00:33:11,399 --> 00:33:13,920
here: There are three PDF
compression levels.

471
00:33:13,920 --> 00:33:18,949
These are called "Normal", "Higher",
and "High". Very marketing appropriate.

472
00:33:18,949 --> 00:33:24,600
So, "Normal" is the mode, that compresses
the most. The reader says:

473
00:33:24,600 --> 00:33:27,649
on "Normal", the error occurs, in the
higher levels it doesn't.

474
00:33:27,649 --> 00:33:34,040
My tests seem to comfirm this. I say it
extra vague here, more on it later.

475
00:33:34,040 --> 00:33:38,340
(pauses to drink)

476
00:33:38,340 --> 00:33:41,190
I promised you to show you the 
moods over this situation,

477
00:33:41,190 --> 00:33:44,890
in case something like it ever happens
to you. And really: In the first moment

478
00:33:44,890 --> 00:33:49,110
my heart dropped into my gut.
I was scared shitless, to be the idiot

479
00:33:49,110 --> 00:33:51,770
that didn't read the manual, yes?
(laughter)

480
00:33:51,770 --> 00:33:55,110
Because there is still no
official Xerox-statement, and I got

481
00:33:55,110 --> 00:33:58,400
a tip from the press, that Xerox says
exactly this in their statement.

482
00:33:58,400 --> 00:34:03,940
Lesson Learned: What's the difference
between inside and outside view?

483
00:34:03,940 --> 00:34:07,809
Exactly this. No? Surely you think:
"Hello? Why is David so agigated,

484
00:34:07,809 --> 00:34:10,310
it's clear that this type of document
error should

485
00:34:10,310 --> 00:34:13,710
never have happened, not even 
unknowingly." But from the inside...

486
00:34:13,710 --> 00:34:18,960
It looks different. Despite being scared,
it's important: Stay calm, act rational.

487
00:34:18,960 --> 00:34:22,349
Because of anxious moments like this,
it's important that previously you

488
00:34:22,349 --> 00:34:26,990
never screech and de-escalate.
Never rabble beforehand.

489
00:34:26,990 --> 00:34:29,670
If you were always sovereign,
you can appear confident,

490
00:34:29,670 --> 00:34:33,410
and in doubt, calmy and publically ask:
"Well, boys? Why did the

491
00:34:33,410 --> 00:34:36,489
support not tell me this
two weeks ago, eh?"

492
00:34:36,489 --> 00:34:41,770
Lesson Learned: Appear professional
from the start, never hate. I'll repeat

493
00:34:41,770 --> 00:34:46,730
that again. So, now,
defense to the front. I presented

494
00:34:46,730 --> 00:34:50,580
the screenshot as a possible workaround
and advised: Turn compression

495
00:34:50,580 --> 00:34:53,801
on "Higher". Additionally I wrote,
that I was wondering a bit,

496
00:34:53,801 --> 00:34:57,250
why the support couldn't say this
to me over the course of a whole week.

497
00:34:57,250 --> 00:35:01,400
I also criticized, that the setting is 
called "Normal". (laughs)

498
00:35:01,400 --> 00:35:04,750
And the possible consequences
I showed to you, of course those stay,

499
00:35:04,750 --> 00:35:07,210
because on the scan you can't 
see, that it might

500
00:35:07,210 --> 00:35:12,579
contain errors. The goal was, to give the
thing a spin, before Xerox fights back.

501
00:35:12,579 --> 00:35:18,310
It follows a telephone conference with
Rick Dastin. (murmur)

502
00:35:18,310 --> 00:35:22,520
I see, he is known in the audience,
the vice president worldwide of Xerox.

503
00:35:22,520 --> 00:35:24,990
And Franics Tse, one of their
chief engineers, that

504
00:35:24,990 --> 00:35:28,570
was handling the image compression.
Guys, the boss does support himself!

505
00:35:28,570 --> 00:35:37,299
(laughter and applause)

506
00:35:37,299 --> 00:35:41,740
Rick Dastin was in fact the first person
that work at Xerox,

507
00:35:41,740 --> 00:35:45,300
that I got officially told by, that
the letter replacement was

508
00:35:45,300 --> 00:35:49,900
in fact already known by Xerox. So,
if you'd like to know, what the

509
00:35:49,900 --> 00:35:53,250
support can't tell you after a week, then
you say: "I want to

510
00:35:53,250 --> 00:35:56,960
talk to Rick Dastin!"
(laughter)

511
00:35:56,960 --> 00:36:00,160
And here, it was revealed that
the theory, that the pattern matching

512
00:36:00,160 --> 00:36:03,680
was at fault, was true. Dastin also
confirmed, that the pattern matching

513
00:36:03,680 --> 00:36:07,880
is only used in "Normal" mode.
So after a bit of discussion, it was

514
00:36:07,880 --> 00:36:12,330
also clear, that the support fucked up,
and the name

515
00:36:12,330 --> 00:36:16,680
"Normal" might be badly chosen. I then
suggested "Experimental".

516
00:36:16,680 --> 00:36:25,250
(laughter and applause)

517
00:36:25,250 --> 00:36:28,970
Maybe here: I'm really in a good mood,
and this is a lot of fun,

518
00:36:28,970 --> 00:36:31,480
and we are all laughing, but in 
that moment I was

519
00:36:31,480 --> 00:36:35,530
just more nervous. Not that you
think it would be different for you.

520
00:36:35,530 --> 00:36:41,190
There I'll be completely honest. And then
comes a clear "RTFM" from Xerox.

521
00:36:41,190 --> 00:36:43,990
First: "Normal" mode, David,
is not even a factory setting!

522
00:36:43,990 --> 00:36:48,030
Dear customers, you're all stupid.
Who would set it to such thing!

523
00:36:48,030 --> 00:36:51,620
Second: That letters can get swapped,
that is explained in the manual,

524
00:36:51,620 --> 00:36:56,490
on two seperate occasions.
Dear customers: double stupid!

525
00:36:56,490 --> 00:36:59,750
For the factory setting: Of course
that's only a half truth. For the

526
00:36:59,750 --> 00:37:03,990
customer, factory setting is, what the 
device gets delivered with. Xerox doesn't

527
00:37:03,990 --> 00:37:08,340
supply to big customers. Those sales
go over third parties.

528
00:37:08,340 --> 00:37:12,309
If you order a Xerox-copier, you do it
over another company,

529
00:37:12,309 --> 00:37:15,789
that isn't Xerox, and they will advise
you and there you can configure

530
00:37:15,789 --> 00:37:19,800
whatever before they ship it. And for the
manual: The notice is in some manuals

531
00:37:19,800 --> 00:37:26,190
indeed. But then I looked closer: On page
107 and 328 in the text, yes?

532
00:37:26,190 --> 00:37:29,800
Now we are all old enough to know, how
many people will read a 300 page

533
00:37:29,800 --> 00:37:34,470
manual, before handling a printer.
(laughter)

534
00:37:34,470 --> 00:37:39,050
I also thought, that copiers generally
shouldn't be designed in a way, so

535
00:37:39,050 --> 00:37:42,679
those errors can occur at all.
That can't be, no one expects that.

536
00:37:42,679 --> 00:37:49,090
(applause)

537
00:37:49,090 --> 00:37:53,330
The answer was: "Yes, it can be!"
(laughter)

538
00:37:53,330 --> 00:37:55,890
"The market wants it this way,
errors would just..."

539
00:37:55,890 --> 00:38:01,100
(laughter)

540
00:38:01,100 --> 00:38:04,810
That was indeed a statement, that
was said exactly like this. I quote here,

541
00:38:04,810 --> 00:38:07,940
but of course that only related to small 
file sizes. And errors

542
00:38:07,940 --> 00:38:11,300
would also be very rare. But I would be
right, you can't prove, that a

543
00:38:11,300 --> 00:38:14,840
document is free of errors. So, all in all
the talk had a nice

544
00:38:14,840 --> 00:38:18,500
atmosphere. They really didn't try to
squash me legally or so.

545
00:38:18,500 --> 00:38:21,740
They listened very nicely, the talk was
super long too, 45 minutes

546
00:38:21,740 --> 00:38:26,530
or so. And then I let myself get
caught by them, like an amateur.

547
00:38:26,530 --> 00:38:29,910
You have to consider, I had never done
anything on a scale like this.

548
00:38:29,910 --> 00:38:34,050
And with a company like Xerox, they have
professionals. I was already wondering,

549
00:38:34,050 --> 00:38:37,849
why we were talking so peacefully for such
a long time. Dastin is the vice

550
00:38:37,849 --> 00:38:41,640
president of a worldwid operating
company after all. And he probably

551
00:38:41,640 --> 00:38:48,960
has other stuff to do. And now it turns
out, during the phone talk,

552
00:38:48,960 --> 00:38:54,670
Xerox published a statement.
Not bad at all. During that time

553
00:38:54,670 --> 00:38:58,590
I couldn't react after all. And it
had the beautiful title "Always listening

554
00:38:58,590 --> 00:39:03,190
to our customers"... right at the moment!
(laughs)

555
00:39:03,190 --> 00:39:06,151
And they write in their statement, for
error free files, please

556
00:39:06,151 --> 00:39:09,590
use a compression setting of
at least "Higher", and the error

557
00:39:09,590 --> 00:39:14,020
would be written about in the manual.
RTFM. Lesson learned: Have someone

558
00:39:14,020 --> 00:39:19,200
watch the side of the enemy.
So I wrote my own article,

559
00:39:19,200 --> 00:39:21,732
about the contents of the phone call,
the one that

560
00:39:21,732 --> 00:39:25,480
I just told you about. Well, and then
I also wrote,

561
00:39:25,480 --> 00:39:29,109
that I don't think they're off
the hook yet. And now?

562
00:39:29,109 --> 00:39:32,371
This could've been over here.
When a single blogger goes up

563
00:39:32,371 --> 00:39:37,230
against a giant company, it
usually ends one of three ways, when

564
00:39:37,230 --> 00:39:40,300
the company shoots back: Either the
blogger gives in after,

565
00:39:40,300 --> 00:39:44,280
or the public sides with the company,
or the public

566
00:39:44,280 --> 00:39:47,040
loses interest, when the company
shot back.

567
00:39:47,040 --> 00:39:53,342
Every one of you can now think of three
stories, where it was like this.

568
00:39:53,342 --> 00:39:56,930
But none of this happened. You see
the giant increase at the bottom. The

569
00:39:56,930 --> 00:40:02,520
story was on the cover of Slashdot.
And the press, luckily,

570
00:40:02,520 --> 00:40:06,320
also had their attention on me.
Here for example, Heise writes, that I

571
00:40:06,320 --> 00:40:09,510
offered the workaround even before Xerox.
(laughs)

572
00:40:09,510 --> 00:40:15,999
(laughter and applause)

573
00:40:15,999 --> 00:40:19,060
I'll exceed my time limit a bit.

574
00:40:19,060 --> 00:40:23,270
Or also, bone dry, "Spiegel". They wrote:
"So so, Xerox knew about the problem

575
00:40:23,270 --> 00:40:27,451
for years?" (laughs dumbly)
That's really... If you sit in

576
00:40:27,451 --> 00:40:30,400
PR of a company, and this
happens to you, I guarantee

577
00:40:30,400 --> 00:40:34,089
you don't need to take vacation
for the rest of the year.

578
00:40:34,089 --> 00:40:39,120
But it gets real funny, when the story
arrives at internet humour.

579
00:40:39,120 --> 00:40:42,420
I won't withhold this from you. I don't 
know who of you has lived in

580
00:40:42,420 --> 00:40:45,750
the US before. In german, we have the
vulgar saying: "Now the shit is

581
00:40:45,750 --> 00:40:51,830
steaming". And the americans say
"Shit hits the fan".

582
00:40:51,830 --> 00:40:55,170
The day after this story is on the
front page of Reddit. The circled

583
00:40:55,170 --> 00:40:58,770
comments brings the most eloquent
version of "Shit hits the fan", that I

584
00:40:58,770 --> 00:41:07,000
have ever seen.
(laughter)

585
00:41:07,000 --> 00:41:09,859
Yes, but what he says, is true. I already
said it earlier.

586
00:41:09,859 --> 00:41:14,530
When a company is depending on document
digitalization, and you think about it,

587
00:41:14,530 --> 00:41:19,390
who isn't these days, then we have a
problem. They can shut down the

588
00:41:19,390 --> 00:41:23,440
company, if they are unlucky. For
example, I was called by the management

589
00:41:23,440 --> 00:41:28,119
of a state archive. They created their 
archive with Xerox devices, and what did

590
00:41:28,119 --> 00:41:30,940
they do then? They thew away the
originals. Ye?

591
00:41:30,940 --> 00:41:32,850
(spiteful laughter)

592
00:41:32,850 --> 00:41:37,760
Now they stand there, with an empty gaze
in front of their scanner fleet, and then

593
00:41:37,760 --> 00:41:42,700
they can check all their documents for
plausability. But even otherwise the

594
00:41:42,700 --> 00:41:45,609
internet humour is amazing.
(laughter)

595
00:41:45,609 --> 00:41:56,189
(applause)

596
00:41:56,189 --> 00:41:58,560
Even the involved provide
the humour themselves.

597
00:41:58,560 --> 00:42:01,890
If you, as the Xerox vice president, 
get the same interviews all day,

598
00:42:01,890 --> 00:42:04,370
maybe mistakes happen.
This one's pretty good. You

599
00:42:04,370 --> 00:42:09,599
don't need to read, I'll read it out real
quick. Of all things, in front of BBC

600
00:42:09,599 --> 00:42:12,171
Dastin tried to explain. He
said: "You know, all this is

601
00:42:12,171 --> 00:42:15,810
half so bad, this "Normal" 
compression mode, it can

602
00:42:15,810 --> 00:42:19,260
produce errors, but almost no one
uses that, only the military or some

603
00:42:19,260 --> 00:42:26,249
oil drilling platform."
(laughter and applause)

604
00:42:26,249 --> 00:42:31,710
Yeah, what could go wrong?
(laughs childlike)

605
00:42:31,710 --> 00:42:34,170
So, now we have...
(laughter)

606
00:42:34,170 --> 00:42:37,089
(laughs)
Now we all noticed,

607
00:42:37,089 --> 00:42:40,790
that errors on oil drilling platforms in
the USA were a bit neglected

608
00:42:40,790 --> 00:42:45,960
lately. Now we all laughed. And I did
say - I want to keep my

609
00:42:45,960 --> 00:42:50,440
word - laughing is ok, but
malice is inappropriate,

610
00:42:50,440 --> 00:42:55,140
even malice is hating. And, try to imagine
you in Dastin's shoes. If you were

611
00:42:55,140 --> 00:42:58,280
interviewed about the same thing for 14
hours, you'd make a mistake too.

612
00:42:58,280 --> 00:43:02,880
And of course, that mistake will be talked
about. Dastin said to me afterwards,

613
00:43:02,880 --> 00:43:06,450
they misquoted him, and I don't have
any reason not to believe him.

614
00:43:06,450 --> 00:43:09,122
Just to protect him a bit here:
He probably didn't have

615
00:43:09,122 --> 00:43:12,780
a good day.
So, let's continue.

616
00:43:12,780 --> 00:43:15,690
This tech-portal is glad that
catpics don't seem to

617
00:43:15,690 --> 00:43:18,900
be affected.
(laughter)

618
00:43:18,900 --> 00:43:23,160
Notice the way it's written, as if they
make sure, yes, as if they don't

619
00:43:23,160 --> 00:43:25,360
know really, maybe catpics are
affected after all.

620
00:43:25,360 --> 00:43:27,910
(murmur)
And here's a new press statement

621
00:43:27,910 --> 00:43:31,670
by Xerox. The public pressure was so
big, that Xerox said:

622
00:43:31,670 --> 00:43:34,520
"Ah well, you know what, maybe we
should rather do a patch

623
00:43:34,520 --> 00:43:38,130
where we remove pattern matching".
Legally recognizing the mistake however,

624
00:43:38,130 --> 00:43:41,380
they never did. Even until now.
Since it was in the manual.

625
00:43:41,380 --> 00:43:44,569
That's how it is by the way. If it's in
the manual, it's ok. For

626
00:43:44,569 --> 00:43:50,960
microwave, it's written, you
can't dry your cat in this.

627
00:43:50,960 --> 00:43:53,960
Here is another newspaper article.
And when you waited so long,

628
00:43:53,960 --> 00:43:57,380
even a patch won't save you from
mockery. Now the newspapers start

629
00:43:57,380 --> 00:43:59,440
including misprints
in titles on purpose.

630
00:43:59,440 --> 00:44:01,510
(laughter)

631
00:44:01,510 --> 00:44:04,190
Let's go back to Xerox's statement,
because they write

632
00:44:04,190 --> 00:44:09,500
a clear, important declaration. You will 
not see letter replacement,

633
00:44:09,500 --> 00:44:14,819
if you set your compression to at least
"Higher", at minimum 200 dpi.

634
00:44:14,819 --> 00:44:18,680
Xerox published documents, in which it
is clearly stated, that pattern matching

635
00:44:18,680 --> 00:44:24,450
is only used in "Normal" compression mode,
and not in the two higher ones.

636
00:44:24,450 --> 00:44:27,500
But now here this whole time I've been 
thinking, I'm sure I also

637
00:44:27,500 --> 00:44:29,930
saw it in the higher modes.
Different readers

638
00:44:29,930 --> 00:44:33,609
told me as well. But I just can't
reproduce it on my two local

639
00:44:33,609 --> 00:44:37,260
devices. But one thing
is for sure:

640
00:44:37,260 --> 00:44:41,319
If letters get replaced in higher modes as
well, then absolutely everyone

641
00:44:41,319 --> 00:44:45,610
would be affected. And Xerox would have
miscommunicated. Then we would

642
00:44:45,610 --> 00:44:50,040
have a much bigger problem worldwide.
So I don't just publish my worry as

643
00:44:50,040 --> 00:44:54,580
a rumour. Decency also dictates
that. So, but now one of my

644
00:44:54,580 --> 00:45:00,220
friends in a company in Bonn, my 
former living place, looked at

645
00:45:00,220 --> 00:45:07,160
his Xerox Workcentre 7545. I'll look up
the numbers later! (laughs dumbly)

646
00:45:07,160 --> 00:45:11,251
And because it was my former place
of residence, we went there and

647
00:45:11,251 --> 00:45:14,300
took my test numbers, and scanned 
them in the mode "Higher",

648
00:45:14,300 --> 00:45:19,080
that's the factory setting, and we even
chose 300 dpi as a resolution,

649
00:45:19,080 --> 00:45:22,700
for text, you'll agree with me, 
that's quite generous.

650
00:45:22,700 --> 00:45:28,280
Zack - The yellow numbers are wrong.
(laughter)

651
00:45:28,280 --> 00:45:31,240
That's not all by the way. I just marked
a few here that I saw.

652
00:45:31,240 --> 00:45:35,739
I won't go through 500.000 numbers
and mark all wrong ones.

653
00:45:35,739 --> 00:45:38,070
But you see, how common the errors are.
I repeat:

654
00:45:38,070 --> 00:45:42,540
In compression mode "Higher" with 300 dpi.
Now we take the blue rectangle and

655
00:45:42,540 --> 00:45:47,270
enhance it. Here are groups of numbers 
marked in red - oh, you only see it

656
00:45:47,270 --> 00:45:52,540
in light pink now, but you see it - 
that are identical to the pixel.

657
00:45:52,540 --> 00:45:56,990
Such thing is very unlikely. If you
scan the same number multiple times,

658
00:45:56,990 --> 00:46:01,970
it will almost always look slightly
diferent. So, pixel identical numbers

659
00:46:01,970 --> 00:46:04,680
in a high quantity means, that numbers 
get reused, that's

660
00:46:04,680 --> 00:46:08,970
a clear sign of pattern matching.
So different from Xerox's statement,

661
00:46:08,970 --> 00:46:13,240
we also have pattern matching that's
used here. One reader once even told

662
00:46:13,240 --> 00:46:17,349
me of an interactive visualization,
that makes same numbers visible.

663
00:46:17,349 --> 00:46:19,449
Yes, let's see if it...
- Yes! - there it is.

664
00:46:19,449 --> 00:46:22,290
And now I can hover over it here with
my mouse pointer, and

665
00:46:22,290 --> 00:46:28,650
we'll make everything red, where
a number was reused.

666
00:46:28,650 --> 00:46:31,290
I won't make it too long, I'm already
a bit in overtime.

667
00:46:31,290 --> 00:46:37,420
It's because you always applaud so nicely.
Which I enjoy. (laughs)

668
00:46:37,420 --> 00:46:42,790
(applause)

669
00:46:42,790 --> 00:46:47,020
But here you can see, how many numbers
can really be wrong.

670
00:46:47,020 --> 00:46:50,360
From here on it's clear: Hundreds of
thousands of devices, on factory

671
00:46:50,360 --> 00:46:54,050
settings are affected, and the fun is
really over. With this you can really

672
00:46:54,050 --> 00:46:57,310
hit a company hard. And I didn't 
want to publish this without

673
00:46:57,310 --> 00:47:00,080
searching a talk first. And I 
wanted to make sure, that I

674
00:47:00,080 --> 00:47:02,960
didn't make a mistake. 
I didn't want to be able to be

675
00:47:02,960 --> 00:47:06,980
sued for millions in stock price here.
So I recorded the whole process

676
00:47:06,980 --> 00:47:10,170
of the wrong number generation on video,
and put it on youtube as an

677
00:47:10,170 --> 00:47:14,030
unlisted video. I sent the link to 
Francis Tse, one of the chief

678
00:47:14,030 --> 00:47:19,190
engineers that I mentioned earlier.
And of course they were

679
00:47:19,190 --> 00:47:22,740
thunderstruck. From here on the thing
is really all encompassing. Francis

680
00:47:22,740 --> 00:47:26,980
confirmed over phone, that I did all
right indeed. And Xerox was cooperative,

681
00:47:26,980 --> 00:47:30,190
but they also wanted me to wait until
they reproduced the error.

682
00:47:30,190 --> 00:47:33,580
But I also remembered, that during our
last telephone call,

683
00:47:33,580 --> 00:47:37,630
I felt a bit fucked with. So I 
said, my people,

684
00:47:37,630 --> 00:47:41,000
it won't be like last time now.
"I have the blog article done,

685
00:47:41,000 --> 00:47:44,720
and the video is already uploaded."
(laughter)

686
00:47:44,720 --> 00:47:47,940
(laughs)
And when you...

687
00:47:47,940 --> 00:47:53,800
(applause)

688
00:47:53,800 --> 00:47:56,840
"Don't take offense, but I request to be
included from now on,

689
00:47:56,840 --> 00:48:00,540
because I also treat you fair."
So we agreed on the thing,

690
00:48:00,540 --> 00:48:03,300
and now you see what it brings to
not hate in advance. If

691
00:48:03,300 --> 00:48:07,870
you shat on them beforehand on Twitter,
it's clear they say "Come, screw you!"

692
00:48:07,870 --> 00:48:11,109
After that, there was about six hours 
back and forth calls. We had

693
00:48:11,109 --> 00:48:15,809
calls over and over. They tried to 
reproduce the error with my help.

694
00:48:15,809 --> 00:48:18,720
For me it was evening, I spent the 
night on the phone in the office

695
00:48:18,720 --> 00:48:22,480
and didn' eat anything but the cookies
that layed around. At some point Francis

696
00:48:22,480 --> 00:48:27,820
calls again, and says completely
dumbfounded "Yep, we reproduced it."

697
00:48:27,820 --> 00:48:31,060
Errors on factory settings, then
there was silence on both sides.

698
00:48:31,060 --> 00:48:35,230
We were just all shocked. 
And you know what was found parallel?

699
00:48:35,230 --> 00:48:39,140
The Code for the compression scan is
eight years old. That's how long the bug

700
00:48:39,140 --> 00:48:43,450
was out in the wild.
Eight years.

701
00:48:43,450 --> 00:48:46,000
Yes, they were a bit dumbfounded. 
And I said: "Here's

702
00:48:46,000 --> 00:48:48,980
my blog article, please read it and
confirm, what legal safety

703
00:48:48,980 --> 00:48:51,310
I have for publishing
this."

704
00:48:51,310 --> 00:48:56,510
(laughter and applause)
(gasps of laughter)

705
00:48:56,510 --> 00:49:01,360
No, so...
this error is extremely dangerous.

706
00:49:01,360 --> 00:49:04,910
I didn't want to wait any longer. Here's
the article, and that's what

707
00:49:04,910 --> 00:49:08,740
they did. And I was allowed to publish
the article before them, even. That's

708
00:49:08,740 --> 00:49:12,180
pretty unique. And you will agree 
with me, don't hate: If that's what

709
00:49:12,180 --> 00:49:15,640
you reach with this, then that's
good. A conversation between adults.

710
00:49:15,640 --> 00:49:20,460
Lesson learned: Negotiate in the 
right moment. This is the next

711
00:49:20,460 --> 00:49:23,960
Xerox press statement. I'll
increase my speed a bit.

712
00:49:23,960 --> 00:49:26,470
Xerox, of course, commented right after
this as well.

713
00:49:26,470 --> 00:49:29,329
They retract their earlier
communication, thank me, and

714
00:49:29,329 --> 00:49:32,550
say, that now first of all they'll see,
how big the thing really is.

715
00:49:32,550 --> 00:49:35,780
And from there on they were always nice
in the statements, and

716
00:49:35,780 --> 00:49:40,040
overall the climate was very constructive.
This is the next Slashdot article.

717
00:49:40,040 --> 00:49:42,340
It's getting surreal, 
just look at the titles!

718
00:49:42,340 --> 00:49:45,560
After the back and forth, it doesn't
matter for be with Slashdot

719
00:49:45,560 --> 00:49:50,630
what Xerox says, but what they
confirm to me. (laughter)

720
00:49:50,630 --> 00:49:54,350
And here again is our snappy
Peter Coy from Business Week.

721
00:49:54,350 --> 00:49:56,809
But now... One more, I do
have on more.

722
00:49:56,809 --> 00:50:00,920
I mean, a compression mode!
(laughter)

723
00:50:00,920 --> 00:50:03,910
Doesn't really matter now. But on
August 11th the proof for the

724
00:50:03,910 --> 00:50:06,810
error also occuring on "Highest" 
mode succeeds.

725
00:50:06,810 --> 00:50:10,310
Even a quality conscious user in the
last eight years, that wanted to

726
00:50:10,310 --> 00:50:13,760
produce beauttiful PDFs, couldn't
avoid it. And to be honest,

727
00:50:13,760 --> 00:50:17,400
after my informations the error 
doesn't occurr on TIFFs.

728
00:50:17,400 --> 00:50:22,040
I don't want to make it look worse than it
is. No one takes TIFFs, of course,

729
00:50:22,040 --> 00:50:26,510
they're gigantic. On August 12th Xerox
admits publically, that it's a matter of

730
00:50:26,510 --> 00:50:30,030
an eight year old system error.
And announces the patch again.

731
00:50:30,030 --> 00:50:34,820
But of course they are deep in the
whole thing, legally. And when it's

732
00:50:34,820 --> 00:50:38,879
midday in the USA, it's night time here.
And so in the middle of the night, when

733
00:50:38,879 --> 00:50:43,680
visitors of this speech are usually awake,
Dastin and Tse called me on

734
00:50:43,680 --> 00:50:48,510
my phone, and wanted to tell me first,
which I have to say, I found incredibly

735
00:50:48,510 --> 00:50:51,510
nice of them, that they found the bug,
and they'll roll out new

736
00:50:51,510 --> 00:50:54,179
software. And there you can see
that the relation really

737
00:50:54,179 --> 00:50:57,819
got better. This is the patch download
page by Xerox. Here you can see

738
00:50:57,819 --> 00:51:01,310
how many devices are affected.
Note the "X"e, that's whole

739
00:51:01,310 --> 00:51:06,549
device families!
(laughter)

740
00:51:06,549 --> 00:51:10,220
So, the press is reporting again.
The computer magazine CT writes

741
00:51:10,220 --> 00:51:14,540
an article, and calls the whole thing
"Scannergate". And here is

742
00:51:14,540 --> 00:51:18,480
one last kick from our
beloved Peter Coy.

743
00:51:18,480 --> 00:51:21,570
He sounds so sarcastic, but
unfortunately he's completely right.

744
00:51:21,570 --> 00:51:25,839
Eight years of production of scanned,
archived documents could contain

745
00:51:25,839 --> 00:51:29,510
these errors, and cause harm until
forever. Hundreds of thousands

746
00:51:29,510 --> 00:51:34,160
of deviced and companies worldwide. We
live in a society, where now,

747
00:51:34,160 --> 00:51:37,770
as we are speaking, the transition
from a world of paper into a mix of

748
00:51:37,770 --> 00:51:41,410
paper and digital is happening.
And the translator

749
00:51:41,410 --> 00:51:44,890
between the two worlds, that's
deviced by Xerox workcentres.

750
00:51:44,890 --> 00:51:47,910
It'll be with us for a long time.
Now the most important thing:

751
00:51:47,910 --> 00:51:51,430
I already said, that Xerox has a 
decentralized supply over third parties.

752
00:51:51,430 --> 00:51:54,869
Personally, I have no reason to believe
that the patch reached

753
00:51:54,869 --> 00:51:59,140
a lot of devices. So: Spread the word!
At the end of this talk there will be URLs,

754
00:51:59,140 --> 00:52:04,440
where you can get more info 
and see more. It's almost

755
00:52:04,440 --> 00:52:08,259
the end... Besides all the "Lessons
learned", there's one "Lesson" that I

756
00:52:08,259 --> 00:52:11,599
haven't mentioned yet.
I always got disbelieving looks,

757
00:52:11,599 --> 00:52:14,730
that I didn't take any money for the
thing. One manager even said,

758
00:52:14,730 --> 00:52:18,950
I'm "pretty dumb". About that, two things.
First, it's generally hard to make money

759
00:52:18,950 --> 00:52:23,899
with something like this. Even if you want
With no proof you won't be taken serious.

760
00:52:23,899 --> 00:52:27,410
And with the proof, you'll mostly just 
find the bugfix directly, and then

761
00:52:27,410 --> 00:52:29,690
you won't get any money either.

762
00:52:29,690 --> 00:52:32,770
And second: Companies don't know
friends. If I had taken money,

763
00:52:32,770 --> 00:52:37,460
it would've somehow been made public
and could've been used against me.

764
00:52:37,460 --> 00:52:40,520
And it would've brought be in a
position hard to negotiate.

765
00:52:40,520 --> 00:52:43,599
But I wanted this error to be fixed.
And last but not least,

766
00:52:43,599 --> 00:52:47,770
the community helped me, and
they didn't get money either.

767
00:52:47,770 --> 00:52:49,520
I'd do it like this
again, but...

768
00:52:49,520 --> 00:52:53,370
(cheering)

769
00:52:53,370 --> 00:52:57,930
...at the end of the day, everyone has
to decide that for themselves. If you

770
00:52:57,930 --> 00:53:01,369
would do it differently, then that's ok.
I just want to say in advance,

771
00:53:01,369 --> 00:53:05,040
you bring yourself in a weaker
negotiation position. That's all

772
00:53:05,040 --> 00:53:09,170
the "Lessons learned" again. I won't
reiterate them again now.

773
00:53:09,170 --> 00:53:13,300
They're here so you can download the
presentation, and still have them.

774
00:53:13,300 --> 00:53:17,100
And now we close the circle to the
start, and with that we are done.

775
00:53:17,100 --> 00:53:21,530
At the start, there's the prologue with
Obama's birth certificate. Here it is,

776
00:53:21,530 --> 00:53:25,700
the "long form birth certificate". Shortly
after the Xerox-saga, journalists from

777
00:53:25,700 --> 00:53:28,809
the "Reality Check" USA wrote me, if
the Xerox bug could've been

778
00:53:28,809 --> 00:53:32,830
the reason for the "forgery".
And they did a whole lot of

779
00:53:32,830 --> 00:53:36,680
detective work. For example, the Obamas
published their tax documents, shortly

780
00:53:36,680 --> 00:53:41,690
before the birth certificate. It was scanned
by a Xerox Workcentre 7655.

781
00:53:41,690 --> 00:53:45,859
Tja, and further technical 
attributes spoke for

782
00:53:45,859 --> 00:53:48,680
a Xerox scanner. And the "Reality 
Check" guys asked me, if

783
00:53:48,680 --> 00:53:53,100
I could ask Xerox about it, since
I had such good contacts. And Xerox...

784
00:53:53,100 --> 00:53:57,090
(laughter)
And Xerox asked for understanding,

785
00:53:57,090 --> 00:54:00,059
that they really didn't want to 
deal with this now... (laughs)

786
00:54:00,059 --> 00:54:03,119
...and I left it alone. And now I'll
prepare for my

787
00:54:03,119 --> 00:54:08,470
congress speech, for this talk today,
yeah, I look in the PDFs again,

788
00:54:08,470 --> 00:54:11,930
and there's the exact copied, yeah
the exact letters in there, that were

789
00:54:11,930 --> 00:54:16,280
a sign by Xerox for pattern matching
back then. And I look on the internet

790
00:54:16,280 --> 00:54:20,000
pages, and there it also says something
about letter doubling. Here's two exact

791
00:54:20,000 --> 00:54:24,420
same boxes. Notice the indents on it.
Now, make your own image here

792
00:54:24,420 --> 00:54:26,839
But I think it could be, that
this conspiracy

793
00:54:26,839 --> 00:54:31,720
is hereby over and done. And with this,
it only remains for me to say thanks, for

794
00:54:31,720 --> 00:54:33,560
spending a whole hour with me!

795
00:54:33,560 --> 00:54:44,019
(applause)

796
00:54:44,019 --> 00:54:50,739
If everyone keeps clapping, it'll
take even longer!

797
00:54:50,739 --> 00:54:55,120
So... (laughs)

798
00:54:55,120 --> 00:54:58,750
Up there you'll find another link for
the Xerox saga. Pass it on!

799
00:54:58,750 --> 00:55:01,260
And down here a link to my page.
There I'll publish

800
00:55:01,260 --> 00:55:05,210
the presentation online. Maybe tomorrow.
I won't go into the WIFI here! (laughs)

801
00:55:05,210 --> 00:55:06,660
(laughter)

802
00:55:06,660 --> 00:55:09,010
And take care of evil copiers!

803
00:55:09,010 --> 00:55:12,730
Herald: Okay, thanks first of all,
for this amazing talk!

804
00:55:12,730 --> 00:55:16,360
I think it was very interesting
for everyone.

805
00:55:16,360 --> 00:55:19,530
Everyone on the way out, please
hurry and close the doors after.

806
00:55:19,530 --> 00:55:25,250
And be quiet.

807
00:55:25,250 --> 00:55:28,030
For the questions, I'd like to start
with the ones from the internet.

808
00:55:28,030 --> 00:55:30,000
From our Signal Angel.

809
00:55:30,000 --> 00:55:33,790
Signal Angel: Thanks!
And a great applause from the internet,

810
00:55:33,790 --> 00:55:38,240
you couldn't hear it now. But there
was a lot of positive feedback.

811
00:55:38,240 --> 00:55:41,460
And also the plea to publish the 
presentation, especially

812
00:55:41,460 --> 00:55:42,859
the symbol images were well 
recived.

813
00:55:42,859 --> 00:55:44,929
Daniel: It will happen, on my page, latest
tomorrow. Definitely.

814
00:55:44,929 --> 00:55:46,720
Signal Angel: Very good, thanks. Two
questions from me.

815
00:55:46,720 --> 00:55:51,520
The first question is, does Xerox have
a technical difference between

816
00:55:51,520 --> 00:55:55,310
Scanning, Printing and Copying?
Or is it always the same thing?

817
00:55:55,310 --> 00:55:58,989
Daniel: So, scanning, there paper comes
in and for printing it comes out, ne?

818
00:55:58,989 --> 00:56:00,439
(laughter)

819
00:56:00,439 --> 00:56:03,540
No, so, for printing, you just 
recieve the printing data.

820
00:56:03,540 --> 00:56:06,570
I don't know about anything being
compressed afterwards again.

821
00:56:06,570 --> 00:56:10,630
Scanning - here there are different modes.
The PDF modes, there are three, that

822
00:56:10,630 --> 00:56:16,869
I mentioned earlier. And copying - In my
view it's not like this, that it always

823
00:56:16,869 --> 00:56:21,640
happens during printing, because there you
don't compress. You see how I mean it, yes?

824
00:56:21,640 --> 00:56:24,930
I'm sure I would have recieved some 
reports if it was like that.

825
00:56:24,930 --> 00:56:27,920
And that's why I don't think the process of
copying itself is affected. But

826
00:56:27,920 --> 00:56:33,180
that wouldn't be so bad anyways, because
there are no documents that get archieved here.

827
00:56:33,180 --> 00:56:37,520
Signal Angel: Okay, and the second question:
Are there any definitive

828
00:56:37,520 --> 00:56:40,430
harms that happened because of this bug?

829
00:56:40,430 --> 00:56:41,790
Did you ever recieve and feedback
regarding this?

830
00:56:41,790 --> 00:56:43,730
Daniel: I have feedback, the ones that
I named earlier.

831
00:56:43,730 --> 00:56:47,220
And of course a few more. I'm of course
not going to say any names.

832
00:56:47,220 --> 00:56:50,559
But... So, I can only say this much:

833
00:56:50,559 --> 00:56:54,910
You have to imagine yourself in the place
of the company that's affected here.

834
00:56:54,910 --> 00:56:58,680
Your files might be good for the trash.

835
00:56:58,680 --> 00:57:01,790
Will you make this public?
No, you will request compensation

836
00:57:01,790 --> 00:57:05,490
from Xerox in silence, and not write
any of this on your

837
00:57:05,490 --> 00:57:08,891
website, because then it will fall back to you,
that our data

838
00:57:08,891 --> 00:57:12,349
is faulty. No one will ask you, if that
was a Xerox copier now.

839
00:57:12,349 --> 00:57:15,400
So I don't expect there to be a grand
reveal now, if it can be

840
00:57:15,400 --> 00:57:17,809
avoided. If some random bridge on
a highway collapses now

841
00:57:17,809 --> 00:57:19,490
that would of course be a different
matter.

842
00:57:19,490 --> 00:57:22,490
Signal Angel: Okay, thanks again!
Daniel: Gern!

843
00:57:22,490 --> 00:57:24,760
Herald: Good, then I'd suggest we continue
at microphone 2,

844
00:57:24,760 --> 00:57:26,190
at the first person.

845
00:57:26,190 --> 00:57:29,609
Question: Just a short question. This is
probably a technique that gets used

846
00:57:29,609 --> 00:57:31,980
by many. Did you ever try this
with devices

847
00:57:31,980 --> 00:57:33,730
from other companies?

848
00:57:33,730 --> 00:57:38,190
Daniel: I had a great quantity of reports
from other companies. But if you

849
00:57:38,190 --> 00:57:41,819
take on a thing of this scale, you'll
become a victim of spin doctoring.

850
00:57:41,819 --> 00:57:44,220
And all of it turned out to be false,
Here, again:

851
00:57:44,220 --> 00:57:48,910
Stay sovereign, don't just pump out
rumours. Here none of it was true,

852
00:57:48,910 --> 00:57:51,579
and in concrete cases it wasn't the 
compression method itself,

853
00:57:51,579 --> 00:57:56,500
but the fact that there was indeed
another bug.

854
00:57:56,500 --> 00:57:58,500
Herald: Good, then 3 please!

855
00:57:58,500 --> 00:58:02,039
Question: Hello? Thanks for the talk,
it was pretty cool.

856
00:58:02,039 --> 00:58:06,410
I just wonder about the thing, the bug
being there somehow for eight years.

857
00:58:06,410 --> 00:58:10,500
Did you look on search engines, did
others... I mean, I can't

858
00:58:10,500 --> 00:58:13,730
imagine that for eight years no one
saw it, because

859
00:58:13,730 --> 00:58:17,670
as you say, on a blueprint, 
there you can see pretty quickly, so...

860
00:58:17,670 --> 00:58:20,400
or maybe other people messaged you,
because they had seen it before,

861
00:58:20,400 --> 00:58:23,220
or maybe they said, hey I noticed this
before, Xerox said,

862
00:58:23,220 --> 00:58:27,650
yes, higher compression, then they were
lucky and it worked.

863
00:58:27,650 --> 00:58:31,120
Daniel: So, it was, first of all hard to
discover. Second of all,

864
00:58:31,120 --> 00:58:36,540
it was known for the mode "Normal".
It was on purpose, they even knew about it

865
00:58:36,540 --> 00:58:41,670
And that's why it was hard to recognize the
real bug, because Xerox...

866
00:58:41,670 --> 00:58:45,470
The support that knew - mine didn't know - 
always blamed it on the

867
00:58:45,470 --> 00:58:48,740
"Normal" setting. And then it's plausible,
then I tell you:

868
00:58:48,740 --> 00:58:50,670
"Yes, you used the "Normal" setting, 
take another one, then

869
00:58:50,670 --> 00:58:55,870
the error will occur less,
you'll probably be lucky there"

870
00:58:55,870 --> 00:58:59,559
So I think, that indeed, that the
bug was discovered for the first time...

871
00:58:59,559 --> 00:59:02,770
Question: So, no one contacted you, with
"Hey, I've seen this before" or so?

872
00:59:02,770 --> 00:59:06,700
Daniel: No, no one. In the whole
storm, no.

873
00:59:06,700 --> 00:59:08,870
Herald: Okay, next up again from
the 2 please.

874
00:59:08,870 --> 00:59:11,660
Question: Moin, thanks for the presentation
from me as well. Was very cool.

875
00:59:11,660 --> 00:59:12,810
Daniel: Sure.

876
00:59:12,810 --> 00:59:15,310
Question: Short question, you said,
you didn't do it for money...

877
00:59:15,310 --> 00:59:16,310
Daniel: Correct.

878
00:59:16,310 --> 00:59:18,380
Question: ...and somehow... I find it
very noble, very cool. But

879
00:59:18,380 --> 00:59:20,550
did they ever offer you something
from their side?

880
00:59:20,550 --> 00:59:21,950
Daniel: No, they didn't.
No one there...

881
00:59:21,950 --> 00:59:23,640
Question: Not even a job or anything?

882
00:59:23,640 --> 00:59:26,062
Daniel: Well, there I can in fact hold 
Xerox a bit. They didn't offer

883
00:59:26,062 --> 00:59:29,089
me anything. I couldn't have accepted
it anyways

884
00:59:29,089 --> 00:59:34,760
by that logic. That's why it was totally 
fine. In that long night, where we

885
00:59:34,760 --> 00:59:37,849
had the phone call, they were ready
to have me fly in. But

886
00:59:37,849 --> 00:59:41,339
I honestly don't know anything about
copiers either. Not my main job.

887
00:59:41,339 --> 00:59:45,220
I can show them the bug, but
I can't repair it. So...

888
00:59:45,220 --> 00:59:46,829
Question: Ok, but if they would have
flown you in, why not

889
00:59:46,829 --> 00:59:49,230
work with them together and try
to solve the thing?

890
00:59:49,230 --> 00:59:53,850
Daniel: Jo, I could've done that. But
I couldn't have contributed anything.

891
00:59:53,850 --> 00:59:56,420
Because, they have to find the bug in
their code themselves. It was clear that

892
00:59:56,420 --> 00:59:59,450
something happened. I can't help with
that. I'd just sit around.

893
00:59:59,450 --> 01:00:01,080
So I also said it just like that.

894
01:00:01,080 --> 01:00:02,849
Question: That makes sense.

895
01:00:02,849 --> 01:00:06,329
Daniel: Yes, and flying 2x intercontinential
for that... I don't know.

896
01:00:06,329 --> 01:00:09,211
Question: Yes, but if they paid I would've
done it.

897
01:00:09,211 --> 01:00:11,420
Daniel: I admit, I also overthought it
again. But I had

898
01:00:11,420 --> 01:00:18,480
also stuff to do job wise, and
it wouldn't have worked out.

899
01:00:18,480 --> 01:00:20,730
Herald: Good, next up 3 again.

900
01:00:20,730 --> 01:00:24,410
Question: Well, I have a copier at home,
and I have a very

901
01:00:24,410 --> 01:00:27,850
intimate relationship with it.
Are there any reports, that

902
01:00:27,850 --> 01:00:30,950
some tried it with their home copiers,

903
01:00:30,950 --> 01:00:32,890
and then went "Oh Sh...?"

904
01:00:32,890 --> 01:00:37,069
Daniel: I don't know of any reports like 
that. It only affected the things that

905
01:00:37,069 --> 01:00:42,380
I just showed. Workcentre, ColourCube.
All big things.

906
01:00:42,380 --> 01:00:44,010
basically.
Question: Okay.

907
01:00:44,010 --> 01:00:47,500
Daniel: This JBig2 in Hardware, 
that's also

908
01:00:47,500 --> 01:00:49,889
I think very expensive to 
implement.

909
01:00:49,889 --> 01:00:52,170
Question: Okay, thanks!
Daniel: Jo!

910
01:00:52,170 --> 01:00:54,900
Herald: And 3 again please!

911
01:00:54,900 --> 01:01:00,270
Question: Maybe a cool crows research
task

912
01:01:00,270 --> 01:01:05,209
Is maybe to look through
those manuals,

913
01:01:05,209 --> 01:01:09,900
to collect. Who had access, which
year does it show

914
01:01:09,900 --> 01:01:14,760
up in the documentation at all, is it
really

915
01:01:14,760 --> 01:01:19,150
that old, so eight years, or maybe 
only four years?

916
01:01:19,150 --> 01:01:23,030
They only noticed four years ago, and 
thought, hm, it's cheaper, we print

917
01:01:23,030 --> 01:01:27,280
new handbooks, and leave the software
as it is. Because it's more expensive,

918
01:01:27,280 --> 01:01:28,580
to roll out new firmware.

919
01:01:28,580 --> 01:01:30,410
Daniel: There's a theory, that here a bug
was declared a feature.

920
01:01:30,410 --> 01:01:33,180
I can confirm that. But I don't have
proof for it. I want to say that very

921
01:01:33,180 --> 01:01:37,540
clearly. But seriously, who would
design a scanner,

922
01:01:37,540 --> 01:01:43,520
that swaps around numbers? Only if it
was just for the military (laugsh)

923
01:01:43,520 --> 01:01:46,200
Herald: Okay, I think one last question.
Then 2 again.

924
01:01:46,200 --> 01:01:48,450
Question: Not really a question, but more
of a suggestion for the presentation,

925
01:01:48,450 --> 01:01:52,330
in case you present it again.
It's really great.

926
01:01:52,330 --> 01:01:55,329
You have this scale, with accesses to
your website at the bottom.

927
01:01:55,329 --> 01:01:57,880
I wondered, during the talk, if maybe
you could also do that

928
01:01:57,880 --> 01:02:00,990
with the stock price of Xerox?
(Daniel laughs)

929
01:02:00,990 --> 01:02:03,580
Daniel: It wasn't that bad. I mean,
that PR section of them

930
01:02:03,580 --> 01:02:08,350
handled it pretty well despite the
world wide attention they had.

931
01:02:08,350 --> 01:02:12,099
I mean, that's really an error, where you
could think, this is

932
01:02:12,099 --> 01:02:14,980
a danger for the whole company. It's their
bread and butter business.

933
01:02:14,980 --> 01:02:17,890
But it didn't turn out that way. We will
see, I could've put

934
01:02:17,890 --> 01:02:20,600
such a live stock price curve in the
presentation. I don't know,

935
01:02:20,600 --> 01:02:24,230
what's happening on the internet right
now. But good suggestion, thanks!

936
01:02:24,230 --> 01:02:26,450
Herald: Okay, we also have questions from
the internet.

937
01:02:26,450 --> 01:02:27,550
Therefore I'd also like to...

938
01:02:27,550 --> 01:02:28,960
Signal Angel: I just have one more
question from the internet. Are

939
01:02:28,960 --> 01:02:33,250
there are statistics or numbers, 
about how high

940
01:02:33,250 --> 01:02:35,420
the likeliness of such an error is?

941
01:02:35,420 --> 01:02:38,210
Daniel: Well, you saw the page I told 
you about. That was the case

942
01:02:38,210 --> 01:02:42,380
with font size 7 or 8. I don't know
anymore, where I got it really

943
01:02:42,380 --> 01:02:44,280
niceöy reproduced. But when...

944
01:02:44,280 --> 01:02:47,279
Signal Angel: But... Numbers, thatr's not a normal page
now is it?

945
01:02:47,279 --> 01:02:49,220
Daniel: It was all numbers, but
of course it's also possible with

946
01:02:49,220 --> 01:02:53,150
similar letters. But it can happen too.
I don't have any statistics.

947
01:02:53,150 --> 01:02:55,880
For the numbers the 6 and 8 are
affected the most. But real

948
01:02:55,880 --> 01:02:58,960
error percentages, I don't have.
But you can see, what's possible.

949
01:02:58,960 --> 01:03:03,039
So I have... I didn't try for
hours on end, until I found the

950
01:03:03,039 --> 01:03:05,950
page with many yellow points. I
scanned ONE page, and then it

951
01:03:05,950 --> 01:03:09,980
was like that. Yeah? So it's not like
you have to look for it forever.

952
01:03:09,980 --> 01:03:12,610
Question: Yes, thanks!

953
01:03:12,610 --> 01:03:15,990
Herald: Alright, I think we are done
then.

954
01:03:15,990 --> 01:03:18,961
Then please another big applause
for the lecturer!

955
01:03:18,961 --> 01:03:20,361
(applause)

956
01:03:20,361 --> 01:03:21,121
Daniel: Thanks!

957
01:03:21,121 --> 01:03:28,721
(longer applause)

958
01:03:28,721 --> 01:03:30,641
31C3 Credits with no audio

959
01:03:30,660 --> 01:03:40,000
Subtitles created on amara.org in the
year 2017 - 2022 by multiple collaborators