[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.00,0:00:14.60,Default,,0000,0000,0000,,{\i1}34C3 preroll music{\i0}
Dialogue: 0,0:00:14.60,0:00:17.87,Default,,0000,0000,0000,,Herald: Please give a warm welcome here.
Dialogue: 0,0:00:17.87,0:00:23.61,Default,,0000,0000,0000,,It’s Franziska, Teresa, and Judith.
Dialogue: 0,0:00:23.61,0:00:26.51,Default,,0000,0000,0000,,Judith, you have the stage, thank you.
Dialogue: 0,0:00:26.51,0:00:28.47,Default,,0000,0000,0000,,Judith Hartstein: Thank you, thanks!
Dialogue: 0,0:00:28.47,0:00:33.71,Default,,0000,0000,0000,,{\i1}applause{\i0}
Dialogue: 0,0:00:33.71,0:00:57.43,Default,,0000,0000,0000,,{\i1}inaudible{\i0}
Dialogue: 0,0:00:57.43,0:01:00.22,Default,,0000,0000,0000,,Judith: We believe that\Nscientific performance indicators
Dialogue: 0,0:01:00.22,0:01:03.27,Default,,0000,0000,0000,,are widely applied to inform\Nfunding decisions and to
Dialogue: 0,0:01:03.27,0:01:08.27,Default,,0000,0000,0000,,determine the availability of career\Nopportunities. So, those of you who are
Dialogue: 0,0:01:08.27,0:01:14.42,Default,,0000,0000,0000,,working in science or have had a look into\Nthe science system might agree to that.
Dialogue: 0,0:01:14.42,0:01:18.84,Default,,0000,0000,0000,,And we want to understand evaluative\Nbibliometrics as algorithmic science
Dialogue: 0,0:01:18.84,0:01:27.64,Default,,0000,0000,0000,,evaluation instruments to highlight some\Nthings that do occur also with other
Dialogue: 0,0:01:27.64,0:01:38.02,Default,,0000,0000,0000,,algorithmic instruments of evaluation. And\Nso we’re going to start with a quote from
Dialogue: 0,0:01:38.02,0:01:44.68,Default,,0000,0000,0000,,a publication in 2015 which reads “As the\Ntyranny of bibliometrics tightens its
Dialogue: 0,0:01:44.68,0:01:49.13,Default,,0000,0000,0000,,grip, it is having a disastrous effect on\Nthe model of science presented to young
Dialogue: 0,0:01:49.13,0:01:58.100,Default,,0000,0000,0000,,researchers.” We have heard the talk of\Nhanno already, and he’s basically also
Dialogue: 0,0:01:58.100,0:02:06.68,Default,,0000,0000,0000,,talking about problems in the science\Nsystem and the reputation by the
Dialogue: 0,0:02:06.68,0:02:14.17,Default,,0000,0000,0000,,indicators. And the question is, is\Nbibliometrics the bad guy here? If you
Dialogue: 0,0:02:14.17,0:02:18.53,Default,,0000,0000,0000,,speak of ‘tyranny of bibliometrics’, who\Nis the actor doing this? Or are maybe
Dialogue: 0,0:02:18.53,0:02:24.95,Default,,0000,0000,0000,,bibliometricians the problem? We want to\Ncontextualize our talk into the growing
Dialogue: 0,0:02:24.95,0:02:30.44,Default,,0000,0000,0000,,movement of Reflexive Metrics. So those\Nwho are doing science studies, social
Dialogue: 0,0:02:30.44,0:02:34.94,Default,,0000,0000,0000,,studies of science, scientometrics and\Nbibliometrics. The movement of Reflexive
Dialogue: 0,0:02:34.94,0:02:41.69,Default,,0000,0000,0000,,Metrics. So the basic idea is to say:\N“Okay, we have to accept accountability if
Dialogue: 0,0:02:41.69,0:02:45.54,Default,,0000,0000,0000,,we do bibliometrics and scientometrics.”\NWe have to understand the effects of
Dialogue: 0,0:02:45.54,0:02:54.22,Default,,0000,0000,0000,,algorithmic evaluation on science, and we\Nwill try not to be the bad guy. And the
Dialogue: 0,0:02:54.22,0:03:03.64,Default,,0000,0000,0000,,main mediator of the science evaluation\Nwhich is perceived by the researchers is
Dialogue: 0,0:03:03.64,0:03:09.63,Default,,0000,0000,0000,,the algorithm. I will hand over the\Nmicrophone to… or I will not hand over the
Dialogue: 0,0:03:09.63,0:03:14.44,Default,,0000,0000,0000,,microphone but I will hand over the talk\Nto Teresa. She’s going to talk about
Dialogue: 0,0:03:14.44,0:03:19.69,Default,,0000,0000,0000,,"Datafication of Scientific Evaluation".
Dialogue: 0,0:03:19.69,0:03:24.18,Default,,0000,0000,0000,,Teresa Isigkeit: Okay. I hope you can\Nhear me. No? Yes? Okay.
Dialogue: 0,0:03:24.18,0:03:25.89,Default,,0000,0000,0000,,Judith: {\i1}mumbling{\i0}
Dialogue: 0,0:03:25.89,0:03:29.48,Default,,0000,0000,0000,,When we think about the science system\Nwhat do we expect?
Dialogue: 0,0:03:29.48,0:03:33.70,Default,,0000,0000,0000,,What can society expect \Nfrom a scientific system?
Dialogue: 0,0:03:33.70,0:03:38.20,Default,,0000,0000,0000,,In general, we would say \Nreliable and truthful knowledge,
Dialogue: 0,0:03:38.20,0:03:41.81,Default,,0000,0000,0000,,that is scrutinized by \Nthe scientific community.
Dialogue: 0,0:03:41.81,0:03:44.32,Default,,0000,0000,0000,,So where can we find this knowledge?
Dialogue: 0,0:03:44.32,0:03:47.14,Default,,0000,0000,0000,,Normally in publications.
Dialogue: 0,0:03:47.14,0:03:51.92,Default,,0000,0000,0000,,So with these publications, \Ncan we actually say
Dialogue: 0,0:03:51.92,0:03:58.67,Default,,0000,0000,0000,,whether science is bad or good? Or is\Nthere better science than others?
Dialogue: 0,0:03:58.67,0:04:03.35,Default,,0000,0000,0000,,In the era of \Ndigital publication databases,
Dialogue: 0,0:04:03.35,0:04:07.28,Default,,0000,0000,0000,,there’s big datasets of publications.
Dialogue: 0,0:04:07.28,0:04:12.33,Default,,0000,0000,0000,,And these are used to \Nevaluate and calculate
Dialogue: 0,0:04:12.33,0:04:16.84,Default,,0000,0000,0000,,the quality of scientific output.
Dialogue: 0,0:04:16.84,0:04:22.72,Default,,0000,0000,0000,,So in general, with this metadata \Nwe can tell you
Dialogue: 0,0:04:22.72,0:04:26.40,Default,,0000,0000,0000,,who is the author of a publication,
Dialogue: 0,0:04:26.40,0:04:30.02,Default,,0000,0000,0000,,where is the home institution \Nof this author,
Dialogue: 0,0:04:30.02,0:04:37.96,Default,,0000,0000,0000,,or which types of citations are in\Nthe bibliographic information.
Dialogue: 0,0:04:37.96,0:04:44.89,Default,,0000,0000,0000,,This is used in the calculation \Nof bibliometric indicators.
Dialogue: 0,0:04:44.89,0:04:51.58,Default,,0000,0000,0000,,For example if you take the\Njournal impact factors,
Dialogue: 0,0:04:51.58,0:04:57.65,Default,,0000,0000,0000,,which is a citation based indicator, \Nyou can compare different journals.
Dialogue: 0,0:04:57.65,0:05:03.75,Default,,0000,0000,0000,,And maybe say which journals\Nare performing better than others
Dialogue: 0,0:05:03.75,0:05:09.36,Default,,0000,0000,0000,,or if the journal factor has increased or\Ndecreased over the years.
Dialogue: 0,0:05:09.36,0:05:15.33,Default,,0000,0000,0000,,Another example would be the \NHirsch-Index for individual scientists,
Dialogue: 0,0:05:15.33,0:05:22.67,Default,,0000,0000,0000,,which is also widely used when\Nscientists apply for jobs. So they put
Dialogue: 0,0:05:22.67,0:05:27.71,Default,,0000,0000,0000,,these numbers in their CVs and supposedly\Nthis tells you something about the quality
Dialogue: 0,0:05:27.71,0:05:36.27,Default,,0000,0000,0000,,of research those scientists are\Nconducting. With the availability of the
Dialogue: 0,0:05:36.27,0:05:45.59,Default,,0000,0000,0000,,data we can see an increase in its usage.\NAnd in a scientific environment in which
Dialogue: 0,0:05:45.59,0:05:52.26,Default,,0000,0000,0000,,data-driven science is established,\Nscientific conduct decisions regarding
Dialogue: 0,0:05:52.26,0:06:04.42,Default,,0000,0000,0000,,hiring or funding heavily rely on these\Nindicators. There’s maybe a naive belief
Dialogue: 0,0:06:04.42,0:06:12.02,Default,,0000,0000,0000,,that these indicators that are data-driven\Nand rely on data that is collected in the
Dialogue: 0,0:06:12.02,0:06:26.61,Default,,0000,0000,0000,,database is a more objective metric that\Nwe can use. So here's a quote by Rieder
Dialogue: 0,0:06:26.61,0:06:32.43,Default,,0000,0000,0000,,and Simon: “In this brave new world trust\Nno longer resides in the integrity of
Dialogue: 0,0:06:32.43,0:06:38.71,Default,,0000,0000,0000,,individual truth-tellers or the veracity\Nof prestigious institutions, but is placed
Dialogue: 0,0:06:38.71,0:06:44.48,Default,,0000,0000,0000,,in highly formalized procedures enacted\Nthrough disciplined self-restraint.
Dialogue: 0,0:06:44.48,0:06:53.15,Default,,0000,0000,0000,,Numbers cease to be supplements.” So we\Nsee a change of an evaluation system that
Dialogue: 0,0:06:53.15,0:07:00.45,Default,,0000,0000,0000,,is relying on expert knowledge to a system\Nof algorithmic science evaluation. In this
Dialogue: 0,0:07:00.45,0:07:05.70,Default,,0000,0000,0000,,change there’s a belief in a\Ndepersonalization of the system and the
Dialogue: 0,0:07:05.70,0:07:15.09,Default,,0000,0000,0000,,perception of algorithms as the rule of\Nlaw. So when looking at the interaction
Dialogue: 0,0:07:15.09,0:07:26.01,Default,,0000,0000,0000,,between the algorithm and scientists we\Ncan tell that this relationship is not as
Dialogue: 0,0:07:26.01,0:07:35.17,Default,,0000,0000,0000,,easy as it seems. Algorithms are not in\Nfact objective. They carry social meaning
Dialogue: 0,0:07:35.17,0:07:43.02,Default,,0000,0000,0000,,and human agency. They are used to\Nconstruct a reality and algorithms don’t
Dialogue: 0,0:07:43.02,0:07:48.29,Default,,0000,0000,0000,,come naturally. They don’t grow on trees\Nand can be picked by scientists and people
Dialogue: 0,0:07:48.29,0:07:55.12,Default,,0000,0000,0000,,who evaluate the scientific system, so we\Nhave to be reflective and think about
Dialogue: 0,0:07:55.12,0:08:04.91,Default,,0000,0000,0000,,which social meanings the algorithm holds.\NSo when there is a code that the algorithm
Dialogue: 0,0:08:04.91,0:08:11.17,Default,,0000,0000,0000,,uses, there is a subjective meaning in\Nthis code, and there is agency in this
Dialogue: 0,0:08:11.17,0:08:17.37,Default,,0000,0000,0000,,code, and you can’t just say, oh, this is\Na perfect construction of the reality of
Dialogue: 0,0:08:17.37,0:08:22.35,Default,,0000,0000,0000,,scientific system. So the belief that this\Ntells you more about the quality of
Dialogue: 0,0:08:22.35,0:08:31.82,Default,,0000,0000,0000,,research is not a good indicator. So when\Nyou think about the example of citation
Dialogue: 0,0:08:31.82,0:08:37.05,Default,,0000,0000,0000,,counts the algorithm reads the\Nbibliographic information of a publication
Dialogue: 0,0:08:37.05,0:08:47.23,Default,,0000,0000,0000,,from the database. So scientists, they\Ncite papers that relate to their studies.
Dialogue: 0,0:08:47.23,0:08:55.74,Default,,0000,0000,0000,,But we don’t actually know which of these\Ncitations are more meaningful than others,
Dialogue: 0,0:08:55.74,0:09:01.32,Default,,0000,0000,0000,,so they’re not as easily comparable. But\Nthe algorithms give you the belief they
Dialogue: 0,0:09:01.32,0:09:11.23,Default,,0000,0000,0000,,are, so relevance is not as easily put\Ninto an algorithm and there is different
Dialogue: 0,0:09:11.23,0:09:19.20,Default,,0000,0000,0000,,types of citations. So the scientists\Nperceive this use of the algorithms also
Dialogue: 0,0:09:19.20,0:09:24.79,Default,,0000,0000,0000,,as a powerful instrument. And so the\Nalgorithm has some sway above the
Dialogue: 0,0:09:24.79,0:09:30.02,Default,,0000,0000,0000,,scientists because they rely so much on\Nthose indicators to further their careers,
Dialogue: 0,0:09:30.02,0:09:38.07,Default,,0000,0000,0000,,to get a promotion, or get funding for\Ntheir next research projects. So we have a
Dialogue: 0,0:09:38.07,0:09:42.97,Default,,0000,0000,0000,,reciprocal relationship between the\Nalgorithm and the scientists, and this
Dialogue: 0,0:09:42.97,0:09:52.40,Default,,0000,0000,0000,,creates a new construction of reality. So\Nwe can conclude that governance by
Dialogue: 0,0:09:52.40,0:09:59.02,Default,,0000,0000,0000,,algorithms leads to behavioral adaptation\Nin scientists, and one of these examples
Dialogue: 0,0:09:59.02,0:10:08.07,Default,,0000,0000,0000,,that uses the Science Citation Index will\Nbe given from Franziska.
Dialogue: 0,0:10:08.07,0:10:12.61,Default,,0000,0000,0000,,Franziska Sörgel: Thanks for the\Nhandover! Yes, let me start.
Dialogue: 0,0:10:12.61,0:10:16.07,Default,,0000,0000,0000,,I’m focusing on reputation\Nand authorship as you can see
Dialogue: 0,0:10:16.07,0:10:21.42,Default,,0000,0000,0000,,on the slide, and first let me\Nstart with a quote
Dialogue: 0,0:10:21.42,0:10:27.49,Default,,0000,0000,0000,,by Eugene Garfield, which says: “Is it\Nreasonable to assume that if I cite a
Dialogue: 0,0:10:27.49,0:10:32.89,Default,,0000,0000,0000,,paper that I would probably be interested\Nin those papers which subsequently cite it
Dialogue: 0,0:10:32.89,0:10:38.63,Default,,0000,0000,0000,,as well as my own paper. Indeed, I have\Nobserved on several occasions that people
Dialogue: 0,0:10:38.63,0:10:45.40,Default,,0000,0000,0000,,preferred to cite the articles I had cited\Nrather than cite me! It would seem to me
Dialogue: 0,0:10:45.40,0:10:51.32,Default,,0000,0000,0000,,that this is the basis for the building up\Nof the ‘logical network’ for the Citation
Dialogue: 0,0:10:51.32,0:11:01.71,Default,,0000,0000,0000,,Index service.” So, actually, this Science\NCitation Index which is described here was
Dialogue: 0,0:11:01.71,0:11:07.97,Default,,0000,0000,0000,,mainly developed in order to solve the\Nproblems of information retrieval. Eugene
Dialogue: 0,0:11:07.97,0:11:16.26,Default,,0000,0000,0000,,Garfield, also founder of this Science\NCitation Index – short: SCI – noted or
Dialogue: 0,0:11:16.26,0:11:22.17,Default,,0000,0000,0000,,began to note a huge interest in\Nreciprocal publication behavior. He
Dialogue: 0,0:11:22.17,0:11:27.47,Default,,0000,0000,0000,,recognized the increasing interest as a\Nstrategic instrument to exploit
Dialogue: 0,0:11:27.47,0:11:33.41,Default,,0000,0000,0000,,intellectual property. And indeed, the\Ninterest in the SCI – and its data –
Dialogue: 0,0:11:33.41,0:11:39.19,Default,,0000,0000,0000,,successively became more relevant within\Nthe disciplines, and its usage extended.
Dialogue: 0,0:11:39.19,0:11:45.80,Default,,0000,0000,0000,,Later, [Derek J.] de Solla Price, another\Nsocial scientist, asked or claimed for a
Dialogue: 0,0:11:45.80,0:11:53.03,Default,,0000,0000,0000,,better research on the topic, as it\Ncurrently also meant a crisis in science,
Dialogue: 0,0:11:53.03,0:11:59.33,Default,,0000,0000,0000,,and stated: “If a paper was cited once,\Nit would get cited again and
Dialogue: 0,0:11:59.33,0:12:05.09,Default,,0000,0000,0000,,again, so the main problem was that the\Nrich would get richer”, which is also
Dialogue: 0,0:12:05.09,0:12:12.03,Default,,0000,0000,0000,,known as the “Matthew Effect”. Finally,\Nthe SCI and its use turned into a system
Dialogue: 0,0:12:12.03,0:12:17.86,Default,,0000,0000,0000,,which was and still is used as a\Nreciprocal citation system, and became a
Dialogue: 0,0:12:17.86,0:12:25.12,Default,,0000,0000,0000,,central and global actor. Once a paper was\Ncited, the probability it was cited again
Dialogue: 0,0:12:25.12,0:12:30.72,Default,,0000,0000,0000,,was higher, and it would even extend its\Nown influence on a certain topic within
Dialogue: 0,0:12:30.72,0:12:38.37,Default,,0000,0000,0000,,the scientific field. So it was known that\Nyou would either read a certain article
Dialogue: 0,0:12:38.37,0:12:48.100,Default,,0000,0000,0000,,and people would do research on a certain\Ntopic or subject. So this phenomenon would
Dialogue: 0,0:12:48.100,0:12:58.57,Default,,0000,0000,0000,,rise to an instrument of disciplining\Nscience and created power structures.
Dialogue: 0,0:12:58.57,0:13:04.53,Default,,0000,0000,0000,,Let me show you one example which is \Nclosely connected to this phenomenon
Dialogue: 0,0:13:04.53,0:13:10.92,Default,,0000,0000,0000,,I just told you about – and I don’t know \Nif here in this room there are any
Dialogue: 0,0:13:10.92,0:13:18.68,Default,,0000,0000,0000,,astronomers or physicists? \NYeah, there are few, okay.
Dialogue: 0,0:13:18.68,0:13:24.58,Default,,0000,0000,0000,,That’s great, actually. \NSo in the next slide, here,
Dialogue: 0,0:13:24.58,0:13:32.99,Default,,0000,0000,0000,,we have a table with a time\Nwindow from 2010 to 2016, and social
Dialogue: 0,0:13:32.99,0:13:41.07,Default,,0000,0000,0000,,scientists from Berlin found out that the\Nco-authorship within the field of physics
Dialogue: 0,0:13:41.07,0:13:50.52,Default,,0000,0000,0000,,extended by 58 on a yearly basis in this\Ntime window. So this is actually already
Dialogue: 0,0:13:50.52,0:13:55.92,Default,,0000,0000,0000,,very high, but they also found another\Nvery extreme case. They found one paper
Dialogue: 0,0:13:55.92,0:14:07.40,Default,,0000,0000,0000,,which had roundabout 7,000 words and the\Nmentioned authorship of 5,000. So, in
Dialogue: 0,0:14:07.40,0:14:15.39,Default,,0000,0000,0000,,average, the contribution of each\Nscientist or researcher of this paper who
Dialogue: 0,0:14:15.39,0:14:28.91,Default,,0000,0000,0000,,was mentioned was 1.1 word. Sounds\Nstrange, yeah. And so of course you have
Dialogue: 0,0:14:28.91,0:14:34.52,Default,,0000,0000,0000,,to see this in a certain context, and\Nmaybe we can talk about this later on,
Dialogue: 0,0:14:34.52,0:14:41.38,Default,,0000,0000,0000,,because it has to do with Atlas particle\Ndetector, which requires high maintenance
Dialogue: 0,0:14:41.38,0:14:46.02,Default,,0000,0000,0000,,and stuff. But still, the number of\Nauthorship, and you can see this
Dialogue: 0,0:14:46.02,0:14:52.52,Default,,0000,0000,0000,,regardless which scientific field we are\Ntalking about, generally increased the
Dialogue: 0,0:14:52.52,0:15:05.36,Default,,0000,0000,0000,,last years. It remains a problem\Nespecially for the reputation, obviously.
Dialogue: 0,0:15:05.36,0:15:12.40,Default,,0000,0000,0000,,It remains a problem that there is such\Nhigh pressure on nowadays researchers.
Dialogue: 0,0:15:12.40,0:15:20.20,Default,,0000,0000,0000,,Still, of course, we have ethics and\Nresearch requires standards of
Dialogue: 0,0:15:20.20,0:15:25.69,Default,,0000,0000,0000,,responsibility. And for example there’s\None, there’s other ones, but there’s one
Dialogue: 0,0:15:25.69,0:15:30.85,Default,,0000,0000,0000,,here on the slide: the “Australian Code\Nfor the Responsible Conduct of Research”
Dialogue: 0,0:15:30.85,0:15:36.69,Default,,0000,0000,0000,,which says: “The right to authorship is\Nnot tied to position or profession and
Dialogue: 0,0:15:36.69,0:15:41.06,Default,,0000,0000,0000,,does not depend on whether the\Ncontribution was paid for or voluntary.
Dialogue: 0,0:15:41.06,0:15:46.41,Default,,0000,0000,0000,,It is not enough to have provided \Nmaterials or routine technical support,
Dialogue: 0,0:15:46.42,0:15:50.97,Default,,0000,0000,0000,,or to have made the measurements \Non which the publication is based.
Dialogue: 0,0:15:50.97,0:15:55.13,Default,,0000,0000,0000,,Substantial intellectual involvement \Nis required.”
Dialogue: 0,0:15:55.13,0:16:03.26,Default,,0000,0000,0000,,So yeah, this is, could be one rule \Nto work with or to work by, to follow.
Dialogue: 0,0:16:03.26,0:16:08.08,Default,,0000,0000,0000,,And still we have this problem\Nof reputation which remains,
Dialogue: 0,0:16:08.08,0:16:11.48,Default,,0000,0000,0000,,and where I hand over to Judith again.
Dialogue: 0,0:16:11.48,0:16:19.74,Default,,0000,0000,0000,,Judith: Thank you. So we’re going to speak\Nabout strategic citation now. So if you
Dialogue: 0,0:16:19.74,0:16:29.54,Default,,0000,0000,0000,,put this point of reputation like that,\Nyou may say: So the researcher does find
Dialogue: 0,0:16:29.54,0:16:36.24,Default,,0000,0000,0000,,something in his research, his or her\Nresearch, and addresses the publication
Dialogue: 0,0:16:36.24,0:16:40.05,Default,,0000,0000,0000,,describing it to the community. And the\Ncommunity, the scientific community
Dialogue: 0,0:16:40.05,0:16:46.27,Default,,0000,0000,0000,,rewards the researcher with reputation.\NAnd now the algorithm, which is like
Dialogue: 0,0:16:46.27,0:16:54.73,Default,,0000,0000,0000,,perceived to be a new thing, is mediating\Nthe visibility of the researcher’s results
Dialogue: 0,0:16:54.73,0:17:00.72,Default,,0000,0000,0000,,to the community, and is also mediating\Nthe rewards – the career opportunities or
Dialogue: 0,0:17:00.72,0:17:04.92,Default,,0000,0000,0000,,the funding decisions etc. And what\Nhappens now and what is plausible to
Dialogue: 0,0:17:04.92,0:17:10.40,Default,,0000,0000,0000,,happen is that the researcher addresses\Nhis or her research also to the algorithm
Dialogue: 0,0:17:10.40,0:17:21.04,Default,,0000,0000,0000,,in terms of citing those who are evaluated\Nby the algorithm, who he wants to support,
Dialogue: 0,0:17:21.04,0:17:29.23,Default,,0000,0000,0000,,and also in terms of strategic keywording\Netc. And that’s the only thing which
Dialogue: 0,0:17:29.23,0:17:33.81,Default,,0000,0000,0000,,happens new, might be a perspective on\Nthat. So the one thing new: the algorithm
Dialogue: 0,0:17:33.81,0:17:40.70,Default,,0000,0000,0000,,is addressed as a recipient of scientific\Npublications. And it is like far-fetched
Dialogue: 0,0:17:40.70,0:17:46.26,Default,,0000,0000,0000,,to discriminate between so-called and\N‘visible colleges’ and ‘citation cartels’.
Dialogue: 0,0:17:46.26,0:17:50.90,Default,,0000,0000,0000,,What do I mean by that? So ‘invisible\Ncolleges’ is a term to say: “Okay, people
Dialogue: 0,0:17:50.90,0:17:56.49,Default,,0000,0000,0000,,are citing each other. They do not work\Ntogether in a co-working space, maybe, but
Dialogue: 0,0:17:56.49,0:18:00.82,Default,,0000,0000,0000,,they do research on the same topic.” And\Nthat’s only plausible that they cite each
Dialogue: 0,0:18:00.82,0:18:06.54,Default,,0000,0000,0000,,other. And if we look at citation networks\Nand find people citing each other, that
Dialogue: 0,0:18:06.54,0:18:12.86,Default,,0000,0000,0000,,does not have necessarily to be something\Nbad. And we also have people who are
Dialogue: 0,0:18:12.86,0:18:18.52,Default,,0000,0000,0000,,concerned that there might be like\N‘citation cartels’. So researchers citing
Dialogue: 0,0:18:18.52,0:18:27.27,Default,,0000,0000,0000,,each other not for purposes like the\Nresearch topics are closely connected, but
Dialogue: 0,0:18:27.27,0:18:35.72,Default,,0000,0000,0000,,to support each other in their career\Nprospects. And people do try to
Dialogue: 0,0:18:35.72,0:18:40.73,Default,,0000,0000,0000,,discriminate those invisible colleges from\Ncitation cartels ex post from looking at
Dialogue: 0,0:18:40.73,0:18:45.99,Default,,0000,0000,0000,,metadata networks of publication and find\Nthat a problem. And we have a discourse on
Dialogue: 0,0:18:45.99,0:18:58.30,Default,,0000,0000,0000,,that in the bibliometrics community. I\Nwill show you some short quotes how people
Dialogue: 0,0:18:58.30,0:19:04.51,Default,,0000,0000,0000,,talk about those citation cartels. So e.g.\NDavis in 2012 said: “George Franck warned
Dialogue: 0,0:19:04.51,0:19:08.96,Default,,0000,0000,0000,,us on the possibility of citation cartels\N– groups of editors and journals working
Dialogue: 0,0:19:08.96,0:19:13.59,Default,,0000,0000,0000,,together for mutual benefit.” So we have\Nheard about their journal impact factors,
Dialogue: 0,0:19:13.59,0:19:23.02,Default,,0000,0000,0000,,so they... it’s believed that editors talk\Nto each other: “Hey you cite my journal,
Dialogue: 0,0:19:23.02,0:19:27.48,Default,,0000,0000,0000,,I cite your journal, and we both \Nwill boost our impact factors.”
Dialogue: 0,0:19:27.48,0:19:32.90,Default,,0000,0000,0000,,So we have people trying \Nto detect those cartels,
Dialogue: 0,0:19:32.90,0:19:37.30,Default,,0000,0000,0000,,and Mongeon et al. wrote that:\N“We have little knowledge
Dialogue: 0,0:19:37.30,0:19:40.64,Default,,0000,0000,0000,,about the phenomenon itself and \Nabout where to draw the line between
Dialogue: 0,0:19:40.64,0:19:46.08,Default,,0000,0000,0000,,acceptable and unacceptable behavior.” So\Nwe are having like moral discussions,
Dialogue: 0,0:19:46.08,0:19:53.56,Default,,0000,0000,0000,,about research ethics. And also we find\Ndiscussions about the fairness of the
Dialogue: 0,0:19:53.56,0:19:58.46,Default,,0000,0000,0000,,impact factors. So Yang et al. wrote:\N“Disingenuously manipulating impact factor
Dialogue: 0,0:19:58.46,0:20:02.70,Default,,0000,0000,0000,,is the significant way to harm the\Nfairness of the impact factor.” And that’s
Dialogue: 0,0:20:02.70,0:20:09.78,Default,,0000,0000,0000,,a very interesting thing I think, because\Nwhy should an indicator be fair? So the...
Dialogue: 0,0:20:09.78,0:20:15.58,Default,,0000,0000,0000,,To believe that we have a fair measurement\Nof scientific quality relevance and rigor
Dialogue: 0,0:20:15.58,0:20:21.81,Default,,0000,0000,0000,,in one single like number, like their\Njournal impact factor, is not a small
Dialogue: 0,0:20:21.81,0:20:29.67,Default,,0000,0000,0000,,thing to say. And also we have a call for\Ndetection and punishment. So Davis also
Dialogue: 0,0:20:29.67,0:20:33.78,Default,,0000,0000,0000,,wrote: “If disciplinary norms and decorum\Ncannot keep this kind of behavior at bay,
Dialogue: 0,0:20:33.78,0:20:39.70,Default,,0000,0000,0000,,the threat of being delisted from the JCR\Nmay be necessary.” So we find the moral
Dialogue: 0,0:20:39.70,0:20:44.48,Default,,0000,0000,0000,,concerns on right and wrong. We find the\Nevocation of the fairness of indicators
Dialogue: 0,0:20:44.48,0:20:51.07,Default,,0000,0000,0000,,and we find the call for detection and\Npunishment. When I first heard about that
Dialogue: 0,0:20:51.07,0:20:56.64,Default,,0000,0000,0000,,phenomenon of citation cartels which is\Nbelieved to exist, I had something in mind
Dialogue: 0,0:20:56.64,0:21:03.74,Default,,0000,0000,0000,,which sounded... or it sounded like\Nfamiliar to me. Because we have a similar
Dialogue: 0,0:21:03.74,0:21:11.13,Default,,0000,0000,0000,,information retrieval discourse or a\Ndiscourse about ranking and power in a
Dialogue: 0,0:21:11.13,0:21:19.16,Default,,0000,0000,0000,,different area of society: in search\Nengine optimization. So I found a quote by
Dialogue: 0,0:21:19.16,0:21:26.83,Default,,0000,0000,0000,,Page et al., who developed the PageRank\Nalgorithm – Google’s ranking algorithm –
Dialogue: 0,0:21:26.83,0:21:32.99,Default,,0000,0000,0000,,in 1999, which has changed since that a\Nlot. But they wrote also a paper about the
Dialogue: 0,0:21:32.99,0:21:42.50,Default,,0000,0000,0000,,social implications of the information\Nretrieval by the PageRank as an indicator.
Dialogue: 0,0:21:42.50,0:21:45.91,Default,,0000,0000,0000,,And wrote that: “These types of\Npersonalized PageRanks are virtually
Dialogue: 0,0:21:45.91,0:21:50.15,Default,,0000,0000,0000,,immune to manipulation by commercial\Ninterests. ... For example fast updating
Dialogue: 0,0:21:50.15,0:21:54.35,Default,,0000,0000,0000,,of documents is a very desirable feature,\Nbut it is abused by people who want to
Dialogue: 0,0:21:54.35,0:22:00.53,Default,,0000,0000,0000,,manipulate the results of the search\Nengine.” And that was important to me to
Dialogue: 0,0:22:00.53,0:22:09.35,Default,,0000,0000,0000,,read because we also have like a narration\Nof abuse, of manipulation, the perception
Dialogue: 0,0:22:09.35,0:22:14.21,Default,,0000,0000,0000,,that that might be fair, so we have a fair\Nindicator and people try to betray it.
Dialogue: 0,0:22:14.21,0:22:21.79,Default,,0000,0000,0000,,And then we had in the early 2000s, \NI recall having a private website
Dialogue: 0,0:22:21.79,0:22:24.56,Default,,0000,0000,0000,,with a public guest book and \Ngetting link spam from people
Dialogue: 0,0:22:24.56,0:22:26.72,Default,,0000,0000,0000,,who wanted to boost their \NGoogle PageRanks,
Dialogue: 0,0:22:26.72,0:22:32.78,Default,,0000,0000,0000,,and shortly afterwards Google\Ndecided to punish link spam in their
Dialogue: 0,0:22:32.78,0:22:37.86,Default,,0000,0000,0000,,ranking algorithm. And then I got lots of\Nemails of people saying: “Please delete my
Dialogue: 0,0:22:37.86,0:22:43.89,Default,,0000,0000,0000,,post from your guestbook because Google’s\Ngoing to punish me for that.” We may say
Dialogue: 0,0:22:43.89,0:22:51.51,Default,,0000,0000,0000,,that this search engine optimization\Ndiscussion is now somehow settled and it’s
Dialogue: 0,0:22:51.51,0:22:57.94,Default,,0000,0000,0000,,accepted that Google's ranking is useful.\NThey have a secret algorithm, but it works
Dialogue: 0,0:22:57.94,0:23:05.25,Default,,0000,0000,0000,,and that is why it’s widely used. Although\Nthat journal impact factor seems to be
Dialogue: 0,0:23:05.25,0:23:13.16,Default,,0000,0000,0000,,transparent it’s basically the same thing\Nthat it's accepted to be useful and thus
Dialogue: 0,0:23:13.16,0:23:17.41,Default,,0000,0000,0000,,it's widely used. So the journal impact\Nfactor, the SCI and the like. We have
Dialogue: 0,0:23:17.41,0:23:24.57,Default,,0000,0000,0000,,another analogy so that Google decides\Nwhich SEO behavior is regarded acceptable
Dialogue: 0,0:23:24.57,0:23:28.48,Default,,0000,0000,0000,,and punishes those who act against the\Nrules and thus holds an enormous amount of
Dialogue: 0,0:23:28.48,0:23:38.67,Default,,0000,0000,0000,,power, which has lots of implications and\Nled to the spreading of content management
Dialogue: 0,0:23:38.67,0:23:44.91,Default,,0000,0000,0000,,systems, for example, with search engine\Noptimization plugins etc. We also have
Dialogue: 0,0:23:44.91,0:23:52.81,Default,,0000,0000,0000,,this power concentration in the hands of\NClarivate (formerly ThomsonReuters) who
Dialogue: 0,0:23:52.81,0:23:59.06,Default,,0000,0000,0000,,host the database for the general impact\Nfactor. And they decide on who’s going to
Dialogue: 0,0:23:59.06,0:24:05.14,Default,,0000,0000,0000,,be indexed in those journal citation\Nrecords and how is the algorithm, in
Dialogue: 0,0:24:05.14,0:24:12.20,Default,,0000,0000,0000,,detail, implemented in their databases. So\Nwe have this power concentration there
Dialogue: 0,0:24:12.20,0:24:21.78,Default,,0000,0000,0000,,too, and I think if we think about this\Nanalogy we might come to interesting
Dialogue: 0,0:24:21.78,0:24:29.77,Default,,0000,0000,0000,,thoughts but our time is running out so we\Nare going to give a take-home message.
Dialogue: 0,0:24:29.77,0:24:34.72,Default,,0000,0000,0000,,Tl;dr, we find that the scientific\Ncommunity reacts with codes of conduct to
Dialogue: 0,0:24:34.72,0:24:39.93,Default,,0000,0000,0000,,a problem which is believed to exist. The\Nstrategic citation – we have database
Dialogue: 0,0:24:39.93,0:24:44.83,Default,,0000,0000,0000,,providers which react with sanctions so\Npeople are delisted from the journal
Dialogue: 0,0:24:44.83,0:24:50.24,Default,,0000,0000,0000,,citation records or journals are delisted\Nfrom the journal citation records to
Dialogue: 0,0:24:50.24,0:24:55.42,Default,,0000,0000,0000,,punish them for citation stacking. And we\Nhave researchers and publishers who adapt
Dialogue: 0,0:24:55.42,0:25:05.16,Default,,0000,0000,0000,,their publication strategies in reaction\Nto this perceived algorithmic power. But
Dialogue: 0,0:25:05.16,0:25:12.47,Default,,0000,0000,0000,,if we want to understand this as a problem\Nwe don’t have to only react to the
Dialogue: 0,0:25:12.47,0:25:19.18,Default,,0000,0000,0000,,algorithm but we have to address the power\Nstructures. Who holds these instruments in
Dialogue: 0,0:25:19.18,0:25:24.21,Default,,0000,0000,0000,,in their hands? If we talk about\Nbibliometrics as an instrument and we
Dialogue: 0,0:25:24.21,0:25:27.83,Default,,0000,0000,0000,,should not only blame the algorithm – so\N#dontblamethealgorithm.
Dialogue: 0,0:25:27.83,0:25:33.20,Default,,0000,0000,0000,,Thank you very much!\N{\i1}applause{\i0}
Dialogue: 0,0:25:37.98,0:25:43.63,Default,,0000,0000,0000,,Herald: Thank you to Franziska, Teresa \Nand Judith, or in the reverse order.
Dialogue: 0,0:25:44.51,0:25:48.45,Default,,0000,0000,0000,,Thank you for shining a light on \Nhow science is actually seen
Dialogue: 0,0:25:48.45,0:25:50.80,Default,,0000,0000,0000,,in its publications.
Dialogue: 0,0:25:50.80,0:25:52.41,Default,,0000,0000,0000,,As I started off as well,
Dialogue: 0,0:25:52.41,0:25:55.65,Default,,0000,0000,0000,,it’s more about\Nscratching each other a little bit.
Dialogue: 0,0:25:55.65,0:25:57.79,Default,,0000,0000,0000,,I have some questions here \Nfrom the audience.
Dialogue: 0,0:25:57.79,0:25:59.74,Default,,0000,0000,0000,,This is Microphone 2, please!
Dialogue: 0,0:25:59.74,0:26:05.32,Default,,0000,0000,0000,,Mic2: Yes, thank you for this interesting\Ntalk. I have a question. You may be
Dialogue: 0,0:26:05.32,0:26:09.55,Default,,0000,0000,0000,,familiar with the term ‘measurement\Ndysfunction’, that if you provide a worker
Dialogue: 0,0:26:09.55,0:26:14.22,Default,,0000,0000,0000,,with an incentive to do a good job based\Non some kind of metric then the worker
Dialogue: 0,0:26:14.22,0:26:19.96,Default,,0000,0000,0000,,will start optimizing for the metric\Ninstead of trying to do a good job, and
Dialogue: 0,0:26:19.96,0:26:26.20,Default,,0000,0000,0000,,this is kind of inevitable. So, don’t you\Nsee that maybe it could be treating the
Dialogue: 0,0:26:26.20,0:26:32.54,Default,,0000,0000,0000,,symptoms if we just react about code of\Nconduct, tweaking algorithms or addressing
Dialogue: 0,0:26:32.54,0:26:37.20,Default,,0000,0000,0000,,power structures. But instead we need to\Nremove the incentives that lead to this
Dialogue: 0,0:26:37.20,0:26:43.67,Default,,0000,0000,0000,,measurement dysfunction.\NJudith: I would refer to this phenomenon
Dialogue: 0,0:26:43.67,0:26:50.74,Default,,0000,0000,0000,,as “perverse learning” – learning for the\Ngrades you get but not for your intrinsic
Dialogue: 0,0:26:50.74,0:27:00.80,Default,,0000,0000,0000,,motivation to learn something. We observe\Nthat in the science system. But if we only
Dialogue: 0,0:27:00.80,0:27:10.44,Default,,0000,0000,0000,,adapt the algorithm, so take away the\Nincentives, would be like you wouldn’t
Dialogue: 0,0:27:10.44,0:27:20.48,Default,,0000,0000,0000,,want to evaluate research at all which you\Ncan probably want to do. But to whom would
Dialogue: 0,0:27:20.48,0:27:32.74,Default,,0000,0000,0000,,you address this call or this demand, so\N“please do not have indicators” or… I give
Dialogue: 0,0:27:32.74,0:27:38.69,Default,,0000,0000,0000,,the question back to you. {\i1}laughs{\i0}\NHerald: Okay, questions from the audience
Dialogue: 0,0:27:38.69,0:27:46.50,Default,,0000,0000,0000,,out there on the Internet, please. Your\Nmic is not working? Okay, then I go to
Dialogue: 0,0:27:46.50,0:27:52.13,Default,,0000,0000,0000,,Microphone 1, please Sir.\NMic1: Yeah, I want to have a provocative
Dialogue: 0,0:27:52.13,0:27:56.69,Default,,0000,0000,0000,,thesis. I think the fundamental problem is\Nnot how these things are gamed but the
Dialogue: 0,0:27:56.69,0:28:01.20,Default,,0000,0000,0000,,fundamental problem is that if we think\Nthe impact factor is a useful measurement
Dialogue: 0,0:28:01.20,0:28:04.60,Default,,0000,0000,0000,,for the quality of science. \NBecause I think it’s just not.
Dialogue: 0,0:28:04.60,0:28:06.95,Default,,0000,0000,0000,,{\i1}applause{\i0}\N
Dialogue: 0,0:28:09.58,0:28:12.11,Default,,0000,0000,0000,,Judith: Ahm.. I..\NMic 1: I guess that was obvious...
Dialogue: 0,0:28:12.11,0:28:13.49,Default,,0000,0000,0000,,Judith: Yeah, I would not say
Dialogue: 0,0:28:13.49,0:28:17.63,Default,,0000,0000,0000,,that the journal impact factor is \Na measurement of scientific quality
Dialogue: 0,0:28:17.63,0:28:23.52,Default,,0000,0000,0000,,because no one has like \Na definition of scientific quality.
Dialogue: 0,0:28:23.52,0:28:28.45,Default,,0000,0000,0000,,So what I can observe is only \Npeople believe this journal impact factor
Dialogue: 0,0:28:28.45,0:28:36.65,Default,,0000,0000,0000,,to reflect some quality. \NMaybe they are chasing a ghost but I…
Dialogue: 0,0:28:36.65,0:28:41.98,Default,,0000,0000,0000,,whether that’s a valid measure \Nis not so important to me,
Dialogue: 0,0:28:41.98,0:28:45.24,Default,,0000,0000,0000,,even if it were a relevant \Nor a valid measure,
Dialogue: 0,0:28:45.24,0:28:52.31,Default,,0000,0000,0000,,it would concern me \Nhow it affects science.
Dialogue: 0,0:28:52.99,0:28:56.50,Default,,0000,0000,0000,,Herald: Okay, question from Microphone 3\Nthere. Please.
Dialogue: 0,0:28:56.50,0:28:59.50,Default,,0000,0000,0000,,Mic3: Thanks for the interesting talk. \NI have a question about
Dialogue: 0,0:28:59.50,0:29:04.40,Default,,0000,0000,0000,,the 5,000 authors paper. \NWas that same paper published
Dialogue: 0,0:29:04.40,0:29:09.45,Default,,0000,0000,0000,,five thousand times or was it one paper\Nwith ten page title page?
Dialogue: 0,0:29:10.05,0:29:15.26,Default,,0000,0000,0000,,Franziska: No, it was one paper ...
Dialogue: 0,0:29:15.26,0:29:19.60,Default,,0000,0000,0000,,... counting more than 7,000 words. \NAnd the authorship,
Dialogue: 0,0:29:19.60,0:29:24.46,Default,,0000,0000,0000,,so authors and co-authors,\Nwere more than 5,000.
Dialogue: 0,0:29:24.46,0:29:31.01,Default,,0000,0000,0000,,Mic3: Isn’t it obvious \Nthat this is a fake?
Dialogue: 0,0:29:31.01,0:29:35.07,Default,,0000,0000,0000,,Franziska: Well that’s \Nwhat I meant earlier
Dialogue: 0,0:29:35.07,0:29:44.05,Default,,0000,0000,0000,,when saying, you have to see this within\Nits context. So physicists are working
Dialogue: 0,0:29:44.05,0:29:52.17,Default,,0000,0000,0000,,with this with Atlas, this detective\Nsystem. As there were some physicists in
Dialogue: 0,0:29:52.17,0:30:01.78,Default,,0000,0000,0000,,the audience they probably do know how\Nthis works. I do not. But as they claim
Dialogue: 0,0:30:01.78,0:30:08.33,Default,,0000,0000,0000,,it’s so much work to work with this, and\Nit, as I said, requires so high
Dialogue: 0,0:30:08.33,0:30:19.12,Default,,0000,0000,0000,,maintenance it’s... They obviously have\Nyeah...
Dialogue: 0,0:30:19.12,0:30:21.72,Default,,0000,0000,0000,,Mic3: So everybody who contributed was\Nlisted?
Dialogue: 0,0:30:21.72,0:30:28.81,Default,,0000,0000,0000,,Judith: Exactly, that’s it. And if this is\Nethically correct or not, well, this is
Dialogue: 0,0:30:28.81,0:30:33.82,Default,,0000,0000,0000,,something which needs to be discussed,\Nright? This is why we have this talk, as
Dialogue: 0,0:30:33.82,0:30:39.91,Default,,0000,0000,0000,,we want to make this transparent, and\Ncontribute it to an open discussion.
Dialogue: 0,0:30:39.91,0:30:44.61,Default,,0000,0000,0000,,Herald: Okay, I’m sorry guys. I have to\Ncut off here because our emission out
Dialogue: 0,0:30:44.61,0:30:49.23,Default,,0000,0000,0000,,there in space is coming to an end.\NI suggest that you guys
Dialogue: 0,0:30:49.23,0:30:52.88,Default,,0000,0000,0000,,find each other somewhere,\Nmaybe in the tea house or...
Dialogue: 0,0:30:52.88,0:30:55.02,Default,,0000,0000,0000,,Judith: Sure. We are around, we are here.
Dialogue: 0,0:30:55.02,0:30:58.15,Default,,0000,0000,0000,,Herald: You are around. I would love to\Nhave lots of applause for these ladies,
Dialogue: 0,0:30:58.15,0:31:03.03,Default,,0000,0000,0000,,for it really lights on\Nhow these algorithms
Dialogue: 0,0:31:03.03,0:31:05.30,Default,,0000,0000,0000,,not or are working. Thank you very much!
Dialogue: 0,0:31:05.30,0:31:06.64,Default,,0000,0000,0000,,Judith: Thank you!
Dialogue: 0,0:31:06.64,0:31:21.63,Default,,0000,0000,0000,,{\i1}postroll music{\i0}
Dialogue: 0,0:31:21.63,0:31:26.74,Default,,0000,0000,0000,,{\i1}subtitles created by c3subtitles.de\Nin the year 2018{\i0}