FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:   || 2 |

«Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting Emilio Delgado López-Cózar1, Nicolás Robinson-García1 ...»

-- [ Page 1 ] --

Manipulating Google Scholar Citations and Google Scholar Metrics:

simple, easy and tempting

Emilio Delgado López-Cózar1, Nicolás Robinson-García1 y Daniel Torres-Salinas2

EC3: Evaluación de la Ciencia y de la Comunicación Científica


Universidad de Granada


Universidad de Navarra

edelgado@ugr.es; elrobin@ugr.es; torressalinas@gmail.com


The launch of Google Scholar Citations and Google Scholar Metrics may cause a revolution in the research evaluation field as it places within every researcher’s reach tools that allow them to measure their output. However, the data and bibliometric indicators offered by Google’s products can be easily manipulated. In order to alert the research community, we present an experiment in which we manipulate the Google Citations’ profiles of a research group through the creation of false documents that cite their documents, and consequently, the journals in which they have published, modifying their H-index. For this purpose we created six documents authored by a faked author and we uploaded them to a researcher’s personal website under the University of Granada’s domain. The result of the experiment meant an increase of 774 citations in 129 papers (six citations per paper) increasing the authors and journals' H-index. We analyse the malicious effect this type of practices can cause to Google Scholar Citations and Google Scholar Metrics. Finally, we conclude with several deliberations over the effects these malpractices may have and the lack of control these tools offer.


Google Citations / Google Scholar Metrics/ Scientific Journals / Scientific fraud / Citation analysis / Bibliometrics / H Index / Evaluation / Researchers Referencia bibliográfica recomendada Delgado López-Cózar, Emilio; Robinson-García, icolás; Torres Salinas, Daniel (2012).

Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting.

EC3 Working Papers 6: 29 May, 2012

1. I TRODUCTIO If the launch of Google Scholar in 2004 (a novel search engine focused on retrieving any type of academic material along with its citations) meant a revolution in the scientific information market allowing universal and free access to all documents available in the web, the launch of Google Scholar Citations (hereafter GS Citations) (Cabezas-Clavijo & Torres-Salinas, 2012) and Google Scholar Metrics (hereafter GS Metrics) (Cabezas- Clavijo & Delgado López-Cózar, 2012) may well be a historical milestone for the globalization and democratisation of research evaluation (Butler 2011). As well as constituting a new threat to the traditional bibliographic databases and bibliometric indexes offered by Thomson Reuters (Web of Science and JCR) and Elsevier (Scopus and SJR), ending with their monopoly and becoming a serious competitor; Google Scholar's new products project a future landscape with ethical and sociological dilemmas that may entail serious consequences in the world of science and research evaluation.

Delgado López-Cózar, Robinson-García & Torres-Salinas. Manipulating Google Scholar … 2 Without considering the technical and methodological problems that the Google Scholar products have which are currently under study (Jacsó, 2008, 2011; Wouters y Costas, 2012; Aguillo, 2012; Cabezas-Clavijo y Delgado López-Cózar, 2012; Torres-Salinas, Ruiz-Pérez y Delgado López-Cózar, 2009) and which will be presumably solved in a near future, its irruption ends with all kinds of scientific control, becoming a new challenge to the bibliometric community. Since the moment Google Scholar automatically retrieves, indexes and stores any type of scientific material uploaded by an author without any previous external control (repositories are only a technical filter as they do not assure a revision of the content), it allows unprincipled people to manipulate their output, impacting directly on their bibliometric performance.

Because this type of behaviour by which one modifies its output and impact through intentional and unrestrained self-citation is not uncommon, we consider necessary to analyse thoroughly Google's capacity to detect the manipulation of data.

This study continues the research line started by Labbé (2010). In his experiment he transformed a faked researcher called Ike Antkare ( ‘I can’t care’) into the most prolific researcher in history. However, in this case we will enquire over the most dangerous aspects of gaming tools aimed at evaluating researchers and the malicious effects they can have on researchers' behaviour. Therefore our aim is to demonstrate how easily anyone can manipulate Google Scholar's tools. But, contrarily to Labbé, we will not emphasize the technical aspects of such gaming, but its sociological dimension, focusing on the enormous temptation these tools can have for researchers and journals' editors, eager to increase their impact. In order to do so, we will show how the bibliometric profiles of researchers and journals can be modified simultaneously in the easiest way possible: by uploading faked documents on our personal website citing the whole production of a research group. It is not necessary to use any type of software for creating faked documents: you only need to copy and paste the same text over and over again and upload the resulting documents in a webpage under an institutional domain. We will also analyse Google's capacity to detect retracted documents and delete their bibliographic records along with the citations they make.

This type of studies by which false documents are created in order to evidence defects, biases or errors committed by authors have been conducted many times, especially in the research evaluation field. The reader is referred to the works of Peters & Ceci (1990), Epstein (1990), Sokal (1996, 1997) or Baxt et al. (1998) when demonstrating the deficiencies of the peer review method as an objective, reliable and valid control tool when filtering scientific papers. Or Scigen1, a programme created by three students from the MIT for generating random papers in the Computer Science field including graphs, figures and references. All of these works raised an intense debate within the research community.

Therefore, this paper is structured as follows. Firstly we described the methodology followed; how were the false documents created and where were they uploaded. Then we show the effect they had on the bibliometric profiles of the researchers who received the citations and we emulate the effect these citations would have had on the journals affected 1 http://pdos.csail.mit.edu/scigen/ Delgado López-Cózar, Robinson-García & Torres-Salinas. Manipulating Google Scholar … 3 if GS Metrics was updated regularly. We analyse the technical effects and the dangers these tools entail for evaluating research. Finally we conclude emphasizing their strengths and we end with some concluding remarks.


In order to analyse GS Citations’ capacity to discriminate academic works from those which aren’t and test the grade of difficulty for manipulating output and citations in Google Scholar and its bibliometric tools (GS Citations and Metrics), we created false documents referencing the whole research production of the EC3 research group (Science and Scientific Communication Evaluation) available at http://ec3.ugr.es in the easiest possible way. This way we intend to show how anyone can manipulate its output and citations in GS Citations.

Figure 1. Fake documents authored by the non-existent researcher MA PantaniContador Following the example set by Labbé (2010), we created a false researcher named Marco Alberto Pantani-Contador, making reference to the great fraud the Italian cyclist became at the end and the accidental causes that deprived the Spanish cyclist from winning the Tour.

Thus, Pantani-Contador authored six documents (figure 1) which did not intend to be considered as research papers but working papers. In a process that lasted less than a half day’s work, we draft a small text, copied and pasted some more from the EC3 research group’s website, included several graphs and figures, translated it automatically into English using Google Translate and divided it into six documents. Each document referenced 129 papers authored by at least one member of the EC3 research group according to their website http://ec3.ugr.es. That is, we expected a total increase of 774 citations.

Delgado López-Cózar, Robinson-García & Torres-Salinas. Manipulating Google Scholar … 4 Afterwards, we created a webpage in html under the University of Granada domain including references to the false papers and linking to their full-text, expecting Google Scholar to index their content. We excluded other services such as institutional or subjectbased repositories as they are not obliged to undertake any bibliographic control rather than a formal one (Delgado López-Cózar, 2012) and we did not aim at bypassing their filters.

The false documents were uploaded on 17 April, 2012. Presumably because it was a personal website and not a repository, Google indexed these documents nearly a month after they were uploaded, on 12 May, 2012. At that time the members of the research group used as case study, including the authors of this paper, received an alert from GS Citations pointing out that someone called MA Pantani-Contador had cited their output.

The citation explosion was thrilling, especially in the case of the youngest researchers where their citation rates were multiplied by six, notoriously increasing in size their profiles.

–  –  –

Figure 2 shows the increase of citations the authors experienced. Obviously, the number of citations by author varies depending on the number of publications each member of the research group had as well as the inclusion of real citations received during the study period. Thus, the greatest increase is for the less-cited author, Robinson-Garcia, who multiplies by 7.25 the number of citations received, while Torres-Salinas doubles it and Delgado López-Cózar experiences an increase of 1.5. We also note the effect on the HDelgado López-Cózar, Robinson-García & Torres-Salinas. Manipulating Google Scholar … 5 index of each researcher. While the most significant increase is perceived in the less prolific profile, the variation for the other two others is much more moderate, illustrating the stability of the indicator. Note how in Torres-Salinas’ case, where the number of citations is doubled, how the H-index only increases by two. On the other hand, we observe how the i10-index is much more sensitive to changes. In Torres-Salinas’ case, the increase goes from 7 to 17, and in Delgado López-Cózar’s case it triples for the last five years, going from 11 to 33.

–  –  –

Also, it is interesting to analyse the effect this citation increase may have on the h-index for journals indexed in GS Metrics. For this, we have considered the two journals in which the members of the research group have published more papers and therefore, more sensitive to be manipulated. These are El Profesional de la Información with 30 papers published in this journal and Revista Española de Documentación Científica, with 33 papers. In table 1 we show the H-indexes for El Profesional de la Información and Revista Española de Documentación Científica according to Google and the increase it would have had if the citations emitted by Pantani-Contador had been included. We must alert the reader that this tool, contrarily to the rest of Google’s products, is not Delgado López-Cózar, Robinson-García & Torres-Salinas. Manipulating Google Scholar … 6 automatically updated and that data displayed dates to the day of its launch, that is, 1 April, 2012 (Cabezas-Clavijo & Delgado López-Cózar, 2012). We observe that El Profesional de la Información would be the one which would have been more influenced, as seven papers would surpass the 12 citations threshold increasing its H-index and ascending in the ranking for journals in Spanish language from position 20 to position 5.

Revista Española de Documentación Científica would slightly modify its position, as only one article surpasses the 9 citations threshold that influence its h-index. Even so and due to the high number of journals with its same h-index, it would go up from position 74 to 54.

–  –  –

After proving the vulnerability of Google’s products when including false documents and showing its effect at the researcher-level and journal-level, on 17 May, 2012 we deleted the false documents and webpage in order to see if Google Scholar would delete the records and the citations received according to GS Citations. However, until this date (29 May) and 17 days after they were removed from the Internet, no modifications have been made whatsoever. The records of the authored documents by our faked researcher are still available when searching its production and, despite being broken links, there is a version of the documents saved by Google.


The results of our experiment show how easy and simple it is to modify the citation profiles offered by GS Citations. This exposes the vulnerability of the product if editors and researchers are tempted to do “citations engineering” and modify their H-index by excessively self-citing their papers or, in a most refined way, sending citations only to the hot zone of their publications, that is, those which can influence this indicator. In the case of El Profesional de la Información, there are 16 documents with 10 to 12 citations for the time period analysed by GS Metrics (2007-2011). These could modify the journal’s position by receiving 1 to 3 citations more.

Pages:   || 2 |

Similar works:

«GENE 303 Tottori Exchange Survival Guide My Eight Months in Tottori, Japan Prepared by: Lily Chai January 15, 2007 Table of Contents 1 Introduction I learned about Tottori Exchange in my 1A semester. Japanese was my second language in high school so I was interested in continuing my studies in the language, as well, I thought it would be a great experience to attend school for a short period of time in the world-leading country in electronics, thus I looked into exchange opportunities in Japan....»

«Sport NZ Submission Ministerial Forum on Alcohol Advertising and Sponsorship Submission from Sport New Zealand 28 April 2014 1. Sport New Zealand (Sport NZ) welcomes the opportunity to make a submission to the Ministerial Forum on Alcohol Advertising and Sponsorship. Executive summary 2. Sport NZ is concerned about the potential of any further restrictions on alcohol sponsorship and associated advertising damaging the financial viability of the sport and recreation sector in challenging...»

«SERVING THE PUBLIC, FIGHTING AGAINST THE STATE LABOR UNREST IN THE PUBLIC SECTOR ACROSS THE EU by Imre Gergely Szabó Submitted tó CENTRAL EUROPEAN UNIVERSITY Dóctóral Schóól óf Pólitical Science, Public Pólicy, and Internatiónal Relatións In partial fulfillment óf the requirements fór the degree óf Dóctór óf Philósóphy Supervisór: Prófessór Bela Greskóvits 31 May 2016 52 875 wórds Declaration I hereby declare that this dissertation contains no materials accepted for any...»

«Philippine Institute for Development Studies Surian sa mga Pag-aaral Pangkaunlaran ng Pilipinas Innovations in Microfinance in Southeast Asia Gilberto M. Llanto and Ryu Fukui DISCUSSION PAPER SERIES NO. 2003-11 The PIDS Discussion Paper Series constitutes studies that are preliminary and subject to further revisions. They are being circulated in a limited number of copies only for purposes of soliciting comments and suggestions for further refinements. The studies under the Series are unedited...»

«ISSN(Online): 2319 8753 ISSN (Print) :2347 6710 International Journal of Innovative Research in Science, Engineering and Technology (An ISO 3297: 2007 Certified Organization) Vol. 4, Issue 5, May 2015 EOG Based Prosthetic Arm-Hand Control Veena J Ukken 1, S.Vaishnodevi 2, S. Mathankumar 3 P.G. Student, Department of Biomedical Engineering, VMKV Engineering College, Salem, Tamilnadu, India1 Assistant Professor, Department of Biomedical Engineering, VMKV Engineering College, Salem, Tamilnadu,...»

«375 Gardens’ Bulletin Singapore 63(1 & 2): 375–383. 2011 Hybrid zone characteristics of the intergeneric hybrid bamboo × Gigantocalamus malpenensis (Poaceae: Bambusoideae) in Peninsular Malaysia K.M. Wong1 and Y.W. Low Singapore Botanic Gardens, 1 Cluny Road, Singapore 259569 wkm2000@gmail.com 1 ABSTRACT. The natural intergeneric hybrid bamboo × Gigantocalamus malpenensis and its parental species, Dendrocalamus pendulus and Gigantochloa scortechinii, were mapped along a 4-km stretch of...»

«IN THE COURT OF JUDICIAL MAGISTRATE 1st CLASS, NAGAON G.R.Case No: 829 of 2008 U/S 406/409, IPC State Vs Sri Ratneswar Bora.. Accused. Present: Mr C.P. Talukdar, AJS Evidence recorded on: 02.05.11, 03.01.12 04.06.12 14.06.13 25.09.13 Arguments heard on : 25.04.14, 03.05.14 Judgment delivered on: 13.05.14 Advocate for the prosecution: Ld A.P.P. Ms R. Ahmed. Advocate for the defence : Ld Adv Mr G.Chandra Bora. Judgment The case of the prosecution in brief is that the informant Sri Lakhyamoni 1....»

«1 STEVEN NEIL MACEACHERN PERSONAL Birthdate: October 24, 1961 Married, two children Work Address: Home Address: Department of Statistics 2820 Andover Road The Ohio State University Columbus, Ohio 43221 1958 Neil Avenue (614) 486-8468 Columbus, Ohio 43210-1247 (614) 292-5843 EDUCATION Ph.D., Statistics, University of Minnesota, 1988, Phi Kappa Phi Dissertation Topic: Sequential Bayesian Bioassay Design B.A., Mathematics, magna cum laude, Carleton College, 1982, Phi Beta Kappa, Sigma Xi POSITIONS...»

«Stories from the brainreels podcast transcript October 1, 2015 [music] Introduction CHERYL: Welcome to Stories from the brainreels monthly podcast about brain injury and disability with a focus on art, culture, and disability pride. [music] My guest this month is documentary filmmaker Mike Turner who has made a film exploring stuttering from inside the stuttering community. He joined me to talk about this film The Way We Talk. I got to see a pre-screening of it and had the great pleasure of...»

«XVII CONGRESO INTERNACIONAL ASOCIACIÓN DE LINGÜÍSTICA Y FILOLOGÍA DE AMÉRICA LATINA (ALFAL 2014) João Pessoa Paraíba, Brasil #2733 FUNCIONALIDADE DE FORMAS VERBAIS DE PASSADO: UMA INTERFACE ENTRE LING Leandra Cristina de Oliveira – UFSC/DLLE/P Alison Felipe Gesser Introdução Assentados nos postulados teóricos da vertente fucionalista da tradução e do funcionalismo linguístic trabalho um olhar contrastivo entre o português brasileiro e o espanhol mexicano, no que diz respeito ao...»

«In‐Depth Analyses of Seabird Bycatch in Individual Marine Stewardship Council Fisheries American Bird Conservancy 4249 Loudoun Ave.  P.O. Box 249 The Plains, VA 20198 USA Tel: 540‐253‐5780  Fax: 540‐253‐5782 www.abcbirds.org February 29, 2012 Table of Contents INTRODUCTION ACKNOWLEDGMENTS POTENTIALLY HIGH RISK FISHERIES British Columbia Pink Salmon Seine, Troll and Gillnet Fishery British Columbia Sockeye Salmon Fishery Canadian Pacific Sablefish (Anoplopoma fimbria) Fishery...»

«FRANKLIN COUNTY RESOURCE INFORMATION Quick Reference County of Franklin. Frankfort Police 502-875-8525 or 911 County Court Clerk 502-875-8702 KY State Police 227-2221 or 800-222-5555 Circuit Court Clerk 502-564-8380 Sheriff 502-875-8740 or 875-8738 County Clerk Voters Reg. 502-875-8704 24-Hr. Abuse Hotline 502-229-9762 Friend of The Court 502-875-2324 Rape Crisis Center 1-800-656-HOPE (4674) Drivers License 502-564-4576 ACCESS Soup Kitchen 502-223-5179 Family Court 502-564-2278 Alcoholics...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.