FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:   || 2 | 3 | 4 |

«Quality assurance and assessment of scholarly research A guide for researchers, academic administrators and librarians May 2010 This ...»

-- [ Page 1 ] --

Quality assurance and

assessment of scholarly research

A guide for researchers, academic administrators and librarians

May 2010


This guide has been produced by The Research Information Network to provide

researchers, academic administrators and librarians with an understanding of

quality assurance processes and some of the current issues surrounding the debate

about quality assurance.

The guide is available at www.rin.ac.uk/quality-assurance

Hard copies can be ordered to distribute to colleagues, please email catherine.gray@rin.ac.uk About the Research Information Network The Research Information Network has been established by the higher education funding councils, the research councils, and the UK national libraries. We investigate how efficient and effective the information services provided for the UK research community are, how they are changing, and how they might be improved for the future. We help to ensure that researchers in the UK benefit from world-leading information services, so that they can sustain their position as among the most successful and productive researchers in the world. All our publications are available on our website at www.rin.ac.uk Contents

1. Introduction 4

2. Why quality assurance and assessment? 6

3. Definitions 7

4. Tools and mechanisms 8

5. Assurance and assessment of research projects and programmes 9

6. Assurance and assessment of researchers, institutions, and 16 scholarly journals

7. Conclusion 20 References and useful resources 22

1. Introduction Quality assurance and assessment play many important roles in the research community. They inform crucial decisions on the funding of projects, teams and whole institutions, on how research is conducted, on recruitment and promotion, on what is published or disseminated, and on what researchers and others choose to read. They underpin trust in the work of the research community.

This document provides an overview of some of the key issues surrounding quality assurance and assessment of scholarly research. It is intended for academic administrators, researchers and librarians who deal with elements of quality assurance and quality assessment as part of their daily work, but who wish to understand more about the broader context of that work.

The various techniques and processes involved in determining quality have evolved over time to meet the changing needs of researchers, institutions and funders. Further development will doubtless be needed in response to changing circumstances and needs, as well as to the opportunities presented by new technologies and services.

Assurance and assessment regimes, tools and techniques will thus continue to evolve in

response to:

• increases in the volume of research. The number of researchers, of the projects they undertake, and of the outputs they produce is growing year by year. This puts increasing strain on existing assurance and assessment systems • increased pressure on costs. Effective assurance and assessment systems are costly, even though some of the most costly parts – including the time spent by peer reviewers – are seldom revealed in cash terms. The need to bear down on costs will continue to exert pressure to find more streamlined systems and techniques.

• increased competition between researchers. Constraints on funding bring increased competition for research grants and contracts, falling success rates, more work for assurance and assessment systems, and more difficult decisions in determining which projects, researchers and institutions succeed in the competition. Similar pressures are evident as researchers compete for space in prestigious publications.

• new kinds of research outputs. Many assurance and assessment regimes are built around outputs that are formally published in journals, monographs and conference proceedings.

But the digital revolution leads to new kinds of outputs – multimedia presentations, working papers, pre-prints, blogs and so on – that can readily be circulated worldwide. And dataintensive research brings a new focus on data as an output in its own right. Assurance and assessment regimes will need to keep up with these new outputs and forms of communication.

New technologies and services have already changed the ways in which assurance and assessment is done. And new opportunities are arising in the form of social tagging and recommendation systems, checks on plagiarism, and facilities for online comments and ratings. New technologies are also facilitating the development of a wide range of sophisticated bibliometric and other measures that can provide useful evidence to support assurance and assessment systems. One of the key issues for the future will be how these new tools and techniques evolve alongside more traditional systems of assurance and assessment of the quality of research.

2. Why quality assurance and assessment?

Quality assurance and assessment are critical at several stages in the research process. Since resources are limited (and likely to be increasingly so) funders need effective mechanisms to ensure that they support only the best projects, researchers and institutions. It is also important that procedures are in place to ensure that research projects are undertaken properly, efficiently and effectively. And once projects have been completed, and results are being prepared for presentation, checks are needed to ensure that what is published and disseminated meets high standards. This is important in order to ensure that the records of research – sometimes referred to as the ‘minutes of science’ – provide an accurate record of what has been found. Such checks also help researchers and other readers who have limited time – and often limited expertise in particular specialised fields or projects – to identify and locate the most high-quality and significant findings of relevance to them.

This document begins with an overview of the closely-related processes of quality assurance and quality assessment. It then examines how these processes manifest themselves at key stages in the

cycle of:

(i) a specific research project or programme (ii) a researcher’s career (iii) a university or research institution (iv) scholarly publications

3. Definitions Quality is of course not a straightforward concept. The Oxford English Dictionary (OED) defines it as the nature or standard of something as measured against other things of a similar kind, and especially the degree of excellence it possesses. In the academic environment, there is widespread agreement that quality involves adherence to key principles such as intellectual rigour, accurate recording and honest reporting of results, and integrity in recognising the work of other researchers.

But there may be legitimate differences of view as to the quality of, for example, notably innovative or groundbreaking work; and notions of quality may in some cases be related to specific contexts such as the scope or remit of a research funder or publisher. Numerous bodies undertake quality assurance and assessment for different purposes; the varying definitions and processes they use may clearly give rise to different results.

It is important also to note that while the terms quality assessment and quality assurance are often used interchangeably, there are important distinctions between them. ’Assessment’ may be defined as the process or means of evaluating academic work. ‘Assurance’, on the other hand, implies a promise or guarantee – a statement that something is of good quality and can be trusted. The relatively modern concept of quality assurance may be defined as the set of procedures designed to ensure a desired level of quality in a service or product.

The key distinction, therefore, is that an assessment process implies a review – involving human judgements and/or quantitative scores – which may find work of varying quality, from the poor or mediocre to the excellent or outstanding. A finding at the end of the process that work is of poor quality does not mean that the process has failed, rather the reverse. Reviewers have no responsibility to improve the work they examine.

Quality assurance, by contrast, implies a process of tests or filters to ensure that products or services pass a quality threshold. If the process is external, quality assurance often also provides a kitemark to indicate that the work has passed that threshold. External processes in particular may also involve an active role for reviewers in ensuring that the work under scrutiny emerges as the best possible version of itself. If work of poor quality emerges at the end of a quality assurance process, that process has failed. It is important to note, however, that in the research environment there can be no guarantee that findings and conclusions are correct and unchallengeable. The advancement of knowledge and understanding depends on challenging, qualifying or even overturning current findings. All that an assurance process can do is to seek to ensure that research is properly conducted, and its results are reported accurately, on the basis of the best currentlyavailable techniques, knowledge and understanding.

The processes of assurance and assessment thus involve a number of different tools and mechanisms which are employed for different purposes, in different contexts, and at different stages of the research lifecycle. The rest of this document focuses, after a brief introduction to the key mechanisms, on how they are applied and with what effect.

4. Tools and mechanisms Some assurance and assessment mechanisms involve quantitative measures such as bibliometrics, while others involve monitoring, review and evaluation by people who are knowledgeable in the relevant field. The principle that judgements should be made by experts who are respected in the field – peer review – is held by most researchers to be fundamental to any effective system for assuring or assessing research quality. Peer review attracts deep and strong support across the research community. But it comes in a number of different forms, and practices vary considerably in different contexts and fields.

Peer review also attracts criticism, on the grounds that it brings delay; that it is not always effective in detecting misconduct and malpractice; that the selection of reviewers may introduce bias into the system; that the judgements made are subjective and inconsistent; that it tends toward conservatism and stifles innovation; and that it disadvantages interdisciplinary research. There are also concerns about the costs of the system (which are largely hidden, since peer reviewers are generally not paid for their time); that the burdens being placed on the reviewing community may become unsustainable as the volumes of research activity and publications continue to increase;

and that there is a need for more training of reviewers to ensure greater consistency.

Such criticisms and concerns have not, however, undermined the value attached to judgements made by respected peers in any system of assurance and assessment. Quantitative measures are generally seen as supplements, not substitutes for such judgements. For further information on the peer review system, the challenges it faces, and how it might change, see the RIN booklet Peer Review: a guide for researchers.1

1 Research Information Network (2010) Peer Review: a guide for researchers

5. Assurance and assessment of research projects and programmes The arrangements for assuring and assessing the quality of research projects varies in some particulars between different fields and disciplines, but may conveniently be divided into six main


(i) programme and project proposals (ii) monitoring and oversight during projects (iii) sharing early findings with colleagues (iv) formal publication (v) data sharing (vi) post-publication assessment and review 5.1 Programme and project proposals Research institutions and funders use a number of checks in seeking to ensure that only highquality research is funded. Many institutions implement mechanisms – through departmental management structures or through institutional bodies such as research offices – to monitor project proposals, thereby ensuring only those that pass quality thresholds are supported by the institution or submitted to external bodies for support.

When external support is sought, Research Councils, along with most research charities and institutions, evaluate project proposals using various peer review mechanisms. Panels of experts are drawn from academia as well as other parts of the public, commercial and voluntary sectors, with members selected to give the panel the necessary range of expertise to assess the proposals they are asked to consider.

Assessment and evaluation may begin even before a specific proposal has been framed. In order to address certain criticisms about the ‘conservatism’ of peer review, some funders are experimenting with new approaches to encourage innovative and cross-disciplinary proposals. The Engineering and Physical Sciences and the Natural Environment Research Councils (EPSRC and NERC) are using ‘sandpits’: 3-5 day workshops, overseen by a director and other facilitators, where researchers from a wide range of disciplines discuss issues and ideas, and then develop, test and refine proposals which may be funded at the end of the process.2 In a more conventional process, researchers prepare proposals with their colleagues. These are then assessed primarily on their scholarly quality, covering such issues as intellectual rigour, creativity, methodology and so on. But many related quality criteria may come into play, including the fit between the proposal and the funder’s priorities, the track record of the applicants, and value for money.

2 See the ESPRC http://www.epsrc.ac.uk/funding/grants/network/ideas/Pages/moresandpits.aspx and NERC http://www.nerc.

ac.uk/funding/assessment websites for more information on sandpits.

Fit with a funder’s priorities may be determined in relation to published strategies which are reviewed from time to time, and for public funding agencies agreed with government. Such strategies and priorities may increasingly include achieving economic, social or cultural impact beyond the research community. Research Councils are collectively committed to increasing the impact of the research they fund.

Pages:   || 2 | 3 | 4 |

Similar works:

«PROCEEDINGS 30 REGULATED INDUSTRIES THE ‘GOVERNANCE CONTRACT’ Doug Andrew Allan Asher Anthony Carey George Dallas David Edmonds Cosmo Graham Nigel Hawkins John Roberts Peter Vass RSM Robson Rhodes REGULATED INDUSTRIES THE ‘GOVERNANCE CONTRACT’ Proceedings of a CRI Conference Chaired by Professor Ralph Turvey held on 16th April 2002 at The National Liberal Club, London ~ supported by RSM Robson Rhodes ~ Edited by Peter Vass Desktop published by Jan Marchant © The University of Bath All...»

«CONGRATULATIONS: 2014 WINNER CIRCLE June 26, 2014, Chicago. Global M&A Network honored the winners from the restructuring, distressed M&A and turnaround communities in fifty awards categories, encompassing variety of restructuring styles and transaction sizes at the annual awards gala ceremony held on the evening of June 24, 2014 at the Standard Club of Chicago. The master of ceremonies, Ms. Lisa Lockwood, the reinvention expert, author and motivational speaker made the evening gala memorable...»

«Metal Forming and Stamping Presses Improve productivity and safety in forming and stamping applications This paper takes a closer look at the automation and information solutions available for metal forming and stamping press systems. 2 | Metal Forming and Stamping Machine Table of Contents Executive Summary Introduction Challenges Solutions & Benefits Conclusions Customer Case Study Resources 3 | Metal Forming and Stamping Machine Executive Summary Press line production runs today are smaller...»

«Curriculum Vitae MOIRA K. AMADO-McCOY PO Box 337 El Prado, NM 87529 251.643.5767 • • moiraamado@gmail.com http://www.moiraamado.com EDUCATION Ph.D. English Miami University, Oxford, OH (August 2003) Primary Area: Rhetoric and Composition Secondary Area: Civic Discourses and Argumentation Kate Ronald, Director Dissertation Title: Rhetoric and Revolution: An Apology for Modesty, Responsibility and Pragmatism in a Post-Fascist Era M.A. English New Mexico State University (July 1999) Primary...»

«FEELING SMALL IN ANOTHER PERSON’S COUNTRY The situation of Burmese migrant children in Mae Sot Thailand “Parents want to see their child ride on an elephant. No one wants to see their child kicked by an elephant” 15 yr old boy attending Migrant High School, Mae Sot Committee for Promotion and Protection of Child Rights (Burma) FEBRUARY 2009 CPPCR 1 Feeling Small in Another Person’s Country: The situation of Burmese migrant children in Mae Sot, Thailand Written and Published by: The...»

«Vol.9(1) March 2007 Peer Reviewed Article Nine drivers of knowledge transfer between universities and industry R&D partners in South Africa A. van Zyl Institute for Technological Innovation University of Pretoria anthea.vanzyl@up.ac.za J. Amadi-Echendu Department of Engineering and Technology Management University of Pretoria joe.amadi-echendu@up.ac.za T.J.D. Bothma Department of Information Science University of Pretoria theo.bothma@up.ac.za This article presents part of the findings of the...»

«TERMINAL POSITION ASSURANCEReliability An Added Layer of Connector APPLIANCES /// TERMINAL POSITION ASSURANCE TERMINAL POSITION ASSURANCEReliability An Added Layer of Connector ADVANCED CONNECTOR DESIGNS INCORPORATE FEATURES DESIGNED TO REDUCE CONTACT BACK-OUT Low-cost “soft shell” connectors featuring crimp/snap technology have been used to address interconnection requirements since the 1960s. Although they provide a cost-effective option for a wide range of everyday applications –...»

«Journal of Forensic & Investigative Accounting Vol. 5, Issue 2, July December, 2013 DOES FINANCIAL REPORTING FRAUD RECOGNIZE BORDERS? EVIDENCE FROM BANK FRAUD IN IRAN Zabihollah Rezaee* Gholamhossein Davani INTRODUCTION Reliable and high quality financial information is the lifeblood of the capital markets and that quality can be adversely affected by the existence and persistence of financial reporting fraud (hereafter, FRF). The persistence of FRF continues to be of significant concern for...»

«[Friday, 12 March 2004] 939 PRIMARY SCHOOL FETE 125. Hon ALAN CADBY to the parliamentary secretary representing the Minister for Education and Training: (1) Is the minister aware of a government primary school advertising a beer garden at its school fete? (2) Are schools required to obtain authority for such activities as a beer garden; and, if yes, from whom? (3) Does the minister consider a beer garden a suitable activity for a primary school fete? (4) Will the minister explain his reasoning...»

«Language in Society 28, 203–223. Printed in the United States of America “Why be normal?”: Language and identity practices in a community of nerd girls M A RY B U C H O LT Z Department of English Texas A&M University College Station, TX 77843–4227 bucholtz@tamu.edu ABSTRACT The introduction of practice theory into sociolinguistics is an important recent development in the field. The community of practice provides a useful alternative to the speech-community model, which has limitations...»

«GJJRMetSbookletsV4 Observing the weather Some advice if you are an amateur, or at a school, and thinking of making weather measurements Geoff Jenkins Royal Meteorological Society February 2010 Contents Introduction Temperature Maximum and minimum temperature Digital thermometers Where outside should a thermometer be put? Soil temperature Humidity Rain and snow Digital rain gauges Snow Pressure How can I calibrate my barometer? Wind Simple anemometers Cup anemometers Where should I try and...»

«Mathematics in Southeast Asia: Country Reports Commission for Developing Countries, International Mathematical Union Mathematics in Southeast Asia: Challenges and Opportunities Country Reports Contents Country report 1: Brunei by Prof Wong Khoon Yoong Country report 2: Cambodia by Prof Chan Roath Country report 3: Cambodia and Laos by Prof Michel Jambu Country report 4: Indonesia (1) by Prof Kiki Ariyanti Sugeng Country report 5: Indonesia (2) by Prof Budi Nurani Ruchjana and Prof Widodo...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.