«Full Report Professor Steven Higgins, ZhiMin Xiao and Maria Katsipataki School of Education, Durham University November 2012 1 Table of Contents ...»
The Impact of Digital Technology on Learning:
A Summary for the Education Endowment Foundation
Professor Steven Higgins,
ZhiMin Xiao and Maria Katsipataki
School of Education, Durham University
Table of Contents
Summary of key points
Approach and methods
Global trends: a move towards increasing scepticism?
Identifying themes in the findings from meta-analysis
Conclusions and recommendations
Appendix 1: Some contemporary myths about digital technologies
Appendix 2: Meta-analyses published between 2000 and 2012
Appendix 3: Meta-analyses published between 1990 and 1999
Appendix 4: Bibliography of Meta-analyses
2 Interpreting the evidence from meta-analysis for the impact of digital technology on learning Overview The aim of this review is to present a synthesis of the evidence from meta-analysis about the impact of the use of digital technology in schools on children’s attainment, or more widely the impact of digital technology on academic achievement. It is divided up into three main sections. The first sets out an overview of the wider research into the impact of technology on learning to set the context and the rationale for the value of this information. The next section reviews the evidence from meta-analysis and other quantitative syntheses of research into the impact of digital technology. A further section looks at trends in the use of digital technology and learning in the UK and internationally, to provide further context for the recommendations which follow.
The purpose of this review is to identify implications for future investment in the use of digital technology for learning in schools. Digital technologies are now embedded in our society.
Focus has shifted from whether or not to use them in teaching and learning, to understanding which technologies can be used for what specific educational purposes and then to investigate how best they can be used and embedded across the range of educational contexts in schools.
Summary of key points Overall, the research evidence over the last forty years about the impact of digital technologies on learning consistently identifies positive benefits. The increasing variety of digital technologies and the diversity of contexts and settings in which the research has been conducted, combined with the challenges in synthesising evidence from different methodologies, makes it difficult to identify clear and specific implications for educational practice in schools.
Studies linking the provision and use of technology with attainment tend to find consistent but small positive associations with educational outcomes. However a causal link cannot be inferred from this kind of research. It seems probable that more effective schools and teachers are more likely to use digital technologies more effectively than other schools. We need to know more about where and how it is used to greatest effect, then investigate to see if this information can be used to help improve learning in other contexts. We do not know if it is the use of technology that is making the difference.
Research findings from experimental and quasi-experimental designs – which have been combined in meta-analyses – indicate that technology-based interventions tend to produce just slightly lower levels of improvement when compared with other researched interventions and approaches (such as peer tutoring or those which provide effect feedback to learners).
The range of impact identified in these studies suggests that it is not whether technology is used (or not) which makes the difference, but how well the technology is used to support teaching and learning. There is no doubt that technology engages and motivates young people. However this benefit is only an advantage for learning if the activity is effectively aligned with what is to be learned. It is therefore the pedagogy of the application of technology in the classroom which is important: the how rather than the what. This is the crucial lesson emerging from the research.
Taken together, the correlational and experimental evidence does not offer a convincing case for the general impact of digital technology on learning outcomes. This is not to say that it is not worth investing in using technology to improve learning. But it should encourage us to be cautious in the face of technological solutions to educational challenges. Careful thought is needed to use technology to best effect.
There is a recurrent and specific challenge in understanding and applying research evidence as it takes time for robust evidence to emerge in education, and the rapid pace of change of technology makes this difficult to achieve.
3 The challenge is to ensure that technology is used to enable, or make more efficient, effective
teaching and learning practices. With this in mind the findings from the synthesis of the metaanalyses indicate the following overall trends:
Collaborative use of technology (in pairs or small groups) is usually more effective than individual use, though some pupils, especially younger children, may need guidance in how to collaborate effectively and responsibly.
Technology can be as powerful as a short but focused intervention to improve learning, particularly when there is regular and frequent use (about three times a week) over the course of about a term (5 - 10 weeks). Sustained use over a longer period is usually less effective at improving this kind of boost to attainment.
Remedial and tutorial use of technology can be particularly practical for lower attaining pupils, those with special educational needs or those from disadvantaged backgrounds in providing intensive support to enable them to catch up with their peers.
In researched interventions, technology is best used as a supplement to normal teaching rather than as a replacement for it. This suggests some caution in the way in which technology is adopted or embedded in schools.
Tested gains in attainment tend to be greater in mathematics and science (compared with literacy for example) though this is also a more general finding in meta-analysis and may be at least partly an artefact of the measurement process. In literacy the impact tends to be greater in writing interventions compared with reading or spelling.
At least a full day’s training or on-going professional inquiry-based approaches to support the introduction of new technology appear the most successful. The implication is that such support should go beyond the teaching of skills in technology and focus on the successful pedagogical use of technology to support teaching and learning aims.
Overall, the over-arching implication is that the technology is solely a catalyst for change. The question is how can technology can bring about improvement and make teaching and learning practices more efficient or effective. Focusing on the change (and the process of change), in terms of learning is essential in supporting effective use.
1. The rationale for the impact of digital technology on teaching and learning needs to be
Will learners work more efficiently, more effectively, more intensively? Will the technology help them to learn for longer, in more depth, more productively? Or will the teacher be able to support learners more efficiently or more effectively?
2. The role of technology in learning should be identified:
Will it help learners gain access to learning content, to teachers or to peers? Will the technology itself provide feedback or will it support more effective feedback from others, or better self-management by learners themselves?
3. Technology should support collaboration and effective interaction for learning:
The use of computer and digital technologies is usually more productive when it supports collaboration and interaction, particularly collaborative use by learners or when teachers use it to support discussion, interaction and feedback.
4. Teachers and/or learners should be supported in developing their use of digital technology to ensure it improves learning.
5. Identify what learners and teachers will stop doing:
The use of digital technology is usually more successful as a supplement rather than as a replacement for usual teaching. Technology is not introduced into a vacuum. It is therefore important to identify carefully what it will replace or how the technology activities will be additional to what learners would normally experience.
Approach and methods This review summarises the research evidence contained in meta-analyses to identify patterns of impact in the accumulating research about the effects of technology on learning, and to identify the extent of the possible impact of technology on learning. A systematic search revealed 48 studies which synthesised primary research studies of the impact of technology on the attainment of school age learners (5-18 year olds). Whilst this presents only a partial and retrospective view of such impact, it is the only approach to allow a systematic comparison of a large number of studies with an estimate of the extent of the effects on learning.
The role of technology in education has been an important question since the potential of computer technology to transform Skinner’s teaching machines was recognised in the 1960s.
It remains an important issue today with debates about the impact of technology on our society, the implications of quick and easy online access to information for knowledge and learning and the effect of technology on young people’s social, emotional and physical development frequently in the news. It is therefore important to take stock of what we know about the impact of digital technology on education from what we have learned over the last fifty years. Appendix 1 sets out a number of these issues in terms of some contemporary myths about the effects of technology.
The main approach used to evaluate the impact of technology on teaching and learning in schools has been where pupils’ attainment across a range of tested curriculum outcomes has been correlated with the quantity or quality of technology which was available or which they experienced in their institutions (see, for example, Watson, 1993; Wenglinsky, 1998; Weaver, 2000; BECTA 2003). In the USA, only a small relationship between computer use in the school curriculum and improvement in pupils’ test scores was found in a longitudinal study (Weaver, 2000). At this very general level, computer use makes very little difference to pupils’ achievement. In the UK, the Impact 2 study (Harrison et al. 2004) identified statistically significant findings positively associating higher levels of ICT use with school achievement at each Key Stage, and in English, Maths, Science, Modern Foreign Languages and Design Technology. An association between high ICT use and higher pupil attainment in primary schools was also reported in an earlier Teacher Training Agency study (Moseley et al. 1999, p 82) though the interpretation by the research team was that more effective teachers (and more effective schools) tended to use more innovative approaches, or chose to use the ICT resources that they had more appropriately, rather than that the technology itself was the cause of the differences in pupil performance.
This connection between technology and learning is found fairly consistently however, and other studies have indicated a stronger association. The ICT Test Bed evaluation identified a link between high levels of ICT use and improved school performance. The rate of improvement was faster in ICT Test Bed Local Authorities (LAs) than in equivalent comparator LAs in KS2 English (Somekh et al. 2007). However, what this association shows is that, on average, schools with higher than average levels of ICT provision also have pupils who perform slightly higher than average. The causal link could be quite the reverse, with high performing schools more likely to be better equipped or more prepared to invest in technology or more motivated to bring about improvement. Fuchs and Woessmann’s (2004) analysis of this link between provision and performance based on the Programme for
International Student Assessment (PISA) data supports this interpretation:
“the initial positive pattern on computer availability at school simply reflects that schools with better computer availability also feature other positive school characteristics. Once these are controlled for, computer availability at school is not related to pupil performance in math and reading.” (p. 13) The Organisation for Economic Co-operation and Development’s (OECD) more detailed analysis of Programme for International Student Assessment (PISA) data indicates a complex picture of association between pupil performance, their access to computers at home and at school together with frequency of use which varies from country to country (OECD 2006, p 51-66). Though as a note of caution the research found that pupils who used computers most widely tended to perform slightly worse on average than those with moderate usage. Overall the analysis suggests that the linkage may not be a simple causal one, nor necessarily a simple linear association. There may be a limit to the amount of technology which is beneficial.
In findings from experimental and quasi-experimental research studies, where gains in knowledge or understanding for groups of pupils using ICT has been compared with gains for groups learning the same content without technology, results again tend to show positive 6 benefits for ICT. These have been reviewed using a narrative approach with consistently positive conclusions (e.g. Parr & Fung, 2000; Andrews et al. 2002; Cox et al. 2004; Hartley,
2007) as well as through quantitative synthesis using meta-analysis (see Appendix 2 (2000and 3 (1990-1999) for more details about these studies, with a full bibliographical list of meta-analyses of the impact of digital technologies on learning in Appendix 4). Again these reviews typically conclude that technology has a positive and measurable effect on learning.