«Taking Targets to Task Revisited: How Indicators of Progress on Access to Education can Mislead Keith M Lewin CREATE PATHWAYS TO ACCESS Research ...»
Consortium for Research on
Transitions and Equity
Taking Targets to Task Revisited:
How Indicators of Progress on Access to
Education can Mislead
Keith M Lewin
CREATE PATHWAYS TO ACCESS
Research Monograph No. 54
University of Sussex
Centre for International Education
The Consortium for Educational Access, Transitions and Equity (CREATE) is a Research Programme
Consortium supported by the UK Department for International Development (DFID). Its purpose is to undertake research designed to improve access to basic education in developing countries. It seeks to achieve this through generating new knowledge and encouraging its application through effective communication and dissemination to national and international development agencies, national governments, education and development professionals, non-government organisations and other interested stakeholders.
Access to basic education lies at the heart of development. Lack of educational access, and securely acquired knowledge and skill, is both a part of the definition of poverty, and a means for its diminution. Sustained access to meaningful learning that has value is critical to long term improvements in productivity, the reduction of inter-generational cycles of poverty, demographic transition, preventive health care, the empowerment of women, and reductions in inequality.
The CREATE partners CREATE is developing its research collaboratively with partners in Sub-Saharan Africa and South Asia. The
lead partner of CREATE is the Centre for International Education at the University of Sussex. The partners are:
The Centre for International Education, University of Sussex: Professor Keith M Lewin (Director) The Institute of Education and Development, BRAC University, Dhaka, Bangladesh: Dr Manzoor Ahmed The National University of Educational Planning and Administration, Delhi, India: Professor R Govinda The Education Policy Unit, University of the Witwatersrand, South Africa: Dr Shireen Motala The Universities of Education at Winneba and Cape Coast, Ghana: Professor Jerome Djangmah, Professor Joseph Ghartey Ampiah The Institute of Education, University of London: Professor Angela W Little Disclaimer The research on which this paper is based was commissioned by the Consortium for Research on Educational Access, Transitions and Equity (CREATE http://www.create-rpc.org). CREATE is funded by the UK Department for International Development (DFID) for the benefit of developing countries and is coordinated from the Centre for International Education, University of Sussex. The views expressed are those of the author(s) and not necessarily those of DFID, the University of Sussex, or the CREATE Team. Authors are responsible for ensuring that any content cited is appropriately referenced and acknowledged, and that copyright laws are respected. CREATE papers are peer reviewed and approved according to academic conventions.
Permission will be granted to reproduce research monographs on request to the Director of CREATE providing there is no commercial benefit. Responsibility for the content of the final publication remains with authors and the relevant Partner Institutions.
Copyright © CREATE 2011 ISBN: 0-901881-61-9
Address for correspondence:
CREATE, Centre for International Education, Department of Education School of Education & Social Work Essex House, University of Sussex, Falmer BN1 9QQ United Kingdom Tel: + 44 (0) 1273 877984 Fax: + 44 (0) 1273 877534 Author email: firstname.lastname@example.org Website: http://www.create-rpc.org Email email@example.com Please contact CREATE using the details above if you require a hard copy of this publication.
Taking Targets to Task Revisited:
How Indicators of Progress on Access to Education can Mislead
1.1 The Policy Context - WCEFA, WEF, and the MDGs
2. Targets and Indicators in Practice
2.1 Gross and Net Enrolment Rates
2.2 The Gender Parity Index
2.3 The Education for All Development Index (EDI)
3. Concluding Remarks
List of Tables
Table 1: Enrolment Rates and GPI
Enrolments, Enrolment Rates and GPI where the Population has Sex parity.................. 14 Table 3: Enrolments, Enrolment Rates and GPI where the Population has 15% Fewer Girls
Table 4: EDI and Uncertainty 2% Average EDI Countries
Table 5: EDI and Uncertainty 5% Low EDI Countries
List of Figures
Figure 1: Enrolments by Grade in Two Countries with similar GERs
Figure 2: Enrolments by Age: Boys and Girls
Figure 3: Values for the EDI and for its Component Parts
Figure 4: Relative Improvement in EDI 1999-2006
Figure 5: Improvement in EDI by EDI Value 1999-2006
iv Acknowledgements The development and discussion of indicators of EFA has a history going back to the Jomtien World Conference on Education for All. This paper has benefited from insights from many academics and development partners who have contributed to the debates about how to assess progress. It has also been informed by opportunities to work closely with development partners and national governments on planning exercises that link resources to performance.
Thanks are therefore due to the many colleagues in development partners and national governments who have shared insights that have contributed to this paper. I am also grateful to faculty and students associated with the Centre for International Education at the University of Sussex who have contributed observations and ideas.
Ten years after governments, bi-lateral development agencies, and United Nations institutions met at in Dakar in to set out a plan to achieve “Education for All” there is still much work to be done. How we measure progress and identify areas for further research and policy depends heavily on the indicators we use. Development agencies use indicators to release or retain funds for EFA. This important monograph selects some of the key indicators used for EFA and offers an analysis of their strengths and weaknesses.
The monograph discusses measurement, application and ambiguities with Gross and Net Enrolment Rates, Gender Parity Indices, and The Education for All Development Index, used by UNESCO and the Global Monitoring Report. These commonly used and cited, influential measures all provide us with valuable insights but also have the potential, as Lewin reminds us, to mislead. Gross and Net Enrolment Rates for example, assess levels of enrolment, but their changing values can provide misleading signals to policy makers since they can go down as well as up when things are improving. GPIs may also have ambiguous directionality.
And the EDI is an aggregate indicator with several component parts which may change in different directions. Unless indicators such as these are properly understood, data can be misunderstood, or in the worst case wilfully misused with the result that policies for EFA can end up being ineffective or poorly targeted.
CREATE has identified widespread problems of over age in grade pupils in schools in many developing countries. It also notes the issues that surround imbalances in the numbers of boys and girls in the population in some countries which be concealed by some uses of GPIs. For this and other reasons better indicators are needed along with more widely understood aspects of the signifiers of progress towards clear goals. This monograph provides essential reading for policy makers, planners, scholars and analysts of education in developing countries who wish to understand critical issues to consider when examining educational targets and also about what is and is not changing as a result of EFA programmes..
Kwame AkyeampongCentre for International EducationUniversity of Sussex
vi Summary Governments, bi-lateral development agencies, and United Nations institutions met at Jomtien in 1990 and Dakar in 2000 to agree support to achieve “Education for All” (EFA).
EFA is now the umbrella that embraces much of the aid for educational development in poor countries and its influence in education policy in many developing countries is substantial. It identifies goals and targets and translates these into indicators which are used to evaluate progress and influence flows of resources. The search for evidence based policy depends on measures of performance that can link cause and effect and that represent real gains in progress towards desired outcomes. However, the indicators widely used for access to education have serious problems. This paper1 selects some of the key indicators used for EFA and offers an analysis of their strengths and weaknesses.
Gross and Net Enrolment Rates (GERs and NERs) are widely used to assess levels of enrolment, but their changing values can provide misleading signals to policy makers. Grade specific enrolment rates may be better. Completion and survival rates for schooling provide improved indications of educational inclusion and exclusion. On schedule graduation rates are better than simple completion rates but are silent on issues of quality and achievement.
Gender Parity Indices (GPI) foreground differences in participation related to children’s sex.
However, most of these indices aggregate and conceal underlying patterns of participation (by age, location, household income) which give insight into causes and opportunities to ameliorate disadvantage. The GPI may also make invisible seriously unequal numbers of boys and girls in populations. The Education for All Development Index (EDI) used by UNESCO and the Global Monitoring Report (GMR) suffers from being a highly aggregated composite index which is difficult to interpret and of limited use since changes over time are often within its margins of error. Simpler indicators may be more useful than composite ones.
The choice of indicators is important. The process of setting and getting targets defined by indicators remains valuable but needs constant interrogation. The homogenising effects of global approaches to EFA has to be mediated by recognition that education policy in practice is national or sub-national at the level of delivering services. Global goals do not always have analogues at country level and conversely national aspirations and realities cannot always be mapped onto global goals. Improving target setting and devising better indicators which allow progress to be assessed and understood requires more nuanced understanding of what indicators do and do not measure, and in appreciating the virtues and vices of setting targets.
1 An abridged, amended and shortened paper based on this Research Monograph has been submitted to the Journal of Educational Policy.
1. Introduction This paper reviews how targets and indicators have shaped Education for All. The World Conference on Education for All (WCEFA) in 1990, the World Education Forum (WEF) in 2000 and the Millennium Development Goals (MDGs) in 2000 set a policy agenda that lies at the heart of planning for improved educational access in low income countries which has defined the focus of much external development assistance. Central to this process have been targets and indicators for a wide range of outcomes e.g. universal enrolment and completion rates for primary schooling, gender parity in educational participation, reductions in illiteracy.
These have been developed at international, regional and national levels. They provide one basis for the rules that mediate relationships between development partners and recipients of development assistance. Thus it has been very common for the financial support for EFA from development partners to be linked to progress towards targets judged through periodic Education Sector Reviews and annual performance reports. Where external assistance is a substantial proportion of government financing, it is easy to see how exogenously determined targets can begin to shape domestic policy. National policy dialogue is likely to be conditioned, and may even be determined by, the targets and indicators of progress that are agreed. Targets thus not only signify desired outcomes but also have practical significance not least in influencing the allocation of resources. But targets need indicators if they are to have operational significance. These indicators are often difficult to devise and easy measure with precision, and are often ambiguous to interpret.
1.1 The Policy Context - WCEFA, WEF, and the MDGs
“Education for All” was unveiled at the World Conference on Education for All (WCEFA) in 1990 at Jomtien, Thailand. This promised a new vision for the 1990s, characterised by five core components – a focus on basic education, an expanded scope for investment in basic education, the mobilisation of new resources, mobilisation of alliances with a range of partners, and an improved understanding of the learning environment and socio-economic context of education and development (Inter Agency Commission, 1990a:30). The final report of WCEFA included a special section on goals and targets. The overarching goal was to “meet the basic learning needs of all children, youth and adults” supported by intermediate goals designed to specify expected attainments and outcomes, focus on excluded groups, and identify observable and measurable indicators of progress (Inter Agency Commission, 1990b:52).