Content uploaded by Jo-Anne Baird
Author content
All content in this area was uploaded by Jo-Anne Baird on Jun 30, 2016
Content may be subject to copyright.
A preview of the PDF is not available
The meaning of curriculum-related examination standards in Scotland and England: a home–international comparison
Abstract
The ways in which examination standards are conceptualised and operationalised differently across nations has not been given sufficient attention. The international literature on standard-setting has been dominated by the psychometrics tradition. Broader conceptualisations of examination standards have been discussed in the literature in England, which has curriculum-related examinations at the end of schooling. There has, however, been little analysis of conceptualisations of examination standards in Scotland. Different education systems and examinations operate in Scotland and England, and the stated value positions and processes relating to examination standards differ markedly. This paper critically examines policy positions on assessment standards in Scotland and England through the lens of recent theories of standard-setting. By analysing public statements on standards, the paper illuminates similarities and differences in conceptual bases and operational approaches, and examines the effects of these on outcomes for candidates. We conclude that both systems are operationalising attainment-referencing, but with different processes in Scotland and England and these practices do not fit within previous examination standards classifications. As such, the paper moves examination standards theory forward by concluding that there is at least one superordinate definitional category that draws upon more than one definitional stance.
... These organisations compete for schools' examination entries for qualifications in a wide range of academic and vocational subjects, under the surveillance of the Office of Qualifications and Examinations Regulation (Ofqual), a non-ministerial government department. In an entrenched class system, educational attainment for improved social mobility has been promoted by a broad spectrum of educational policymakers (Baird & Gray, 2016). While the General Certificate of Secondary Education (GCSE) is conceived as an "inclusive" assessment, there are criticisms that participation, accessibility, and appropriateness have been prioritised over breadth of curriculum, especially for students with special educational needs and disabilities (e.g. ...
In the midst of a range of unprecedented global events, geopolitical developments, and health, climate, socio-economic, and humanitarian crises, which have challenged assumptions about the nature and purpose of education, this chapter frames the book as an attempt to engage with a window of opportunity for shaping educational trajectories for the future. The purpose of the chapter is to introduce the principal themes of the book, and the research on which this work is based, through a nuanced and meticulous unpacking of the paradoxes and dilemmas arising between the assessment and inclusion agendas in education. The chapter argues that an analysis of these two agendas offers insights into how assessment and inclusion constitute and epitomise internal and external agendas as well as the multiplicity of dimensions associated with education. Finally, the chapter briefly introduces the empirical case contexts analysed in the book and presents the chapter structure of the book.
... With this historical background, it is easy to understand the rationale for the SRR approach of the HKDSE, focussing on standard maintenance across years and intersubject comparability which have been part of the HKCEE and HKALE legacy. This reference to previous standards is, of course, a common practice when setting new standards, and special care is needed to ensure that the approach is politically acceptable and robust to challenge (Baird & Gray, 2016). As compared to the previous so-called 'norm-referenced system', more expert judgement is incorporated in the grading procedure for validation and for enhancing stakeholder confidence in the newly-established HKDSE standards. ...
In alignment with the New Academic Structure, the Hong Kong Diploma of Secondary Education Examination (HKDSE) was launched in 2012 to replace the former Hong Kong Certificate of Education Examination (HKCEE) as certification for completion of secondary education, and the Hong Kong Advanced Level Examination (HKALE) as the main credentials for university admission in Hong Kong. Standards-referenced reporting is adopted for the HKDSE with the objective of reporting candidates’ results against a set of prescribed levels of achievement based on typical performances at those levels. Clearly defined standards facilitate learning and teaching as well as enable users of the qualification, including tertiary institutions and employers, to set appropriate entrance/job requirements. The standards are set and maintained by expert judgement supported by psychometric data to ensure fairness and consistency of standards across subjects and across cohorts. Systemic and implementation issues and their resolutions are discussed in the context of the education reform in Hong Kong.
This article conceptualises the relationship between exam board insider research and the policy-making context in which they operate. Exam board researchers are constrained by commercial and political interests in disclosing their knowledge. and face pressures in disseminating research, butalso find themselves working in contexts where calls to ‘evidence-based policy-making’ are ubiquitous. This can deprofessionalise and disenfranchise the researcher.. This article will depict the context faced by exam board researchers attempting to influence policy before portraying possible responses, evaluating how these can be applied to exam board research, with reference to research on standard-setting. The article will build on a conceptualisation of successful exam board insider research as the creation of Habermasian ‘communicative spaces’, applying lessons from research–policy interface literature to that conceptualisation. Inapplying those lessons, the article will suggest possible solutions to the problems faced by that group in their attempts to influence policymakers.
Psychometrics is a scientific discipline concerned with the construction of measurement models for psychological data. In these models, a theoretical construct (e.g., intelligence) is systematically coordinated with observables (e.g., IQ scores). This is often done through latent variable models, which represent the construct of interest as a latent variable that acts as the common determinant of a set of test scores. Important psychometric questions include (1) how much information about the latent variable is contained in the data (measurement precision), (2) whether the test scores indeed measure the intended construct (validity), and (3) to what extent the test scores function in the same way in different groups (measurement invariance). Recent developments have focused on extending the basic latent variable model for more complex research designs and on implementing psychometric models in freely available software.
Politicians and civil servants are very much involved in examination developments in many countries. Policy development and implementation is notoriously difficult to unpick in terms of decision-making, roles and responsibilities. Nonetheless, three systemic examination failures are used to illustrate the problems caused by the policy context - in Scotland 2000, New Zealand 2004 and England 2008. Taking these cases and the literature together, it is argued that features of the policy environment conspire to generate latent errors: 1) evolving policy and competing perspectives; 2) lack of role clarity and diffusion of responsibility and 3) timeframe slippage. Human error theory indicates that to try to reduce errors we must understand their fundamental causes and that these usually run deeper than the first stories that are told. Understanding the full reasons for particular systemic examination errors is difficult because politics is slippery, and many perspectives have to be sifted.
This thesis reports a study of the processes by which public examination grades are awarded. Following a review of the purposes of public examinations, new theoretical analyses are given of the issues of norm and criterion-referencing, the nature of public examination standards, the problems of defining comparable standards across widely disparate assessment domains and the more technical matters of aggregating marks and examiners' judgements. The main empirical work investigated conventional public examination grade awarding using a combination of participant observation of examiners making judgements and statistical analysis of examination outcomes. Two additional experiments are also reported; one on grade, rather than mark, aggregation methods and one on the use of strong criterionreferencing to award grades. The main conclusions of the study are as follows: 1. Examination standards are social constructs created by special groups of judges, known as awarders, who are empowered, through the examining boards as governmentregulated social institutions, to evaluate the quality of students' attainment on behalf of society as a whole. 2. As a result, examination standards can be defined only in terms of human evaluative judgements and must be set initially on the basis of such judgements. 3. The process by which awarders judge candidates' work is one in which direct and immediate evaluations are formed and revised as the awarder reads through the work. At the conscious level, it is not a computational process and it cannot, therefore, be mechanised by the use of high-level rule-bound procedures and explicit criteria. 4. Awarders' judgements of candidates' work are inadequate, by themselves, as a basis for maintaining comparable standards in successive examinations on the same syllabus. The reasons for this are related both to the social psychology of awarding meetings and to the fundamental nature of awarders' judgements. 5. The use of statistical data alongside awarders' judgements greatly improves the maintenance of standards and research should be carried out into the feasibility of using solely statistical approaches to maintain standards in successive examinations on the same syllabus. 6. A broadening of the range of interest groups explicitly represented among judges initially setting standards should also be considered.
Scotland, in common with many countries internationally, has been learning how to align ideas from research with policy and practice. This article considers what Scotland learned from large-scale evaluations of its Assessment is for Learning (AifL) programme and the extent to which this evidence was used to inform future learning within the national programme. More recently, the policy focus in Scotland has shifted to the creation of a new curriculum, Curriculum for Excellence, subsuming AifL. Merging curriculum and assessment innovations brought new challenges in the alignment of curriculum, pedagogy and assessment. Drawing on a Scottish Government-funded research project, Assessment at Transition, designed to identify and explore emerging gaps between practice in schools and local authorities and national curriculum and assessment policy aspirations, the article argues that assessment is learning and explores how formative approaches to evaluation at a national level might be used to prevent countries repeating past mistakes.
"Construct validation was introduced in order to specify types of research required in developing tests for which the conventional views on validation are inappropriate. Personality tests, and some tests of ability, are interpreted in terms of attributes for which there is no adequate criterion. This paper indicates what sorts of evidence can substantiate such an interpretation, and how such evidence is to be interpreted." 60 references. (PsycINFO Database Record (c) 2006 APA, all rights reserved).
Scotland, in common with many other countries internationally, has paid considerable attention to the development of assessment for learning. Currently, schools in Scotland are engaged in a major programme of curriculum and assessment reform, entitled Curriculum for Excellence. As part of the reform process, there is concern amongst practitioners, researchers and policy-makers about ‘consistency’ and ‘standards’. In this article, we explore international issues of consistency and standards through a Scottish lens. In particular, we focus on how standards, and the idea of consistency of judgements and standards, are understood and applied in practice. We draw on international research and policy, and reflect on how that evidence relates to the findings from a recent government-funded research project in Scotland, ‘Assessment at Transition’. We conclude by identifying what the different communities need to do to help build an integrated, assessment-capable system that will be sustainable in the longer term.
This article examines variations among England, Wales, and Scotland in the association between social origin and educational attainment and the role that different national educational policies may have played in shaping these variations. The findings show that country variation in the association between origins and attainment was mostly or entirely due to variations in overall levels of attainment. Moreover, inequality was the highest where the proportions attaining a particular threshold were the highest—upper secondary school or higher in Scotland. The authors propose a refinement of Raftery and Hout's theory of maximally maintained inequality that takes into account that the trajectory of inequality is not linear: inequality can widen in the initial phase of expanding opportunity, en route to an eventual contraction, because the most advantaged groups are the first to exploit any new opportunities that policy changes offer. The results show that country differences in educational policy have not yielded different changes over time in the association between origin and educational attainment.
Public examination results are used in a variety of ways and the ways in which they are used dictate the demands that society makes of them. Unfortunately, some of the uses to which British examination results are currently being put make unrealistic demands. Government, in particular, deems it necessary to measure the progress of ‘educational standards’ across decades in time and assumes that this can be achieved to some extent with reference to pass rates from public examinations: hence, it demands that precisely the same examining standards must be applied from one year to the next. Recently it has been suggested that this demand is not being met and, as a consequence, changes in pass rates may give us a misleading picture of changing ‘educational standards’. Unfortunately, this criticism is ill‐founded and misrepresents the nature of examining standards, which, if they are to be of any use at all, must be dynamic and relative to specific moments in time. Thus, the notion of ‘applying the same standard’ becomes more and more meaningless the further apart the comparison years. While, to some, this may seem shocking, the triviality of the conclusion is apparent when the following are borne in mind: (a) the attempt to measure ‘educational standards’ over time is not feasible anyway; (b) the primary selective function of examination results is not affected by the application of dynamic examining standards.