This article has been authored by Marina Shapira and Mark Priestley, University of Stirling
Context – PISA 2022
One of the more interesting features of the regular Programme for International Student Assessment is the ways in which its results are selectively appropriated to support pre-existing ideological positions on Scottish education. The release this week of the latest scores has produced contrasting narratives. According to the government press release, ‘Scotland has maintained its international standing’. Conversely, other media reports have contained hyperbolic headlines about results falling ‘off a cliff edge’. The reality falls somewhere in between – Scotland has clearly suffered a long term decline it in its overall PISA scores in reading, mathematics and Science. This decline predates the introduction of Curriculum for Excellence (CfE) by some years, with occasional upward trends (notably in reading in both 2012 and 2018. These long-term declines are not atypical across OECD countries, with similar tends evident elsewhere, including in Finland (often held up as a shining example of a successful educational system). Many countries have experienced a decline in PISA scores between 2018 and 2022. The impact of global pandemic on this decline cannot be dismissed, although of course this is not the sole reason. Moreover, the decline is quite gradual over time, with shorter term changes between different PISA cycles – although the long term picture looks more dramatic.
For many commentators, the issue is the failure of CFE, and that the answer is to replace it an English-style, so-called ’knowledge-rich’ curriculum. While CfE undoubtedly has many problems, related to its structure and coherence, lack of attention to knowledge and its implementation, and badly needs reform, we do not believe that CfE is the sole, or even the primary issue affecting the PISA results. It is worthy of note that the top-performing Estonian curriculum – a competency-based model – has much more in common with CfE than it does with the English approach. Our belief is that the issues in Scotland are much more systemic, including the following:
- The shift from the 1990s onwards from a system premised on support to one premised on measurement. A focus on performance indicators, curriculum standards and benchmarks, reflecting broader neoliberal tendency in public governance, has created perverse incentives and a culture of performativity (often evidenced in a short-term focus on improving data and formulaic teaching to the test). This is a business rather than an educational rationale for schooling.
- A continued over-emphasis on qualifications in the secondary school, to the detriment of curriculum making that considers the broader question of what it means to become educated. This issue was, of course, identified by the OECD (2021) in its recent report and has been a key focus of the 2023 Hayward review.
- Poorly specified processes for supporting the implementation of CfE. These have led to a piecemeal approach, and often poor understanding of the core aims and principles the curriculum, in the lack of clear processes (and time) for practitioners to make sense of policy in relation to their own contexts.
- Poor resourcing of schools in an age of austerity, including limited teacher non-contact time and a lack of available cover for teachers to attend courses.
It is necessary to take a more nuanced systemic view of Scotland’s education system – but, inevitably, what we are seeing is the usual politicised point scoring.
Curriculum making in secondary schools – our recent research
In this context, it is useful to consider our recent study, funded by the Nuffield Foundation. This explored curricular practices and the experiences of young people in secondary schools across Scotland under CfE. This study is considered in relation to the earlier 2018 PISA findings, as we have not yet undertaken a detailed analysis of the 2022 results.
Among our findings, we discovered a decline in National Qualifications entries in S4 under CfE, known in Scotland as curriculum narrowing. Additionally, we observed evidence of social stratification in subject entry patterns in S4, indicating a greater decline in entries for students from disadvantaged areas, limiting their subject choices.
The findings from our study also suggest that the curriculum decisions made in schools are primarily driven by the demand for better attainment data, particularly in National Qualifications during the Senior Phase (school years S4-S6), with less emphasis on what it means for an individual to become an educated person in a modern and complex society (Shapira et al 2023[1]).
Considering the above, we argue that using attainment as the primary measure of curriculum success is unhealthy, counter-educational, and contradicts the goals of CfE (ibid). Given the pressure placed on schools by the government, local authorities, and school inspectors to raise attainment, to evaluate whether CfE meets its goals and provides comprehensive educational experiences, additional indicators are necessary.
The PISA dataset, while not directly linked to specific curriculum national features, evaluates 15-year-olds’ ability to apply school-acquired knowledge in real-life situations (OECD, 2020[2]; Schleicher, 2020[3]). PISA’s measures assess skills and competencies across an expanding range of countries, making it an invaluable resource for informing national educational policies and evaluating education system performance (although with the caveat that narrow use of its findings can lead to knee jerk reaction in policy, following so-called ‘PISA shock’ engendered by what has become in many respects a self-defeating global arms race).
It is important to note that CfE architects emphasised that attainment wasn’t the curriculum’s sole focus (Scottish Executive, 2006). CfE purposes are defined through the Four Capacities, aiming to enable learners to become successful learners, confident individuals, responsible citizens, and effective contributors through a framework emphasising learning experiences, skills for life and work, and individual potential maximization (Education Scotland, n.d.).
The OECD’s measures of global competencies, available in PISA’s 2018 data, align with CfE’s vision. The Global Competences framework emphasises young people’s multidimensional capacity for understanding global issues, appreciating different perspectives, engaging in effective cross-cultural interactions, and contributing to collective well-being and sustainable development (OECD, 2020[4]). This alignment justifies the use of PISA measures to evaluate whether CfE has indeed achieved its goals and enables young people with developing the Four Capacities.
PISA’s global competences encompass students’ cultural awareness, problem discussion, and recognition of their ability to contribute to well-being. Thus, the 2018 PISA dataset, combined with Scottish administrative education data, offers a chance to compare outcomes across various competences assessed by PISA and those measures of attainment and outcomes available in administrative education data (such as subject entries, attainment in National level qualifications, and short-term destinations of school leavers). This would contribute to our understanding of the Scottish education system and its alignment with the goals outlined in the “four capacities” framework. Additionally, PISA data includes a rich set of family and school-level characteristics that are absent from administrative education data, providing valuable insights into how individual, family, and school-level attributes impact various outcomes for young people.
In our study, we used the 2018 Scotland PISA dataset linked with the administration education data. We explored the relationship between the breadth of the secondary curriculum experienced by 15–16-year-old students, the level of attainment across National 5 qualifications and the outcome measures available in the PISA dataset.
We found (for detailed findings, see Shapira et al. 2023) that students who attended schools with a broader S4 curriculum were developing better competence in using general language, mathematics, and other knowledge and skills obtained in schools for solving real-life problems.
Furthermore, students exposed to a broader upper secondary curriculum not only achieved better academic outcomes but also developed a better understanding of the complexities of modern societies, self-awareness, resilience, and active citizenship. In schools with a broader curriculum in S4, students scored more highly on the OECD measures of global competences. For example, students attending schools with more subject entries for National 5 level qualifications in S4 were more likely to feel a sense of belonging to the school, learn about different cultures, feel proud of their accomplishments, and feel empowered to do something about the world’s problems.
Higher scores in international tests such as PISA English and Maths, and higher scores for OECD measures of global competences correlate positively with the schools that allow 15-year-old students to enrol in the larger number of National 5 qualifications in school year S4 and with the schools that demonstrate higher attainment in national qualifications at SCQF level 5 (National 5).
In summary, using the PISA data in conjunction with other data about a national education system can offer additional invaluable insights about national education, that can be used for informing national educational policies and evaluating education system performance. Our findings indicate that the narrowing of the Scottish secondary curriculum under CfE not only correlates with poorer attainment in Scottish National qualifications and less successful transitions to positive destinations, but also has adverse effects on a broader range of student outcomes. Specifically, it positively correlates with measures available in PISA datasets, such as reading, mathematics, and science competencies, as well as those associated with the Global Competences Framework. Considering the alignment between the Global Competences Framework and the ‘Four Capacities’, we can conclude that employing additional outcome measures, such as those found in PISA data, provides additional valuable insights into how CfE achieves its objectives. It demonstrates that in schools offering a broad secondary curriculum, young people are more likely to acquire enhanced learning experiences, life skills, and realise their maximum individual potential. Given that curriculum narrowing disproportionately impacts disadvantaged learners attending schools in socioeconomically challenged areas, our findings from the analysis of 2018 Scotland’s PISA data reinforce our other findings obtained from the analysis of administrative data, as well as the data collected by our study, about the necessity of developing policies that enhance the range and structure of subjects offered to 15-year-olds to enhance their educational experiences and outcomes.
[1] See the main public report for details: Choice, Attainment and Positive Destinations: Exploring the impact of curriculum policy change on young people (nuffieldfoundation.org)
[2] OECD (2020) Global Release of PISA Global Competence Assessment Results. 2020 Global Conference, AFS Intercultural Programmes. 23-24 October.
[3] Schleicher, A. (2020) PISA 2018: Results and Interpretations. OECD. Retrieved from: PISA 2018 Insights and Interpretations FINAL PDF.pdf (oecd.org)
[4] OECD (2020) Global Release of PISA Global Competence Assessment Results. 2020 Global Conference, AFS Intercultural Programmes. 23-24 October.
Professor Mark Priestley 