Curriculum for Excellence and attainment in National Qualifications

Curriculum for Excellence and attainment in National Qualifications

Marina Shapira, Camilla Barnett, Tracey Peace-Hughes, Mark Priestley and Michelle Ritchie

[Please note – the web version of this post does not include the charts. To see these, download the PDF] Shapira et al

Over the past week, we have witnessed a reigniting of the rancorous debate about qualifications in Scotland’s Curriculum for Excellence. This debate has surfaced periodically since 2015, and has focused on allegedly falling attainment and a narrowing of curriculum choice in the Senior Phase of secondary schooling (e.g. see TES, 2018). The annual publication of qualifications data relating to National Qualifications has tended to stoke the debate, but another factor has been the periodic release of unpublished research by University of Dundee Honorary Professor Jim Scott.

Extensive media coverage has at times assumed the proportions of a moral panic, talking in polemical terms of the ‘crisis’ that is ‘putting the education of a generation of pupils at risk’, and which has spread like ‘a virus’ (The Herald, 2019. Also see: Times Educational Supplement, 2018; The Scotsman, 2019). Most recently further controversy has been sparked by the coverage of a claim in Scott’s latest report that the percentage of those who gain national qualifications in Scotland is going sharply down since the introduction of the new curriculum. Initial coverage in the Telegraph was picked up and broadly disseminated across the media, including The Times, with eye catching headlines ‘Ministers admit curriculum failure as grade hits disadvantaged’ and ‘Failed curriculum’. This has subsequently provided fuel for attacks on government education policy by opposition politicians.

This debate has been of great interest to us, given the focus of our recent publications (Shapira & Priestley, 2018, 2019). We acknowledge the role of independent research in offering a systematic critique of policy and practice in education, and we recognise Scott’s important role in drawing attention to the issue of curriculum narrowing. We also note that we too have been critical of many aspects of CfE – articulation of policy, implementation and particularly the trend towards curriculum narrowing in the senior phase. Nevertheless, we also are cognisant of the dangers of using research to support political agendas, as appears to be the case in the current furore about qualifications. For that reason, it is important to have rigorous independent research to support both policy formation and critique of that policy. One of the arguments that we have been making in our recent work is that there is simply not enough evidence on the impacts of the Curriculum for Excellence.

Scott’s reports have been described, quite rightly, as independent. We do, however, have some concerns about their methodological rigour, and this in turn calls into question the findings and conclusions that are drawn from them. For example, in our own analysis of the attainment data (on the level of secondary schools in Scotland for years 2011-2017) we have not seen evidence that the attainment at National 5 level and Higher level has deteriorated under the new curriculum, as is claimed by Scott. On the contrary, the attainment levels have risen, both in terms of the overall percentage passes (out of total number of entries into qualifications, grades A-C), as well as in terms of percentages of pupils who attained 5 A-C grades at National 5 and Higher levels.

So what is going on? How is it possible to have two quite conflicting interpretations from the same data? We suggest that the issue lies in a lack of robust methodology in the underpinning research; this in turn then produces results – and subsequent claims – that are at best dubious, and which at worst misrepresent the data. We cannot address the full range of claims made in the report, but we offer two examples to illustrate claims which are problematic.

Falling attainment?

Let us first address one of the claims made in the media based on the report.  It says

Looking at national, local authority and individual school data, he found attainment in Scottish national qualification levels three to five in S4 pupils has dropped by at least 32.9% for each level since CfE was introduced in 2013.

How could this profound conclusion be reached based on the figures presented in Scott’s report? The report uses the figures obtained from the Scottish Qualifications Authority official statistics. These show 335,397 passes in 2018-2019, compared to 503,221 passes in 2012-2013.  Simple maths thus suggests that the total number of passes in 2018-19 stand at 66.6% of the total number of passes in 2012-2013.  It does technically mean that there was a 33.3% reduction in passes in year 2018-2019 compared to year 2012-2013. But could we conclude based on that that attainment on National 3-5 levels dropped by 33.3%? The answer is no, and it is necessary to explain why.

When we look at the first two tables reproduced from Scott’s reports, it is not clear what Scott is presenting as ‘Attainment’. The reader is not signposted and without going back to the original SQA data we are unsure whether the figures presented in the column for each year refers to the number of pupils who received a pass at A-C level or whether he is referring to the total number of passes (including passes at D level).  When we do go to the original, publicly available SQA data we find the brief definitions for both ‘Attainment’ and ‘Entries’

  • ‘Attainment’ refers to entries with successful results.
  • ‘Entries’ are the entries for a year (e.g. 1/8/17 – 31/7/18), that is the centre estimates that the learner will complete the award within that time period.

In other words, ‘attainment’ is counted as all exam/coursework passes, across all year groups. Young people who sit multiple qualifications, and pass, are counted multiple times. When we look at the tables available from the SQA, for example, we also see that ‘attainment’ at National 5 level is considered to be a pass at grades A-C. Thus attainment is conceptualised as the total number of qualifications gained in a year across all subjects.

There is a major caveat here. Claiming that the total number of qualifications achieved is not the same as saying that grades have fallen. To claim the former as a fall in attainment is misleading. The reason for this is that a drop in the total number of qualifications achieved is not necessarily evidence of a decline in standards; it may simply be that fewer qualifications are being taken, and there are various factors that need to be considered when analysing this. To simply compare raw numbers from year to year will not account for these.

Over time comparisons are important because they allow us to understand these factors. There are different methodologies for doing such comparisons.  One methodology would be to include a baseline year (or period) and then compare the rest of the data to that baseline year. This is what Scott apparently does in his report. Yet, a selection of the baseline year should be justified and one should make sure that such comparison is meaningful, comparing ‘like with like’, and avoiding spurious comparisons. A good example of unsuitable choice of baseline year would be the one from the former Soviet Union, where the baseline year 1913 was used routinely by the authorities to make comparisons that proved the advances of a planned socialist economy (e.g. we produce more washing machines in 1980 than we produced in 1913). Taking this logic, there is a need to reflect on both demographic issues (e.g. declining school rolls) as well as changes in school practices after the introduction of the new curriculum, as these may explain the total number of passes at National levels 3-5 after the introduction of the CfE.

Prior to the introduction of new national qualifications under the CfE there was a wide-spread practice of double counting of passes at SCQF levels 4 and 5. For example, schools often entered pupils for qualifications simultaneously at both Standard Grade Credit and Standard Grade General to minimise the number of fails and number of pupils who do not receive any qualifications.  The widespread practice of fall-back from Higher (level 6) to Intermediate 2 (level 5) in school year S5 meant that thousands of students were routinely entered for level 5 qualifications when they already had one (Standard Grade Credit) in the same subject from the previous year. Enrolment patterns have thus changed since the introduction of new single track qualifications under CfE, which makes it problematic to directly compare pre-and post-CfE qualification numbers.

Furthermore, there was a continuous reduction in the size of the 12-18 years old cohort  (and especially in the size of the 15-16 year cohort, see charts 1 and 2) therefore a part of reduction in the number of passes is due to the  demographic changes. Taking into account these (and other) changes, the comparison between the total number of passes before and after the introduction of the CfE is essentially meaningless.

Another significant reason for the reduction in the number of qualifications achieved is the narrowing of the secondary curriculum documented in our recent research papers (Shapira & Priestley, 2018, 2019). Whereas previously 8 subjects was the norm for the first tier of qualifications, many schools now offer as few as 5 or 6 subjects at SCQF levels 4/5. The main reduction in the number of subject entries and the number of subject choices took place during from 2012-2013 and 2013-2014 during the transition period from the former to new qualifications. Therefore using the year 2012-2013 as a base line year is simply wrong, because this doesn’t take into account this subject choice reduction (for more detailed discussion of the  reasons behind the curriculum narrowing see our paper: Shapira & Priestley 2019).

Therefore, to meaningfully compare the number of passes over time, we must do so as a proportion of the total number of entrants or awards for each year.  Using the data we can calculate the proportion of students attaining at each level over time, but only if we calculate the proportion at each level using that specific year’s total as a base.  Then we can compare the trends in the size of the proportion of those who achieve qualifications at a certain level. When we do so one can see (Chart 3) that after the introduction of CfE there was a reduction in the proportion of passes on National 3 and National 4 levels but a 15% increase in the proportion of passes  (from 53% in 2011-2012 to 67% in 2018-2019) at National 5 level qualifications (Chart 3). This strongly suggests at attainment is actually rising (in that students in S4 are taking qualifications at a higher level than previously).

Leaving school without qualifications

Let us now address another headline derived from the report – that there is 50% increase in number of people who leave school without qualifications. Looking the figures presented in the report, one can see that technically this is true. The proportion of young people leaving school without qualifications was 2.8% in 2007-8, gradually went down to 1.5%  in 2011-2012 and then gradually increased to 2.2% (which is indeed 50% increase compared with 2011-2012). Of course the evidence that 2.2% of young people left  school without qualifications should be of a great concern, as this is about life chances of 995 young people (the number of young people left school without qualifications in 2012-13 was 1005, reflecting the larger size of the school leaver cohort in that year). Scott’s report clearly identifies an issue of concern that requires action.

Chart 4: Percentage of initial leavers by stage of leaving, 2009/10 to 2017/18

However, to present this as evidence of the failure of CfE is misleading. These figures need to be examined not in isolation, but in relation to other statistics about school leavers. Thus, the Scottish Government published data shows that over the period of the CfE introduction there was also a decrease in the proportion of young people leaving school in S4 (from 12.6% in 2013/14 to 11.3% in 2017/18) and an increase in the proportion of young people leaving school in S5 (from 24.5% in 2013/14 to 25.9% in 2017/18) (see Chart 4). There was also an increase in proportion of young people leaving schools with at least SCQF level 5 or better in literacy (from 63% in 2012-13 to 80% in 2018-19) and numeracy (from 56.6% in 2012-2013 to 70% in 2018-19; Charts 5 and 6)[1]. Finally, a school leaver’s destination is a very important outcome by which curriculum impacts may be accessed. In Chart 7 we present trends in initial (3 months after leaving school) destinations of young people. The trends show that over the period there was an increase of the percentage of young people who made transitions to Higher Education (an increase from about 36% to 41% over the entire period, and an increase from about 38% to 41% after the introduction of CfE), as well as increase in employment (from 18.5% to 22.7% over the entire period). At the same time, the proportion of school leavers who were unemployed and looking for work decreased.

Finally, official Scottish Government data (Chart 7) show that that at levels 4, 5 and 6 attainment gaps between the  most advantaged and most disadvantaged SIMD quintiles getting smaller (see yellow, dark blue and grey lines on the chart).

Conclusions

Overall, we believe that studying the impact the curriculum reform is complex. It needs to consider a broad range of indicators that assess the outcomes of young people. These outcomes include a breadth of education, the range and configuration of subjects they study, the number of A-C passes at National 5 and Higher level qualifications, the overall level and range of qualifications achieved, the transitions made and destination reached after leaving school, and outcomes in the later life (for example occupational outcomes). Not all these outcomes can be studied now, however we should focus our attention on those indicators that are already available and analyse them using rigorous methods to produce an evidence which is comprehensive and reliable. At the same time we need to fill gaps in the data which is currently missing but which could potentially offer insights about the way the curriculum is made in schools, the role of different actors – local authorities, teachers, parents and learners in these processes, and their impacts on young people. Our new two year research project, funded by the Nuffield Foundation, will go some way towards addressing these gaps in Scotland. This comprehensive study will combine an analysis of available administrative data sources at the nuanced level of individual pupils and schools in Scotland with new data collection though social surveys, interviews and focus groups in secondary schools in Scotland. The goal is to understand curriculum making processes, curriculum provision and outcomes in terms of subject choice, attainment and destinations of young people in final phase of secondary education.

References

Shapira, M. & Priestley, M. (2018). Narrowing the Curriculum? Contemporary trends in provision and attainment in the Scottish Curriculum. Scottish Educational Review, 50[1], 75-107.

Shapira, M. & Priestley, M. (2019). Do schools matter? An exploration of the determinants of   lower secondary school subject choices under the Scottish Curriculum for Excellence. Review of Education.

[1] Attainment in literacy and numeracy refers to all students who achieved a pass in relevant units (not full qualifications) which contribute towards attainment in literacy and numeracy at that particular level.