Inter-disciplinary curriculum: why is it so difficult to develop? (part two)

In my previous blog post, I set out some of the reasons why Inter-disciplinary learning (IDL) is advocated in modern curricula, and highlighted some of the problems that have stemmed from this. These include poorly conceptualised policy guidance, and patchy understanding of the nature of inter-disciplinarity, particularly its grounding in disciplinary knowledge. A result of these issues can be poor quality provision that is often difficult to see as inter-disciplinary; more often than not, what we see emerging is multi-disciplinarity with poor connections between discipline-based knowledge, or weakly conceptualised cross-curricular study that does not draw upon more than one discipline. I note here that while there is a general assumption that primaries do IDL, and secondaries do not, this is not necessarily the case. In this second post on this theme, I explore two issues – conceptual development and creating propitious conditions for IDL – both of which are essential if meaningful and coherent inter-disciplinary approaches to curriculum are to become possible. The insights here are equally applicable for primary and secondary education.

Conceptual understanding

At a general level, with wider implications for curriculum making as I have argued elsewhere, practitioners need to develop more expansive concepts of curriculum. This includes, inter alia, viewing curriculum as more than simply content, but instead as the social practices (including pedagogy and assessment) that constitute curriculum making. It involves practitioners taking a holistic view and seeing how their part of the curriculum fits into the whole, to ensure coherence and progression – both vertically across the age range, and horizontally across the breadth of the curriculum.

This, I believe, entails school-level curriculum planning that looks systematically at the content that forms the curriculum. It means shifting from the question of ‘what subjects should we teach?’, instead asking the question ‘what knowledge, skills and attributes are required to become an educated person, capable of thriving in a modern, complex democratic society?’.  As I have argued elsewhere in this blog, this is not a case of simply specifying content; it is instead part of a process of asking what education is for, which should rightly start with consideration of the purposes of education. In the case of Scotland these are set out to some extent in the big ideas – the attributes and capabilities – that form the Four Capacities.

This will probably involve a realisation that the current range of subjects contains (and conceals) considerable gaps in the required knowledge. It requires a shift from seeing subjects as not ends of education, somehow set in stone as was the case in the famous parable of the sabre-tooth curriculum1, but instead viewing them as a means of apportioning curricular content5 (with alternative means available). This means understanding that knowledge is not the same as disciplines, and disciplines are not the same as school subjects.

At an IDL specific level, we need to develop better understandings of concepts that relate to inter-disciplinarity. Part of this lies in the principles that underpin IDL, which might be seen as learning that draws knowledge (substantive/propositional and procedural2) from two or more disciplines in a connected way. Thus, disciplinary knowledge should always be the major foundation of school content, whether the approach is via subjects (based loosely or otherwise on disciplines) or integrated/inter-disciplinary provision.

According to Repko (20073), an inter-disciplinary curriculum should have four key elements:

  • addressing a complex problem or focus question that cannot be resolved by using a single disciplinary approach
  • drawing on insights generated by disciplines, inter-disciplines, or schools of thought, including non-disciplinary knowledge formations
  • integrating insights
  • producing an inter-disciplinary understanding of the problem or question.

It also lies in drawing upon the rich theoretical models that have formed the basis for inter-disciplinary curriculum in other parts of the world. Prominent amongst these is work by James Beane4 and Robin Fogarty5. Fogarty’s work is especially useful for schools seeking to develop a more integrated curriculum, offering a continuum of practice, including:

  • Fragmented – no joint planning or link making between subjects
  • Sequenced – arranging teaching so that related topics are taught concurrently within different subjects (e.g. allowing the study of the First World War in History to coincide with the study of war poetry in English).
  • Shared – joint planning of related disciplines (e.g. identifying commonalities between Science and Geography).
  • Webbed – the use of thematic approaches to bring content from different disciplines together (e.g. an Africa week when all curriculum areas focus on this single theme).
  • Threaded – a cross-curricular approach where big ideas (e.g. citizenship, thinking skills) are coherently planned across the curriculum.
  • Integrated – largely an interdisciplinary organisational approach, which breaks down traditional subject boundaries – either partially (e.g. hybrid subjects) or fully (e.g. the US middle school approach)

And of course, all of the above requires systematic sense making by teachers, not just an articulation of ideas in curricular guidance, which may or may not be read by practitioners.

Cultural/Structural

Enhancing understanding of concepts associated with IDL, amongst those seeking to develop new approaches, is only part of the process, and may actually be a waste of time if such innovation is impeded by formidable barriers. Thus sense making to develop understanding should be accompanied by actions to address the conditions that promote and impede the development of inter-disciplinary curricula. A major issue lies in the ways in which secondary teachers are educated as subject specialists and primary teachers as generalists, as these assumptions constitute a major cultural barrier to IDL. If one accepts the argument, for instance, that IDL is an appropriate approach across the primary/secondary BGE transition phase from P5-S2, then it seems sensible to educate specialist teachers to teach across this phase. This is starting to happen in Scotland, via primary teaching degrees with a specialism, and PGDE/Master’s level programmes that prepare teachers to practice across the primary/secondary transition. More needs to be done here, for example, educating generalists with a subject or domain specialism (e.g. general science, modern languages or social studies) that spans the transition. Such a workforce might help address a lack of specialist teachers in the upper primary years, and an overly fragmented approach in the junior secondary years that currently relies on input from specialists educated in more narrow, discipline-based specialisms. This would of course involve some system-level change, including GTCS accreditation of new ITE routes. Other system level changes might include reconfiguration of qualifications systems to reward the development of IDL; current approaches serve to lock prevailing subject-based provision patterns in place.

Another issue relates to resourcing. Under CfE, there has been a general assumption that schools will make their own curriculum in ways that suit local needs. While I agree in principle with the notion of subsidiarity in curriculum making, this should not mean each school reinventing the wheel in isolation. The predominant approach for CfE has been to provide guidance and exemplification and ask schools to get on with it. That, in my view, is no substitute for national or regional support and resourcing for curriculum making, including curriculum leadership by expert teachers, systematic processes such as professional enquiry and the development of national resources that can be adapted in schools. I note here that the latter approach has a long pedigree in the UK, notably in the work of the Schools Council projects; for example, Schools History Project (including well-established GCSE programmes) is an enduring – and popular – testament to their success.

In summary, curriculum making does not happen in a vacuum. Meaningful IDL requires attention to both conceptual development and the conditions that support emerging practice. This is something that requires thoughtful and systematic leadership and resourcing from the centre as well as school-based curriculum making.

Endnotes

  1. https://users.ugent.be/~mvalcke/OWK_1415/toetsing/thesabertoothcurriculumshor.pdf
  2. Substantive for propositional knowledge refers to knowing that (not simply facts relating to a discipline, to which the content of schooling is often reduced, but also the ways of knowing and ordering that knowledge. Procedural knowledge refers to knowing how.
  3. https://go.galegroup.com/ps/i.do?p=AONE&sw=w&u=googlescholar&v=2.1&it=r&id=GALE%7CA165912657&sid=googleScholar&asid=4eaedccb. While Repko was writing about Higher Education, his insights are highly applicable for schools.
  4. https://books.google.co.uk/books?id=XxkBDAAAQBAJ
  5. https://pdfs.semanticscholar.org/fc84/06745befdf07ad521450d7434df379c72c48.pdf. For more detail, see Fogarty, R. & Pete, B. (2009). How to Integrate the Curricula. Corwin.
Advertisements

‘There be dragons’: redrawing the curriculum map in Wales

Here is the text of my post on the Welsh curriculum, originally posted at https://curriculumforwales.gov.wales/

The 2015 publication of Successful Futures marked a watershed moment in the history of education in Wales. The proposed new curriculum is a radical departure from recent top-down, teacher proof policy. It moves schools away from prescriptive content-led approaches to teaching, and affords teachers and schools considerable autonomy in developing a school-based curriculum to meet local needs.

The new Curriculum for Wales is typical, in many ways, of recent worldwide ‘new curriculum’ policy. It emphases the centrality of the learner, and the importance of developing so-called 21st century skills, to equip young people to thrive in modern complex democratic societies and in the workplace. It recognises that subjects, the ubiquitous approach to segmenting the secondary curriculum, may not always be the best way of organising teaching to ensure that young people develop the knowledge required to thrive in the modern world.

Moreover, like other ‘new curricula’ in countries such as Scotland and New Zealand, the new curriculum is open to critique, and faces considerable challenges in its enactment in schools. These curricula have been attacked for downgrading knowledge, blurring the well-established boundaries between everyday knowledge and disciplinary knowledge. Critics have derided their alleged focus on fuzzy skills and child centred learning. They have often attracted the pejorative label ‘progressive’. Furthermore, curricula in Scotland and elsewhere have suffered implementation problems. Andreas Schleicher of the OECD stated in December 2016 that, while Scotland had developed a bold and visionary approach, it still needed to move from an intended to an implemented curriculum. Our research suggests that a major issue lies in a gap between teachers’ prevalent practices and beliefs about education, and the implicit aims of the new curriculum.

Despite these anxieties, I believe that Wales is different. First, Wales is heeding the lessons from other countries, and has solicited the advice of researchers in some of the countries already developing this type of curriculum. Second, the Welsh curriculum developers have actively sought to put in place principles and processes that address some of the criticisms. The importance of knowledge has been foregrounded in the curriculum guidance. An explicit process of developing the curriculum from purposes of education – articulated in the Four Purposes and the ‘What matters?’ statements for each AoLE – has been set out clearly. The role of Pioneer Teachers will prove to be significant – as writers of the AoLE statements, and as facilitators of school-based curriculum development as the curriculum is translated into practice over the coming years. A major source of tension ion many new curricula – the practice of defining the curriculum via thousands of learning outcomes – will not happen in Wales, where the What Matters? Framework is a far more constructive approach to developing practice in schools.

All of the above should not detract from the challenges faced as schools step into the uncharted terrain of the future. Nevertheless, a few principles should help guide this journey of exploration. First, the starting point for curriculum development is not the content (or subjects) to be taught, but instead should be the purposes of education set out in the curriculum. Sense-making – through extensive professional dialogue – is an essential part of this process; if teachers do not understand the new curriculum, then they will not develop practices that are fit-for-purpose. Knowledge and skills – powerful knowledge – need to be taught with these purposes in mind. Similarly, educational methods need to be fit-for-purpose. Powerful pedagogies are as important for developing intellectual capacity as is powerful knowledge. The role of the Pioneer teachers and the regional consortia will be vital in developing the infrastructure to support curriculum development. And significantly, Wales will need to develop approaches to accountability and qualifications that serve rather than drive school’s practices.

If the above issues are addressed – and I am confident that the will is there to address them – then the new Welsh curriculum may well herald successful futures. The new curriculum is different to what came before, and will require different approaches and working patterns. Because it offers greater local flexibility and autonomy, it will require active engagement by all teachers in Wales. Experience from Scotland suggests that those teachers and schools that engaged early in process, making sense of CfE and developing a vision for it, were the same schools and teachers that made the most of its potential. It is worth remembering the words of curriculum scholar Lawrence Stenhouse – that there can be no curriculum development without teacher development. Both will be required to maximise the opportunities afforded by Successful Futures.

Stenhouse, L. (1975). An Introduction to Curriculum Research and Development (London, Heinemann).

The school curriculum in the UK: divergence on the Celtic fringe

This is the original version of the article published today in The Conversation (https://theconversation.com/in-britains-battle-over-school-curriculum-celtic-nations-have-got-it-right-90277) – before all the editorial to-ing and fro-ing, and with its original title.

The National Curriculum introduced by Margaret Thatcher’s governments in the 1980s was a seminal development in UK education history. Applying to England, Northern Ireland and Wales (but not to Scotland, which has a tradition of educational independence from Westminster), the new curriculum was highly controversial. Content-rigid and overcrowded, this teacher-proof curriculum was widely decried by education experts as badly theorised and damaging to young people. These criticisms seemed to be borne out in practice, as the new curriculum was subject to review and revision throughout the 1990s. By the early years of the new millennium, new curricular forms were starting to emerge, first in Scotland and Northern Ireland in 2004.

These curricula were primarily characterised by a move away from the detailed specification of content to be taught, and involved a significant shift to school and teacher autonomy in terms of what should be taught. England, under New Labour, initially appeared to be heading in the same direction, following a major review of the National Curriculum (2007-2008). Following the election of the coalition government in 2010, however, New Labour’s reforms were ditched in favour of a more traditional approach to defining the curriculum, widely described as knowledge-rich and influenced by ideas about cultural literacy. Wales, on the other hand, has followed the other Celtic nations, announcing its own new curriculum in 2015.

The new curricula have been widely attacked. According to critics, they downgrade knowledge, effectively dumbing down learning, and over-emphasise skills, particularly those required for the workplace. They are derided as being progressive, an apparently pejorative term in today’s educational climate. They are criticised for blurring the boundaries between subjects, and thus undermining the foundations of all that is great and noble in British education.

While such criticisms invariably contain some truth, they have been unhelpful in defining and operationalising good education in British schools. They have created unhelpful dichotomies of traditional versus progressive, knowledge versus skills, and the teacher as a ‘sage on the stage’ versus the teacher as a ‘guide on the side’. A good – and balanced – education should attend to all of these dimensions.

The new Celtic curricula are in fact helpful for a number of reasons. They are all grounded in clearly specified purposes of education. In Scotland these are articulated as attributes and capabilities, set out under four headings known as the Four Capacities: Successful Learners, Responsible Citizens, Effective Contributors and Confident Individuals. In Northern Ireland, detailed learning objectives are set out under three headings, developing young people as: Individuals; Contributors to Society; and Contributors to the Economy and Environment. These statements of purpose seek to set out clearly what an educated young person should look like at the end of a stage of education, and are greatly preferable in my view to a curriculum apparently devoid of purposes, and framed solely as specification of content. Clear specification of purposes should enable schools to define content and methods that are fit-for-purpose, to develop the knowledge and skills necessary for successful lives as an adult – including active and critically engaged citizenship and successful career trajectories.

If this is progressive education, then I do not take issue with the term progressive. Indeed, the father of progressive education, the American philosopher John Dewey emphasised the importance of engaging with the accumulated wisdom of mankind.  I would further argue that it is the non-progressive elements of the new curricula that have been responsible for their patchy implementation and for some of the issues raised by critics. Foremost amongst these is the framing of the curricula as detailed learning outcomes – hundreds of statements arrayed into hierarchical levels. These are a throwback to the original National Curriculum in England, with its simplistic assumptions that learning is a neat linear progress, to be measured at every stage, rather than a messy and emergent developmental process that varies between individuals. In Scotland, the learning outcomes are, in my opinion, largely responsible for the rather patchy implementation of Curriculum for Excellence. As the OECD’s Andreas Schleicher stated on BBC news in 2016, Scotland needs to move from an intended curriculum to an implemented curriculum. Detailed learning outcomes have been linked to heavy duty accountability processes; they can encourage risk aversion and tick-box approaches to curriculum development in schools.

It is, therefore, really interesting to see the new iteration of this sort of curriculum emerging in Wales. The developers of a Curriculum for Wales seem to be cognisant of the problems afflicting these curricula elsewhere. Development materials have emphasised the importance of clearly identifying and making sense of educational purposes. They have highlighted the need for knowledge – as well as skills – to be prominent in the thinking of teachers, as they enact the curriculum in schools, while recognising that traditional subjects are only one way of articulating this knowledge; not handed down to Moses on tablets of stone, but nevertheless still a useful means of dividing the curricular cake along with more integrated approaches. And, crucially, the Welsh process acknowledges the importance of both teacher involvement in all stages of developing the new curriculum from policy to practice, and the need to reframe accountability processes that distort teacher decision-making. This highlights the vital role played by teacher sense-making, as well as the mechanisms and processes that support this.

The report launching a Curriculum for Wales was called Successful Futures. Time will tell whether this was prescient.

Regional Improvement Collaboratives: a new strengthened middle in Scottish education?

The recent and long-awaited publication of the Scottish Government’s review of governance (http://www.gov.scot/Resource/0052/00521075.pdf) is, in many respects, a welcome development. The practical steps outlined in this review have the potential to transform the ongoing development of Curriculum for Excellence by explicitly addressing some of the weaknesses in the curriculum development process to date. In particular, the establishment of Regional Improvement Collaboratives provides a constructive response to the OECD’s call for Scotland to ‘strengthen the middle’; to establish a meso-level infrastructure that will (to quote the review) ‘mean that hands on advice, support and guidance can flow directly to schools to support improvement’ (p.7). This in turn will facilitate more meaningful engagement in schools with the core principles of CfE, to address, in the words of the OECD’s Andreas Schleicher, the need for Scotland to move from an ‘intended curriculum to an implemented curriculum’ ( BBC news on 6 December 2016).

In this context, the newly proposed Collaboratives look promising. According to the review:

New Regional Improvement Collaboratives […] will provide teams of professionals who have the singular focus of helping teachers to improve their practice. These teams will include sector and curriculum area specialists as well as additional support for learning experts such as educational psychologists (p.23).

This regional approach will involve decentralising some Education Scotland resources to support improvement closer to schools. It will also involve local authorities sharing resource at a regional level to ensure an enhanced improvement capability (p30).

This ‘national vision or framework to support collaboration’ (p.21) is supported by reference to the much cited McKinsey and co. report (https://tinyurl.com/ycl7jsdr), which states:

As the school systems we studied have progressed on their improvement journey, they seem to have increasingly come to rely on a “mediating layer” that acts between the centre and the schools. This mediating layer sustains improvement by providing three things of importance to the system: targeted hands-on support to schools, a buffer between the school and the centre, and a channel to share and integrate improvements across schools. (p. 6)

Readers of this blog will know that I have long argued for the development of a strong meso-level for supporting curriculum development (as opposed to meso-level structures that focus on audit and documentation; e.g. see https://tinyurl.com/y9eadhvy). I therefore very much welcome the general direction signalled by this review. Nevertheless, we should be aware of a number of potentially problematic issues as we take forward the recommendations of the review.

First, handing responsibility to teachers (both within the new Collaboratives and in schools) also means enhancing teachers’ agency (see https://tinyurl.com/y9vcccf4). This requires more than just rhetoric about autonomy. It requires establishing professional trust and developing contexts where teachers can exercise professional judgments, free from risk, and supported by access to cognitive, relational and material resources. A key issue here relates to the focus of the Collaboratives. Will they focus primarily on audit or on support? If they simply become beefed up local authorities focusing mainly on auditing performance against KPIs, rather than hands on leadership of curriculum development, then the new structures will not achieve their purpose.

Second, we need to take a nuanced look at how insights from other systems work in Scotland. International cherry-picking of other people’s policy is now a well-established international phenomenon. The governance review makes reference to how we might learn from other systems, for example: ‘countries such as Finland and Canada display strong overall performance and, equally important, show that a disadvantaged socioeconomic background does not necessarily result in poor performance at school’ (p.15). The important insight here is that we should be looking beyond copying the structures of these countries, and instead seek to emulate the processes that lead to improvement – in the case of Finland this includes a lack of inspections (which reduces the risk of performativity in the system), clear processes for sense-making in relation to changed policy, and leadership for reform/innovation. It can be argued that Finland’s social practices of curriculum-making shape its success as a system, as much as the ways in which its policies are framed.

Third, eyebrows will have been raised at the continued, and enhanced, role of Education Scotland in leading the development of the curriculum. The separation of the inspection and development functions of Education Scotland was widely predicted, and the fact that this has not happened will create some challenges for the system as it adapts to the new structures. I know, from my numerous conversations with teachers, that many will question putting the organisation responsible for the current state of affairs in CfE in charge of remedying its perceived ills. Of course views about the success (or otherwise) of CfE to date are contested, but regardless of where on stands on this question, it is clear that Education Scotland will face considerable challenges as it provides leadership within the new structures. A number of issues will need to be addressed, if the new structures are to be successful in the stated aim of developing improved practices across Scotland’s schools.

There is a need, in my view, to develop expertise – capacity – in curriculum development. We should be mindful of the observation by Lawrence Stenhouse (40 years ago) that there can be no curriculum development without teacher development – and that such insights apply to organisations leading curriculum development nationally/regionally as much as they apply to schools. We need to develop alternative methodologies for curriculum development. The evolution of CfE has been shaped to a large extent by the predominance of inspection/audit methodologies for curriculum development (in schools, in local authorities, and within Education Scotland). Such methodologies tend to focus on the evaluation of outputs, rather than considering the quality of inputs and processes; and I reiterate here that the success of Finland has been in a large part due to its focus on the latter. Linked to this is the issue of research literacy. A notable feature of organisations such as the SLO curriculum development agency in the Netherlands, or Ireland’s National Council for Curriculum and Assessment, is that their personnel are research literate: they engage with educational research in a way that often seems absent in Scotland; and in many cases they undertake their own research, for example through doctoral study. Engagement with research is essential if we are to break beyond the bounds of what is habitual and familiar, and to explore alternative repertoires for educational practice. Finally, we need a culture, in our new governance structures, that is focused on future improvement, rather than one that is wedded to maintaining the sacred cows presented by past structures, methods and guidance. As my colleague Walter Humes has pointed out in TESS this week, changing structures is, on its own, insufficient; we also need to address the cultures that frame educational practice in Scotland.

 

CfE and PISA: ‘holding our nerve’

The publication this week of the triennial PISA results has produced the usual phenomenon of the PISA shock in various countries. In the UK, England has maintained its position relative to other countries, and this is a source of disappointment to a government that staked its reputation on improving its performance. In Wales, indifferent performance and a disappointing set of results in science are a source for concern, but the political message is, in the words of one headteacher, to ‘hold our nerve’ and see through the current curricular reforms. In Scotland, a dip in performance relative to England is more difficult to stomach, and has raised inevitable questions about whether the decline is due to Curriculum for Excellence. According to Professor Lindsay Paterson, a long-time critic of CfE, the decline in Scotland’s relative and absolute performance on PISA is ‘shocking’ (see http://www.itv.com/news/border/2016-12-06/education-professor-calls-scotland-figures-shocking/). Professor Paterson states that the pupils tested in the current PISA round have been entirely educated under CfE, suggesting that CfE is the problem.

I have some sympathy with some of his well-known criticisms of CfE. I have consistently been on record as supporting the broad general direction represented by the curriculum – local flexibility, student-centred approaches and teacher autonomy – but would agree with him in his critique of the lack of attention to knowledge within CfE. To my mind, a progressive curriculum should not preclude, as stated by John Dewey, the learning of ‘the accumulated wisdom of the ages’; it should not mean that teachers should neglect issues of knowledge. I too regularly hear educators telling me that ‘we do not need to teach knowledge anymore because pupils can google what they need to know’, and that ‘education is all about skills now’. In my view, a curriculum should be knowledge-rich, and this entails teachers posing the right questions in their curriculum design about what knowledge is of most worth.

Nevertheless, I would disagree with the notion that CfE is to blame for the decline in PISA scores experienced in Scotland. This seems to be a simplistic explanation, which ignores the complexity of educational reform and of the multi-layered terrain of education in Scotland. Instead I would point to what the OECD’s Andreas Schleicher stated on BBC news on 6 December 2016 – that Scotland needs to move from an intended curriculum to an implemented curriculum. While I have warned for some years that the problem with CfE is not CfE per se, despite its weaknesses in the area of knowledge (see http://hdl.handle.net/1893/17866) and tensions within its structure (see http://dx.doi.org/10.1080/03054980903518951), I would argue that the original vision for the curriculum was sound, with its basis in a set of clear purposes – attributes and capabilities – to be developed by education. Indeed, CfE sought to provide in Scotland exactly the sort of rich educational experience that is evident in highly performing education systems such as Singapore and Finland, and is typical in many ways of curriculum policy in many such countries.

Instead, the problems lies in its enactment – its translation from policy to practice, as clearly indicated by the OECD in its 2015 review of CfE. There are various issues here, all of which add to a highly complex enactment of policy to practice. They include:

  • The specification of curricular content as detailed learning outcomes, which have encouraged audit approaches and strategic compliance with CfE, rather than full engagement with its principles. The new benchmarks offer more of the same, and will require a great deal of care in their implementation if we are to avoid a continuation of such approaches.
  • A lack of clarity in CfE guidance about processes for curriculum development (see http://hdl.handle.net/1893/22518) and the sheer volume of CfE documentation; the latter issue has contributed to a lack of clarity amongst schools and teachers, especially as the key messages have often been obscured, and in some cases changed over time. Again, the OECD identified this issue in their call for a simplified narrative.
  • The persistence of accountability mechanisms that have acted counter to the spirit of CfE, often encouraging performativity in schools (see http://hdl.handle.net/1893/20761) and its accompanying bureaucracy.
  • A teaching workforce that, while highly dedicated and technically skilled,  has often struggled to make sense of a new and different curriculum, in the absence of sustained programmes to engage them with its principles and develop theories of knowledge that are consonant with this approach. I continue to meet teachers who admit to being baffled by CfE. A further, and related issue is cultural; implicit teacher philosophies about education do not always sit easily with CfE, and the lack of adequate spaces for sense-making does not allow this issue to be readily addressed.
  • A paucity of craft knowledge around curriculum development across the system – what Michael Apple has described as a ‘lost art’.

The net result has been an incomplete (at best) enactment of CfE, and a tendency to address new curricular problems through existing practices and assumptions. This was evident in our research in 2011 and 2012 (e.g. http://hdl.handle.net/1893/11356), and I have seen little evidence since, in my extensive work with teachers, that the situation has improved. Thus the issue with declining scores in PISA is, in my opinion, likely to be due to a failure to enact CfE adequately, rather than being a problem with CfE as a curricular approach.

So how do we address this? A good starting point is the OECD review, which provides legitimation for a revision of CfE in its call to be bold. In all of this we need to remember that the curriculum should not be set in stone as a sabre tooth curriculum (see http://users.ugent.be/~mvalcke/OWK_1415/toetsing/thesabertoothcurriculumshor.pdf); instead it should be subject to regular review, and such a process should not be framed as a climb-down or u-turn by policy makers, but simply a part of the normal process of updating the curriculum to adapt to changing societal needs. This means a rationalisation of existing documentation, in my view, to provide the simplified narrative called for by the OECD. It requires the establishment of a strengthened middle – a mid-system leadership stratum that provides support and facilitation for curriculum development, using tried and tested methods of teacher/curriculum development such as collaborative professional enquiry (see http://hdl.handle.net/1893/24179). Above all, we should ‘hold our nerve’ with CfE, and enact it fully in the spirit of its original aspirations (avoiding the political temptation to ape the testing regimes so familiar in England). The CfE model is much admired around the world – but we need to make it a reality in our schools.

Some commentary on the new CfE Science benchmarks

The following is a guest post from Dr Laura Colucci-Gray, University of Aberdeen. It is somewhat longer than the usual blog post, but has been posted here in full because of the nuanced way it explores the new Science benchmarks.

The newly released assessment benchmarks – as they are presented in the draft consultation document –  aim to clearly set out what learners ‘need to know and be able to do’, moving from Early to First through Fourth Level. Quoting from the draft document, the benchmarks should be used to ‘monitor progress towards achievement’ and “to provide guidance, in a single, key resource to support teachers’ professional judgement”. In line with the expectations of developing successfully learners, one of the hallmarks of the Curriculum for Excellence, students’ learning is presented here as some kind of advancement, a striding forward, towards a clear goal.

I must recognize that such statements make an impact on me, as a reader, for their neatness and apparent simplicity. Three aspects appear to be of greatest importance:

  1. Specified outcomes, or benchmarks, which operate as a proxy for the learning process;
  2. Progression, implying the existence of a gradient or spectrum along which learning gains can be evidenced;
  3. Teachers’ professional judgement, perhaps the most human and possibly less predictable aspect of all, but which is firmly situated at the end of the learning process, after both outcomes (1) and progression (2) have been clearly and truly specified.

I wish to take a closer look at the validity and feasibility of such a plan, its potentially contested relationships with the overall aims of the Scottish Curriculum, and the implications this may have for science education.

The Scottish Curriculum as it was first analysed in the lucid work of Priestley and Hume (2010) is fundamentally a hybrid model, seeking to combine socio-political and economic drivers[1]. In many respects, it seems to wish to combine earlier attempts, by making the science curriculum more relevant for students, seeking to engage them as ‘active contributors’ – while developing their basic literacy and skills of scientifically informed citizens. Yet, the design and guidelines for implementation have already generated concerns and criticism from science education researchers in Scotland. Bryce and Day (2013) argued for further clarity, alluding to the inevitable risks of the hybrid condition producing confoundment of purposes and professional confusion[2]. A hybrid condition brings in itself a form of dualism: which way to turn? Towards critical competencies one way, or towards skills and knowledge for work another way? It is hard not to recognize that a curriculum built around the design of ‘learning experiences’ which are supposed to lead towards specified ‘outcomes’ contains an in-built direction of travel relentlessly moving from the variability of people’s experience to the singularity of results, from diversity to sameness and from openness to closure. So, let’s take a closer look at what happens during such progression, and what might be the expected ‘learning gains, when the learning outcomes are turned into benchmarks.

Progression as simplification

I noted that in the letter signed by the learned societies, a concern was being expressed about the level of detail at which the outcomes – for particular subjects – were being described. One of such subjects is the Planet Earth. As newly formed human beings, we join the biodiversity and the web of interdependences amongst living and non-living things on the Planet. Such concept is first captured in the benchmarks for the Early level science, as follows:

I have observed living things in the environment over time and am becoming aware of how they depend on each other.

SCN 0-01a

 

·         Explores and sorts objects as living, non-living or once living.

·         Describes characteristics of livings things and how they depend on each other, for example young dependent on parents.

 

 

 

Relationships of dependence are foregrounded to refer to a broader set of caring, feeding, exchanging actions, within supportive or unsupportive relationships. Then when we reach level 1, the language almost suddenly changes, becoming directive and specific, moving from ‘explores and describes’ to ‘explain and uses’.

 

I can explore examples of food chains and show an appreciation of how animals and plants depend on each other for food.

SCN 1-02a

 

 

●      Explains that the sun is the main source of energy.

●      Explains that energy can be taken in by green plants to provide the major source of food for all living things.

●      Uses the terms ‘producer’ and ‘consumer’ correctly.

●      Uses vocabulary correctly including ‘predator’, ‘hunter’, ‘prey’ and ‘hunted’.

●      Uses and constructs a simple food chain showing energy flow.

 

This learning outcome at Level 1 is supposed to be clear and simple, building progressively from the previous one at early level. In this passage, we note that the emphasis on the broader set of relational dependencies and interdependencies of the early level has been reduced to the more recognisable concept of ‘food chain’, whereby relationships are instrumental to materials exchanges and living organisms are given the stultified roles of producers and consumers. What is more, an additional layer of meaning is supposedly being provided by the introduction of the ecological concepts, as a means to extend on the production and consumption model. So, in the second bullet point before last, producers and consumers are now turning into prey and predators. Not only, the prey-predator model applies to a very specific category of beings, the carnivore animals, but in the effort to simplify content, scientific ideas are being distorted to suit a particular view of the world. Mechanistic models of the biosphere – exemplified by the consumption chain – seem to be preferred, although these ideas have been long overtaken by modern biology. Current thinking favours metaphors such as the web, the tree, the hand along with relational models of symbiosis, mutualism and cooperation.

Progression as determinism

Another example is from Fourth level science where at the level of the generic skills, to develop scientifically literate citizens, we find:

  • Demonstrates understanding of the impact of science on society.
  • Discusses the moral and ethical implications of some scientific developments.

Later, in the more specific outcomes, we cannot fail to notice a certain determinism in fact-finding related for example to agricultural production:

  • Uses information about essential plant nutrients to design a fertiliser;
  • States possible impacts of the use of fertilisers, for example, eutrophication and algal blooms.

The use of fertilisers is controversial; eutrophication and algal blooms are not simply the effects of fertilisers in the water but the consequences of mass food production for commercial purposes. At no point such contestation is hinted at or even supposed. While we cannot make assumptions without hearing directly from the authors of the document, it is the language of the specific outcomes which is of concern here. Words are used carefully to delineate factual knowledge but shy away from any critical appraisal of the surrounding cultural and social context.

Progression as Anthropocentrism

As science develops within a cultural, historical and social context, the language of its expression carries connotations about the ways in which human societies – at different point in history – have looked at the bio-physical world. As was mentioned earlier, in relation to the history of curriculum innovation in science education, also the teaching of science needs to be framed within the cultural and value narratives of the time.

The extract below – from Third Level Science – is focussing on viruses and microbes. We recognise almost immediately the underlying violent frame – carried by words such as ‘defence, barriers and the breaching of barriers’. A particularly westernised view of the world, which locates human beings – and their bodies – in direct and adversarial contrast with the living world upon which they depend is forcefully transferred through the power structures of the curriculum.

I have explored how the body defends itself against disease and can describe how vaccines can provide protection.

SCN 3-13c

●      Explains how microbes, for example, bacteria and viruses, can cause disease and infection.

●      Describes the barriers to infection as a first line of defence, for example, skin, mucus and stomach acids.

●      States how the immune system protects the body against disease if the first line of defence is breached, for example, white blood cells and production of antibodies.

●      Explores and explains how vaccinations can protect individuals and populations from disease.

Arguably, one might suggest that the factual phrasing of learning outcomes precludes the possibility to disclose alternative cultural frameworks. Here I refer to the heritage of pupils holding alternative views of the living world, such as the more animistic or religious views. As we know from many years of research in science education, learning progression in science comes from the opportunity to navigate alternative understandings and to recognise how scientific concepts are defined through ongoing dialogue, within a community of researchers and learners. Science education pedagogies should thus focus more on the elicitation of such alternative understandings, as opposed to the fast delivery of the ‘right answer’. That said, looking at the next example from First level:

 

By investigating forces on toys and other objects, I can predict the effect on the shape or motion of objects.

SCN 1-07a

●     Uses vocabulary to describe forces, for example, pushing, pulling, stretching, squashing and twisting.

●     Demonstrates understanding of how a force can make an object change speed, direction or shape.

●     Investigates balanced forces and can explain that if a push and pull are equal then there is no change in movement.

●     Investigates how shape is linked to motion and stability.

 

… the benchmark document is strict and ‘forceful’ in the prescription of how children are supposed to talk in physics, even at the basic level of describing the motion of objects in front of them. The words that are being used are also forceful in themselves – pushing; squashing… – not dissimilar from the metaphors of power highlighted above. And finally, forces – second bullet point above – can make an object change shape… forces are being anthropomorphised, extracted from the wider, bio- physical system of interactions and interdependencies… if another view of the world was adopted, one which would focus on a variety of experiences and alternative value-frameworks, one which would let students to play and explore, through bodies, hearts and minds, we would might have been able to see other words coming through in relation to forces: supporting, coming together, balancing or attracting…

 

So, returning to three key aspects emphasised by the document, I would like to share some of the concerns which have now emerged.

 

1&2. Turning outcomes into benchmarks and using benchmarks as proxy for learning; By setting out the learning process as a movement from experiences to outcomes and from outcomes to benchmarks, I observed progressive reduction, narrowing of linguistic frames and selective use of value-frameworks. As opposed to the meaning of progressive as reformist or broad-minded, like the CfE wished to be known to the world, I see stultification and closure to dialogue. Institutionalised reductionism is built-in within a machine for knowledge delivery. The progression is perceived more like an inexorable and unavoidable rush, forward-tracking learning to inevitable conclusions.

(I am reminded at this point of the popular film Cassandra Crossing, with Sophia Loren, back in the Seventies… has anybody seen it? The train carrying a sick spy with a contagious illness is made to rush blindly towards an old, crumbling bridge, while Sophia Loren is attending to the illness-stricken passengers, kept in the back carriage, in the desperate attempt to prove the authorities they can recover from the bug…)

 

  1. Teachers’ professional judgement. Within the simplified picture of the benchmarks document we are then expected to find the voices or judgement of the teachers, who are instrumental in the certification of learning. Yet within a scenario of progressive reduction, what are the teachers expected to do? Is there any room for them to exert their professional judgement? When all they are asked to do is to lead the pupils towards specific ‘benchmarks’ which are simplified and stripped of any personal signification, what is the teacher’s role meant to be? I am perplexed. Are the teachers like the passionate and professionally trained Sophia Loren who is working her socks off at the back of the train to treat the ill passengers… or more like the old-fashioned tram-controllers, stamping the tickets of the people who come on board? Neither metaphor seems to satisfy here…

 

So where to next? Addressing the trajectory….

Science, like all other subjects, is a body of knowledge which has been accumulated over time but which has also progressively changed over time. Behind science as a set of disciplinary knowledge there are people – the scientists – as well as policy-makers, citizens and tax-payers, including the merchants and the merchandise. Indeed, since the last energy transition from coal to oil, we have witnessed an explosion of knowledge thanks to the power of technology and computing machines. Consequently, science has changed dramatically from being the craft of a single individual to the interconnected activities of interdisciplinary teams operating within an extended web of public and private funding. Such transformations have two important implications. As science is increasingly emmeshed with political and economic agendas, the public is called to interrogate the allocation of funding and the ethical dimension of new ventures. New terms such as post-normal science, citizen science and even DIY science, are pointing to hybrid forms of knowledge sharing and knowledge forming calling for inclusion of different voices, participation, democratisation of science and critical appraisal.  Secondly, if the participation of the public is harnessed, evoked or even feared, education is called upon the task of preparing citizens for ethical, public dialogue, moving from knowledge to complex dialogical competencies which are linguistic, social, imaginative and creative.

In this open-ended and contested scenario, which progresses through debate and radical uncertainties arising from the new frontiers of science, the teacher has a key role in terms of preparing young people to interrogate the knowledge we need and to elaborate own models of living.  I find some notable parallels with the comments produced by Dr. Joe Smith in the earlier blog about history, saying that “progression in history refers not to a more complete understanding of the past (of which most of us know very little), but a more sophisticated one”.  Clearly this business of complexification is tricky for science education. It is well known that in terms of language, scientific terms – like food chain or food webs – are specific, retaining the root of everyday language but encompassing a singular and precise meaning, defined by the discipline.  So, we can see how the preoccupation with specification arises and how it can be justified and legitimised. However, we can recognise that we are amidst a contradiction here. If on the one hand, a simpler set of benchmarks carries the hopes of freeing teachers from the task of sifting out exuberant content; on the other hand, the specified nature of the content demands a critical interrogation of the selections that have been made, recovering the motives, purposes and value-frameworks that accompany any form of knowledge.

What hopes and what possibilities?

Research in science education has repeatedly pointed to the problems of resisting naïve views and perceptions of science held by both teachers and students at different levels of education. Learning and teaching science is equated to a protocol, which through the right sequence of steps, will lead to the right answer. Much has been contributed by science education researchers in terms of pedagogies to address such problems. My own research conducted with colleagues in International contexts, has shown that Scottish teachers are interested in innovation, often taking risks in the implementation of creative pedagogies in science. However, leadership ethos in the school is not supportive of such attempts, and students are preoccupied with attainment and performance, thus contributing to a progressive reinforcement of a transmissive pedagogy and old-fashioned beliefs (Gray et al., 2016[3]).

An important element of innovation and hope in the newly published set of benchmarks however lies with the emphasis on play as a form of scientific inquiry and discovery. Recent understandings of cognition as an embodied process point to play as the first and fundamental process of sense-making. The engagement with spaces, objects and materials can be paralleled to what happens during a scientific investigation and for this reason, it can provide the first point of access for young children into science. Most importantly however it is a process which sustains the development of analogical and metaphorical language supporting increasing levels of conceptual thinking and abstraction. For this reason, I would like to see more emphasis on play and imaginative play throughout the curriculum and into the draft benchmark document. I would like to see further opportunities for students at third and fourth level to be sensitised to alternative value-frameworks and to grapple with the ambiguities and the uncertainties which characterise a genuine scientific investigation!

Similarly, I wish to see teachers as animators of playful interactions. Students and teachers of science can come together as a team of inquirers and interpreters of the ways in which science and technology shape our actions in society. The science laboratory is the wider world and the classroom can afford a space of possibilities, in which we are all actors… in an unfolding play.

(This contribution for the blog has benefitted from the ongoing conversations with colleagues in the School of Education at Aberdeen, Dr. Kirsten Darling and Dr. Donald Gray and from the long-standing affiliation with the Interdisciplinary Research Institute on Sustainability, based at Turin University, www.iris.unito.it)

References

[1] Priestley, M. & Humes, W. (2010) The Development of Scotland’s Curriculum for Excellence: amnesia and déjà vu, Oxford Review of Education 36 (3): 345-361.

[2] Day, S. and Bryce, T. (2013). Curriculum for Excellence science: vision or confusion? Scottish Educational Review, Vol. 45 , No. 1, 2013, p. 53 – 67.

[3] https://www.esera.org/media/eBook%202015/eBook_Part_8_links.pdf

 

The endless quest for the Holy Grail of educational specification: Scotland’s new assessment benchmarks

Teachers in Scotland are presently witnessing the phased publication of a series of draft assessment benchmarks. These are linked to the call in last year’s OECD review of Scottish education to simplify the narrative of the curriculum in response to OECD recommendations. The first benchmarks, for literacy and numeracy, were published in August 2016 (https://tinyurl.com/zjtogmb). They have subsequently been followed by draft benchmarks in a range of subjects such as Science (see https://tinyurl.com/zaj4s93), Expressive Arts and Social Studies, with more to follow for each curriculum area by the end of the year. Each set of benchmarks comprises around 50 pages of text, with groups of Experiences and Outcomes (Es & Os) listed alongside sets of benchmarks related to the applicable outcomes. If early drafts are any indication, we can expect to see around 4000 benchmarks covering the whole curriculum. The example below, from the draft Third Level Social Studies benchmarks, provides a flavour of this new approach.

People, past events and societies  

I can use my knowledge of a historical period to interpret the evidence and present an informed view.             SOC 3-01a

I can make links between my current and previous studies, and show my understanding of how people and events have contributed to the development of the Scottish nation.

SOC 3-02a

 

I can explain why a group of people from beyond Scotland settled here in the past and discuss the impact they have had on the life and culture of Scotland.                     SOC 3-03a

 

I can explain the similarities and differences between the lifestyles, values and attitudes of people in the past by comparing Scotland with a society in Europe or elsewhere.

SOC 3-04a

 

I can describe the factors contributing to a major social, political or economic change in the past and can assess the impact on people’s lives.

SOC 3-05a

I can discuss the motives of those involved in a significant turning point in the past and assess the consequences it had then and since.            SOC 3-06a

Through researching, I can identify possible causes of a past conflict and report on the impact it has had on the lives of people at that time.      SOC 3-06b

 

·       Evaluates a range of primary and secondary sources of evidence, to present valid conclusions about a historical period.

·       Draws on previous work to provide a detail explanation of how people and events have contributed to the development of the Scottish nation.

·       Provides reasons why a group of people from beyond Scotland settled here.

·       Describes the impacts immigrants have had on life and culture of Scotland.

·       Provides an account with some explanation as to how and why society has developed in different ways comparing Scotland to another society in Europe or elsewhere.

·        Describes factors which contributed to a major social, economic or social change in the past.

·       Draws reasoned conclusions about the impact on people’s lives of a major social economic or social change in the past.

·       Draws reasoned conclusions about the motives of those involved in a significant turning point or event in history.

·       Provides a justifies view of the impact of this significant historical event.

·       Identifies possible causes of past conflict, using research methods.

·       Presents in any appropriate form on the impact of people at that time.

It is immediately clear that the benchmarks add a new layer to the existing specification of Curriculum for Excellence. This is difficult to reconcile with the stated desire to simplify the narrative of the curriculum. It is thus hardly surprising that the benchmarks have been met with considerable scepticism by teachers on social media, and this week saw the publication of a thoughtful and considered, yet highly critical response from a group of STEM Learned Societies (see https://t.co/B3mDnglL9B). So what exactly is happening here, when a call to simplify the curriculum is met with a further spiral of specification (Wolf, 1995)? And what is wrong with this approach in any case?

Attempts to specify curriculum and assessment in detailed ways are not new. It is around hundred years since Bobbitt published his taxonomy of educational objectives. More recently in the United Kingdom, we have seen the emergence of the competency-based model that has underpinned vocational qualifications such as those produced by NCVQ in England and Scotvec in Scotland. Related to this has been the genesis and subsequent development of national curricula: from 1988, England’s National Curriculum set out attainment targets, expressed as lists of detailed outcomes, arrayed into hierarchical levels. Subsequent worldwide curriculum developments (for example, Scotland’s 5-14 curriculum, New Zealand’s 1993 Curriculum Framework, CfE in Scotland) have exhibited similar thinking. This approach has an instinctive appeal to those concerned with measuring attainment and tracking a school’s effectiveness. It provides a superficially neat way of categorising and measuring learning. The approach also attracted some support (especially in its early days) from some educationists. For example, Nash has talked of enabling learners “to have a sense of direction through planned and well-defined learning targets which are in turn based on defined criteria in terms of knowledge, skills and understanding”( Nash, in  Burke, 1995, p.162). Gilbert Jessup, the architect of the GNVQ competency-based model, stated that “statements of competence set clear goals for education and training programmes” and that “explicit standards of performance …… bring a rigour to assessment which has seldom been present in workplace assessment in the past” (Jessup, p.39). Jessup saw little difference between the competency-based model for vocational education and the emerging models of outcomes-based national curriculum, predicting that the National Curriculum would “result in more individual and small group project work, and less class teaching” (Jessup, 1991, p.78). Subsequent experience has of course demonstrated quite the opposite effect.

So what are the problems associated with this approach? I list some of well-documented issues here, focusing on generic critique of the model, rather than on a detailed analysis of specific benchmarks/subjects. Further posts on this blog will look at some of the subject areas such as social studies and science, offering a more finely focused critique of particular sets of outcomes.

  • The approach is complex, jargon-ridden and lends itself to bureaucracy. This criticism was levelled at the NCVQ model by Hyland who said that the model was “labyrinthine” in complexity and entirely “esoteric,” and as a consequence of all these factors, the model has proven to be unwieldy and difficult to access for both students and assessors (Hyland, 1994, p.13). Such issues have certainly been evident in Scotland in the creeping development of time-consuming, bureaucratic processes, and the subsequent exhortations for schools to reduce bureaucracy.
  • Specification of learning in this way has been shown to narrow learning, reducing the focus of lessons to what has to be assessed. Critics of this approach such as Hyland (1994) and Kelly (2004) were quick to point out that far from encouraging learner autonomy and flexibility in learning, the model inhibits it because of the prescriptive nature of many of the outcomes. Recent research in New Zealand (Ormond, 2016) indicates that specification of assessment standards has seriously narrowed the scope of the curriculum. Ormond provides an example of the Vietnam War, where some teachers omitted to teach the role of the USA in the war, while still meeting the requirements of the assessment standard.
  • Where assessment standards/benchmarks are too specific, they reduce teacher autonomy by filling lessons with assessment tasks and associated teaching to the test. Teaching thus becomes assessment-driven. In turn, this places great pressure on both teachers and students to perform – to meet the demands of the test. Performativity has been well-documented in the research. Its effects include stress on students and teachers, pressure to fabricate school image and manipulate statistics, and even downright cheating (see Priestley, Biesta & Robinson, 2015, chapter 5)
  • Focusing on ticking off benchmarks encourages an instrumental approach to curriculum development. Our research in Scotland documented instances of strategic compliance – box-ticking – with the Es & Os (e.g. see Priestley & Minty, 2013). There is a tendency to only visit an area of learning until enough evidence has been gathered that it has been covered, then to move onto to another required area. This is not an educational approach designed to build deep understanding or construct cross-curricular links. Instead it atomises learning.
  • There are philosophical arguments about whether it is ethical in a modern democracy to define in detail what young people should become. The assessment benchmarks can be framed as narrow behaviourist statements of performance, which mould people to behave in particular ways – as such, they can be seen as being more about training (at best) and indoctrination (at worst), rather than as educational (see Kelly, 2004).

The above objections to the specification of tightly specified assessment criteria suggest that it is extremely unwise for Scotland to take Curriculum for Excellence in this direction, which moves the practical curriculum yet further from the aspirational goals set out in early documentation. It is clear that such specification has political appeal, offering the (arguably spurious) opportunity to track achievement; moreover, it can be framed as a response to those teachers who have long decried the Es & Os for being too vague. Nevertheless, this spiral of specification is dangerous, and Scotland would do well to learn from prior history of curriculum reform. A salutary example lies in the GNVQ model: initially this was specified as Units, Elements and Performance criteria; later specification added range statements and evidence indicators, as curriculum designers engaged in a Holy Grail quest to achieve total clarity. The result was anything but clear; instead teachers experienced all of the issues outlined above, as courses became increasingly complex, bureaucratic and difficult to teach.

References

Burke, J. (ed.) (1995). Outcomes, Learning and the Curriculum: Implications for NVQs, GNVQs and other qualifications. London: Falmer Press.

Hyland, T. (1994). Competence, Education and NVQs: Dissenting Perspectives. London: Cassell.

Kelly, A.V. (2004). The Curriculum: theory and practice, 5th edition. London: Sage.

Jessup, G. (1991). Outcomes: NVQs and the Emerging Model of Education and Training. London: Falmer Press.

Ormond, B.M. (2016, in press). Curriculum decisions – the challenges of teacher autonomy over knowledge selection for history. Journal of Curriculum Studies. (http://dx.doi.org/10.1080/00220272.2016.1149225).

Priestley, M., Biesta, G. & Robinson, S. (2015). Teacher Agency: An Ecological Approach. London: Bloomsbury Academic.

Priestley, M. & Minty, S. (2013). Curriculum for Excellence: ‘A brilliant idea, but. . .’. Scottish Educational Review, 45 (1), 39-52.

Wolf, A. (1995). Competence-Based Assessment. Buckingham: Open University Press.