Towards informed judgement

Edited extract from Boud, D. (2007) Reframing assessment as if learning were important, in D. Boud and N. Falchikov (eds) Rethinking Assessment for Higher Education: Learning for the Longer Term, London: Routledge, 14-25.

(with permission of the publisher)

[A] productive way of reframing assessment discourse is around the theme of informing judgement (cf. Hager and Butler, 1996). That is, informing the capacity to evaluate evidence, appraise situations and circumstances astutely, to draw sound conclusions and act in accordance with this analysis. This is an idea that focuses on learning centrally —learning to form judgements—but also on the act of forming judgements about learning—which may be used for validating purposes. This notion has the potential to incorporate a forward-looking dimension—informing judgement for future decision-making about learning. At one level this is what university courses have always been about and therefore it is not a substantial change. However, with regard to the discourse of assessment it marks a significant shift of focus away from the process of measuring effects and artefacts towards what education is intended to be for - that is, the formation of a capable person who can engage in professional work and contribute to society as an informed citizen.

Informing judgement as the central idea of assessment has a multiple emphasis. It relates both to the judgement of others in processes of certification and aiding learning and to informing the judgment of the learner in processes of presenting themselves for certification processes and for learning in the short and long term. It encompasses the dual focus of summative and formative assessment. However, formatively, it is directed towards informing learners’ judgement as a key function of assessment as distinct from the present almost exclusive focus on informing others. It has the potential to orient assessment towards consequences in that it makes a desired consequence explicit: informing the judgement of learners. It acknowledges the importance of reflexivity and self-regulation through acknowledgement of the centrality of judgement as a process. And it has the potential to contextualise assessment in practice, as judgement always has to be for a particular purpose.

Concerns about measurement, objectivity, standards and integrity are integral to a notion of informing judgement, but in themselves they are secondary to the act of becoming informed. It is this shift from foregrounding measurement to contextualising it as integral to processes of informing judgement we regard as important. It is not a matter of rejecting the concerns that are presently dominant, but viewing them within an appropriate educational frame that more adequately allows concerns about learning to come forward. A view of assessment that places informing judgement as central is able to include key graduate learning attributes as an intrinsic part of what assessment is for. It gives prominence to students making judgements about their own learning as a normal part of assessment activities, not as a special add-on that, unfortunately, student self-assessment has sometimes come to be.

New ways of inscribing assessment cannot in themselves change assessment and assessment practice, however. A shift will only occur if there is sufficient desire to want to move from one form of discourse to another and if a possible alternative is sufficiently generative to enable other desires to be focused through it. 

Implications of a focus on informing judgement

What then needs to be taken into account in such a view of assessment? 
Strong connections between assessment activities and what students learn have been accepted for some time. Assessment activities signal what is to be learned, they influence the approaches to learning that students take and they can indicate the levels of achievement that are required for any given unit of study. In short, assessment frames what students do. It provides an indication of what the institution gives priority to in making judgements, it provides an agenda more persuasive than a syllabus or course outline and it therefore has a powerful backwash effect on all teaching and learning activities. In some settings it has a stronger instrumental effect than others, but it has an effect on learning in all of them. We simply have to imagine how different the experience of a course would be if all the formal assessment activities were removed or changed to become discretionary to begin to see the influence. 

While these effects of assessment influence learning, they do not constitute educational features. Our view is that assessment can have a crucial role to play in an educational framing of learning of all kinds and in all settings and that an excessive officially sanctioned focus on grading and classification has distracted attention from what this role can be. The features we intend the notion of informing judgement to address are threefold.

The first feature connects assessment and learning directly. It takes up the [issue] of consequential validity and places the consequences of acts of assessment as central. Do assessment acts actively promote development of students’ capacity to make judgements about their own work throughout the course? Are there inadvertent effects of certification and grading on students learning that must be countered? 

Considerations of the consequential effects of assessment, the influences of assessment beyond the point of graduation, must also be taken into account. Are students better equipped to engage in learning in professional practice? Are they able to deploy assessment strategies to positively influence their own continuing learning? Are they able to utilise the resources of others in this process? Unless it is possible to claim that assessment within higher education contributes in a significant way to these goals, in principle and in practice, then it has an insufficient learning focus. 

The second feature of the desirable view of assessment we propose is a focus on fostering reflexivity and self-regulation. These represent two similar ideas, but come from different traditions—reflexivity from social theory (eg. Bourdieu and Wacquant, 1992; Giddens, 1991) and self-regulation from psychology (Karoly, 1993). What they share is a view that a key to learning in complex settings is to be able to ‘look again’, to monitor one’s own performance, to see one’s own learning in the context in which it is deployed and to respond with awareness to the exigencies of the tasks in which one is engaged. Reflexivity and self-regulation are not just about skills, although there is obviously a skill element to them. They involve dispositions and an orientation to both work and learning. They also have an affective dimension. They involve confidence and an image of oneself as an active learner, not one solely directed by others. A focus on reflexivity and self-regulation is a key element in constructing active learners, as these features need to be constructed both by teachers and examiners and by themselves.

There have been some assessment initiatives that operate in this general territory. The use of self- and peer-assessment practices as a formal part of a course of study is an example (e.g. Boud, 1995). However, not all self- and peer-assessment activities have a substantial impact on reflexivity and self-regulation, particularly those of a procedural kind, and reflexivity and self-regulation encompass more than is normally subsumed in self-assessment practices, at least as documented in much of the literature. Peer-assessment is sometimes used as a proxy for assessment by staff in the generation of grades and some conceptions of self-assessment are not even associated with the promotion of self-regulation (see Tan 2007). The seeking and utilising of feedback from multiple sources can be part of reflexivity and self-regulation, but if information from feedback is not used explicitly for learning to complete the feedback loop and to contribute to building an understanding of how to effectively utilise information from others, then it is very limited. Some aspects of self- and peer-assessment clearly have a part to play in a new conception of assessment for learning, but it is only part of the picture. Unless the fostering of reflexivity and self-regulation are part of the normal curriculum and are a fundamental part of course design, the impact of assessment on them is likely to be limited. The building of skills for reflexivity and self-regulation through assessment acts is not currently a strong feature of courses.

The third feature of a new focus of assessment is recognition of the variety of contexts in which learning occurs and is utilised. Dominant views of assessment have arisen from traditions in which what is taught is also assessed. In this, assessment is an act of measuring what has been learned from a course. The context of assessment then is the course itself, typically within an academic program in an educational institution. Contexts of application in work or professional practice may be used for illustrative purposes, but their influence on assessment is secondary, they are effectively used for background colour. Application of what is learned typically takes place post-graduation, or at least post- the unit of study. It occurs after assessment and cannot therefore be fully part of it. This creates an issue, which is often referred to as the problem of transferability of learning. That is, application to any given context is postponed and treated as a separate stage of learning that is not the concern of the educational institution, (Boud, 2001: Bowden and Marton, 1998). The context of assessment in higher education is often taken to be the world of the course, not the world of practice, despite lip service being paid to the latter. 

The world of the course privileges individual judgments. Acts of assessment judge individuals. Subject content is reified. The role of peers in learning and the nature of practice are downplayed. And knowledge is separated from the world in which it is used (Hager, 2004). Of course, in professional programs this is often less marked than in programs in which there is no obvious site of application. Indeed, one of the most significant innovations in professional education, problem-based learning, makes a particular feature of assessing learning in the context of application to problems as authentically constructed as can be achieved within an educational institution (Nendaz and Tekian, 1999). 

Again, there are many examples of a partial engagement with the contexts in which knowledge is used, most notably in the increasing use of what is termed authentic assessment (Wiggins, 1989). This refers to assessment practices that are closely aligned with activities that take place in real work settings as distinct from what are the, often artificial, constructs of university courses. This move has progressed a very long way in some vocational education and training systems in which the only legitimate forms of assessment are those which are based upon performance in a real work setting outside the educational institution and judged by an expert practitioner (see Kvale 2007). While there are limitations to an authentic assessment approach taken to extremes, the notion that assessment tasks should acknowledge and engage with the ways in which knowledge and skills are used in authentic settings is a useful one.

Conclusion

What might be some of the issues that arise from this articulation and how might they be addressed? Using a new term does not in itself make a difference to practice. However, if the notion of informing judgement is accepted as the guiding principle in all aspects of assessment, it leads to a new set of questions that enable us to determine whether any given set of assessment practices is addressing the central question of improving judgements of both assessors and learners. Practices that may otherwise demonstrate qualities of reliability and validity, may not meet the requirement of informing judgement of learners.

If the notion of informing judgement is taken up, the question arises of how assessment that exhibits this feature might differ from assessment that does not. As is implicit in the discussion given above, a discourse of informing judgement will always have the dual qualities of a focus on the learning needs of students as well as the legitimate needs of others to have an account of students’ learning outcomes. This points to an important shift of emphasis. If judgement is to be informed, what is the language that is to be used to do the informing? The current dominant discourse uses the language of marks and grades, as if there were a common understanding of how they might be interpreted. This language is one that destroys detail and obscures meaning. What information is conveyed in knowing that a person gained a ‘B’ or a Credit in a given unit of study? It only has meaning in the context of comparison with other students within the same context because it has its origins in a norm-referenced system of assessment that is becoming outmoded. Grades can be used in a standards-based system but only when grade descriptors are present. We suggest that if informing judgement is widely adopted, it will be necessary to move to a language of judgement that is both more transparent and more varied than the language of current assessment discourses. While there are still problems in the use of portfolios and qualitative descriptors of outcomes, and while further refinement of them is necessary, this may be worth undertaking, as the alternative has even less to offer.

References

Boud, D. (1995) Enhancing Learning through Self Assessment, London: Kogan Page.
Boud, D. (2001) Knowledge at work: issues of learning, in D. Boud and N. Solomon (eds) Work-Based Learning: A New Higher Education?, Buckingham: SRHE and Open University Press, 34-43.  
Bourdieu, P. and Wacquant, L. (1992) An Invitation to Reflexive Sociology, Chicago: University of Chicago Press.
Bowden, J. and Marton, F. (1998) The University of Learning: Beyond Quality and Competence in Higher Education, London: Kogan Page.
Giddens, A. (1991) Modernity and Self-Identity: Self and Society in the Late Modern Age, Cambridge: Polity Press.
Hager, P. (2004) The conceptualisation and measurement of learning at work, in H. Rainbird, A. Fuller and A. Munro (eds) Workplace Learning in Context, London: Routledge.
Hager, P. and Butler, J. (1996) Two models of educational assessment, Assessment and Evaluation in Higher Education, 21, 4: 367-378.
Karoly, P. (1993) Mechanisms of self-regulation: a systems view, Annual Review of Psychology, 44: 23-52.
Kvale, S. (2007). Contradictions of assessment for learning in institutions of higher learning. In D. Boud and N. Falchikov (eds) Rethinking Assessment for Higher Education: Learning for the Longer Term, London: Routledge, 57-71.
Wiggins, G. (1989) A true test: toward more authentic and equitable assessment, Phi Delta Kappan, 71, 9: 703-713.
Nendaz, M.R. and Tekian, A. (1999) Assessment in problem-based learning medical schools: a literature review, Teaching and Learning in Medicine, 11: 232–243.
Tan, K. (2007) Conceptions of self-assessment: what is needed for long-term learning?, in D. Boud and N. Falchikov (eds) Rethinking Assessment for Higher Education: Learning for the Longer Term, London: Routledge, 114-127.