Communications

The context for this review

This review is of a blended second year (or elective) subject, taken by a mixture of undergraduate and postgraduate students from across different faculties in the university. In this blended subject lectures and tutorials are delivered mainly face to face, supplemented by some virtual classroom work with class discussions and two of the three assignments involving intensive online work by the students. The teacher wanted to address questions about the assessment of students’ online work which she had been developing and improving over several years. The review focused on the strategies used to encourage greater student directed and managed participation in the online wikis that were part of an assignment. This is the review that is the subject of the following case study.

This case study is an example of how useful PR of assessment can be. The review process developed into four rounds of reviewing with three reviewers. When this review was completed it was decided to continue the process by doing a second review of the same subject, by the same reviewer. The second review went on to evaluate the team based online work and class discussions and the individual reflective assignments that followed this task. A third round of review was done by the tutor in the subject and a fourth cross-disciplinary review cycle was undertaken with a colleague from the health sciences faculty who also taught in blended mode. After completing reciprocal reviews this pair plan to do further reviews together. This review developed an ongoing review process of discussing and reviewing teaching. This case study summarises the first round of review.

Review participants

The teacher in this case is had been internationally recognised for this innovative subject. After completing two rounds of peer review she successfully applied for a national teaching citation.

The reviewer had been part of the teaching team for this subject in previous semesters and was also part of the project team, therefore time was available to converse with the reviewee and to document what emerged.

Reason for the review for this teacher one of the key motivations for engaging in peer review was to provide a way of triangulating the student survey results from this innovative blended subject. Another was concern about the sustainability of the assessments from the point of view of the teacher, and related to this succession planning for this subject in future years. The review provided a way to document aspects of this subject that had previously been implicit.

What happened in this review

Aspect of the subject focused on for this review was the student directed and managed online teamwork to create resources and then facilitate online class discussions.

Forms of evidence reviewed were course documents and the online resources and discussions created by students in the learning management system, as well as the structure of this overall site by the teacher.

Pre-review briefing was done face to face using project documentation

Review Process

These Reviews were extensive pre-review reflections by the reviewee and review comments by reviewer can be seen on the review criteria document (pdf 139kb)

Debrief meetings took place as several face to face meetings before or after other events with additional follow up email and exchanges of documents that incorporated further questions and answers and reflections.

Time the total time taken to complete this initial review was one day. The pre-review time was (thinking and completing questions) 1.5 hours and then 1 hour for the briefing meetings. The review itself took approximately 3 hours (iterating between ‘visiting’ site; subject outline; tutorial handouts and noting feedback against the criteria). A debriefing meeting took 1 hour with emails before and after to clarify questions from the reviewer. Documentation time included finalising the feedback for each of the criteria and then summarising this into the summative review template (pdf 184kb) took about 2.5 hours in total.

Outcomes of the review

The next steps for this teacher were to implement the reviewer’s main suggestion, by aligning recent changes made to the weekly delivery of the program and the assignments with all of the documentation associated with the subject, so that each was an exact reflection of the other.
A request for a further round of review with the same reviewer, in order to investigate an assignment closely linked to the online activities, using the outcomes available now the semester had ended (eg student assignments, feedback and grades). The teacher had concerns that her intentions for assessment (and the subject) were clear to the students and thus evident in their assignment work. An extended communication case study (pdf 496kb) describes these two stages of review and use of this projects framework and resources.
Planning a third round of review for the next delivery of the subject, this time by the new tutor who would be taking a greater a role with the students. A continuing Action Plan would be created based on those findings.

Points to note from this review

This review is very detailed as the reviewee wanted to solve specific issues so that she could hand over more of the subject to other teachers and she wanted to investigate these subject in some depth. The reviewer was available to do this.
This was also a very comfortable review pair who were used to working productively together over many years.
This case study is particularly interesting because it shows the value of the formative process of Peer Review in a fully blended subject for the teaching practice of experienced and innovative teachers.
It also demonstrates that the same review can be used for an ultimately summative purpose. Teachers may choose to engage in a review process that is formative and at the same time has a summative final goal.
The process of review is often ongoing, continuing beyond the expected trajectory of a the project or other initiative

More from the Reviewee

"I have found it invaluable because it’s provided me with a real sense of empowerment and that’s why I really use that word. Because until going through that review process I never felt confident… But also to see the comments made by the reviewer about where there was room for improvement. So those things where I struggled to work out how to make changes, or you have a sense that there’s something that isn’t quite right, but you can’t put your finger on it…it’s been a wonderful foil for ongoing reflection about what’s happening. So it’s an articulation process."

"I struggle to see a disadvantage…You know - I mean it's all win/win for the reviewee…I’m generally a reflective practitioner. So the sort of thing that I was asked to reflect on was no different from what I’ve done all the time anyway anyway. So in that case there’s no extra work as far as I could see."

"Work out ahead of time how much time you can commit and more importantly how much time the reviewer can commit to look at this."

"You’re always asked to show impact or to show how do you know you’ve done a good job this year? Well you know here, this subject that I’m developing has been reviewed by someone other than me. I think it has helped in terms of negotiating workload…Yes and it’s been useful in terms of trying to make arguments about designing subjects, or speaking with a certain authority about ideas that I have that I want to implement in the faculty…directions I feel we should be taking but don’t always get listened to."

"I think what I like about the peer review process the way that this is set up as opposed to the kind of feedback that you get through SFS is that you have that conversation going back and forth. So that you have a peer reviewer who says here are the things and it’s documented…Then you have the space and in fact I like the fact that the documentation encourages you to talk about what you might do. How might you address these or what might be your priorities? It’s about showing that you are constantly improving. It’s a lot easier to demonstrate that in the peer review process than it is to have student feedback where numbers rule."

"One of the things that became really clear is the process of being engaged in the peer review and writing. So articulating that in a way that documents it on paper meant that it was a lot easier to show evidence of impact than it has been in the past…It’s really important because a lot of times that is where - I speak only for myself but that’s where I struggle. So that there’s a lot that’s like around in my head but actually articulating that and putting that into words is really tricky…it takes a long time. That’s why I’d hate to not have that as something that you do. But also the reality is that you can only do that so many times."

"You know that whole movement from beyond spoken into the articulated that is an incredibly painful, arduous, but essential part of the process. You can’t really communicate unless you do that…You can’t document for yourself. You can’t document for anyone else until you go through that…knowing the value that I got out of that and that the documentation has been able to provide me in a number of different contexts which is why I was really happy...For instance one of the things that I also had in the back of my mind was the amazing value of my peer review, which continues to serve a purpose because I now have to write a entirely new subject."

More from the Reviewer

“This experience of PR has highlighted the difficulty of keeping a balance between consideration of the subject as whole with the aspect under review (whole versus parts), especially in a BLE. Using a framework to structure the review was important to support and guide consideration of the aspects of learning and teaching that may be ‘hidden’.”

“Being a reviewer you are the foil that provides that clarity, it’s the questions that you ask that prompt the teacher to start to