Engineering

The context for this review

This fully online technical subject relates theoretical principles to practical application by using case studies and uses continuous online assessment that has been intensively developed over several years. Although originally designed for postgraduates, the subject now caters to an extremely diverse range and dispersed cohort of students and is offered at undergraduate and pre-degree level and internationally in distance mode. Some prior consultation with an academic developer about redesigning it for online delivery had taken place. At the time of the review the university planned to share the subject with other institutions.

Review participants

The teacher in this case is an Engineering academic who had come to teaching from industry and had not been involved with peer review before.

The reviewer is an academic developer; originally from a different discipline who was already familiar with this subjects development.

Reason for the review was that this teacher wanted a formal summative review of his considerable long term development of the subject, particularly the online assessment, in order to make the rationale of this course design explicit as part of the course sharing project, as well as to provide evidence for a promotion application. Another interesting motivation for this review was that this teacher had longstanding concerns about using student feedback as evidence because he sees students as just one of the stakeholders in his practice-oriented discipline.

What happened in this review

Aspect of the subject that was focused on for this review was the continuous online formative & summative assessment developed by this academic over many years for different streams of students.

Forms of evidence reviewed along with the quizzes in the online environment the course documents, student evaluations, assessed quizzes and grade distribution were considered. The review was conducted after subject ‘delivery’ using the online records.

Review Process

Pre-review briefing included in this case early preliminary meetings place with the reviewer asking highly focused questions based on her previous visits to the subject site and making notes. The briefing for the review itself was done as an interview using the briefing guide with the reviewer asking for the teacher’s thoughts about the criteria and documenting these using her own adaptation of the template. These were finalised after the meeting and sent back to the teacher who added further comments.

The Review included considering the very extensive pre-review reflections by the reviewee alongside the online subject itself.  These and the review feedback from the reviewer can be seen in the review criteria document (used Feb 2009 version of the template, pdf 165kb). After completing the formal review this was documented in an ePortfolio and shared with the teacher

A debrief meeting was held face to face and this is where an Action Plan was developed.

Time taken to complete all parts of this review was approximately one day (half a day each for both reviewee and reviewer). The elapsed time between the initial conversations and the debriefing meeting was in this case extended over several months as part of an academic development and guided curriculum evaluation process

What were the outcomes of this review

1. An Action Plan that included an ongoing evaluation process using the actions suggested by the review. The value of a ‘guided’ review process introduced into the ongoing development and curriculum review and the benefits of a different perspective was strongly emphasised by this teacher and the reviewer. "It was really mostly about…what I want to achieve, what I want to check, how I can do this. It was very focused pedagogical type of questions and I had to scratch my head to give the answers…how I can achieve that and how I can validate what I want to achieve."
2. The teacher will work towards another summative review as part of gathering evidence about his teaching.
3. A promotion application will be considered for the next round review. The use of a scholarly Peer Review structure, and related materials, will enable this teacher to plan how to provide evidence for a future promotion application.

Points to note from this review

• Possible improvements were identified which the teacher could consider for the next phase of subject development (diversifying assessment; providing feedback from quizzes to students; drawing on students own experiences and use of discussion forums to support all students), "It’s very interesting to have someone outside a specialty and you have all sorts of questions to make you think. People from other professions see things differently."
• The considerable contribution of time and energy this teacher had invested, see for example the extensive comments the teacher included prior to the review, was identified and recognised by this review. The teacher realised the value of reflecting and documenting their intentions on an ongoing basis, as well as getting reviewers feedback.