Online examinations
Students find a number of benefits in online examinations. It is well known that many students suffer from anxiety when faced with large, end-of-semester exams. Sambell, Sambell and Sexton (1999) found that their students reported less exam anxiety when computer assessments were a component of continuous assessment throughout the semester as each test did not represent a high level of risk for the students and they were able to build their confidence in their abilities along the way. Students particularly appreciated the opportunity to do practice tests online, reporting that they assisted them in knowing what to study and they helped them to pace their revision so they did not just cram for the final exam.
Harvey and Mogey (1999) suggest the advantage of online assessment is the possibility to introduce more variables into the structure of the assessment pattern so that computer-based tests can be offered at different times, locations or even different tests to different students. As issues of security, access and equity, support and training for students are resolved summative online testing will be one more strategy for managing large volumes of marking and administration that has the potential to generate efficiencies that can ease academic workloads.
This guide is intended as a resource for UTS staff to become familiar with the issues behind online examinations. It draws from a number of recent studies on online assessment and on interviews with staff and students on the assessment practices in different sections of UTS. This guide presents information and examples on using question banks and student authentication. A second guide on online assessment deals with assessing online discussions. Strategies for setting up an online examination are listed at the end of this guide.
The idea that computers can automate marking is one of the attractions of putting assessments online. As attempts to grade student essays are still in their infancy (Shermis, Mzumara, Olson & Harrington, 2001), the major focus of online testing has been on multiple-choice questions (MCQ). MCQs have a long association with computer-assisted assessment and are relatively easy to convert to an online format. As with all MCQs it is the question design that makes multiple-choice tests effective. An important but time-consuming procedure in MCQ examinations is to produce a test that differentiates between levels of intellectual ability. These questions take more time to create but once produced they offer flexibility in the delivery of examinations and efficiency can be gained through a computer calculating and recording student marks. Pritchett (1999) cautions that even with careful question design it is difficult to assess the highest-level outcomes, like synthesis and evaluation, by using multiple-choice responses.
As with all assessment, MCQs should test clearly identified learning outcomes and be integrated into the course rather than treated as an afterthought. An advantage of using MCQs online is the ease in scheduling an examination once you have a pool of questions. Known as "question banks", these are a store of large numbers of questions in one subject area (The wall, 2000). For any particular test questions can be selected from the bank at random. Questions in the bank are often organised by level of difficulty or different values within a standard question randomised to create slightly different questions. Having randomised questions with the same structure and level but different content is considered to also deter cheating as there is no advantage by seeing another student's exam paper (Harvey and Mogey, 1999). For this to be effective the question bank needs to be large enough to prevent high levels of repetition.
Dealing with computer crashes
As a general rule a well maintained computer system is less likely to fail than one used only occasionally for online examinations. UTSOnline places a range of online assessment technologies within a single framework familiar to the students that is available to students 24 hours a day and has technical backup for the situation when an emergency does happen. Regardless of what computer system you use, you will need a contingency plan in case there are any technical faults in the middle of the examination period that may prevent student from completing the exam and disadvantaging their results.
The key to any plan is to keep the students and colleagues informed of what action you intend to take and what action they are expected to take. Depending on how long the problem exists Harvey and Mogey (1999) suggest some solutions to computer failures is to add the time missed to the end of the exam, reducing the number of questions or allocate students some marks in compensation. Zakzewski and Christine (2000) suggest ensuring there are spare machines available in the laboratory in case of machine failure so that a student can quickly be moved to a new machine. If the crash occurs later in the examination they recommend the student complete paper copies of the exam. They also propose that when there are multiple sessions scheduled the student can do the exam with another group or be referred to the Faculty's Appeals Committee for special consideration. Should the whole network fail the examination needs to be rescheduled.
Authentication and security
On the whole lecturers are still wary of setting examinations online. When the students are off-site their main concern is with student authentication. They want to be sure they know who exactly is 'pushing the buttons' in the exam. Even though there is no way to ensure the identity of a student at a remote assessment site the greater concern to many lecturers is how to ensure that students did not receive outside assistance to improve their test score. One approach is to make all off-site online examinations formative and open-book exams.
Where examinations are conducted in a laboratory environment it needs to be recognised that computer labs are rarely set up for testing. Wintington (1999) argues that all computers to be used under exam conditions need to be audited. Computers connected to the Internet are difficult to make secure and he cautions that occupational health and safety issues are particularly important in longer exams. If the timing of the exam is automated then consideration needs to be made for students with special needs. Finally, to maximise securing the questions in any summative examinations should only be stored online at the last possible moment.
