Automated marking in an online role play simulation
Keywords: simulation, authentic assessment, practice.
Faculty: Engineering and IT
Stuart Nettleton teaches finance to 300 engineering students. It is a core part of the engineering program in which students learn accounting and financial essentials for engineers, such as costing, net present value and investment mathematics. These can be extremely technical topics for a non-specialist core, and Stuart uses a version of the Harvard case study model to give his students difficult, real-world assignments that demonstrate the relevance of accounting for operating a successful business.
In the place of a formal lecture series, the subject has a series of assignment decisions due each week. The three hours of classroom time are used to inform the students’ assignment decisions as they play a competitive business simulation game. Students participate in a different type of teaching and learning activity each week, ranging from lectures and tutorials, video lectures with discussions of the videos and laboratory sessions to learn specialist accounting software. The video lectures cover key accounting principles that comprise about half of all content that is delivered.
The live lectures contain the anecdotes and discussions of complex topics that cannot be easily learned through videos, such as business ethics. All of the topics are designed to help students develop strategies for competing in the business simulation, which is the central assessment task in the subject.
Assessment activity
The business simulation requires students to work together in groups of three to run a digital camera company. Each week teams enter data into the simulation’s web site to tackle the same kind of business decisions that would be made by actual digital camera companies. Students need to apply the accounting principles they have learned in that week’s class into running their company as they borrow money, do R&D, decide to sack or hire people, or train staff so they are able to sell their cameras in different markets around the world. The teams need to record their decisions on the different camera components, production benchmarks, assets, liabilities and dividends they will pay their shareholders, all of which is calculated automatically by the simulation to give the team it overall score. The score is presented using standard industry measures for success, like earnings per share, return on investment, company stock price, and credit rating.
The online simulation is structured as a competition between companies run by other class members. All of the participating teams are ranked against each other and the students receive immediate feedback on the impact of their business decisions. The winners from each university are invited to participate in the international play-offs with 128 teams competing from around the world. The play-offs are not a requirement of the subject and invited teams can elect not to play, although UTS teams that have competed usually fall within the top 5 in the competition.
The students’ performance in the international business game is complimented by reflections on playing in the competition and their team performance that form part of a larger eportfolio of reflections. Each team has to create two business plans and a report on their direction for the company that includes all of their meeting minutes and planning documents. At the end of the simulation there is a company presentation by the winners of each industry group to share their business strategy with the rest of the class.
There is also a learning assurance report in which students show they have demonstrated leadership, strategic planning, self and peer reviews of their team skills that are linked to the faculty’s graduate attributes. The students complete two quizzes at the start and about half way through the session all of which goes into the eportfolio and the system automatically gives the percentage allocations to the assessment tasks, taking into account any late penalty and bonus marks to calculate the final mark for students.
Benefits for staff and students
The assignments the students do are holistic which allows everyone in a group to use the strengths they have in the assignment. At the same time the overall task still requires everyone to work as a team and when a problem arises the whole team needs to find a way to sort it out. The assignments are highly motivating because they are so domain based it gets them thinking about being in an engineering industry. The simulation cements all of the smaller pieces of the subject together so that it makes sense to the students. All of the information from the assignments is mapped into the graduate attributes in the eportfolio. Students receive direct and detailed feedback on their performance on every aspect of their graduate attributes.
Tips and advice
- There are always a small number of students who refuse to engage with the assessment tasks. It is important to reinforce to the students that the secret to passing the subject is to engage with assignments right from the start.
- A well functioning team is the key to success. Rigorous processes are required to ensure that everyone in the team is performing. Self and peer assessment of group processes helps to address issues before they impact on group performance.
- Most of the time was invested is setting up the ePortfolio system that records the student progress through the subject. Students don't want to learn a new system as well as play the game and the ePortfolio has to be simple for them to navigate. Provide students with 5-10 minute step-by-step instruction videos on how to use the system.
Photo by: Anna Zhu
