Improving sample text plus peer discussion (Civil Law)

This Writing Activity with Writing Analytics (WAWA) was developed as part of the doctoral research conducted by Shibani Antonette, as part of the Academic Writing Analytics project at the University of Technology Sydney’s Connected Intelligence Centre.

Academic writing is a key skill required for higher education students, which is often challenging to learn. A promising approach to help students develop this skill is the use of automated tools that provide formative feedback on writing. However, such tools are not widely adopted by students unless useful for their discipline-related writing, and embedded in the curriculum. This recognition motivates an increased emphasis in the field on aligning learning analytics applications with learning design, so that analytics-driven feedback is congruent with the pedagogy and assessment regime. The current design shows the implementation of a pedagogic intervention that was developed for law students to make use of the AcaWriter tool for improving their academic writing. By developing an effective learning analytics intervention design for writing instruction, I exemplify the integration of writing analytics technology into the classroom in authentic contexts. To augment automated feedback with human insights and improve the learning design, I’m also studying the impact of an additional peer discussion component in the design. The design can be implemented using a learning analytics tool which is developed to facilitate the intervention and provide analytic capabilities by collecting learner data. The validated design could be potentially transferrable across different writing instruction contexts with the development of standardized abstractions.

Learning Design

License: Creative Commons BY-SA 4.0 

Developed by: Shibani Antonette, Simon Knight, Simon Buckingham Shum and Philippa Ryan (University of Technology Sydney)

Learning design for the civil law context included a set of online writing tasks designed for students by embedding the use of AcaWriter as below:

 

 

 

 

 

 

 

 

 

The introduction video is a playlist consisting of few videos where the instructor who co-designed the tasks explains the motivation of this activity. It is available publicly on Youtube:

The whole intervention is facilitated by a tool called AWA-Tutor which acts as an online platform for the activities. Students are redirected to AcaWriter to use its automated feedback for their revision task. This intervention acts as an induction for students to understand the concept of rhetorical moves and how they can use AcaWriter to get feedback on their writing. After they complete the activity, students are provided with a Quick start guide to start using AcaWriter to get feedback on their own draft essays.

The AcaWriter Quick start guide is available for download below.

.

Analytics Genre Profile

This describes the Genre module in AcaWriter that has been developed to support this activity.

License: Creative Commons BY-SA 4.0 

Developed by: Shibani Antonette (University of Technology Sydney)

Version: 1.0

Based on: Analytical Writing (Standard)

Purpose: Highlights sentences that appear to show hallmarks of good academic writing for a UTS Civil Law essay and to provide specific feedback on possible improvements that can be made in that context.

Textual Features: The analytical report highlights rhetorical moves from the following list that AcaWriter identified in the text:

S:  Summarises or signals the author’s goals

P: Perspective or stance

E: Emphasis to highlight key ideas

N: Novel improvements in ideas

C: Contrasting idea, tension or critical insight

B: Background information and previous work

S: Surprising or unexpected finding

Q: Question or gap in previous knowledge

T: Trend or tendency related to ideas

These moves are tagged at the sentence level. There can be more than one rhetorical move in a sentence.

Feedback: Feedback consists of three tabs: Analytical Report, Feedback and Examples (see screenshot).

The Analytical Report tab shows highlighted moves for reflection:

The Feedback tab provides more specific feedback on missing rhetorical moves in a law essay context.  This tab first displays a cautionary message shown for all texts, followed by feedback on specific missing moves in the essay, mapping them to the Civil Law essay assessment criteria. The feedback also includes suggestions on possible improvements that can be made, e.g.

This is dynamically updated by checking for the moves whenever the student uses the Get Feedback button. Students may receive zero or more additional feedback messages depending on the rhetorical moves AcaWriter identified in their text.

The Examples tab provides examples of sentences from a law essay by mapping it to rhetorical moves identified by AcaWriter. This tab remains static to help students relate the rhetorical moves to their essay assessment criterion to better make use of the feedback from AcaWriter.

 

Research

Here’s what we’re learning, and more details on the rationale underpinning the design and implementation of (i) the analytics tools, and (ii) the student activities:

Antonette Shibani, Simon Knight, Simon Buckingham Shum and Philippa Ryan (2017). Design and Implementation of a Pedagogic Intervention Using Writing Analytics. In Proceedings of the 25th International Conference on Computers in Education. New Zealand: Asia-Pacific Society for Computers in Education.

Antonette Shibani (2018). AWA-Tutor: A Platform to Ground Automated Writing Feedback in Robust Learning Design (Demo). In Companion Proceedings of the Eighth International Conference on Learning Analytics & Knowledge (LAK ’18), Sydney, Australia.

Knight, S., Buckingham Shum, S., Ryan, P., Sándor, Á., & Wang, X. (2017). Academic Writing Analytics for Civil Law: Participatory Design Through Academic and Student Engagement. International Journal of Artificial Intelligence in Education, 28, (1), 1-28.