Improving pharmacy students’ reflective writing
This Writing Activity with Writing Analytics (WAWA) was developed as part of research conducted by Cherie Lucas (University of Technology Sydney, Graduate School of Health, Discipline of Pharmacy), in collaboration with the Academic Writing Analytics project at the UTS Connected Intelligence Centre.
![](/sites/default/files/styles/video_thumb/public/video_thumbnails/vl9fpKtbLfY.jpg?itok=pqbTUsDk)
Reflection is a graduate attribute that we have here at UTS and our students are required to reflect on their practice. Reflection is very hard to teach, and it’s difficult to learn, but it can be done. With the theoretical framework that underpins AcaWriter, it’s come from evidence. The tool works by using natural language processing techniques, which means that the algorithms behind the tool extracts certain pieces of information that is related to the human reflection response. And those extractions are tagged up, and then the feedback comes with that tagging. They’re highlighted in different colours and different shapes, and each tag relates to different areas of reflection so they can actually see what areas of reflection they fall down on. It is actually a good tool for them to see and visualise in front of themselves.
Now the output is actually very, very quick, it’s immediate output. Now, when you have students who are getting immediate feedback, it’s looking good. They can actually self-critique once they get that immediate feedback.
Basically, we’re going to go into AcaWriter and we’re going to copy and paste into this section over here, and we’ll get the feedback over here. There we go. There’s the feedback, it’s immediate feedback. As you can see there is a key that is situated over here, and these have different prompts. You can see there’s tags; there’s a square tag, there’s a circle tag, there’s bold writing.
You need a number of different learning modes when you’re dealing with lots of different students, and because the tags are very easy to recognise, and they’re colourful and there’s keys that protrude in this particular tool, that showcases different feedback mechanisms. I mean when you’ve got a number of different things going on at once you’re sort of targeting lots of different learning modes.
So, the different tags relate to the different elements of the reflective writing process. The blue tag, the square, looks at the initial thoughts, feelings, reactions about a significant experience. The circle tag looks at the challenges of new, surprising or unfamiliar idea or any problems, and the bold writing, for example, is the deeper reflection, personally applied. And ideally, towards the end, we are trying to aspire to have one of these triangles, which is a shift in a perspective and how new knowledge can lead to change in some particular way. There are lots of different things that the student can look at in terms of that sort of feedback.
Now you’ve got the remember reflections they’re quite personal, there’s subjective elements in the reflective process. But when I come summative assessment, there are objective elements. Those objective elements of reflection are transferrable across all the health disciplines and obviously across other disciplines, in business, in law, in engineering as well. Also, it is something they can use many, many times, on their own, have that self-critique immediately, even while they’re out on placement. I send students to rural placements, overseas placements, I have international placements, and they can still use this tool without the academic actually guiding them through the tool. Once they’ve got the facilities to use the tool, and it’s open-source here at UTS, they can use that tool as many times as they want.
As you can see, if there’s something there that they want to change they can come over this side and, in real time, change the writing on this side and then access another feedback report. We’re working closely to include more written feedback for each of the sections in the reflection so they so have some more written feedback and comments as well. And they can also download the PDF as well.
And like I said, this is a personal reflection to them, we don’t see it on the other side, they can access this as many time as they want. If they feel that it’s not really up to standard, we’re not looking at that at all. What we do see is what they submit at the end of the semester through another process. So this is for formative feedback, immediate formative feedback.
So when I introduced this to the pharmacy students, I gave them the option whether to use this program or not, and interestingly, they all used it, even after hours which is quite amazing, and they found it quite valuable. It worked well, and it’s not embedded in my curriculum. I’m finding it useful, the students are finding it useful, it’s an open source tool, it’s a no brainer.
![](/sites/default/files/styles/video_thumb/public/video_thumbnails/srsqzhZwfyo.jpg?itok=x_EsAtvz)
How to use AcaWriter - Cherie Lucas
Learning Design
Developed by: Cherie Lucas (School of Pharmacy, University of Technology Sydney)
License: Creative Commons BY-SA 4.0
Analytics Genre Profile
Reflective Writing (Standard)
This describes the Genre module in AcaWriter that has been developed to support this activity.
License: Creative Commons BY-SA 4.0
Summary: Highlights sentences that appear to show hallmarks of good academic reflective writing.
Developed by: Andrew Gibson, Adam Aitken, Rosalie Goldsmith, Simon Buckingham Shum (University of Technology Sydney), Ágnes Sándor (Naver Labs Europe)
Version: 1.0
Based on: None
Purpose: Highlights sentences that appear to show hallmarks of good reflective writing.
Textual Features: The model of reflective writing is a distillation of scholarship into the teaching of reflective writing (often about professional practice). This analytics draws attention to sentences that appear to demonstrate three key moves, signalled by icons:
![Reflective writing_markup](/sites/default/files/styles/wysiwyg_generic_large_x1/public/2022-05/Reflective%20writing_markup.jpg?itok=j6GkkaJR)
Feedback: The Feedback frame annotates the text using the above conventions, as illustrated in the screenshots below, illustrating how different annotations can be shown/hidden:
![Acawriter feedback tab](/sites/default/files/styles/wysiwyg_generic_large_x1/public/2019-06/feedback%20shot.png?itok=6UHkQ7X7)
Research
Project homepage: University of Technology Sydney, Connected Intelligence Centre, Academic Writing Analytics project
Focus: pharmacy case study
Lucas, C., Gibson, A. and Buckingham Shum, S. (In Press). Utilization of a novel online reflective learning tool for immediate formative feedback to assist pharmacy students’ reflective writing skills. American Journal of Pharmaceutical Education. https://doi.org/10.5688/ajpe6800
Focus: the revised model of reflection and its implementation:
Gibson, A., Aitken, A., Sándor, Á., Buckingham Shum, S., Tsingos-Lucas, C. and Knight, S. (2017). Reflective Writing Analytics for Actionable Feedback. Proceedings of LAK17: 7th International Conference on Learning Analytics & Knowledge, March 13-17, 2017, Vancouver, BC, Canada. (ACM Press). DOI: http://dx.doi.org/10.1145/3027385.3027436. [Preprint] [Replay]
Focus: the first design iterations of the reflective parser, leading to the Gibson et al. model
Buckingham Shum, S., Á. Sándor, R. Goldsmith, R. Bass and M. McWilliams (2017). Towards Reflective Writing Analytics: Rationale, Methodology and Preliminary Results. Journal of Learning Analytics, 4, (1), 58–84. http://dx.doi.org/10.18608/jla.2017.41.5