Clinical anatomy e-cases: a five-year follow-up of learning analytics

Anatomy is one of the foundational subjects in biomedical sciences education1 and health science students spend a significant amount of time learning this subject. For the students to develop critical clinical judgment skills, learning anatomy often requires a clinically oriented problem-solving approach.9 However, inadequate resources such as cadaver dissection,3,4 non-availability of skilled tutors5 and inadequate time to cover wide range of concepts6 have drastically diminished anatomy teaching hours across many institutions.5–11 This reduction in classroom teaching time has driven university teachers to explore new ways that help students learn outside the traditional classroom setting.12 Such initiatives also include the exploration of online learning environments.13–16 While students exercise various degrees of self-directedness and take full responsibility of their learning in such non-formal learning environments,17 only their active interaction could promote knowledge retention.7,18–22 It should also be noted that there is no guarantee that the time students spend on online resources are of benefit to them.23 Although such self-directed learning is becoming a prominent learning environment in different domains, there is limited supplementary material available for supporting students in learning clinical anatomy.24–27

The departmental anatomy resource collection at the University of Otago lodges a considerable number of digital materials—three-dimensional atlases, e-books and videos for anatomy education. While most resources provide excellent information on gross anatomy, they are not targeted to meet the needs of the current curriculum at our university. Considering these limitations, a tailor-made web-based clinical anatomy learning resource (the clinical anatomy e-cases) was developed to supplement the Regional and Clinical Anatomy (RCA) block module for third-year medicine course in 2011. Following its first trial, and positive student feedback,28 subsequent e-cases have been developed and integrated into other anatomy modules, data from which were periodically extracted and evaluated over the years. The current study is a part of the ongoing wider project; the objective is to evaluate how well the students engaged with the supplementary learning resource and the extent to which they use and value the resource throughout their course.

Methods

Development of a clinical anatomy e-case

A typical clinical anatomy e-case begins with a short clinical presentation, followed by an exploration of the gross, surface and radiological anatomy and the anatomical basis of clinical procedures related to a particular disease condition. Each case is presented with progressive exposure of multiple interactive questions of different formats—true or false, matching, free hand drawing, labelling, short answers, etc. (Figure 1). Since the subject of anatomy is visually rich, animated movie clips and links to external resources were provided. A hint, reference or brief explanation is provided at the end of each question followed by a formative answer. Care has been taken not to overload each component with text or questions; and not exceeding a working time of 10–15 minutes.

Figure 1: Screen shot from an “Abdomen” e-case showing a labelling task on an E12 cross-sectional plastination slice (from the department of anatomy collection).

The instructional design process of the e-cases involved reviewing the laboratory manuals for identification of potential topics: clinically relevant anatomy topics related to general medical practice, scope for utilisation of audio-visual resources (angiography, ultrasound videos) and sectional anatomy tools (e-12 plastination slices, MRI) to enhance both student interaction and engagement. Further, emphasis was made on surface landmarks essential for physical examination and clinical procedures. The embryological basis of congenital anomalies was also introduced. Paper-based quizzes from the lab manual were removed and included into the e-cases, allowing more time for the students to get hands-on experience during the laboratory sessions. The cases were open to be accessed any time and any number of times during the academic year.

Distribution of the e-cases

The first version of clinical anatomy e-cases was developed (2011) using the free e-learning authoring tool CourseLab v2.7 (Websoft, Moscow, Russia). This was revised in 2013 and deployed on Moodle 2.7.2+ (Moodle HQ, Perth, Australia). Since 2011, 22 e-cases have been developed and introduced into the third year medicine anatomy course. In the head and neck section, two e-cases were distributed following each practical session whereas one e-case was provided in all the other sections (thorax, abdomen and pelvis) of the RCA module. All subsequent e-cases were distributed via the online learning system: Moodle. A formal feedback on the e-cases was obtained from the 2011 cohort of students, based on which further modifications on the resource were made.

Research design

This was an exploratory study aimed at examining how students engage with a series of interactive e-cases. The process involved introducing the clinical anatomy e-cases to all third year medical students as supplementary learning resources. The cases from the head and neck section were also open to second year dentistry course. Usage data from all e-cases was extracted from Moodle and analysed to ascertain user behavioural traits. The project was approved by the University of Otago human ethics committee.

Data sets and analytics

The following description is from the learning analytics based on students’ access and performances, extracted from the 2015 cohort of third year medicine course (n=282). The data was compared with analytics obtained from 2011 to 2015. The datasets generated from the user-usage analytics were:

1. Resource access analytics: number of visits by individual students
   
2. Repetition analytics: number and frequency of repeated accesses
   
3. Duration analytics: time taken to complete each case (in minutes)
   
4. Timeline analytics: access time of the day (across 24-hour period) and period of the week (across semesters)
   
5. Score analytics: A formative score to assess the performance and to compare them with their formative examinations
   

Feedbacks on the content and efficiency of the e-cases were obtained from the students during 1) A formal university’s Higher Education Development Centre course analysis in 2011, 2) The medical school’s periodic reviews in 2011 and 2014, 3) E-mail communications from students on the standard of the e-cases, suggestions to modify and requests to develop more e-cases.

Results

On average, each e-case was accessed 282 times at the end of the academic year. While 85.4% of those attempts were completed with a mean formative score of 82.8%, 14.6% of the attempts were left incomplete at some stage (Table 1).

Table 1: Quantitative variables analysed in the academic year 2015 (student number= 282). The values presented are the means calculated for a single e-case in the given section.

‍

During the academic year 2015, 73.3% of the class population (207 of 282 students) used the resource with 26.6% students involved in repeated usage, some in spite of scoring 100% in earlier attempts. Students solved the e-cases at shorter duration and obtained better formative scores during their repetitions when compared to their previous attempts. The number of repetitions was not related to the number of questions in the case or their formative scores. The e-cases were accessed most days of the academic year including holidays, exceptions being formative examination days, community events and long vacations (Figure 2).

Figure 2: A calendar timeline showing student participation across 2015. Spikes show increased participation every time a new e-case was uploaded (arrow—beginning of semester 2).

Although there were individual variations, the mean time taken to complete the cases was 14 minutes, which was within the proposed time range (10–15 minutes). There was a steady decline in the number of e-case accesses from first through the last case of each section. A steep rise in usage was observed during the last few days of the course, towards the university final examinations (Figure 3). The quantitative data collected from students during the early part of the study clearly showed that the e-cases were useful, short and not overloading the students’ routine study tasks. Student feedback also included requests for more e-cases for their 4th and 5th year of medicine course. Given the study was exploratory in nature, there was no attempt made at this stage to statistically explore any variations in resource usage patterns.

Figure 3: Increased utilisation of the e-cases resource during examination period (red) compared to the end of teaching period (blue) in 2015.

Highlights of observations in the e-case access patterns

1. The analytics showed resource access at three phases: before, during and after examinations, supporting a life long learning pattern.
   
2. Supplementary resources are utilised after classroom hours, weekends and holidays.
   
3. Increased engagement is seen towards examinations.
   
4. Student participation is maximal at the beginning of semesters, and towards examinations.
   

E-learning analytics from a five-year dataset

The user-usage data was obtained from Moodle at the end of each academic year from 2011 to 2015 and the usage pattern was analysed. This analysis showed a significant reduction in the number of students accessing the resource from 2011–2012, but a marked hike in usage was observed from 2013.

Discussion

Previous research has shown that supplementary learning resources of this kind are more effective when combined with conventional class teaching.26,29–31 For this reason, the clinical anatomy e-cases were designed and implemented in a manner suited to support the current teaching program, by augmenting the already existing standard of anatomy education in the university. This study explored the degree to which this is being achieved by examining student engagement with e-cases over a five-year period.

Overall results suggest that students have high engagement with these resources. The increased resource usage, repeated participation and improvement in formative scores offered a positive sign of student engagement with the resource. However, this positive sign could not be considered as a sole factor to determine the success of the educational approach.32 Data analysis within modules revealed 1) the frequency of access often decreased towards the end of each section, and 2) the sections with more e-cases (two cases distributed per practical session) showed a steeper fall in access rate than shorter sections. This would suggest that one case per practical session is adequate.

The results also showed that e-cases attempted but left incomplete were not contingent on time of day or the number of questions within the case. On the other hand, students re-accessing the cases throughout the year had often previously scored 100% (formative) in the cases. Earlier studies on student engagement showed the students’ interests dropping after 15–30 minutes33 and the varying interference of social media or procrastination on online engagement.34–37 Considering these factors, the e-cases were made short, which demanded a work time not more than 10–15 minutes. To maintain a stress-free learning environment, no attempt was made to provide a time limit to answer the questions; learning strictly at the students’ pace was encouraged. In spite of the ease of access and availability, not all the students utilised the resource, a finding that could relate to their lack of motivation and not necessarily the standards of the resource.29

For a supplementary learning material to be effectively used, it should not overload the students’ regular study practices. They should be well integrated into the study material and stimulated their interest towards the subject.28 Adapting these features, the clinical anatomy e-cases were seen as complimentary to the class requirements and served as a good revision tool towards examinations.

Five-year data follow up and comparison of analytics

In the year 2011, the project was started with 13 cases for thorax, abdomen and pelvis sections of the RCA module, and was later expanded to other modules and courses. This allowed a progressive expansion of the resource obtaining suggestions from colleagues and students.33 The variation and reduction in the access pattern observed from 2011 to 2012 might be explained on the basis of two factors—popularity (marketing the resource) and technical (browser compatibility). With 2011 as the first year of e-case development, presentation sessions outlining the importance of supplementary resource to cover the clinical material were provided during lab sessions (marketing), which reduced to a mere reminder in their lab manual in the following years. Also, the e-cases, which were initially developed using CourseLab software, had compatibility issues in most browsers (Chrome, Safari and Internet explorer, etc.) as Moodle was updated. This browser issue, coupled with decline in marketing, could have contributed to the decline of students’ usage of the material. When both issues were rectified in 2013, a marked improvement in the utilisation of e-cases was noticed.

The Moodle e-cases supported most browsers and provided more scope for extracting analytics.27,38 These e-cases were more user-friendly, compatible via smart phones and tablets, a more popular medium among doctors and medical students.39 Immediate feedback and individualised formative scores stimulated student interaction than the previous years. Fuelled with good marketing, the students’ access graphs reached considerable heights in a few days for each section. The academic year 2013 also viewed a change in the teaching staff, with three different lecturers handling the thorax, abdomen and pelvis modules, showing a variable level of usage across the module compared to earlier years (Figure 4).

Figure 4: Comparison of students’ participation across a five-year period (2011–2015). Student numbers are shown within brackets. (‘Head and neck’ and ‘nervous system’ e-cases were introduced from the years 2012 from 2014 respectively).

Limitations

While the online usage data and student feedback responses showed a positive effect, there were limitations highlighted. The resource usage analytics could have been aligned to student exam score. This would have provided useful information on the impact of the resources on the academic performance of the students. Qualitative data obtained possibly through interviews or focus group studies could have been included to offer a deeper understanding of the student’s perspective on the strengths and weaknesses of the resource. The usage data was collected from Moodle until the day of final university examinations, after which the third year medical students moved to different campuses for their clinical study; this was the major obstacle for interviewing the students. In addition, no attempt was made to divide the students into control and trial groups, thus restricting the resource availability to the controls only, as the main intention of the project was to ensure all students benefit from the resource.

Conclusion

The reduction in teaching time coupled with increasing teacher-student ratios can be mitigated through the use of quality supplementary online resources, provided they are aligned with the existing curriculum. The analysis of e-cases usage data showed students were interested in the supplementary material if it was modulated into short, self-paced sessions that were not compulsory. The annual timeline of resource usage also suggests that supplementary resources would become beneficial if introduced in the beginning of the semesters when students’ levels of engagement are high. Accepting these factors, offering students continuous open access to tailor-made interactive online resources that supplement their course work appears to be a useful and valuable addition to anatomy education.

‍