Information about the Project background
Students learn best when they have insight in their own study behavior and motivation and know how to self-regulate. Self-regulation means that students are actively involved in their study before, during and after learning. It includes study behaviors such as planning and goal setting, monitoring progress, and reflecting on learning and study behavior (see e.g. Zimmerman, 2002). The Thermos project gives students insight in their study behavior and motivation by means of Learning Analytics. Learning Analytics revolves around gathering data about learners and their learning processes to better understanding and where possible improve them. It is often defined as “the measurement, collection, analysis and reporting of data, about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Siemens & Baker, 2012). Presenting students with data about their study motivation and study behavior may help them improve on aspects they choose. However, simply gathering and visualizing data may not necessarily work, as interpreting and acting upon data (visualizations) is not always easy (Jivet et al., 2020). Implementing a Learning Analytics Dashboard (LAD) into a meaningful educational context can increase the value for learners (Wise et al., 2016).
Description
From academic year 18/19 to 20/21, the Thermos project developed a Learning Analytics Dashboard (LAD) to support students in their studies. This dashboard gives insight in several aspects of study behavior (e.g. motivation and group work skills) and study progress (e.g. ECTS). It also provides users with actionable feedback to engage with these aspects such as individual exercises, and additional referrals to (for example) Skills lab workshops or an online studycoach.
In academic year 20/21, the dashboard was piloted in 6 study programs from different faculties of UU and the UMCU. In this pilot we explored the dashboard’s usability and usefulness for students, and ways to implement it in a study program.
In the academic year 21/22 the follow-up project ‘Thermos for UU’ will start, which focusses more on the process of implementing the dashboard into tutorprograms. Within this follow-up project, the dashboard will be implemented in more study programs across both UU and UMCU.
Goals
The overall goal is to support students in improving aspects of study behavior by: helping them to assess important aspects of study behavior, to communicate students’ study behavior profile to provide them with insight, and to help them develop through actionable feedback.
The goals of ‘Thermos for UU’ are:
1) Further researching the process of implementation and development of implementation guidelines. These guidelines can be used by study programs who want to implement the dashboard in the future.
2) Further development of the dashboard itself, based on findings from the implementation.
3) Further integration of the dashboard in the UU-infrastructure, both content wise and technical.
Results
The dashboard was implemented in the tutorprogram of six studyprograms across UU and UMCU in academic year 20/21. This pilot evaluated the dashboard’s usability, usefulness, and the way in which it was implemented across study programs. Overall, the dashboard’s usability was already good, students understood and were able to interpret the different parts. As it was suggested that usability could be improved upon by developing a Dutch version of the dashboard, this will be available at the start of the Thermos for UU project. The dashboard’s experienced usefulness varied, with both positive (e.g. remarks such as “I learned more about my study behavior and some useful tips”) and less positive evaluations (“At this moment I’m not sure in which way the dashboard could help my study behavior”). One of the goals for the Thermos for UU project is further investigation of the dashboard’s usefulness and potential improvements. The implementation process varied across study programs, as well as the reasons for these differences. Based on interviews and a focusgroup, a concept implementation format was developed, aimed to help study programs make deliberate choices about the manner of implementation. Further investigation of the implementation process is the main aim for the Thermos for UU project.
Log-in to the dashboard at https://thermos.science.uu.nl