Figure 2 - uploaded by Martin Ebner
Content may be subject to copyright.
Source publication
As Learning Analytics (LA) in the higher education setting increasingly transitions from a field of research to an implemented matter of fact of the learner's experience, the demand of practical guidelines to support its development is rising. LA Policies bring together different perspectives, like the ethical and legal dimensions, into frameworks...
Context in source publication
Context 1
... at this point in the process, where the personal data from the LA service should be used for other purposes, it is necessary to look at this transformation: how to get from personal data to anonymous or aggregated data. Figure 2 depicts this process. The HEI, as a processor, already would have the stored data at its physical disposal. ...
Citations
... There is no sense of students as co-researchers, or as owners of the data harvested from their skin, their eye movements, their heart rates and perspiration, emotions, and postures. In the broader field of LA research there are glimpses of students as co-researchers, as controllers of their data and as having sovereignty over their data (eg, Gosch et al., 2021;Sun et al., 2019). ...
Since the emergence of learning analytics (LA) in 2011 as a distinct field of research and practice, multimodal learning analytics (MMLA), shares an interdisciplinary approach to research and practice with LA in its use of technology (eg, low cost sensors, wearable technologies), the use of artificial intelligence (AI) and machine learning (ML), and the provision of automated, mostly real‐time feedback to students and instructors. Much of MMLA takes place in experimental and laboratory settings, researching students' learning in in‐between spaces—between research and classroom application, in‐between students' learning in private and public spaces as researchers track students' learning both in their use of social media and connected devices, and through the use of context‐aware and adaptive devices; and lastly, in‐between respecting students' privacy while increasingly using intrusive technologies. This study seeks to establish what is known about MMLA in terms of rationale for applications, the nature and scope of data collected, the study contexts, evidence of commercial interests and/or downstream uses of students' data, and consideration of ethics, privacy, and the protection of student data. This systematic review analysed 192 articles using a search string consisting of various combinations of multimodal (data) and learning analytics. The main findings include, inter alia, that though MMLA provide insights into learning and teaching, there is little evidence of MMLA findings successfully being applied to classroom settings, at scale. Given that the nature of MMLA research often necessitates the use of a range of (intrusive) sensors and recording technologies and can include children in its samples; the encroachment of students' right to privacy is a huge concern that is not addressed. There is also a need to reconsider the rationale for collecting multimodal data, the conditions under which such data collection will be ethical and in service of students' wellness, and the boundaries that should protect their (multimodal) data.
Practitioner notes
What is already known about this topic Experimental educational research predates both multimodal learning analytics (MMLA) and educational data mining (EDM).
MMLA has been an integral part of learning analytics as research focus and practice since its inception.
The increased digitisation and datafication, advances in technology (eg, sensor technology, geo‐tracking, etc.) and a growing normalisation of wearable technologies provides greater scope for collecting multimodal data.
There is a vast body of published MMLA research providing a range of insight into students' and educators' behaviours aiming to increase the effectiveness of teaching.
What this paper adds While there are several studies providing insights into the state of MMLA research, this study provides insight to a selected range of factors in MMLA research such as the type of research (empirical/conceptual); the nature and scope of data collected; the sample populations (pre‐higher education, higher education, etc.); evidence of commercial interests or consideration of downstream uses; issues pertaining to consent, privacy, data protection and ethics; and evidence of how findings were used to improve teaching and learning.
The vast majority of MMLA research targets higher education, is empirical in nature and is based on relatively small samples of participation in experimental settings.
Confirms previous research that found the predominance of small samples, and a lack of replicability and, as a result, lacking scalability.
That there is very little explicit discussion of the ethical and privacy implications and data protection, either at design stage or for future implementation.
Similarly, that there is little consideration of potential commercial interests or downstream uses of data.
Implications for practice and/or policy MMLA in its essence requires interdisciplinary approaches and teams.
For MMLA to move beyond small‐scale, experimental settings to application in real (classroom) settings, larger, replicable studies should be conducted together with ways to make study findings actionable for teachers and students.
Ethical issues, commercial interests and downstream uses of collected data must be considered within the design and approval of MMLA research.
... The first limitation ("freely given") immediately causes problems due to the power imbalance between students and their HEIs: students are inclined to say yes to decisions put forward by their HEI, since getting their degree depends on it [78]. At the very least, this makes the use of consent within the context of education highly questionable [79]. To truly make a decision for online examination with proctoring "freely given", a viable (offline) alternative must be offered to students [25]. ...
... Moreover, we consider that higher education institutions should offer LA technology as a service that students can decide to use or not. Also, students should control their data instead of assuming the role of data subjects (Gosch et al. 2021). This paradigm shift empowers students and amplifies students' responsibility to use LA services and manage their data. ...
Human-centred design is a well-established approach within research fields such as human-computer interaction, ergonomics, and human factors. Recently Learning Analytics (LA) researchers and practitioners have manifested great interest in exploring methods and techniques associated with this approach to manage the design process in ways that can enhance human interaction with LA technology. The project “Learning Analytics – Students in Focus” aims to use student-related data to support the learning and teaching process in a higher educational context. Our interdisciplinary team investigates LA tools that leverage students’ academic success by acquiring or developing self-regulated learning skills. We adopted a Human-Centred Learning Analytics (HCLA) approach involving students, teachers, and other educational stakeholders in the iterative design of our LA tools. This article contributes to the discussion on how to design LA tools using a human-centred approach. We describe the analysis, design, implementation, and evaluation process of three LA tools comprised in our students’ dashboard, i.e., the planner, the activity graph, and the learning diary. In addition, we present key results gained in several empirical studies which had an implication on the tools’ design. Finally, we provide insights about our experience with the HCLA approach, pointing out benefits and limitations in practice.KeywordsHuman-centred learning analyticsSelf-regulated learningLearning analytics dashboard
... We proposed a normative framework in the form of an interdisciplinary Criteria Catalog that guided our design choices to build trustworthy LA tools [14]. We also questioned if asking for consent before starting to use LA is enough and explored LA being offered as a service in higher education institutions rather than an intervention [5]. Community building. ...
Learning Analytics (LA) researchers and practitioners are growing interested in applying human-centred design methods and techniques to design LA technology. This approach finds solutions by involving the perspectives of students, teachers, and other educational stakeholders in all process steps. It enables the creation of technology that resonates and is tailored to the end-users needs. The "Learning Analytics-Students in Focus" project aims to support the learning and teaching process in the higher education context. Our interdisciplinary team focuses on LA technology that facilitates acquiring and developing students' self-regulated learning skills, such as goal setting, planning, monitoring progress, and reflecting. We embraced a Human-Centred Learning Analytics (HCLA) approach since the start of our project, and it helped us to understand students' points of view and needs and find solutions together. This article summarises the design process of a LA tool named Planner, which aims to support students in planning and monitoring coursework. We share our experience with various methods and techniques applied in our research and present insights about the benefits and limitations of the HCLA approach. Finally, we highlight how the HCLA approach helped to build a LA community at our university and promote trust towards LA.
... HEIs must facilitate and promote the responsible use of new educational technologies, such as LA. However, HEIs should offer learning analytics as a purely voluntary service [8], which teachers and students can adopt to enhance the teaching and learning processes. In this context, HEIs are responsible for offering further training measures for teachers and providing didactic support to ensure the pedagogically meaningful use of LA data. ...
Learning analytics (LA) is an emerging field of science due to its great potential to better understand, support and improve the learning and teaching process. Many higher education institutions (HEIs) have already included LA in their digitalisation strategies. This process has been additionally accelerated during the COVID-19 pandemic when HEIs transitioned from face-2-face learning environments to hybrid and e-learning environments and entirely relied on technology to continue operating. Undoubtedly, there was never a time when so much student data was collected, analysed, and reported, which brings numerous ethical and data protection concerns to the forefront. For example, a critical issue when implementing LA is to determine which data should be processed to fulfil pedagogical purposes while making sure that LA is in line with ethical principles and data protection law, such as the European General Data Protection Regulation (GDPR). This article contributes to the discussion on how to design LA applications that are not only useful and innovative but also trustworthy and enable higher education learners to make data-informed decisions about their learning process. For that purpose, we first present the idea and methodology behind the development of our interdisciplinary Criteria Catalogue for trustworthy LA applications intended for students. The Criteria Catalogue is a new normative framework that supports students to assess the trustworthiness of LA applications. It consists of seven defined Core Areas (i.e., autonomy, protection, respect, non-discrimination, responsibility and accountability, transparency, and privacy and good data governance) and corresponding criteria and indicators. Next, we apply this normative framework to learning diaries as a specific LA application. Our goal is to demonstrate how ethical and legal aspects could be translated into specific recommendations and design implications that should accompany the whole lifecycle of LA applications.
... To do so, this study addresses one of the data subject stakeholders in LA, namely students, whose direct engagement in the design of LA services is critical to their improved learning in education settings (Ochoa & Wise, 2021). Engaging them as data subjects, especially regarding a thorough understanding of their concerns about LA privacy implications, is an essential first step toward the development of effective privacy-enhancing practices that would protect students and ultimately empower them by enabling their agency in LA (Ahn et al., 2021;Gosch et al., 2021;Roberts et al., 2016;Tsai et al., 2020). ...
... This is important to the successful implementation of LA systems that are directly dependent on student data (Li et al., 2021). Accordingly, higher education institutions need to further develop and adopt effective privacy-enhancing student practices and management strategies (eg, through relevant policy work) that mitigate students' perceptions of privacy risks and enhance perceptions of their ability to control their personal information, which will empower them as data controllers as compared to data subjects (Gosch et al., 2021). ...
Understanding students' privacy concerns is an essential first step toward effective privacy‐enhancing practices in learning analytics (LA). In this study, we develop and validate a model to explore the students' privacy concerns (SPICE) regarding LA practice in higher education. The SPICE model considers privacy concerns as a central construct between two antecedents—perceived privacy risk and perceived privacy control, and two outcomes—trusting beliefs and non‐self‐disclosure behaviours. To validate the model, data through an online survey were collected, and 132 students from three Swedish universities participated in the study. Partial least square results show that the model accounts for high variance in privacy concerns, trusting beliefs, and non‐self‐disclosure behaviours. They also illustrate that students' perceived privacy risk is a firm predictor of their privacy concerns. The students' privacy concerns and perceived privacy risk were found to affect their non‐self‐disclosure behaviours. Finally, the results show that the students' perceptions of privacy control and privacy risks determine their trusting beliefs. The study results contribute to understand the relationships between students' privacy concerns, trust and non‐self‐disclosure behaviours in the LA context. A set of relevant implications for LA systems' design and privacy‐enhancing practices' development in higher education is offered.
Practitioner notes
What is already known about this topic Addressing students' privacy is critical for large‐scale learning analytics (LA) implementation.
Understanding students' privacy concerns is an essential first step to developing effective privacy‐enhancing practices in LA.
Several conceptual, not empirically validated frameworks focus on ethics and privacy in LA.
What this paper adds The paper offers a validated model to explore the nature of students' privacy concerns in LA in higher education.
It provides an enhanced theoretical understanding of the relationship between privacy concerns, trust and self‐disclosure behaviour in the LA context of higher education.
It offers a set of relevant implications for LA researchers and practitioners.
Implications for practice and/or policy Students' perceptions of privacy risks and privacy control are antecedents of students' privacy concerns, trust in the higher education institution and the willingness to share personal information.
Enhancing students' perceptions of privacy control and reducing perceptions of privacy risks are essential for LA adoption and success.
Contextual factors that may influence students' privacy concerns should be considered.
... There is no sense of students as co-researchers, or as owners of the data harvested from their skin, their eye movements, their heart rates and perspiration, emotions, and postures. In the broader field of LA research there are glimpses of students as co-researchers, as controllers of their data and as having sovereignty over their data (eg, Gosch et al., 2021;Sun et al., 2019). ...
This book explores and further expands on the rich history of theoretical and empirical research in open and distributed learning, and addresses the impact of the “data revolution” and the emergence of learning analytics on this increasingly diverse form of educational delivery. Following an introductory chapter that maps the book’s conceptual rationale, the book discusses the potential, challenges and practices of learning analytics in various open and distributed contexts. A concluding chapter briefly summarises the chapters before providing a tentative future research agenda for learning analytics in open and distributed environments.
... As the students' needs vary, the way the distance learning courses are offered should be adapted to these needs and also to the students' prior knowledge. In conventional education, the instructors and the auxiliary staff, guide and assist the students through the learning procedure by using different types of learning material [1][2][3]. In distance learning, the students schedule their study time by themselves and the successful completion of each course depends on the students' effort but also on additional parameters, like their educational background, the psychological support of their family, and their job obligations. ...
Educational Data Mining has turned into an effective technique for revealing relationships hidden in educational data and predicting students’ learning outcomes. One can analyze data extracted from the students’ activity, educational and social behavior, and academic background. The outcomes which are produced are, the following: A personalized learning procedure, a feasible engagement with students’ behavior, a predictable interaction of the students with the learning processes and data. In the current work, we apply several supervised methods aiming at predicting the students’ academic performance. We prove that the use of the default parameters of learning algorithms on a voting generalization procedure of the three most accurate classifiers, can produce better results than any single tuned learning algorithm.
The digitalisation of learning, teaching, and study processes has a major impact on possible evaluations and uses of data, for example with regard to individual learning recommendations, prognosis, or assessments. This also gives rise to ethical issues centered around digital teaching and possible challenges of data use. One possible approach to this challenge might be to install a Digital Ethics Officer (DEO), whose future profile this paper outlines for a Educational Technology unit of a Higher Education Institution (HEI). Therefore, an introductory overview of the tasks and roles of Ethics Officers (EO) is given based on the literature. The authors then describe the current ethics program of a university of technology and collect current and potential ethical issues from the field of educational technologies. Based on this, a first professional profile for a DEO at an educational technology unit of a university is described. From the authors’ point of view, the article thus prepares important considerations and steps for the future of this position.
While learning analytics frameworks precede the official launch of learning analytics in 2011, there has been a proliferation of learning analytics frameworks since. This systematic review of learning analytics frameworks between 2011 and 2021 in three databases resulted in an initial corpus of 268 articles and conference proceeding papers based on the occurrence of "learning analytics" and "framework" in titles, keywords and abstracts. The final corpus of 46 frameworks were analysed using a coding scheme derived from purposefully selected learning analytics frameworks. The results found that learning analytics frameworks share a number of elements and characteristics such as source, development and application focus, a form of representation, data sources and types, focus and context. Less than half of the frameworks consider student data privacy and ethics. Finally, while design and process elements of these frameworks may be transferable and scalable to other contexts, users in different contexts will be best-placed to determine their transferability/scalability.
