Figure 1 - uploaded by Rhonda R Christensen
Content may be subject to copyright.
Conceptual assessment framework for analysis of user actions 

Conceptual assessment framework for analysis of user actions 

Source publication
Article
Full-text available
A rationale for using a simulated teaching environment to train pre-service teacher candidates is presented, followed by the key components of the simSchool dynamic simulator created to accomplish this task. Results of analyses of two sets of data, for the areas of pedagogical practices and teaching skills, are used to illustrate that changes in pr...

Contexts in source publication

Context 1
... modelling paradigm in simSchool works by computing a time series evolution of the classroom as a system. This modelling allows novel dynamics to evolve moment by moment as the user, a teacher candidate, makes decisions. simSchool promotes thinking on one's feet because class time waits for no one. The experimental logic model framework (Figure 1) is also relevant in simSchool but instead of each state of the system simSchool: an online dynamic simulator for enhancing teacher preparation 205 waiting upon a user's action as in a customer satisfaction degree (CSD) model, in simSchool, the classroom evolves whether or not the teacher takes actions. The dynamic modelling approach uses initial conditions, attractors, and multiple layers of dynamic interactions to simulate learning by individuals in a ...
Context 2
... a result of the dynamic modelling approach, there is a continuous production of moment-by-moment evidence of what the teacher candidate is attempting to do as he or she 'teaches' the class. A conceptual assessment framework ( Mislevy et al., 2003) guides the analysis of that evidence so that inferences about the growth and development of teaching skill can be made based on the evidence of 'game play' in simSchool (Figure 1). The task model as well as the student model has been presented as 'landscapes' of factors that are changing over time. The evidence model is comprised of the actions that each pre-service teacher uses while playing simSchool, as well as the analysis we bring to understanding the teachers intention and their actions' impacts on the classroom. The cognitive model of the student in simSchool is built around a three-tiered model of the physical, emotional and academic performance variables of learning as explained above. No other hidden variables exist, so all of the effects of the user's decisions are directly attributable to interaction effects, as opposed to randomly generated settings as in CSD model. The down side of this approach is the extra cost in forming an analysis, because for any particular resulting end condition at any point in time, all of the previous actions have had some causative impact. Luckily, a simple visual interface can directly present the results of the pre-service teacher's actions on the simulated students for reflection and summation ( Figure 2) where a quantitative analysis is sometimes less helpful. ...

Similar publications

Article
Full-text available
simSchool, an online simulator that has been used to enhance teacher preparation since 2003, models different types of students and provides virtual practice sessions for teachers to assign tasks and interact with students. In this article the authors (a) examine changes in preservice teacher perceptions of teaching confidence and teaching experien...

Citations

... Another advantage is that there is no risk of affecting real students negatively (Foley & McAllister, 2005). Studies have shown that the use of a simulation-based learning approach in teacher education increases PTs' self-efficacy and improves their communication skills, classroom management, and lesson-planning skills (Badiee & Kaufman, 2014;Bautista & Boone, 2015;Chernikova et al., 2020;Christensen et al., 2011;Dawson, 2016;Levin et al., 2023;Mahon et al., 2010). Furthermore, virtual classroom simulations enhance the effective use of technology for instructional proposes (Aksoy & Kelleci, 2023;Bayer & Öner, 2022;Polikarpus et al., 2023), a process that is known as Technological Pedagogical Content Knowledge (TPACK) (Koehler & Mishra, 2009). ...
Article
Full-text available
Before taking full responsibility for a real classroom, pre-service teachers (PTs) can safely experience a rich learning atmosphere in a simulated virtual classroom environment and receive feedback on their lesson planning and teaching performance. This research aims to support and examine the process of structuring PTs’ teaching skills with a multi-source feedback system integrated with simulation-based teacher training (SimInTeach-FBS). A phenomenological approach was employed in order to understand the nature of PT’s experiences from a feedback system focusing on teaching skills. Seventy PTs, twelve mentor teachers (MTs), and six teacher educators (TEs) participated in the study. Various data collection tools were used to gain a deeper understanding of the participants' experiences: reflective journals, open-ended feedback forms, and observation notes. The results of the content analysis showed that teaching experience with SimInTeach-FBS raises self-awareness and improves PTs’ teaching skills. It was revealed that teaching skills were mutually structured, feedback given to the PTs multiplied the effect of the development process, and teaching experience facilitated the transfer of acquired teaching skills to real life.
... For example, SimSchool incorporates the OCEAN model of personality (e.g. openness, conscientiousness) in their virtual students (Christensen et al., 2011). Their virtual students' personalities can be randomly generated or customised according to the OCEAN model. ...
... Digitale Simulationen kommen bereits in verschiedenen Kontexten in der Lehrkräftebildung (Christensen et al., 2011;Sullivan et al., 2020) zum Einsatz. Sie wirken positiv auf die Selbstwirksamkeit sowie internale Kontrollüberzeugungen angehender Lehrkräfte (Christensen et al., 2011). ...
... Digitale Simulationen kommen bereits in verschiedenen Kontexten in der Lehrkräftebildung (Christensen et al., 2011;Sullivan et al., 2020) zum Einsatz. Sie wirken positiv auf die Selbstwirksamkeit sowie internale Kontrollüberzeugungen angehender Lehrkräfte (Christensen et al., 2011). Insgesamt besteht jedoch noch ein großer Bedarf an Kontrollgruppenstudien. Dieser Beitrag stellt eine digitale, problemorientierte Simulation zur Förderung diagnostischer Kompetenz von Lehramtsstudierenden vor und vergleicht sie mit einem textbasierten, problemorientierten sowie einem lehrendenzentrierten Setting. ...
Article
Die Förderung diagnostischer Kompetenz in der Lehrer*innenbildung birgt zwei Herausforderungen: Erstens bleibt in großen Lehrveranstaltungen wenig Raum für die notwendige Anwendung theoretischen Wissens, zweitens können Fehlentscheidungen in realen diagnostischen Situationen gravierende negative Folgen für Schüler*innen haben. Eine digitale Simulation kann beiden Herausforderungen begegnen. In diesem Beitrag wurde eine digitale, problemorientierte Simulation zur Förderung diagnostischer Kompetenz von Lehramtsstudierenden evaluiert und mit einer textbasierten, problemorientierten und einer lehrendenzentrierten Kontrollgruppe verglichen. Die Ergebnisse zeigen positive Effekte für die digitale Simulation hinsichtlich der selbstbestimmten Motivation, aber einen geringeren Wissenszuwachs im Vergleich zum lehrendenzentrierten Setting. Die Selbstwirksamkeit nahm in allen drei Gruppen zu. Eine digitale Simulation bietet in der untersuchten Form somit einen, wenn auch geringen, Mehrwert für die Lehrkräftebildung im Bereich der Kompetenzentwicklung.
... As Theelen et al. (2019) note in their systematic literature review, simulations contribute to the professional development of future teachers in bridging the gap between teacher education and educational practice. Students may apply the skills they have, or try out new teaching strategies (Rayner & Fluck, 2014), and they may enhance their sense of instructional self-efficacy more rapidly (Bautista & Boone, 2005;Christensen et al., 2011). Simulations also have considerable potential in the teaching of subject literacy to both novice and experienced teachers (Ferguson, 2017;Kelleci & Aksoy, 2020;Sahi, 2019). ...
... Simulations also have considerable potential in the teaching of subject literacy to both novice and experienced teachers (Ferguson, 2017;Kelleci & Aksoy, 2020;Sahi, 2019). In particular, they provide a safe environment in which students may practise their role as teachers (Dalgarno et al., 2016;Rayner & Fluck, 2014), and serve as catalysts for reflection on and discussion about classroom teaching practice (Christensen et al., 2011). Experiencing virtual pupils acting in unpredictable ways, and noticing the impact of subtle changes teachers make during lessons, could raise awareness among student teachers of the complexity in the classroom (Dalgarno et al., 2016;Ferry et al, 2005). ...
... Many of the reflections were positively related to a) having an overview of the alternatives and b) being able to go back and forth among them, both of which stimulated reflection and encouraged the students to explore didactic choices from a teaching perspective. In the words of Christensen et al. (2011), VES acts as a catalyst for reflection and discussion on classroom teaching practice. It also functioned as a safe place between theoretical studies and practice, where students could try things out before going into the practice period (see also Dalgarno et al., 2016;Rayner & Fluck, 2014). ...
... In der Forschung zu digitalen Simulationen zeichnet sich ein heterogenes Bild ab, das von äußerst unterschiedlich gestalteten Simulationen, vielen qualitativ-explorativen und eher wenigen Prä-Post-Kontrollgruppenstudien geprägt ist. Diverse Forschungsergebnisse zeigen jedoch, dass sich digitale Simulationen insbesondere auf motivationaler Ebene positiv auswirken können: Sie wecken Interesse (Bartel & Roth, 2020) und unterstützen die Selbstwirksamkeit (Christensen et al., 2011;McPherson et al., 2011). Außerdem sind sie authentisch (Stürmer et al., 2021;Thompson et al., 2019) und fördern das Selbstvertrauen in Bezug auf die Trainingsbereiche der Simulation (Gebreheat et al., 2022;Girod & Girod, 2006;Kaka et al., 2021). ...
Thesis
Full-text available
Um Schüler*innenmerkmale korrekt zu beurteilen benötigen Lehrkräfte diagnostische Kompetenz. Insbesondere die genaue Einschätzung nicht-kognitiver Merkmale von Schüler*innen fällt Lehrkräften eher schwer. Dazu zählen auch mögliche Anzeichen für Lern- und Verhaltensauffälligkeiten. Maßnahmen zur Förderung diagnostischer Kompetenz fokussieren bisher größtenteils die Diagnostik fachspezifischer, situativer Urteilsgegenstände, wie Fehlkonzepte im Mathematikunterricht. Systematisch evaluierte Fördermaßnahmen, die das Erkennen und Beurteilen von Lern- und Verhaltensauffälligkeiten bei Schüler*innen fokussieren, existieren bisher nicht. Aus diesem Anlass wurde die digitale, problemorientierte Simulation GEProS zur Förderung diagnostischer Kompetenz von Lehramtsstudierenden entwickelt und systematisch in drei längsschnittlichen, (quasi-)experimentellen Kontrollgruppenstudien evaluiert. Die Ergebnisse zeigen, dass die GEProS einen Mehrwert für die Förderung diagnostischer Kompetenz insbesondere im Bereich selbstbestimmter Motivation bieten kann. Ein integriertes Quiz mit Feedback kann mittels testing effect darüber hinaus der Erwerb deklarativ-konzeptionellen Wissens als zentrale Disposition diagnostischer Kompetenz in GEProS unterstützen. Die Ergebnisse tragen durch das entwickelte Integrative Modell diagnostischer Kompetenz zur Theoriebildung sowie zur Forschung und Entwicklung im Bereich der Förderung diagnostischer Kompetenz bei, indem eine Maßnahme zur Förderung diagnostischer Kompetenz für einen fächerübergreifenden, zeitlich stabilen Urteilsgegenstand systematisch evaluiert wurde. Durch die Ableitung und Berücksichtigung wesentlicher Merkmale digitaler Simulationen trägt diese Arbeit außerdem zur theoretischen Fundierung digitaler Simulationen bei. In order to accurately assess student characteristics, teachers need diagnostic competence. Especially the accurate assessment of non-cognitive student characteristics tends to be difficult for teachers. These can also include possible signs of learning and behavioral problems. Measures to promote diagnostic competence have so far largely focused on the diagnosis of subject-specific and temporary objects of judgment, such as misconceptions in mathematics education. So far, there are no systematically evaluated measures that focus on the recognition and assessment of learning and behavioral problems in students. For this reason, the digital, problem-based simulation GEProS was developed to promote diagnostic competence in pre-service teachers. GEProS was systematically evaluated in three longitudinal, (quasi-)experimental control group studies. The results show that GEProS can provide added value for promoting diagnostic competence in university teacher education, especially in the area of self-determined motivation. Furthermore, by means of an integrated quiz with feedback, the testing effect can be used in GEProS to support the acquisition of declarative-conceptual knowledge, a central disposition of diagnostic competence. By developing an Integrative Model of Diagnostic Competence, this thesis contributes to theory building as well as to research and development in the area of promoting diagnostic competence of (pre-service) teachers by systematically evaluating a measure for promoting diagnostic competence for a subject-general, temporally stable object of judgment. In addition, key features of digital simulations were defined and served as a basis for the development of GEProS. This work thus also contributes to the theoretical foundation of digital simulations.
... All participants completed simulated field experiences focused on teaching techniques and classroom management. SimSchool virtual student profiles were based on academic profiles (e.g. a student is learning at the expected level for his or her age group, or a student is functioning at a lower level than expected), and five aspects of student emotions: openness to experience, conscientiousness, extroversion, agreeableness, and neuroticism (Christensen et al. 2011). Student behaviours change as the teacher candidate assigns tasks, shifts their tone of voice, and employs classroom management strategies. ...
... Compared to traditional classrooms, simulated classrooms led to higher response scores related to self-efficacy and working with students with learning disabilities. Along with prior research (Christensen et al. 2011), this finding indicates simulated field experiences may be an effective tool to develop teacher candidates' abilities to accommodate and engage diverse learners and create inclusive classrooms. ...
Article
Full-text available
New teachers often work in culturally diverse schools after practicum at culturally homogenous schools. Simulated field experiences could ensure that they are better prepared. This study measured gains in culturally responsive pedagogy among teacher candidates after a semester of simulated field experiences. Teacher candidates completed a pre- and post-test evaluation. Findings revealed significantly higher culturally responsive pedagogy, specifically in cultural value of diversity and self-regulation support. Simulations may be a promising complement to traditional field experiences in education; policymakers and faculty should explore incorporating them into educator preparation programs.
... In the study by Christensen et al. (2011), the basic components of the simSchool dynamic simulator, which is a simulated teaching environment to train prospective teachers, are presented. The research results suggested that prospective teachers gained a faster sense of instructional self-efficacy (confidence in their efficacy) by using the ...
Article
Full-text available
This study examines the scientific studies on the use of virtual environments and simulation programs in teacher training programs. Firstly, a search was made in the Web of Science database using the keywords simSchool and TeachLive. Four of the studies obtained as a result of the screening were excluded for various reasons (such as duplicate or off-topic publication) and the study was conducted with 22 scientific publications. The research results showed that virtual environments and simulations, which are widely and effectively used in many disciplines, also have a crucial potential in teaching and training. Most of the studies subject to analysis suggested that the use of virtual environments and simulation in education has positive effects such as self-confidence, time management, and classroom management. The research findings were discussed within the framework of the relevant literature, and suggestions were made.
... Few simulated classroom environments have been developed to measure and promote action-oriented facets of PK (e.g., Bautista & Boone, 2015;Christensen et al., 2011;Dalgarno et al., 2016), but none that cover ePCK. We were successful in validating the SCR Bio in German language so far, but still aim to use and compare the instrument in other countries and expand the range of participants. ...
Article
Full-text available
Background The professional knowledge of pre‐service teachers is highly important for effective and successful teaching. In recent years, many research groups have been engaged in developing simulated classroom environments to capture especially the pedagogical knowledge (PK) of pre‐service teachers, neglecting the content‐related facets of professional knowledge such as pedagogical content knowledge (PCK). Objectives In the present study, we describe the development of a simulated classroom environment—the Simulated Classroom Biology (SCRBio)—and provide evidence regarding its validity to assess pre‐service biology teachers' action‐oriented PCK in the area of evolution. Methods This study examined the evidence supporting the validity of using the SCRBio to investigate action‐oriented PCK of pre‐service biology teachers. The (1) evidence based on test content (expert ratings) and the (2) evidence based on relation to other variables (known‐groups comparison) was obtained. We tested the SCRBio with N = 76 German pre‐service biology teachers. Results and Conclusions Our results show the successfully operationalized PCK in the SCRBio through explicit allocation of specific misconceptions to each virtual student's answer and the valid measurement of pre‐service biology teachers' action‐oriented PCK. This results in a validated simulated classroom environment for pre‐service but also in‐service teachers. In the future, the SCRBio will be developed from an assessment instrument to a training tool to simulate explicit teaching situations. This allows to complement the predominantly theoretical components of university‐based teacher education with practice‐based simulated classroom environments.
... This type of simulation involves the use of software to simulate the learning environment. For example, ClassSim is an online simulation that focuses on training teachers to work with students with special needs (Ferry et al., 2005); SimSchool offers preservice teachers an opportunity to experience digital-based practice (Badiee & Kaufman, 2014;Christensen et al., 2011;Gibson, 2007); and Teacher-Moment is an online platform that uses digital scripted characters to practice communication skills in the educational arena (Thompson et al., 2018). In the latter, which was recently used in the context of the COVID-19 pandemic (Sullivan et al., 2020), human figures are present in the digital simulation, yet the interactions are not performed by an actual human being. ...
Article
This paper presents a novel simulation model, developed amidst COVID-19, given the need to adapt the (ordinarily used) human-based clinical simulations to an online platform. The Virtual Sim(HU)lation (VSH) Model is a human-based virtual simulation. First, we conceptualize the model, its features and utilization, addressing the distance-learning-related challenges. Then, we present findings of a qualitative case study, reviewing the implementation of the VSH model in 37 workshops (N=412). Analysis revealed three categories regarding the opportunities and obstacles of the VSH model. This study contributes to methodological, theoretical, and practical knowledge of designing online simulations for both routine and emergent use.
... The potential of simulation platforms to facilitate an array of skills, knowledge, understandings, and approaches that are of import for preservice teachers is well highlighted in the literature [17] (p. 1). The unique features of simulation platforms and their affordances, benefits, and efficacy of deployment, as well as their limitations in teacher preparation contexts, have also been well documented [3,18,19]. Christensen, Knezek, Tyler-Wood, and Gibson [18], for example, described in a paper on simSchool the various features and benefits of using the platform, e.g., it provided a safe space for practicing techniques, especially approaches that addressed learning styles (p. 27). ...
... The unique features of simulation platforms and their affordances, benefits, and efficacy of deployment, as well as their limitations in teacher preparation contexts, have also been well documented [3,18,19]. Christensen, Knezek, Tyler-Wood, and Gibson [18], for example, described in a paper on simSchool the various features and benefits of using the platform, e.g., it provided a safe space for practicing techniques, especially approaches that addressed learning styles (p. 27). ...
... In some instances, preservice teachers are afforded the opportunity and time for repeated, reflective practice [17]. In an early paper on Teacher Education with simSchool, Gibson [18] described the potential benefits of simulations thusly: ...
Article
Full-text available
Authentic practice in pedagogical approaches is essential for preparing teachers to design effective learning experiences that foster student engagement during this digital era. There is an opportunity to explore novel and effective designs of virtual experiences that may augment or better prepare preservice teachers for field placements in physical classrooms. We proffer that virtual classroom simulations can and should be further explored and leveraged, now more than ever. In this paper, we examined a model of the impact of perceptual variables on instructional effectiveness that can enhance teaching efficacy and outcome expectancy when preservice teachers engage in practice teaching experiences in a virtual classroom simulation. The relationships between perceptual variables (presence, instructional time, and engagement) and teaching efficacy and outcome expectancy, as they relate to instructional effectiveness, were analyzed using the structural equation modeling approach. The results supported all of the hypothesized relationships. For example, presence and instructional time strongly and positively influenced engagement. Engagement was strongly related to instructional effectiveness. Instructional effectiveness was strongly and positively related to teaching self-efficacy, as well as outcome expectancy. The results support that virtual world classroom simulations can be an effective space for practice teaching for prospective mathematics teachers. Furthermore, this study provides insights for teacher educators, developers, and instructional designers interested in designing and utilizing practice-based simulations.