Wolfgang Schnotz

Universität Koblenz-Landau · Department of General and Educational Psychology

Multimedia messages use combinations of texts, pictures, maps, and graphs as tools for communication. This book provides a synthesis of theory and research about how people comprehend multimedia. It adopts the perspectives of cognitive psychology, semiotics, anthropology, linguistics, education, and art. Its central idea is that information display...

Text and pictures serve different purposes in multimedia comprehension. Conceptual processing of texts and pictures results in propositions, whereby text-based propositions and picture-based propositions specialize in different kinds of information. These propositions are merged into an overarching conceptual semantic network guiding mental model c...

This chapter aims at clarifying basic concepts related to multimedia: communication, comprehension, and learning. Multimedia communication is considered as the intentional creation, display, and reception of multiple kinds of signs in order to convey messages about some content. It entails two subprocesses: meaning and comprehension. Multimedia mea...

Multimedia communication design is a form of complex problem-solving. It requires heuristics which take into account addressees’ cognitive abilities and prior knowledge, the complexity of the subject matter, processing conditions, time limits, and other factors. Multimedia designers have to be aware of the asymmetry between texts and pictures in te...

Human history has created a large variety of sign systems for communication. These systems were developed at different times for different purposes. While oral language has developed as part of human biological evolution, written texts, realistic pictures, maps, and graphs are cultural inventions. Human oral language might have originated from gest...

Comprehension can be the starting point for further cognitive activities such as thinking and problem-solving. Productive thinking requires a specific interaction between descriptive and depictive representations combining representational power and inferential power. The interaction takes place through processes of mental model construction and mo...

Multimedia messages use combinations of texts, pictures, maps, and graphs as tools for communication. This book provides a synthesis of theory and research about how people comprehend multimedia. It adopts the perspectives of cognitive psychology, semiotics, anthropology, linguistics, education, and art. Its central idea is that information display...

Sign systems help to create descriptive and depictive representations. Descriptive representations operate on symbols. They are based on conceptual analyses identifying objects or events as well as attributes and interrelations. Attributes and relations are ascribed by predications to entities according to syntactic rules resulting in so-called pro...

Pictures are two-dimensional depictive representations. They include static pictures and animations. The latter are defined as pictorial displays that change their structure or other features over time and trigger perception of a continuous change. Static and animated pictures can display static as well as dynamic content. Both can have an envision...

Text comprehension and picture comprehension can be synthesized into a common conceptual framework which differentiates between external and internal descriptive and depictive representations. Combining this framework with the human cognitive architecture including sensory registers, working memory, and long-term memory leads to an integrated model...

Spoken or written texts are coherent sequences of sentences. Text comprehension is equivalent to the construction of multiple mental representations in working memory. It is based on an interaction between external text information and internal prior knowledge information stored in long-term memory. Mental representations include a text surface rep...

Using multiple external representations is advocated for learning in STEM education. This learning approach assumes that multiple external representations promote richer mental representations and a deeper understanding of the concept. In mathematics, the concept of function is a prototypical content area in which multiple representations are used....

This study examined the relationship between grade and the ability of text-picture integration in terms of task completion and the pattern of using textual and pictorial information. Children ( N = 144) from secondary schools were recruited from grade 5 and grade 8. Analyzing the time spent with the multimedia unit prior to an incorrect response, t...

Digital and online learning is more prevalent than ever, making multimedia learning a primary objective for many instructors. The Cambridge Handbook of Multimedia Learning examines cutting-edge research to guide creative teaching methods in online classrooms and training. Recognized as the field's major reference work, this research-based handbook...

This article investigates whether goal-directed learning of pictures leads to multiple mental representations which are differently useful for different purposes. The paper further investigates the effects of prompts on picture processing. 136 undergraduate students were presented maps of a fictitious city. One half of the participants were instruc...

Regulation of distance to the screen (i.e., head-to-screen distance, fluctuation of head-to-screen distance) has been proved to reflect the cognitive engagement of the reader. However, it is still not clear (a) whether regulation of distance to the screen can be a potential parameter to infer high cognitive load and (b) whether it can predict the u...

Prior knowledge, motivational factors, and metacognition have been long considered basic elements for successful problem-solving process. However, the interplay of these variables with context-based task characteristics has rarely been explored in research. The present study aimed at investigating how context-based task characteristics influence th...

In this paper we present a laboratory experiment in which 157 secondary-school students learned the concept of function with either static representations or dynamic visualizations. We used two different versions of dynamic visualization in order to evaluate whether interactivity had an impact on learning outcome. In the group learning with a linea...

In this study we examined the different functions of text and pictures during text–picture integration in multimedia learning. In Study 1, 144 secondary school students (age = 11 to 14 years; 72 females, 72 males) received six text–picture units under two conditions. In the delayed-question condition, students first read the units without a specifi...

Proper understanding of and learning from physics phenomena and experiments requires—among other competencies—flexible and coherent use of multiple representations (MRs). These can include everything from the “enactive” or “operational” manipulation of the experimental devices and materials to the most abstract level of a mathematical formulation o...

For an adequate comprehension of scientific phenomena, concepts, and experiments, it is necessary to understand several different representations and their interconnections. From a scientific point of view, representational competence (RC) can be understood as students’ ability to correctly generate, interconnect, and translate several representati...

This study describes the problem solving behaviour of bilingual (German - Turkish) primary school students, their approaches and preferences while solving the word problems. Four different types of mathematical word problems (combinations, movement, subtraction and addiction) were given successively to four pupils (one girl and three boys) and thei...

(Original Full-text legally free available via Springer Nature Shared-It: http://rdcu.be/D8Rm ) This paper presents an experiment that investigated the effect of forms of representation on students’ quantitative and qualitative reasoning on functions. The forms of representation we used were tables, augmented bar charts, and graphs. Research assume...

Both interest and flow are identified as key features that can foster learning and positively influence performance. However, there is only a little body of research that has attempted to examine how the two variables relate to each other, and as a result, impact performance. On the one hand, conceptually, interest is considered to be a preconditio...

Conjoint processing of text and pictures is assumed to possess an inherent asymmetry, because text and pictures serve fundamentally different but complementary functions. Conjoint processing is assumed to start with general, coherence-oriented mental model construction. When certain tasks have to be solved, the mental model is adjusted to the task...

Ein Zugang zu einem mathematischen Objekt ist lediglich über eine externe Repräsentation möglich (Duval, 2006). Dabei wird angenommen, dass das Lernen mit multiplen Repräsentationen Vorteile bietet und „zu einer höheren Flexibilität mathematischen Verstehens und Denkens“ (Schnotz, 2014, S. 52) beiträgt. Auf der anderen Seite stellt sich die Frage,...

A possible explanation for why students do not benefit from learner-controlled instruction is that they are not able to accurately monitor their own performance. The purpose of this study was to investigate whether and how the accuracy of metacognitive judgments made during training moderates the effect of learner control on performance when solvin...

Women are still underrepresented in engineering courses although some German universities offer separate women’s engineering courses which include virtual STEM learning environments. To outline information about fundamental aspects relevant for virtual STEM learning, one has to reveal which similarities both genders in virtual learning show. Moreov...

We review results from two related studies on two aspects of multiple representations (MRs) which are considered to be essential for the learning from physics experiments, namely representational coherence ability (RCA) and the representation related conceptual change (RCC) of the learners. Both knowledge coherence and conceptual change are hard to...

375 students from 24 randomly selected classes of a three-tier secondary school system were tested in a longitudinal study for their text-picture integration and pure reading competence as well as verbal and spatial intelligence at grades 5 to 7. Data were analyzed according to the integrated model of text and picture comprehension using hierarchic...

Visitors to modern science museums are likely to encounter exhibitions presenting conflicting information, such as risks and benefits of new scientific developments. Such exhibitions encourage visitors to reflect upon different sides of a story and to form or adjust their attitudes toward the topic on display. However, there is very little evidence...

Many recent research studies have compared dynamic visualizations with static graphics in the expectation that one of these display types is superior to the other. The research reported here challenges this black-and-white view by focusing instead on whether the aim of the intended learning is to produce a perceptual or a cognitive mental represent...

Students are frequently required to integrate text and picture information into coherent knowledge structures. This raises the questions of how students deal with texts and how they deal with graphics when they try to integrate the two sources of information, and whether there are differences between students from different school types and grades....

Teaching and learning from texts with integrated pictures are challenging tasks for teachers and students. Nevertheless, this kind of material is universal in secondary school as well as in elementary school and holds huge potential for student learning, if instruction and presentation are appropriate. The project “BiTe” investigates teachers’ and...

In der Sekundarstufe I werden Schülerinnen und Schüler zunehmend mit Lehrmaterialien konfrontiert, in denen Texte mit verschiedenen Arten von Visualisierungen (Graphiken, Diagrammen etc.) kombiniert sind. Lehrkräfte sollten die Text-/Bild-Integration von Schülerinnen und Schülern im Unterricht daher systematisch fördern. Dafür benötigen sie spezifi...

This paper presents the results of an empirical study on phrasal verbs with to come, to give, to go, to get, and to take and up, down, out as complements. Presuming that in many instances the overall meaning of PVs cannot be adequately inferred from the senses of their constituent parts (e.g., to make up for ⇒ to compensate), it is hypothesized tha...

Research on learning and instruction focuses on structural and procedural characteristics of guided human learning as well as its internal and external constraints including the embeddedness of human learning into social systems. Instruction (which is in this context just as a synonym for teaching) needs to be aligned with these characteristics of...

Ein mathematisches Objekt ist nur über eine Repräsentation zugänglich (Duval, 2006). Diese Erkenntnis induziert die Frage, ob es unterschiedlich " vorteilhafte " Repräsentationsform für den Umgang mit mathematischen Objekten gibt. Nach Larkin und Simon (1987) ist eine Repräsentation nutzungseffizienter, wenn die relevanten Informationen schneller u...

Bereits seit den Anfängen der Informationstechnologie besteht die Hoffnung, dass computergenerierte dynamische Visualisierungen das Lernen unterstützen und erleichtern können. In den letzten zwei Jahrzehnten wurden daher interaktive Mathematiksoftware (z.B. Dynamische Geometriesysteme, Computeralgebrasysteme) entwickelt, mit denen der Benutzer anim...

In science and science education,1 it is well known to use representations, often in multiple interconnected forms (multiple representations; MR) as an essential means of domain-specific reasoning; they are the cornerstone for adequately understanding scientific experiments, phenomena, and concepts (Gilbert & Treagust, 2009; Schnotz, Baadte, Müller...

A particular difficulty in physics learning is the fact that pupils’ “intuitive” concepts are often resistant to instruction. This article reports empirical results from two related studies within an interdisciplinary project of physics education and educational psychology in ray optics. Two different kinds of treatment groups (TG A and TG B), both...

Comprehension of graphics can be considered as a process of schema-mediated structure mapping from external graphics on internal mental models. Two experiments were conducted to test the hypothesis that graphics possess a perceptible surface structure as well as a semantic deep structure both of which affect mental model construction. The same cont...

There is a long tradition of museum research assessing visitors' personal background. In this article, we suggest an insightful way to enhance and intensify visitor analyses and adopt a more integrative approach. To this end, we draw attention to Latent Class Analysis (LCA), a classification method that allows us to investigate visitor profiles rat...

Although there is only a small number of English prepositions, they play a substantial role in language use and thus appear frequently: In English, one in every eight to ten words is a preposition (cf. Svartvik, 1988). English may therefore be considered a language of prepositions. Furthermore, English prepositions are characterized by polysemy: On...

In the context of fast technological development and the widespread use of learning technologies in education, the need for students to regulate their own learning processes has become increasingly important (Bannert & Reimann, 2012; Winters, Greene, & Costich, 2008). Research in the last decade has revealed that students have to possess specific s...

The importance of language skills in multinational Europe is beyond dispute. For this reason, the Council of Europe has set plurilingualism as a desirable goal (Common European Framework of Reference: Learning, Teaching, Assessment, 2001). In school education, this objective means that, alongside pupils learning at least two foreign languages, teac...

The scientific discussion of heurism concentrates on the solving of tasks. One of its specific aims is to emphasize the general reasons for choosing certain aspects of a problem that helps us to find the solution (Pólya, 1964). The elements that are detected in such analyses are known as “heuristic operations” or “heuristics” (Bruder & Collet, 2011...

Internationale Forschungsergebnisse belegen, dass für ein angemessenes Verständnis von naturwissenschaftlichen Experimenten, Phänomenen und Konzepten unterschiedliche Repräsentationsformen und deren Kohärenz ein wesentlicher Faktor sind. Die Fähigkeit, Informationen von verschiedenen Repräsentationsformen kohärent aufeinander beziehen zu können, is...

Inaccurate judgments of task difficulty and invested mental effort may negatively affect how accurate students monitor their own performance. When students are not able to accurately monitor their own performance, they cannot control their learning effectively (e.g., allocate adequate mental effort and study time). Although students' judgments of t...

(Basingstoke: Palgrave Macmillan)

This study tested the assumption that the effectiveness of feedback with regard to performance, motivation, and affect is moderated by the learners’ self-concept. A total of 72 sixth-graders completed a web-based interactive learning program. Half of the sample received feedback and the other half received no feedback. Differential feedback effects...

Reading for learning frequently requires integrating text and picture information into coherent knowledge structures. This paper presents an experimental study which aimed at analyzing the strategies used by students for integrating text and picture information. Four combinations of texts and pictures (text-picture units) were selected from textboo...

Until recently, museums mainly communicated well-established knowledge. Current science, however, is characterized by a rapid knowledge increase, so that we often have to deal with fragile and inconsistent knowledge. In order to develop exhibitions that encourage visitors to process information in a differentiated way, museums need to know how visi...

Abstract Book Multimedia comprehension is the construction of mental representations of some content in the mind of a recipient based on the interplay of different sign systems with the human perceptual and cognitive system. The sign systems have different biological and cultural histories. They are used to create external representations for mult...

The potential educational benefits of animation arise from its capacity to portray temporal change directly and explicitly. However, these benefits come with associated costs that can pose considerable challenges to learners. The processing requirements of conventionally designed instructional animations are too often poorly aligned with the charac...

Recent research has shown the importance of multiple representations for an adequate understanding of scientific experiments and phenomena. A basic skill for efficient work with multiple representations is that students are able to generate coherent representations. However, students' level of representational coherence is low. In addition, they le...

Despite numerous studies on reading and multimedia comprehension, the usage of text and picture with different reading strategies has rarely become a focus of research. The current study aims to explore whether the usage of text differs from the usage of picture when readers follow different strategies of knowledge acquisition. In a within-subjects...

When students read for learning, they frequently are required to integrate text and graphics information into coherent knowledge structures. The following study aimed at analyzing how students deal with texts and how they deal with graphics when they try to integrate the two sources of information. Furthermore, the study investigated differences be...

When reading conflicting science-related texts, readers may attend to cues which allow them to assess plausibility. One such plausibility cue is the use of graphs in the texts, which are regarded as typical of ‘hard science’. The goal of our study was to investigate the effects of the presence of graphs on the perceived plausibility and situation m...

Museums as free choice learning settings are increasingly confronted with the challenge to present conflicting information in an effective and motivating manner. To solve this task, knowledge about visitor profiles based on personality traits (epistemological beliefs, tolerance of ambiguity, self-efficacy) which are relevant for processing conflict...

When dealing with the concept of function, secondary school teaching mainly focuses on the input-output view. In this view, every value of the domain corresponds to a precise value of the range. Researchers stress that a second aspect – the aspect of covariation – is not sufficiently implemented in mathematical curricula (e.g., Thompson, 1994). Cov...

There is a broad consensus among science education researchers and cognitive scientists that learners need to be able to work with multiple representations in order to comprehend scientific concepts. Representational competence means the ability to represent knowledge as sentences, equations or formulas, and diagrams or drawings (Dolin, 2007). It i...

Learning materials incorporating written texts as well as instructional pictures are the basis for learning in many subjects. However, text–picture integration makes high cognitive demands of learners, and it seems plausible that the development of this competence is influenced by teachers’ instructional skills. The present studies investigated fir...

Three experiments with students from 7th and 8th grade were performed to investigate the effects of decorative pictures in learning as compared to instructional pictures. Pictures were considered as instructional, when they were primarily informative, and as decorative, when they were primarily aesthetically appealing. The experiments investigated,...

High quality learning is extensive, well integrated, deep, and supports the use of knowledge in new situations that require adaptation of what has been learned previously. This book reviews current research on the nature of high quality learning and the factors that facilitate or inhibit it. The book addresses relationships between quality of learn...

Reading for learning frequently requires integrating text and picture information into coherent knowledge structures. To investigate how students’ competencies for text-picture integration develop, a representative sample of integration task was selected from schoolbooks and used for the construction of items, which were presented to 1060 students...

Der Projekttitel BiTe steht für ein Forschungsvorhaben, das die schulische Kompetenzentwicklung im Bereich der integrierten Verarbeitung von Bild- und Textinformationen untersucht. Ausgangspunkt sind dabei kognitionspsychologische Überlegungen zu den mentalen Prozessen, die dem Wissenserwerb mit Texten und Bildern zugrunde liegen. So lässt sich anh...

The modality effect states that students learn better from multimedia instructional messages when the text is spoken rather than when it is written. The effect is considered as a derivate of the multimedia effect, because its rationale is to take full advantage of text-picture combinations by maximizing temporal contiguity of verbal and pictorial i...

This study investigated the relations between teachers’ pedagogical beliefs and students’ self-reported engagement in learning from texts with instructional pictures. Participants were the biology, geography, and German teachers of 46 classes (Grades 5–8) and their students. Teachers’ instructional behaviors and students’ engagement in learning fro...

Dynamic subject matter can be portrayed to learners in the form of either static or dynamic depictions. Two plausible bases for choosing a depiction format are alignment with the subject matter's dynamics, or the specific affordances a depiction provides for performing a particular learning output task. Experimental participants viewed an ordered s...

Learning environments commonly offer aids to address learners’ lack of knowledge or skills. However, learners are not or sub-optimally using these aids when these are non-embedded or segregated (Aleven, Stahl, Schworm, Fischer, & Wallace, 2003; Clarebout& Elen, 2006). This lack of optimal usage has been related to variables as self-regulation and p...

Während die Urteilsakkuratheit von Lehrerurteilen als solche vielfach untersucht wurde, ist über die Reliabilität der eingesetzten Urteilsmaße noch nicht viel bekannt. Die vorliegende Studie untersucht die Paralleltestreliabilität und die interne Konsistenz diagnostischer Urteilsmaße bei der Einschätzung von Aufgabenschwierigkeiten. Die Datengrundl...

Der Problemlöseprozess ist ein wichtiger Teil der mathematischen Bildung (Stanic & Kilptrick, 1988). Zahlreiche Forschungen in der mathematischen Bildung zielen darauf ab zu verstehen wie Schüler zur Lösung gelangen und welche Strategie sie in dem jeweiligen Arbeitsablauf verwendet haben (vgl. Pape & Wang, 2003). Derzeitige Studien befassen sich mi...