Content uploaded by Vasiliki Papageorgiou
Author content
All content in this area was uploaded by Vasiliki Papageorgiou on Feb 16, 2019
Content may be subject to copyright.
MULTIMODAL TEACHING AND LEARNING WITH THE
USE OF TECHNOLOGY: MEANINGS, PRACTICES AND
DISCOURSES
Vasiliki Papageorgiou and Petros Lameras
Coventry University, Priori Street, Coventry, United Kingdom, CV1 5FB
ABSTRACT
The aim of this paper is to report on teachers’ experiences of, and approaches to, multimodality in teaching and learning.
A small-scale online survey with closed and semi-structured questions has been deployed to school and university
teachers (n=68) for eliciting their experiences in multimodal teaching and learning. Thematic analysis has been adopted
as the overarching methodology for reporting patterns in the data from the survey. The results from the analysis showed
that experiences of multimodality are discerned as: (1) imparting information, (2) enacting collaborative learning and (3)
preparing students for exploring concepts. The process of meaning making is exemplified through a developmental
progression from more teacher-directed modes through oral, written and visual representations to more student-centered
through gestural representations as means of connecting and combining different modes triggered via visual
communication, collaboration and exploration.
KEYWORDS
Multimodality, Technology-Enhanced Learning, Collaborative Learning
1. INTRODUCTION
There is an increasing body of evidence, which suggests that multimodality in learning is an active,
student-centred approach in which students select the resources mostly relevant to them (Mayer, 2001). This
means that students are responsible for organising learning content such as words and images into coherent
verbal and visual models comprising their mental schemata and conceptual structures (ibid.). The essence of
multimodality therefore is to provide different types of resources to the student for stimulating learning in
meaningful ways within and across disciplines. To this end, multimodality is described as an interdisciplinary
approach drawn with an emphasis on communication and representation (Jewitt, 2013). This is directly
relevant to teachers in terms of using current theories of learning to engage students with student-centred
pedagogies and resources of learning (Hassett & Curwood, 2009). Multimodality in today’s classrooms refers
to ‘multiple’ modes of representation, with combined elements of print, visual images and design (Jewitt,
2008). Multiple modes of representation include capabilities of combinations of oral and written language,
visual, gestural, tactile and spatial representations (Cope & Kalantzis, 2009). This transitional shift from
print-based education to multimodal education indicates the need to rethink how teaching and learning is
conceived, approached and practiced.
A substantial body of research has reported on interventions, case studies, conceptual frameworks and
design of teaching and learning activities using multimodal technologies and not on the actual multimodal
teaching practices mediated by technologies currently used by teachers (Bell et al., 2010; Cope & Kalantzis,
2009). In this paper, it is argued that multimodality is based on the process of creating meaning through
connecting and combining teaching strategies, approaches to teaching and learning with technologies that
afford exploration, investigation and participatory learning. Against this background, meanings, practices and
discourses were investigated in conceptualising and practicing multimodality in teaching and learning with
the use of associated teaching approaches, technologies and resources. The research questions addressed in
this study were the following:
14th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2017)
133
1. What multimodality means for teaching and learning with the use of technology?
2. How multimodality is approached and practiced with the use of technology for enhancing teaching
and learning?
The paper starts by contemplating on ways of understanding multimodality and elucidates on
multimodality and digital technologies for teaching and learning. It then describes pedagogical approaches
such as collaborative learning in tandem with more activity-led teaching advocating context-specific and
personal construction of meaning. It continues with the methodology and research design adopted for the data
collection and analysis. The results of the survey analysis are presented in the next section along with a
discussion on connecting findings with research questions and implications for teaching practice and
research. Finally, conclusions and future research are discussed based on the evidence from the survey.
2. WAYS OF UNDERSTANDING MULTIMODALITY
The way multimodality is perceived and enacted is in tandem with how technologies are deployed and used
for designing and delivering learning content (Miller & McVee, 2013). Thus, it is less the technology itself
and more about the individual and collaborative practices applied with technology including the mechanisms
that teachers incorporate to create meaning to students. In essence, multimodality emphasises situated action,
considering the social context as the key factor for meaning making, with special focus on how people use
the resources available based on their cultural practices, personal beliefs and institutional contexts; rather
than emphasising the attributes, behaviours and the system of the available resources (Kress, 2001). This
introduces new possibilities for investigating, analysing and understanding the different ways which people
use multimodality for creating, sustaining and transferring meaning to inter-related ecosystems and social
contexts.
3. MULTIMODALITY, PEDAGOGICAL APPROACHES AND DIGITAL
TECHNOLOGIES FOR TEACHING AND LEARNING
The advent of digital technologies for supporting teaching and learning has supplemented or amplified
conventional non-digital activities (Beetham & Sharpe, 2013). Digitisation of administrative and routine
tasks such as storing, transferring and retrieving information supplements traditional teaching and learning in
the sense that the digital modalities used do not resemble or offer something novel to the way current
teaching and learning processes and strategies are practiced. Indeed, learning technologies should help
students to increase their capacities for innovation, leadership, multi-and inter-disciplinary collaboration,
emotional intelligence, critical skills and collective problem solving in a participatory digital learning
environment (Greenhow, et al., 2009). Multimedia resources and tools in these environments may include for
example, interactive videos and images, recorded lecture presentations, online quizzes, discussion forums
(synchronous and asynchronous), visual representations of student data to depict progress and on what the
student is doing to learn (Sharples, et al., 2016). Currently many teachers tend to use digital technologies to
support teacher-directed approaches with the aim of improving the quality of lecture presentations by using
Interactive Whiteboards; PowerPoint for lecture notes and asynchronous discussion forums for the recreation
of face-to-face tutorial discussions (Beetham & Sharpe, 2013). The affordances of the aforementioned
technologies in these cases are mainly exploited to explain and visualise content knowledge or for engaging
students in activities that require to visually absorb and manipulate information more actively than before.
Mayer (2001) argues that student’s learning becomes more meaningful when an array of interactive tools and
resources are deployed rather using text alone. Moreover, the visual representation of content is vital for
communicating subject matter and improving students’ understanding (ibid.).
There is increasing research on collaborative multimedia learning in different subject domains (Bell, et
al., 2010). Studies have shown that collaboration can enhance the quality of the learning process, hence the
importance of achieving specific learning outcomes combining multimedia content with collaborative
learning may lead to engaging, interactive and powerful multimedia learning environments (Dillenbourg,
1999). Students working collaboratively in groups have the opportunity to share their thoughts and prior
ISBN: 978-989-8533-68-5 © 2017
134
knowledge. Collaborative dialogue supports learning by clarifying thinking and consolidating ideas
(Hmelo-Silver, 2002). The “classroom learning communities” approach seeks to operationalize the benefits
of learning through participation in communities of practice (Lave & Wenger, 1991). Multimodal
collaborative learning promotes the idea of creating a learning community with a shared purpose of making
sense of scientific ideas and practices (Harris & Rooks, 2010). Teachers, although still reluctant in using
technology in pedagogically driven ways, have slowly started to integrate new educational teaching and
learning practices (Miller & McVee, 2013) including but not limited to serious games, blogging,
collaborative editing and online media manipulation. In this study, multimodality for teaching and learning is
examined in terms of experiences on using different modalities (oral and written language, visual, gestural
and tactile representations), technologies and teaching strategies (e.g. transmitting information, collaborative
learning, informal learning) capturing more holistically the way multimodality is perceived and enacted by
teachers.
4. METHODOLOGY AND RESEARCH DESIGN
A thematic analysis approach was adopted as the overarching methodology for this study. Thematic analysis
is a method for identifying, analysing and reporting themes or patterns within data. Thematic analysis was
selected because it offers a dynamic and flexible method to analysing qualitative data and it is ideal for
thematic synthesis of primary research (Braun & Clarke, 2006; Boyatzis, 1998). Deciding to use thematic
analysis was interlinked with our attempt in coding the different primary data on meanings, practices and
discourses for multimodal teaching and learning. The data collection process started by designing an online
survey for eliciting teachers’ beliefs, intentions and actions in using multimodality for teaching and learning.
The VARK1 for ‘Teachers and Trainers’ questionnaire on multimodality has been adapted to encompass
questions aligned towards identifying variation in ways of deploying teaching strategies, digital technology
and modes of multimodal meaning as comprehended by teachers in three European countries. In particular,
30 teachers from Germany, 20 from Finland and 18 from Denmark (n=68) participated in this small-scale
study. Purposive sampling aimed at selecting teachers from different disciplines for the facilitation of
generalisability and validity.
The online survey has been translated in German, Finnish and Danish for ease of use for the participants
of each country. The survey included 28 closed and semi-structured questions as to enable participants to
instantly select the option mostly relevant to them (i.e. closed question) as well as to prompt for a more
recursive process where participants had to go back and forth their descriptive answer for fine-tuning,
refining and reflecting (i.e. semi-structured question). An informed consent form and information sheet were
included in the survey for completion prior to main body of questions for seeking participants’ agreement to
take part in the study; and be ensured that all information provided would be treated in confidence.
The processes of analysis were adopted from a thematic analysis perspective for discovering the themes
embodied in the evolving meanings of the data. In line to this, the following processes were carried out: (1)
familiarization with the data, (2) generation of initial codes, (3) searching for themes, (4) review of themes,
(5) definition of themes and (6) synthesis of themes. Repeating this process for all data-sets the data were
categorised while considering connections and interconnections between codes and themes. Then, eight
codings were generated (e.g. experiencing multimodality as imparting information) which were mainly
emerged and resembled within the different sections of the survey.
5. RESULTS
This section reports on the results of the online survey for eliciting teachers’ beliefs, intentions and actions in
using multimodality for teaching and learning. The themes emerged from data analysis are presented below.
1 http://vark-learn.com/the-vark-questionnaire/teaching-questionnaire/
14th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2017)
135
5.1 Experiencing Multimodality as Imparting Information
The majority of the participants from Germany (53.3%) and Finland (50%) define multimodality as “a
complex of modes including talk, visual communication, action, gesture, posture and movement”, which
shows a clear understanding of what is multimodality and in line with contemporary definitions revised in the
literature (see Figure 1.). A considerable proportion of the teachers perceived that multimodality describes
communication practices using modes to develop and deliver content, hence transferring information and
content was the focus for multimodal teaching. Most of the teachers used technology for transferring
information in static and linear way using voice and gestures via lectures and imparting knowledge via
assignments and tests although there was a tendency to explain multimodality as a more interactive,
constructive and interactive mode of teaching.
Figure 1. Teachers’ definition of multimodality for teaching and learning
5.2 Experiencing Multimodality as Enacting Collaborative Learning
Teachers from Finland and Denmark (95% and 100% respectively) indicated that they use technology for
collaborative learning practices. This may shows that the use of collaborative practices mediated by
multimodal technologies is prevalent, as teachers increasingly seem to be confident in integrating
collaborative activities that necessitate participatory tools and multiple modes. Participants were also asked to
specify the modalities they use to enhance collaborative practices. Collectively the data show that the use of
collaborative projects mediated by multimodal technologies was the most prominent choice between the
others for the teachers from the three surveyed countries. Formative assessment seemed to be an important
part of collaborative teaching strategies since the focus is to help the student understand comprehensively not
only how to improve individually but also how the overall performance of the team could be enhanced to
communally produce meaningful ideas. Interestingly, mostly the teachers from Finland and several teachers
from Denmark appeared to use a variety of digital modalities (e.g. synchronous/asynchronous tools, social
networking) to enhance collaboration and create a more dialogic and supportive learning culture. This result
illustrates the importance of feedback and the ways that feedback is visualised to students for enhancing
engagement (see Figure 2.).
43,3%
53,3%
36,7%
3,3%
16,7%
30%
40%
50%
30%
0%
15%
35%
33,3%
38,9%
38,9%
5,6%
16,7%
38,9%
0% 10% 20% 30% 40% 50% 60%
It describes communication practices using modes
to develop and deliver content
A complex of modes including talk, visual
communication, action, gesture, posture and …
Processing information through multiple modalities
Print-based reading and writing
Computer software and multimedia
All of the above
What do you think multimodality for teaching and learning is?
Teachers from
Germany
Teachers from
Finland
Teachers from
Denmark
ISBN: 978-989-8533-68-5 © 2017
136
Figure 2. Modalities used by teachers for collaborative learning
5.3 Experiencing Multimodality as Preparing Students for Exploring Concepts
Participants were asked to make explicit their understanding concerning the strategies they would prefer to
use in different teaching and learning situations. The teachers were enquired “How would you ideally prepare
the first session of your course for your students?” Participants from Germany (46.7%) tend to introduce their
session by an oral statement that stresses what students need to learn, absorb and memorise alluding to a
behaviouristic approach to teaching. This result illustrates teachers’ preference in more traditional-based
practices via the use of the predominant oral communication modes. It also shows that less emphasis is given
to the students’ ideas and interests but mainly on teacher’s organisation of content-based sessions and
activities. Creating activities for students to collaboratively brainstorm on what they want to learn in this
course via Google docs was the most popular option of the educators from Finland and Denmark (See Figure
3.). This approach highlights the teachers’ learning design decisions informed by the students’ personal
interests, prior knowledge and interaction with peers; exemplifying a visual communication multimodality
approach. Participants from Germany (30%), Finland (15%) and Denmark (38.9%) would ideally use a
PowerPoint presentation showing examples and applications of how aspects of the course are beneficial for
the society. Teachers attempted to use visual communication tools for explaining and clarifying concepts
guided and directed by them and in line with the subject matter. This possibly reveals that teachers had
difficulties in creating novel and complex multimodal artefacts for introducing concepts and ideas beyond the
curriculum for initiating discussions that would help students to connect prior knowledge with novel
concepts.
Figure 3. The pedagogical choices by the teachers for introducing their first session of a course
7,7%
76,9%
0%
23,1%
15,4%
61,5%
68,4%
84,2%
57,9%
42,1%
68,4%
15,8%
38,9%
72,2%
50%
5,6%
27,8%
38,9%
0% 50% 100%
Synhronous ( e.g. real time chat tool) and asynhconous online
(e.g. email) dicussion tools
Collaborative projects
Collaboration authoring tools (e.g. Google docs, Wiki)
Blogs / podcasts for collaborative writing
Social networking (e.g. Facebook, Twitter)
Formative assessment
If you use technology for enhancing collaboration, what modalities do you use?
Teachers from
Germany
Teachers from
Finland
Teachers from
Denmark
46,7%
3,3%
30%
6,7%
13,3%
15%
10%
15%
15%
45%
0%
5,6%
38,9%
16,7%
38,9%
0% 10% 20% 30% 40% 50%
An oral statement with what students need to learn,
absorb and memorise
By referencing students to read the outline of the course
available on the LMS
A power point presentation showing examples and
applications of how aspects of the course are beneficial …
Explaining to students through visual diagrams the
importance of the course and the connection between …
Encouraging students to collaboratively brainstorm via
Google docs
How would you ideally prepare the first session of your course for your students?
Teachers from
Germany
Teachers from
Finland
Teachers from
Denmark
14th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2017)
137
Responding to the question: “You have organised a field trip for your students to explore a topic in
question. How would you prepare students?” Participants (see Figure 4.) perceived the usefulness of
supporting students to design their own field trip preparation guidelines. Teacher’s role is to provide the
context for students to prepare and suggest resources and tools (mobile devices with gestural interfaces) to be
used by students to plan their scientific investigations. The provision of different web links / web-quests for
the preparation of the trip was also an option chosen by many teachers from Finland and Denmark,
illustrating the vital role of multimodal technologies to empower students to enquire and conduct experiments
in informal learning situations. However, oral means of communication still remain one of the central modes
of content delivery as many teachers stated that they would orally suggest ways of preparing (as opposed to
instructing students to follow a specific direction imparted by the teacher) that needs to be made by students
in the classroom – because technology seemed to be obsolete when face-to-face teaching is the primary
teaching mode. It was clear that the use of technology for multimodal teaching is experienced as a
supplementary tool for face-to-face teaching for cementing a blended learning setting.
Figure 4. Strategies used by teachers for the exploration of a specific topic during a field trip
6. IMPLICATIONS FOR PRACTICE AND RESEARCH
Connecting the 3 emerging categories of experiencing multimodality as: imparting information, enacting
collaborative learning and preparing students for exploring concepts to the two overarching research
questions led to interesting experiences and practices for understanding how multimodality is approached
with the use of technology. This section aims to provide a reflection on implications for multimodal practice
and research in tandem to the research questions.
6.1 What Multimodality Means for Teaching and Learning with the use of
Technology?
It is evident from the findings that multimodality is conceived as a cluster of modes increasingly being used
as a communication mechanism for delivering information and content. This is in conjunction to using oral,
written and visual representations for meaning making. The choice of modes, being used by participants
include oral language (lecture), written language and visual communications (diagrams, PowerPoint). This
creates an inventory of the meanings when technology is used in relation to storing content in a repository for
rote learning. Essentially, building on the notion of using oral, written and visual descriptions, teachers’
primary awareness is on constructing a system for imparting their knowledge and experiences through linear
ways of representation. A demand of learning the foundations of the module based on subject matter or
63,3%
3,3%
16,7%
16,7%
63,3%
30%
10%
10%
55%
50%
50%
22,2%
38,9%
50%
61,1%
0% 20% 40% 60% 80%
Orally describe the preparation that needs to be made by
them
Create a question and answer online session
Prepare and upload an analytical guide with prepration
instructions
Provide different web links of how to prepare for a field
trip
You allow students to design their own field trip
preparation guides
You have organised a field trip for your students to explore a topic in question. How
would you prepare students?
Teachers from
Germany
Teachers from
Finland
Teachers from
Denmark
ISBN: 978-989-8533-68-5 © 2017
138
offering to students’ information pre-selected, reviewed and evaluated by the teacher is the distinct aspect of
meaning through the combination of oral/written and visual modes of meaning.
There were instances where multimodality via using technology was experienced as engaging students in
activities for individual and collaborative construction of meaning and for preparing students to explore via
field trips. An array of more ‘developed’ modes were evidenced such gestural for constructing meaning
making processes that required students to take control of their learning by preparing to carry out
investigations in field trips through smart devices. This demonstrated a more ‘complete’ understanding of
multimodality in a more developmental sense, starting from a fragmented perception, that of using
oral/written and visual descriptions for accessing and transferring information to a more cohesive- that of
using gestural (via mobile devices for carrying out explorations during field trips) coupled with written/oral,
audio and visual.
6.2 How Multimodality is Approached and Practiced with the use of
Technology for Enhancing Teaching and Learning?
The findings revealed that there is a connection between ‘espoused theories’ and ‘theories in use’. For
example, participants that perceived multimodality as a linear contingent process emanated via oral/written
language, the use of technologies that supported this view was more prevalent. Information transfer through
lectures, presentations, learning repositories and written hand-outs were widely used. This was in tandem to
influences derived from personal beliefs about teaching and learning, the role of the teacher in the classroom
as well from institutional policies that might have pushed teaching to a certain pedagogical direction. It is
however inconclusive to suggest that teachers’ perceptions on ‘fragmented’ ways of using multimodal tools
for teaching and learning are influenced only from their conceptions of teaching and learning.
More systematic research is needed for understanding, the connections between multimodal conceptions
of learning and teaching with the way technology is used. Participants who felt that multimodality is enacted
via more gestural, processes for meaning making, the use of technology was experienced as a process of
engaging students in activities that could not be realised with conventional technology (PowerPoints). For
example, teachers who experienced multimodality as an active process, the technologies deployed were more
participatory, interactive and adaptive to student’s interests and meaning making processes. Activities that
involved collaborative projects and preparing for carrying out investigations through field trips encompassed
a blending learning approach for synchronous and asynchronous communications, and the use of mobile
devices for investigations, explorations and inquiry.
7. CONCLUSION
This paper investigated experiences of multimodality and their association to teaching and learning with the
use of technology. Thematic analysis has been used to analyse the findings based on a survey with 68
teachers for eliciting experiences of multimodal teaching. The results showed a relation between multimodal
meaning making descriptions, teaching approaches and technologies used for teaching and learning.
Limitations of this research were on the basis of discerning more detailed accounts of connections and
influences that determined a specific way of conceiving the phenomenon in question. Sample restrictions
were also a factor that prevented from getting more nuanced descriptions, particularly for identifying
relations between ways of understanding multimodality, teaching strategies and technologies being used.
More-over, due to the small-scale nature of this study, the findings are not generalizable to other contexts but
rather may be used to provide insights on how teachers experience multimodal teaching and learning in
specific contexts grounded to this study. Finally, it is acknowledged that multimodality is not a new concept
for teaching and learning, however, this study might help to better understand how multimodality could work
by using pedagogically driven approaches to teaching and learning mediated by technology that would help
practitioners to design and support teaching and learning activities for enhancing students’ learning
experiences. Future research should closely investigate relations between multimodal meaning making and
associated use of technology for discerning theories, practices and discourses. This will help the
technology-enhanced learning research community to investigate and interpret multimodality in specific
digital environments hence gauging larger-scale qualitative studies as means to surmount its impressionistic
endeavour.
14th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2017)
139
ACKNOWLEDGEMENT
The research leading to these results has received funding from the European Union Erasmus + programme,
Key Action 2 Strategic Partnerships under grant agreement 2016-1-FR01-KA204-024178, STEAM project.
REFERENCES
Beetham, H. & Sharpe, R., 2013. Rethinking Pedagogy for a digital age: Designing for 21st century learning.. London:
Routledge.
Bell, T., Urhahne, D., Schanze, S. & Ploetzner, R., 2010. Collaborative inquiry learning: Models, tools, and challenges.
32(3). International journal of science education, 32(3), pp. 349-377.
Boyatzis, R., 1998. transforming Qualitative Information. Thematic Analysis and Code Development. Cleveland: Sage.
Braun, V. & Clarke, V., 2006. Using thematic analysis in psychology. Qualitative research in psychology, 3(2),
pp. 77-101.
Cope, B. & Kalantzis, M., 2009. “Multiliteracies”: New literacies, new learning. Pedagogies: An international journal,
4(3), pp. 164-195.
Dillenbourg, P., 1999. What do you mean by collaborative learning? . In: Collaborative-learning: Cognitive and
computational approaches. Oxford: Elsevier, pp. 1-15.
Greenhow, C., Robelia, B. & Hughes, J. E., 2009. Learning, teaching, and scholarship in a digital age Web 2.0 and
classroom research: What path should we take now?. Educational researcher, 38(4), pp. 246-259.
Hennessy, S., 2011. The role of digital artefacts on the interactive whiteboard in supporting classroom dialogue. Journal
of Computer Assisted Learning, 27: pp. 463–489.
Harris, C. J. & Rooks, D. L., 2010. Managing inquiry-based science: Challenges in enacting complex science instruction
in elementary and middle school classrooms. Journal of Science Teacher Education, 21(2), pp. 227-240.
Hassett, D. D. & Curwood, J. S., 2009. Theories and practices of multimodal education: The instructional dynamics of
picture books and primary classrooms.. The Reading Teacher, 63(4), pp. 270-282.
Hmelo-Silver, C. E., 2002. Collaborative ways of knowing: Issues in facilitation.. Boulder, Colorado, s.n.
Jewitt, C., 2008. Multimodality and literacy in school classrooms. Review of research in education, 32(1), pp. 241-267.
Jewitt, C., 2013. Multimodal methods for researching digital technologies. In: The Sage handbook of digital technology
research. Los Angeles: SAGE, pp. 250-265.
Kress, G., 2001. Multimodal teaching and learning: The rhetorics of the science classroom.. London: Continuum .
Kress, G. & van Leeuwen, T., 2001. Multimodal discourse: The modes and media of contemporary communication.
London, UK: Edward Arnold.. London, UK: Edward Arnold.
Lave, J. & Wenger, E., 1991. Situated learning: Legitimate peripheral participation.. Cambridge: Cambridge University
press.
Mayer, R. E., 2001. Multimedia learning. New York: Cambridge University Press.
Miller, S. M. & McVee, M. B., 2013. Multimodal composing in classrooms: Learning and teaching for the digital world..
London: Routledge.
Redecker, C. et al., 2009. Learning 2.0: The impact of Web 2.0 innovations on education and training in Europe (No.
JRC55629)., s.l.: Institute for Prospective and Technological Studies, Joint Research Centre..
Sharples, M. et al., 2016. Innovating Pedagogy 2016: Open University Innovation Report 5, Milton Keynes: Open
University .
ISBN: 978-989-8533-68-5 © 2017
140
