Evaluating Listening and Speaking Skills in a Mobile Game-Based Learning Environment with Situational Contexts.

Abstract

Game-based learning activities that facilitate students’ listening and speaking skills were designed in this study. To participate in learning activities, students in the control group used traditional methods, while students in the experimental group used a mobile system. In our study, we looked into the feasibility of mobile game-based learning activities. One experiment was carried out and the results revealed that the experimental-group students significantly outperformed the control-group students on the verbal post-test. However, the performance of the two groups was equal on the listening post-test. Two variables (the average score for the interactive jigsaw game and the number of designed cards) were found to be the most important factors for influencing students’ performance in the verbal post-test. Furthermore, most students had positive perceptions toward learning activities that are supported by a mobile system. These results suggest that game-based learning activities can significantly improve students’ speaking skills if driven by a mobile system. Furthermore, these results suggest that learning activities with a mobile system foster students to (1) practice speaking English as a foreign language (EFL) more frequently as well as to reflect on their speech; (2) create meaningful sentences and speak with greater accuracy and confidence; and (3) practice speaking EFL in an authentic context.

Full text

Evaluating listening and speaking skills in a mobile game-based
learning environment with situational contexts
Wu-Yuin Hwang
a
, Timothy K. Shih
b
, Zhao-Heng Ma
b
, Rustam Shadiev
c
*and
Shu-Yu Chen
b
a
Graduate Institute of Network Learning Technology, National Central University, Jungli, Taiwan;
b
Department of Computer Science and Information Engineering, National Central University,
Jhongli, Taiwan;
c
Department of Engineering Science, National Cheng Kung University,
Tainan, Taiwan
Game-based learning activities that facilitate students’ listening and speaking skills
were designed in this study. To participate in learning activities, students in the
control group used traditional methods, while students in the experimental group used
a mobile system. In our study, we looked into the feasibility of mobile game-based
learning activities. One experiment was carried out and the results revealed that the
experimental-group students significantly outperformed the control-group students on
the verbal post-test. However, the performance of the two groups was equal on the
listening post-test. Two variables (the average score for the interactive jigsaw game
and the number of designed cards) were found to be the most important factors for
influencing students’ performance in the verbal post-test. Furthermore, most students
had positive perceptions toward learning activities that are supported by a mobile
system. These results suggest that game-based learning activities can significantly
improve students’ speaking skills if driven by a mobile system. Furthermore, these
results suggest that learning activities with a mobile system foster students to (1)
practice speaking English as a foreign language (EFL) more frequently as well as to
reflect on their speech; (2) create meaningful sentences and speak with greater
accuracy and confidence; and (3) practice speaking EFL in an authentic context.
Keywords: mobile game-based learning; situational context; EFL listening and
speaking
1. Introduction
Research on teaching and learning English as a foreign language (EFL) suggests that lis-
tening and speaking are core elements of interaction and facilitate language acquisition
(Cohen, 2012; Ghoneim, 2013). However, there are limited opportunities to speak
English or listen to a speech in English in countries where the native language is not
English (Shadiev, Hwang, Huang, & Liu, 2015). According to Cheon (2003), Shadiev
et al. (2015), and Tsou (2005), some factors that lead to the lack of verbal and aural prac-
tice are the absence of an English context, deficiency of good spoken English examples,
and excessive dependence upon traditional teaching techniques.
To address this issue, related literature suggests constructing surroundings or envi-
ronments in which students have an opportunity to use the foreign language by com-
municating and practicing their EFL listening and speaking skills (Hwang & Chen,
2013). Furthermore, it is suggested that students’ EFL learning experiences can be
extended through the use of mobile (Crescente & Lee, 2011; Huang, Huang, Huang,
*Corresponding author. Email: rustamsh@gmail.com
Ó2015 Taylor & Francis
Computer Assisted Language Learning, 2015
http://dx.doi.org/10.1080/09588221.2015.1016438
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

& Lin et al., 2012) and digital game-based applications (Prensky, 2012). For example,
mobile technology and games provide opportunities for real communication, encour-
age student interaction and feedback, and engage students in collaborative learning
(Griva, Semoglou, & Geladari, 2010; Kinzie & Joseph, 2008). With mobile technol-
ogy, students may learn anywhere and anytime (Crescente & Lee, 2011), and learning
content can be adapted to the context a student finds himself/herself in (Norbrook &
Scott, 2003). On the other hand, digital games make learning fun and relaxing (Sand-
berg,Maris,&deGeus,2011) and increase students’ attention, interest, and effort in
the language learning process (Kinzie & Joseph, 2008;Liu&Chu,2010).
Our literature review shows that not much attention has been paid to relevant research
on improving EFL speaking and listening skills by using a combination of mobile- and
game-based applications. Therefore, some game-based and context-situated learning
activities to facilitate listening and speaking skills were designed in this study. Control-
group students learned and participated through traditional teaching methods, while
experimental-group students used a mobile system. This study tested how feasible
mobile-or game-based learning activities can be to enhance English learning.
2. Literature review
2.1 Foreign language learning theory and teaching methods
Various theories were proposed to explain how people learn a foreign language. Harmer
(2007) distinguished two functions of the language learning process: language input
when information is received (listening and reading) and language output when informa-
tion is applied (speaking and writing). Harmer (2007) further claimed that both meaning-
ful and comprehensible input and output are essential (if not sufficient) elements for
foreign language acquisition, and a balance should be kept between them.
Cohen (2012) has argued that communicative ability is an important factor in the target
language acquisition, and thus more focus should be given to it. Communicative ability
includes, among many other aptitudes, the ability to speak, listen, and interact. In order to
enhance communicative ability, various teaching methods have been proposed. One
approach is the communicative teaching method (Celce-Murcia, 2007), which is based on
the idea that learning a language successfully lies in knowing not only the structure and
forms of the language, but also its function and purposes. This approach emphasizes mean-
ing-based communication rather than practice of grammatical forms in isolation (Light-
bown & Spada, 2006). That is, when students are involved in real communication, their
natural strategies for language acquisition will be used, and this will allow them to learn
how to use the language (Celce-Murcia, 2007). Therefore, to facilitate communicative abil-
ity, Harmer (2007) recommended that instructors design such learning tasks in which stu-
dents have an opportunity to use language as a vehicle for communication. Furthermore, a
situation in which the language is used needs to be realistic. That is, all words and senten-
ces must grow out of some real situation and the meanings of words should be tied up with
the situations in which they are used. Related literature suggests that different real situa-
tions can be created outside of the classroom, in surrounding contexts (i.e., the environ-
ment, the situation, and body of information that elicits the use of language). Hwang,
Chen, Shadiev, Huang, and Chen (2012) suggested that the surrounding learning context
should be rich for language usage and it should envelop students’ lives on a daily basis.
Learning activities based in surrounding contexts to enhance students’ communicative
abilities were designed in this study. In the learning activities, students created sentences,
2W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

spoke them aloud, and listened to their own and peers’ sentences. The surrounding con-
text in this study was the school district; all students lived in the same school district. Stu-
dents could use this surrounding context to describe it in English, to create learning
material (i.e., game cards), to find clues when playing games, and to interact with that
context in English.
2.2 Mobile learning
Mobile technology provides various advantages for learning in terms of learning any-
where and anytime, while considering one’s preferences (Crescente & Lee, 2011;
Sandberg et al., 2011). Furthermore, mobile technology provides the possibility to adapt
learning content to the context in which a student finds himself (Norbrook & Scott,
2003). Therefore, mobile-assisted language learning has been successfully implemented
in many studies.
The HELLO learning environment was created by Liu (2009) to enhance students’
language abilities through their participation in various learning activities. Evaluation
results showed that the HELLO environment and associated learning activities improved
students’ EFL listening and speaking skills. The Ubiquitous English Vocabulary Learning
System was developed by Huang et al. (2012). This system aimed to help students experi-
ence a systematic vocabulary learning process. The Language-learning Outside the
Classroom with Handhelds environment was proposed by Ogata et al. (2008) for students
to integrate the knowledge acquired in the classroom with the real needs in their daily
lives. Using this environment, students ventured around town to fulfill tasks related to
typical daily activities and then shared their individual knowledge and experiences.
Hwang, Huang, Shadiev, Wu, and Chen (2014) designed learning activities supported by
the EngLearn mobile system to enhance elementary school students’ EFL listening and
speaking skills. Hwang and his colleagues examined the differences in students’ practice
methods and how these variances influenced their language proficiency.
Informed by previous research, this study utilized mobile technology to conduct lan-
guage learning via familiar surrounding contextual support. That is, mobile-based learning
methods enabled students to learn English in and make contact through a familiar context.
2.3 Digital game-based learning
Kinzie and Joseph (2008) defined a game as “a voluntary and enjoyable activity in which
a player pursues a challenging goal based on the game rules.” Prensky (2012) proposed a
term, “Digital Game-Based Learning,” to denote the use of computer games in delivering
educational content. Educational digital games came along with many advantages that
were highly recognized in previous studies. For example, gaming contexts are potentially
rich venues to deliver language learning (Belz & Reinhardt, 2004), and games provide
opportunities for real communication and encourage meaningful interaction and helpful
feedback among student participants (Thorne, Black, & Sykes, 2009; Shadiev, Hwang, &
Huang, in press). Reinhardt (2014), Shadiev et al. (2015), and Thorne et al. (2009) argued
that educational games bridge the gap between the classroom and the real world. Besides,
digital games are highly motivating; thus, they make learning fun and relaxing (Sandberg
et al., 2011).
A considerable amount of literature has discussed applications of digital game-based
language learning. Griva et al. (2010) focused on developing students’ EFL skills through
interactive and physical activities. Students played and interacted through games, and
Computer Assisted Language Learning 3
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

practiced new language patterns. Evaluation results indicated the positive effects of the
game on students’ EFL verbal skills development. Sandberg et al. (2011) studied the
added value of mobile technology for game-based EFL learning. Different game types
included a quiz, a memory game, and a jigsaw puzzle. Evaluation results showed that stu-
dents who used mobile applications improved the most. Liu and Chu (2010) examined
how high school students learn EFL and how digital games affect their learning achieve-
ment and motivation. Evaluation results showed that incorporating games into the EFL
learning process could assist in the achievement of better learning outcomes and motiva-
tion compared to merely using a non-gaming method.
According to Gardner (2010), learning motivation is an inner process, which can sus-
tain a certain behavior. If students’ learning motivation could be aroused, they may listen
more carefully in class and consciously review lessons after school to achieve better per-
formance. Therefore, learning motivation is considered as the most influential factor in
foreign language acquisition. Thus, maintaining motivation for longer periods of time
may develop students’ learning interests (Keller, 2010).
Game-based learning activities for EFL learning were designed in this study following
general recommendations of previous related research. This study utilized games to pro-
vide rich context for language learning, to enable students’ communication in the target
language, and to increase learning motivation.
3. Method
3.1 Participants and procedure
The experimental design was adopted in this study. A total of 40 female students from
one class in a girls’ senior high school were randomly assigned to control and experimen-
tal groups with 20 students in each. The experimental procedure is shown in Figure 1. All
students took a pre-test before the experiment. Four 30-minute EFL classes were con-
ducted weekly over a three-week period. During these classes, the same instructor taught
both groups the same learning content. The instructor provided both groups with direc-
tions and guidance to perform the learning activities. After class, both groups practiced
and applied what they learned during the class, but used different methods: the control
Figure 1. The experimental procedure.
4W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

group used a paper-and-pen method, while the experimental group used a mobile system.
The students in the experimental group were trained to use a system before engaging in
the learning activities. After the experiment, all students took a post-test, while the exper-
imental group also completed a questionnaire survey and participated in one-on-one
semi-structured interviews.
3.2 Learning activities
3.2.1 Week 1
3.2.1.1 Individual learning. In the beginning, students learned new words. New vocabu-
lary terms along with definitions in English and Chinese and related pictures were dis-
played for students. Pronunciation of the new vocabulary words was practiced. After that,
a sentence was displayed and read out loud (e.g., “I often play basketball during summer
vacation”) and nine cards were displayed. Cards showed related and unrelated pictures of
the vocabulary words (Figure 2(a)). The students had to select the pictures that matched
the words they heard in a sentence. They then formed a sentence by assembling selected
cards (Figure 2(b)).
3.2.1.1 “Jigsaw” game. In this game, students executed the same activity displayed during
individual learning. However, in the jigsaw game, a sentence was not displayed; students
were given a specific timeframe to complete the game, and their performance was scored.
3.2.2 Week 2
3.2.2.1 Interactive learning. Students were grouped in pairs and each pair was provided
with cards for the activity. Cards were pre-designed by the instructor; however, students
were allowed to design their own cards. Divided into pairs, Student A had to make a sen-
tence by assembling several cards and speak that sentence aloud to Student B. Student B
then had to identify words from the sentence, select cards that matched the words from
the sentence, and assemble cards together to form a sentence. Students swapped roles
after every spoken sentence.
Figure 2. Cards for the activity (a) and making a sentence with cards (b).
Computer Assisted Language Learning 5
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

What is very important in this activity is that students’ EFL learning experiences were
extended to a situational context. That is, students created their own cards with content
captured from real-life scenarios. Students used their own cards and surrounding context
to make sentences and to speak them aloud. Figure 3 shows that one student created the
sentence: “My aunt goes shopping once a week” by using her own cards (her aunt, a local
department store, and phrase “once a week”). Therefore, students were able to acquire the
language in the classroom as well as to apply it in daily life situations.
3.2.2.1 “Interactive jigsaw”. In this game, what students did was similar to that carried
out during interactive learning sessions. However, in this interactive jigsaw game, a sen-
tence was not displayed, but spoken aloud, and yet students were given a specific time-
frame to complete game, and their performance was scored.
3.2.3 Week 3
3.2.3.1 Card design and game. Students created their own cards from a surrounding
context. In this game, students were asked questions that could be answered by assem-
bling cards to form a sentence. Then students spoke their answers aloud. Other students
tried to identify words from the sentence and selected their own cards to assemble the sen-
tence. The student who identified the most cards in sentences spoken by other students
was the winner.
To ensure participation in the above-mentioned learning activities, the instructor
employed a traditional method for the control group and a mobile system for the experi-
mental group. For example, during week 1, vocabulary terms and sentences were printed
out on a piece of paper and read out loud by the instructor for control students. Mean-
while, experimental students could see and hear the same learning material via the use of
a mobile system.
3.3 Mobile system
This study developed a mobile system by using Apache, PHP, and MYSQL. The client
platform ran on Android and a Linux-based open source operating system, and the server
platform ran on a WindowsÒServer 2003. Figure 4 depicts an interface of learning activi-
ties on the mobile system.
Figure 3. Created sentence with cards designed by a student.
6W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

3.4 Data collection
EFL listening and speaking abilities were assessed before the experiment with a pre-test
and after the experiment with a post-test. The tests were developed by an experienced
senior high school teacher based on General English Proficiency Test (GEPT) guidelines
at the elementary level. Those who pass the elementary level of the GEPT have mastered
what the English textbooks of junior high schools teach (Shih, 2008), which is English
level equivalent to A2 on the Common European Framework of Reference for Languages
(CEFR). The items of the pre-test were similar to the items of the post-test, yet different
in content. The teacher considered the items’ difficulty levels for the tests’ designs and
developed test items with a similar difficulty index. The listening part of the test con-
tained three tasks all in multiple-choice question format: a picture description, a question
or statement response, and a short conversation. In the speaking part of the test, students
were asked to (1) listen to a sentence and then repeat it; (2) read aloud several sentences;
and (3) answer pre-recorded questions after they were broadcast twice. A sample of ques-
tions from the listening and speaking parts is provided in Appendix 1. The listening test
was scored on a 100-point scale and the speaking test was scored on a 21-point scale.
Three raters were involved in the marking process and big differences in the assessment
were resolved through discussions and obtaining a consensus. The teacher used the test
items in the same school for measuring the learning achievement of students of the same
age for several years, and the assessment results were consistent. Thus, this study suggests
that the test items were reliable under such conditions.
Experimental students’ learning behaviors and performances during the learning activi-
ties were recorded by the system. The following variables were derived from the data:
NWSL: the number of words a student listened to during the experiment
NSSL: the number of sentences a student listened to during the experiment
NSSP: the number of sentences students practiced in the jigsaw game
NJG: the number of times a student played the jigsaw game
ASJ: the average score for the jigsaw game
NSPI: the number of times a student played the interactive jigsaw game
ASIJ: the average score for the interactive jigsaw game
NOPRS: the number of original pictures on cards a student replaced by pictures
from the surrounding context
NC: the number of designed cards
As students in both groups could freely practice their skills after class outside of
school for two weeks, the data related to the duration of their practice was not available
for this study, but it is promising and useful information to be used in a future study.
The questionnaire survey (see Appendix 2) was developed by following the general
recommendations of Hwang, Shadiev, and Huang (2011), Keller (2010), and Shadiev
Figure 4. An interface of individual learning (a), the jigsaw game (b), and the interactive jigsaw (c).
Computer Assisted Language Learning 7
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

et al. (2014). The first part of the questionnaire included three dimensions, and it aimed to
explore experimental students’ perceptions and behavioral intentions:
Ease-of-Use (Items 111) using the system is free of physical and mental effort
Usefulness (Items 1221) the system is useful for learning
Behavioral Intention (Items 2225) involves two factors of how users perceive
the system: ease-of-use and usefulness (the behavioral intention is hypothesized to
be a major determinant of whether or not a student would continue to use the
system)
The second part of the questionnaire focused on the learning motivation of the experi-
mental-group’s students, which included four dimensions (Keller, 2010):
Attention (Items 15) is aroused and sustained due to learning activities
Relevance (Items 610) of learning content to tasks
Confidence (Items 1115) to complete learning tasks
Satisfaction (Items 1618) about outcomes in an effort to complete learning tasks
This study aimed to explore how students perceived the mobile system and whether
their learning motivation levels were high in a mobile game-based learning environment
with situational context. Therefore, the questionnaire survey was administered to the
experimental-group students only.
A total of 20 valid answer sheets to the questionnaire were obtained from 20 students
in the experimental groups. To analyze the questionnaire, this study utilized a five-point
Likert scale, anchored by the end-points: “strongly disagree” (1) and “strongly agree” (5).
Cronbach ato assess the internal consistency of the survey was adopted and the values
exceeded 0.73 in all dimensions, which demonstrated a satisfactory level of the items’
reliability.
One-on-one semi-structured interviews and subsequent data analyses were conducted
with all experimental-group students to explore their learning experiences with the sys-
tem, and to make relevant and logical inferences to the findings. During the interviews,
students were asked open-ended questions about how they felt learning to use the mobile
system. Each interview took approximately 30 minutes.
4. Results and discussion
An independent-samples t-test was conducted to compare scores of the control and exper-
imental groups on the EFL speaking and listening pre-test and post-test. The mean scores,
standard deviations, and results of the t-test are illustrated in Table 1. Levene’s test con-
firmed the equality of variances of both groups’ scores on the listening (FD0.275, pD
0.603) and speaking (FD0.320, pD0.575) portions of the pre-test and on the speaking
(FD0.386, pD0.538) post-test portion. Given a violation of Levene’s test for the homo-
geneity of the scores’ variances on the listening post-test (FD5.169, pD0.029), a t-test
was calculated without assuming homogeneous variances.
There was no significant difference in scores of the control (MD6.15, SD D2.85)
and experimental (MD6.80, SD D2.12) groups on the verbal pre-test; tD¡0.818, pD
0.418. This result suggests that two groups had equivalent EFL speaking abilities before
8W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

the experiment. However, the experimental group (MD10.40, SD D3.33) significantly
outperformed the control group (MD7.60, SD D3.08) on the verbal post-test; tD
¡2.758, pD0.009. Similar results were observed when the level of learning gained in
speaking by the experimental group was compared to the control group by using an inde-
pendent-samples t-test. This result suggests that students who learned using the support
of the system showed more prominent improvement in EFL speaking abilities than those
who learned using only the support of traditional means. The traditional means that con-
trol students used for learning were more cumbersome to perform and to use in situational
contexts compared to the mobile system. Therefore, experimental-group students prac-
ticed and applied new knowledge in situational contexts more efficiently and frequently
than evidenced by the control-group students. The benefits of the mobile system to the
experimental-group students’ EFL acquisition are discussed in the following four parts.
The mobile system enabled students of the experimental group to learn and practice
new vocabulary, pronunciation, and example sentences, yet these students also applied
that new knowledge in daily life situations. When experimental-group students verbalized
vocabulary words and sentences, they could record their own voices and share audio-
recorded files with peers. Later, the experimental-group students listened to their own
recorded files to evaluate their own performance. These students listened to their peers’
recorded files to learn from them, to compare them with their own audios, and to identify
mistakes in their own recorded speech. After mistakes were identified, students could
modify, improve, and rerecord their audios. This learning process was based on the com-
municative teaching methods, and it enabled students to practice their speaking skills fre-
quently, to reflect on their own speaking abilities, and to improve them. Celce-Murcia
(2007) and Lightbown and Spada (2006) suggested that executing the key elements of the
communicative teaching method, such as practicing, imitating, contrasting, and conclud-
ing, facilitates language acquisition.
The mobile system enabled experimental-group students to share their audio-recorded
files with peers. Students knew that their peers and the instructor would listen to the shared
files; therefore, students tried to make grammatically correct, meaningful sentences and
verbalize them very clearly. In this way, peers could comprehend audio-recorded speech
easily and provide some feedback. Hwang et al. (2014) and Lightbown and Spada (2006)
suggested that EFL students try to speak to others as well as to make their speech and inter-
action patterns meaningful and understandable; such a process can aid language compre-
hension and promote its acquisition. Furthermore, Celce-Murcia (2007) and Lightbown and
Table 1. The results of the assessment and t-test for EFL speaking and listening abilities.
Control group
(nD20)
Experimental
group (nD20) Sig.
Assessment Mean SD Mean SD FSig. t(two-tailed)
Speaking
Pre-test 6.15 2.85 6.80 2.12 0.320 0.575 ¡0.818 0.418
Post-test 7.60 3.08 10.40 3.33 0.386 0.538 ¡2.758 0.009
Listening
Pre-test 70.95 12.83 69.30 13.67 0.275 0.603 0.394 0.696
Post-test 67.83 17.88 71.83 9.08 5.169 0.029

¡0.893 0.377
Computer Assisted Language Learning 9
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

Spada (2006) suggested that, according to the communicative language teaching method,
students need to focus not only on the structure and forms of a language, but also on the
function and purposes that a language serves in different communicative settings.
Because the mobile system was easy to carry to surrounding contexts, it enabled experi-
mental-group students to practice what they learned in the classroom setting and apply new
knowledge in real situations. Surrounding contexts helped experimental-group students to
create meaningful learning material (game cards). With such learning material, students
were willing to make more interesting sentences and to practice speaking more frequently.
Furthermore, the experimental-group students acquired new knowledge in the classroom
and later applied that new knowledge in real life scenarios outside of school. When stu-
dents played a game, clues from the surrounding context helped them to finish it. Such a
learning process was based on the situational language teaching method (Celce-Murcia,
2007). Students in the study of Hwang et al. (2011), Hwang and Chen (2013), and Hwang
et al. (2014) learned in a familiar context, and it helped students to reflect on what they
learned; students were exposed to that context more frequently. Moreover, when students
acquired knowledge in the contextual scenarios, they were more inclined to learn and, in
turn, applied that knowledge to solve daily life problems. In the study of Huang et al.
(2012), students in the contextual environment learned more effectively since they could
use contextual cues to define the meaning of new words and sentences.
Finally, the students’ levels of learning motivation were high during the learning activi-
ties due to elements of a game and surrounding context; thus, students kept practicing their
speaking skills (Gardner, 2010;Keller,2010). Such a learning process leads to more fre-
quent practicing of speaking skills, learning with meaningful contextual material, producing
comprehensive output, and surely to enhancing students’ speaking abilities. This finding is
consistent with other related studies. In the study of Huang et al. (2012), students’ perfor-
mance and interests were promoted through learning and practicing the target language by
using technology. Learning objects in a familiar context and using elements of a game, as
introduced in the study of Hwang et al. (2012) and Hwang et al. (2014), helped to maintain
students’ interest in situated learning scenarios, and inspired students to describe the target
objects clearly and thoroughly, which further enhanced performance.
There was no significant difference in scores of the students in the control (MD70.95,
SD D12.826) and experimental (MD69.30, SD D13.669) groups on the listening pre-
test; tD0.394, pD0.696. There was also no significant difference between the control
(MD67.83, SD D17.878) and experimental (MD71.83, SD D9.079) groups on scores
of the listening post-test; tD¡0.893, pD0.377. Results of the t-test showed similar
results when the level of learning gained in listening was compared between the control
and experimental students. This result suggests that there was no difference in EFL listen-
ing abilities between the control and experimental groups before and after the experiment
and that no learning as gained. The main reason for such a finding is that students could
not easily identify words in sentences spoken aloud by peers without contextual support.
That is, without being told in what context cards were designed, students had no clues of
the context to help them finish a game. Our future study will address this issue. Particu-
larly, when students design cards, they will be instructed on how to make a related context
more explicit to peers. Furthermore, a multi-stage game will be designed in the future,
e.g., the first stage will take place in the classroom, the second stage will occur at the
entrance of the school building, and the third stage will be situated in the school’s court-
yard. Perhaps, students will identify the context more easily by using clues, and they can
therefore finish a game more successfully. In our future study, we will employ a smart
sensor, e.g., a global positioning system (GPS) (Huang et al., 2012; Ogata et al., 2008), to
10 W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

identify the surrounding context in which the learning material was created. Furthermore,
the mobile system may suggest pairing partners to play games, particularly individuals
who are close in location. The future mobile system may also suggest a game based on
players’ profiles and surrounding contexts.
According to the results, listening scores on the pre-test are higher, though not signifi-
cantly, compared to the listening scores on the post-test. Perhaps, this result is due to lim-
ited opportunity that control-group students had to practice their listening skills; they
could only practice in class and for a short time. Since the control-group students did not
practice enough, their scores on the listening post-test were lower compared to the scores
indicated on the pre-test.
Descriptive statistics of the experimental-group students’ learning behaviors during
learning activities are shown in Table 2.
A relationship between the independent research variables (i.e., NSSP, NJG, ASJ,
NSPI, ASIJ, NOPRS, and NC) and the verbal post-test scores was explored by employing
a Pearson product–moment correlation coefficient. According to the results (Table 3),
there was a correlation between the verbal post-test scores and two variables: the average
score for the interactive jigsaw game (rD0.597, pD0.005) and the number of designed
cards (rD0.511, pD.021). Furthermore, a Pearson product–moment correlation coeffi-
cient showed that only the number of designed cards had a significant correlation with the
learning gained in speaking, rD0.506, pD0.023.
Independent research variables were used in a stepwise multiple regression analysis to
predict verbal post-test scores. The correlation of the variables is shown in Table 4.In
step 1 of the analysis, the average score for interactive jigsaw game (ASIJ) was entered
into the regression equation, and was significantly related to the verbal post-test scores, F
(1, 18) D9.974, p<0.01. The multiple correlation coefficient was 0.59, indicating
Table 2. Means and standard deviations of experimental students’ learning behaviors.
Variable Mean SD
The number of words students listened to 280.70 295.57
The number of sentences students listened to 124.85 119.76
The number of sentences students practiced in the jigsaw game 138.85 90.21
The number of times students played the jigsaw game 13.25 7.71
The score for the jigsaw game 385.92 171.80
The number of times students played the interactive jigsaw game 12.80 5.03
The score for the interactive jigsaw game 170.84 78.48
The number of original pictures on cards replaced by pictures from
surrounding context
27.80 12.17
The number of designed cards 6.65 6.17
Table 3. The results of a Pearson productmoment correlation coefficient.
ASIJ
a
NC
b
Speaking post-test scores Pearson .597 .511
Sig. (two-tailed) .005 .021
Notes:
a
ASIJ: the average score for the interactive jigsaw game.
b
NC: the number of designed cards.
Computer Assisted Language Learning 11
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

approximately 35.7% of the variance of verbal post-test scores could be accounted for by
ASIJ. Other variables did not enter into the equation in step 2 of the analysis. Thus, the
regression equation for predicting the verbal post-test scores was
Predicted verbal post-test scores D.025 £ASIJ ¡6.070.
The interactive nature of the jigsaw game could explain why ASIJ significantly corre-
lated with verbal post-test scores as well as why ASIJ was found as a predicting variable
for the verbal post-test scores. First, students could indirectly interact with one another. Stu-
dents created cards, made sentences, and spoke them aloud for peers so that peers could lis-
ten to recorded speaking and identify words in a sentence. Both the students who created
the learning material and the students who played a game tried their best; the former tried
to make the learning material more understandable, and the latter tried to successfully finish
a game. Second, during the interactive game, students interacted with the learning material
through the mobile system, particularly, with photos taken in daily life situations and ele-
ments of the surrounding context. Such interaction helped students to efficiently finish the
interactive jigsaw game and later to perform well on the verbal post-test.
The number of designed cards (NC) has a significant correlation with verbal post-test
scores as well as with speaking learning performance. The following is a possible expla-
nation of this finding. Students designed cards in advance; they interacted with real-life
scenarios and took pictures of the surrounding context for their cards. During the card-
designing process, students thought about how to describe pictures they had taken and
how to verbally verbalize these descriptions. Students tried to create more cards so that
later they used as many as they could for creating sentences and speaking aloud during
learning activities. Such a learning process created an opportunity for students to learn
actively and to practice EFL speaking more frequently. As a result, students could learn
and remember more vocabulary words, master their speaking skills, and become more
confident in their English-speaking skills. This finding is in line with other related studies.
In the study of Liu (2009) and Ogata et al. (2008), students who performed well usually
learned actively, took full responsibility for their learning, and engaged in metacognitive
processes. In the study of Hwang et al. (2012) and Hwang et al. (2014), students who
practiced more frequently and put effort into their practice sessions could easily apply
what they learned in class. They also performed very well on the exam.
Statistical analysis showed that although the average score for the interactive jigsaw
game was significantly correlated with the verbal post-test scores, it had no significant
correlation with the speaking improvement. On the other hand, the number of designed
cards significantly correlated with the verbal post-test scores and learning gained in
speaking. However, it could not predict speaking abilities on the post-test.
From interviews with the experimental students, it is implied that students’ prior
knowledge levels are the explanation for this finding. Students mentioned that typically
those who have higher prior knowledge levels are more likely to obtain high scores on
the interactive jigsaw game and on the post-test. However, not all students with high prior
knowledge levels exerted greater efforts to create more cards and to make more senten-
ces. Such learning behavior was not always helpful in attaining better learning outcomes.
Table 4. The results of stepwise multiple regression.
Model Predictive variable FSig. RR
2
BbtSig.
1 ASIJ
a
9.974 .005 .597 .357 .025 .597 3.158 .005
Notes:
a
ASIJ: the average score for the interactive jigsaw game.
12 W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

Students also mentioned that, on the other hand, those with lower levels of prior knowl-
edge usually do not perform well on the interactive jigsaw game and on the post-test.
However, those students attempted to design more cards and to make more sentences.
Such learning behavior helped students with low prior knowledge levels to enhance learn-
ing. Therefore, learning acquisition is a very important variable when it comes to consid-
ering students’ prior knowledge levels.
Appendix 2 shows the results of the questionnaire survey analysis. According to the
results, most students gave the items of Ease-of-Use, Usefulness, and Behavioral Inten-
tion high scores. It demonstrates that, in general, students perceived that the mobile sys-
tem was easy-to-use, and it was useful in learning; yet, most of the students responded
affirmatively to questions about using the system for learning in the future. Results of the
interviews’ data analysis also strengthened the findings of students’ high recognition for
the mobile learning system. The following content was abstracted from five interviews:
I can practice my listening and speaking skills by playing Interactive Jigsaw Game.
It’s easier to understand and remember new words if I take related to them photos.
Taking photos for designing cards in surrounding context is a nice way to learn in real life
scenario.
Scenarios I used for designing cards could help me to recall sentences I made.
I think my English speaking ability has improved.
There were two items ranked the lowest: item 8, “Card design is easy for me” (MD
3.4, SD D0.80) and item 20, “Card design can help me understand words with context
and improve my ability of making sentence effectively” (MD3.40, SD D0.49). With
regard to item 8, in this study, after a student made a sentence by using cards, those
vocabulary cards could not be rearranged or replaced in a sentence. Students felt that the
system is not convenient, particularly when used in a situation when they wanted to revise
a sentence or add a more interesting and meaningful card. As for item 20, students could
not use the system for learning activities as long as they liked due to the time limit. There-
fore, some students learned or played less, and some students did not create cards at all.
We will address these issues in our future study. Particularly, the system functions will be
modified so that students will be able to edit cards or change the order in a sentence.
Learning activity design will also be improved to ensure that students have enough time
to learn by using the mobile system.
According to Appendix 2, most students gave high scores to the items related to their
motivation. This suggests that, in general, students had a high degree of motivation to learn
in a mobile game-based learning environment with a situational context. This finding was
confirmed by interview results. The following content was derived from four interviews:
It is interesting to learn by using tablet PC and this way to learn increases my interest in English.
It’s funny to play “Interactive Jigsaw Game,” it is like playing a real game; the game moti-
vates to play it and learn English.
Interactive Jigsaw Game” helps me to develop my confidence to speak in English.
I am happy to see that others can understand what I said and we can correct own mistakes.
Meanwhile, it was found that item 7, “I had enough time to learn on this system” was
ranked the lowest (MD3.25, SD D0.54). Students were asked to design cards and to
Computer Assisted Language Learning 13
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

play interactive games after class time. Some students complained that there was a lot of
homework assigned to other subjects so that they had no time to go out and take pictures
for their cards. Another issue that some students brought up was Internet access. To play
interactive games with others, students needed access to the Internet; however, not many
places with access to the Internet were available. These issues will be addressed in our
future research. Perhaps, it will be effective if several Internet access points are created in
areas around the school community for students to use. In such cases, students could take
pictures, create cards, and play individual games wherever they please, and they could
use provided access points to play interactive games.
5. Conclusion
Game-based learning activities in situational contexts were designed in this study to facili-
tate students’ listening and speaking skills. In learning activities, students created sentences,
spoke them aloud, and listened to their own and their peers’ spoken sentences. More impor-
tantly, students used surrounding contexts to design meaningful learning material (game
cards). When playing games, students could find clues in surrounding contexts to help fin-
ish the games. Therefore, the proposed approach enabled students to acquire the language
not only in the classroom, but also outside of it, in the surrounding context. The feasibility
of game-based learning activities was investigated in this study. Evaluation results revealed
that there was no significant difference between the experimental and control groups in the
listening post-test. However, the experimental group significantly outperformed the control
group in the verbal post-test. The average score for the interactive jigsaw game and the
number of designed cards were found as the most important variables in the relationship
with verbal post-test scores. Based on the results, this study concludes that game-based
learning activities in situational contexts, supported by the mobile system, effectively facili-
tated students speaking skills in the following ways: (1) students practiced EFL speaking
frequently and reflected on their speaking; (2) students created meaningful sentences and
spoke them aloud more correctly and comprehensibly; and (3) students practiced their
speaking skills in surrounding contexts. Finally, students’ learning motivation levels were
high and most students demonstrated positive perceptions toward the novel approach.
One may argue that many other studies (Huang et al., 2012; Hwang et al., 2014; Liu,
2009; Ogata et al., 2008) have also applied similar mobile technologies and obtained the
same results. This study differs from others in the way that mobile technology was
applied; this study utilized it to conduct game-based language learning. Apart from taking
pictures and describing them outside of school in a familiar surrounding context, our
application has also required playing, interacting in, and practicing the target language
through games. In contrast to other studies on game-based learning (Griva et al., 2010;
Liu & Chu, 2010; Sandberg et al., 2011), students in this study have designed their own
learning material for the game with content captured in a familiar surrounding context.
Such approach, coupled with elements of the game, resulted in increased students’ lan-
guage learning motivation.
This study made three main contributions. First, in order to make EFL learning more
meaningful and interesting, this study combined the communicative and situational lan-
guage teaching methods to foster learning through game-based learning activities with sit-
uational contexts in traditional and mobile learning environments. Second, this study
compared the differences in students’ performances with and without technology during
learning activities to explore the effectiveness and pedagogical potential of the novel
14 W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

approach. Third, several guidelines and suggestions for a future design and research were
provided in this study.
A few limitations need to be acknowledged about this study and addressed in the future.
The first limitation of this study was a relatively small sample size was used of only female
participants, which may limit the broad generalization of the results. The long-term impact
of the instruction in the traditional and mobile learning environments was not considered in
this study. In our future study, we will address these limitations as well as extend the capac-
ity of the system. For example, in our future study, students will be provided with feedback
so that they can immediately correct mistakes in the learning materials or output they con-
struct themselves. Locations in which learning materials were created will be recorded and
shared so that students will not be confused about the surrounding context during learning
activities and, perhaps, their listening skills will also be enhanced. Finally, adding new
vocabulary terms from surrounding contexts to learning activities and learning new gram-
mar rules by using the system will be considered in our future study.
Disclosure statement
No potential conflict of interest was reported by the authors.
Funding
This research is partially supported by the “International Research-Intensive Center of Excellence
Program” of NTNU and the Ministry of Science and Technology, Taiwan, R.O.C., under Grant no.
NSC 103-2911-I-003-301, NSC 102-3113-P-006-019-, MOST 103-2511-S-006-007-MY3, and
MOST 103-2511-S-006-002-MY3.
Notes on contributors
Dr. Wu-Yuin Hwang is a distinguished professor at the Graduate Institute of Network Learning
Technology, National Central University, Taiwan. His research interests include computer assisted
language learning, HCI, and knowledge construction.
Dr. Timothy K. Shih is a professor at the National Central University, Taiwan. He was the dean of
College of Computer Science, Asia University, Taiwan and the Department Chair of the CSIE
Department at Tamkang University, Taiwan. He is a fellow of the Institution of Engineering and
Technology (IET). In addition, he is a senior member of ACM and a senior member of IEEE. Dr.
Shih also joined the Educational Activities Board of the Computer Society. His research interests
include e-learning systems, video processing, and interactive multimedia and music.
Zhao-Heng Ma is a doctoral student at the Department of Computer Science and Information Engi-
neering, National Central University, Taiwan. His research interest includes e-learning and game-
based learning.
Dr. Rustam Shadiev is a postdoctoral research fellow at the Department of Engineering Science,
National Cheng Kung University, Taiwan. His research interests include computer assisted lan-
guage learning, human-computer interaction for collaboration, and applications of speech to text
recognition for learning.
Shu-Yu Chen graduated from Master program of the Department of Computer Science and Infor-
mation Engineering, National Central University, Taiwan. Her research interest includes e-learning,
game-based learning, and learning English as a foreign language.
Computer Assisted Language Learning 15
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

References
Belz, J.A., & Reinhardt, J. (2004). Aspects of advanced foreign language proficiency: Inter-
netmediated German language play. International Journal of Applied Linguistics, 14(3),
324362.
Celce-Murcia, M. (2007). Rethinking the role of communicative competence in language teaching.
In E.A. Soler, & M.P.S. Jorda (Eds.), Intercultural language use and language learning
(pp. 4157). Dordrecht: Springer.
Cheon, H. (2003). The viability of computer mediated communication in the Korean secondary EFL
classroom. Asian EFL Journal, 5(1), 161.
Cohen, A.D. (2012). Comprehensible pragmatics: Where input and output come together. In M.
Pawlak (Ed.), New perspectives on individual differences in language learning and teaching
(pp. 249261). Berlin, Heidelberg: Springer.
Crescente, M.L., & Lee, D. (2011). Critical issues of m-learning: Design models, adoption pro-
cesses, and future trends. Journal of the Chinese Institute of Industrial Engineers, 28(2),
111123.
Gardner, R.C. (2010). Motivation and second language acquisition: The socio-educational model.
New York, NY: Peter Lang.
Ghoneim, N.M.M. (2013). The listening comprehension strategies used by college students to cope
with the aural problems in EFL classes: An analytical study. English Language Teaching, 6(2),
100112.
Griva, E., Semoglou, K., & Geladari, A. (2010). Early foreign language learning: Implementation of
a project in a game-based context. Procedia Social and Behavioral Sciences, 2(2),
37003705.
Harmer, J. (2007). The practice of English language teaching. London: Longman.
Huang, Y.M., Huang, Y.M., Huang, S.H., & Lin, Y.T. (2012). A ubiquitous English vocabulary
learning system: Evidence of active/passive attitudes vs. usefulness/ease-of-use. Computers &
Education, 58(1), 273282.
Hwang, W.Y., & Chen, H.S. (2013). Users’ familiar situational contexts facilitate the practice of
EFL in elementary schools with mobile devices. Computer Assisted Language Learning, 26(2),
101125.
Hwang, W.Y., Chen, H.S.L., Shadiev, R., Huang, Y.M. & Chen, C.Y. (2012). Improving English as
a foreign language writing in elementary schools using mobile devices in familiar situational
contexts. Computer Assisted Language Learning. Advance online publication. doi:10.1080/
09588221.2012.
Hwang, W.Y., Huang, Y.M., Shadiev, R., Wu, S.Y., & Chen, S.L. (2014). Effects of using mobile
devices on English listening diversity and speaking for EFL elementary students. Australasian
Journal of Educational Technology, 30(5), 503516.
Hwang, W.Y., Shadiev, R. & Huang, S.M. (2011). A study of a multimedia web annotation system
and its effect on the EFL writing and speaking performance of junior high school students.
ReCALL, 23(2), 160180.
Keller, J.M. (2010). Motivational design for learning and performance: The ARCS model approach.
New York, NY: Springer.
Kinzie, M.B., & Joseph, D.R. (2008). Gender differences in game activity preferences of middle
school children: Implications for educational game design. Educational Technology Research
and Development, 56(56), 643663.
Lightbown, P.M. & Spada, N. (2006). How languages are learned. Oxford: Oxford University Press.
Liu, T.Y. (2009). A context-aware ubiquitous learning environment for language listening and
speaking. Journal of Computer Assisted Learning, 25(6), 515527.
Liu, T.Y., & Chu, Y.L. (2010). Using ubiquitous games in an English listening and speaking course:
Impact on learning outcomes and motivation. Computers & Education, 55(2), 630643.
Norbrook, H., & Scott, P. (2003). Motivation in mobile modern foreign language learning. In J.
Attewell, G. Da Bormida, M. Sharples, & C. Savill-Smith (Eds.), MLEARN: Learning with
mobile devices (pp. 5051). London: Learning and Skills Development Agency.
Ogata, H., Hui, G.L., Yin, C., Ueda, T., Oishi, Y., & Yano, Y. (2008). LOCH: Supporting mobile
language learning outside classrooms. International Journal of Mobile Learning and Organiza-
tion, 2(3), 271282.
Prensky, M. (2012). Digital natives, digital immigrants. On the Horizon, 9(5), 16.
16 W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

Reinhardt, J. (2014). Special issue commentary: Digital game and play activity in L2 teaching and
learning. Language Learning & Technology, 18(2), 28.
Sandberg, J., Maris, M., & de Geus, K. (2011). Mobile English learning: An evidence-based study
with fifth graders. Computers & Education, 57(1), 13341347.
Shadiev, R., Hwang, W.Y., & Huang, Y.M. (in press). A pilot study of facilitating cross-cultural
understanding with project-based collaborative learning activity in online environment. Aus-
tralasian Journal of Educational Technology.
Shadiev, R., Hwang, W.Y., Huang, Y.M., & Liu, C.J. (2015). Investigating applications of speech to
text recognition for face to face seminar to assist learning of non-native English participants.
Technology, Pedagogy and Education. doi: 10.1080/1475939X.2014.988744
Shadiev, R., Hwang, W.Y., Yeh, S.C., Yang, S.J.H., Wang, J.L., Lin, H., & Hsu, G.L. (2014).
Effects of unidirectional vs. reciprocal teaching strategies on web-based computer program-
ming learning. Journal of Educational Computing Research, 50(1), 6795
Shih, C.M. (2008). The general English proficiency test. Language Assessment Quarterly, 5(1),
6376.
Thorne, S.L., Black, R.W., & Sykes, J. (2009). Second language use, socialization, and learning in
Internet communities and online games. Modern Language Journal, 93, 802821.
Tsou, W. (2005). Improving speaking skills through instruction in oral classroom participation. For-
eign Language Annals, 38,4655.
Appendix 1
Sample questions from the test
Listening part
C. Short conversation
(Man) I have an appointment with Dr. Smith at two o’clock. My son Tim has a cold.
(Woman) Please have a seat.
Question: Where did this conversation take place?
Computer Assisted Language Learning 17
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

Please select a correct answer:
A. In a concert.
B. In a market.
C. In a doctor’s office.
Speaking part
A. Listen to a sentence and then repeat it
My father has worked in a drug store for six years.
B. Read aloud several sentences
The post office is across from the train station.
C. Answer prerecorded questions after they are broadcasted twice.
What are you going to do today?
Appendix 2
A. Perceptions and behavioral intentions
No. Items Mean SD
1. The overall operation of the system is not difficult for me. 4.00 0.45
2. It’s easy to understand interactions of the system and it’s very clear. 3.95 0.59
The jigsaw game in learning mode is easy for me. 3.90 0.62
3. The jigsaw game in game mode is easy for me. 3.65 0.65
4. The interactive jigsaw game in offline mode is easy for me. 3.65 0.65
5. The interactive jigsaw game in online mode is easy for me. 3.45 0.74
6. “Cards Edit” is easy for me. 3.65 0.73
7. “Card Design” is easy for me. 3.40 0.80
8. “Vocabulary pronunciation” is easy for me. 3.55 0.59
9. “Sentence Pronunciation” is easy for me. 3.55 0.67
10. “Vocabulary Search” is easy for me. 4.10 0.70
11. The jigsaw game can help improve my English listening ability. 3.85 0.57
12. “Vocabulary Search” can help me learn other words. 3.95 0.50
13. The jigsaw game can help me remember words and sentences. 3.95 0.67
14. The jigsaw game can help improve my English listening ability. 3.95 0.59
15. The interactive jigsaw game can help improve EFL speaking ability. 3.95 0.50
16. The interactive jigsaw game can help me be brave to speak English. 4.00 0.55
17. The interactive jigsaw game can help me remember words and sentences. 3.75 0.54
18. “Cards Edit” can help me remember words and understand words more
deeply.
3.55 0.50
19. “Card Design” can help me understand words with context and improve
my ability of making sentence effectively.
3.40 0.49
20. In general, the system improved my interest and confidence of learning
English.
3.65 0.48
21. I would like to use this system to learn English. 3.70 0.46
22. I hope this system can be embedded into English learning course in
school.
3.80 0.40
23. I would like to continue using the system for learning in the future. 3.70 0.46
24. In general, I feel satisfied with the system to learn English. 3.80 0.40
18 W.-Y. Hwang et al.
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015

B. Learning motivation
No. Items Mean SD
1. This system increases my learning interest. 3.75 0.54
2. Different features of this system can catch my learning attention. 3.80 0.40
3. The games of the system catch my attention. 3.85 0.48
4. The interface and icons of the system catch my attention. 3.70 0.56
5. Learning activities of the system catch my attention. 3.75 0.54
6. What I learned from this system was helpful to improve my English ability. 3.85 0.48
7. I had enough time to learn on this system. 3.25 0.54
8. I knew what to learn every time I used the system. 3.8 0.51
9. What I learned on this system is practical and closely linked to my daily life. 3.75 0.62
10. What I learned on this system will be helpful for me in the future. 3.70 0.64
11. The difficulty level of the system fit me well. 3.90 0.62
12. The scoring of the system was fair. 3.55 0.59
13. My good performance on this system was due to my hard work. 3.90 0.44
14. I had confidence to learn on this system. 3.65 0.48
15. I had confidence to perform better on this system. 3.60 0.49
16. I’m satisfied with what I learned on this system. 3.70 0.46
17. I actively learned on this system. 3.70 0.46
18. Learning on this system made me happy. 3.65 0.73
Computer Assisted Language Learning 19
Downloaded by [National Cheng Kung University] at 18:30 02 March 2015