Content uploaded by Boglarka Brezovszky
Author content
All content in this area was uploaded by Boglarka Brezovszky on Sep 26, 2018
Content may be subject to copyright.
Running head: EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT
Effects of a Mathematics Game-Based Learning Environment on Primary School Students’ Adaptive
Number Knowledge
Boglárka Brezovszky1, Jake McMullen1, Koen Veermans1, Minna M. Hannula-Sormunen1,2, Gabriela
Rodríguez-Aflecht 1, Nonmanut Pongsakdi1 Eero Laakkonen1 & Erno Lehtinen1
University of Turku
Author Note
1Centre for Research on Learning and Instruction and Department of Teacher Education, University
of Turku, Finland
2 Department of Teacher Education and Turku Institute for Advanced Studies, University of Turku,
Finland
Published in Computers & Education 128 (2019) 63-74
https://doi.org/10.1016/j.compedu.2018.09.011
Corresponding Author:
Boglárka Brezovszky
Centre for Research on Learning and Instruction, Department of Teacher Education
Turun Yliopisto, 20014, Finland.
E-mail: bogbre@utu.fi
Funding
This work was supported by the Academy of Finland [grant number 274163; PI: Erno Lehtinen] and
the Academy of Finland DECIN project [grant number 278579; PI: Minna Hannula-Sormunen].
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 1
1. Introduction
Flexible mathematical thinking and the adaptive use of arithmetic strategies have been
highlighted by researchers of mathematics education (Baroody, 2003; Nunes, Dorneles, Lin, &
Rathgeb-Schnierer, 2016) and flexibility is an important aim of national mathematics education
standards and curricula in many countries (e.g., Finnish National Agency for Education, 2014; NCTM
2014). Flexibility is necessary to apply problem solving procedures in new contexts and across
different representations, which is crucial for a transferable mathematical proficiency to everyday life
(NCTM 2014; Nunes et al., 2016). However, pedagogical recommendations and models for enhancing
flexibility and adaptivity are rare and mostly focused on teaching the use of a few strategies
(Verschaffel, Luwel, Torbeyns, & Van Dooren, 2009), which are rarely applied in practice in spite of a
strong curricular emphasis (Hickendorff, 2017). One explanation for this disconnect could be that, in
order to flexibly apply arithmetic strategies in varying situations, it is not enough to teach these
strategies, but students need to develop a well-connected mental representation of the natural
number system which makes it possible for them to notice when and which strategies are applicable
(Baroody, 2003; Brezovszky et al., 2015; Lehtinen, et al., 2015; Threlfall, 2002, 2009; Verschaffel et al.,
2009).
Indeed, regular classroom instruction may struggle to provide students with the opportunity to
experience the rich relations of numbers and operations (Baroody, 2003; Threlfall, 2009) and the type
of practice needed for the development of flexible and adaptive mathematical skills (Lehtinen,
Hannula-Sormunen, McMullen, & Gruber, 2017). Thus, it is important to study if game-based learning
could be used in enhancing this highly emphasized, but pedagogically under-supported, aim of
mathematics education. The open-ended and flexible nature of game-based learning environments
can be ideal to support this intensive practice in an engaging way (Devlin, 2011; Ke, 2009; Squire,
2008), and the technological affordances allow the training to be scalable (Rutherford et al., 2014).
These features of the game-based learning environment could provide teachers with an ideal
complementary tool for enriching their methods in developing flexibility and adaptivity with
arithmetic problem solving. The present study explores the potential of the Number Navigation
Game (NNG) in strengthening fourth, fifth, and sixth grade students’ flexible and adaptive arithmetic
skills. Gameplay with the NNG is expected to develop a more well connected representation of the
arithmetic relations between natural numbers in students, which should enable them to become
more adaptive in their arithmetic problem solving.
1.1. Developing flexibility and adaptivity with arithmetic problem solving by strengthening
students’ adaptive number knowledge
The definition of flexibility varies greatly, but in general it is described as: (a) having a rich
repertoire of arithmetic problem solving procedures, (b) being able to flexibly switch between these
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 2
procedures, and (c) being adaptive with selecting strategies efficiently (Heinze, Star, & Verschaffel,
2009; Star & Rittle-Johnson, 2008; Verschaffel et al., 2009). It is problematic however that the
meaning of flexible and adaptive can be very subjective, as it might depend on the characteristics of
the problem, on personal preferences, or the context (Verschaffel et al., 2009), and efficient solutions
can sometimes be described as elegant and sometimes as quick or fast (Heinze et al., 2009). With so
much variability in what it means to be flexible and adaptive with arithmetic problem solving it is
equally unclear what type of instructional design can support the development of this knowledge and
skills.
It is suggested that instead of focusing on building a strategy repertoire, more emphasis should be
given to supporting students in developing an understanding of the qualities of numbers and
numerical relations (Threlfall, 2009). Studies show that both mathematically proficient primary school
students (Blöte, Klein, & Beishuizen, 2000; Blöte, Van der Burg, & Klein, 2001) and expert
mathematicians (Star & Newton, 2009) are likely to take into account the characteristics of numbers
in a problem when deciding on solution strategies. This approach emphasizes the subjective nature of
flexibility and shifts the focus towards the process of problem solving during which efficient strategies
emerge and are applied adaptively as important features of a problem become visible (Baroody, 2003;
Schneider, Rittle-Johnson, & Star, 2011; Threlfall, 2002, 2009; Verschaffel et al., 2009). From a more
practical perspective, it is suggested that for students to recognize important features of a problem
they need to develop a well-connected understanding of numerical relations for which they need
opportunities to practice with various number combinations and numerical relations making their
own discoveries (Baroody, 2003; Verschaffel et al., 2009).
The present study tested the efficiency of a game-based learning environment for strengthening
these numerical relations defined as adaptive number knowledge. Adaptive number knowledge is a
component of adaptivity with whole-number arithmetic and it refers to the well-connected, network-
like knowledge of numerical characteristics and arithmetic relations between numbers (McMullen et
al., 2016, 2017). Adaptive number knowledge enables the recognition and use of key numbers in
adaptive arithmetic problem solving. This can include recognizing numbers with many factors or
multiples, recognizing numbers in equations that are close to other numbers that are easy to work
with, or the use of the base-ten structure and basic arithmetic principles like associativity or
commutativity (McMullen et al., 2017). Adaptive number knowledge is related to arithmetic fluency
and knowledge of arithmetic concepts and is a unique predictor of pre-algebra skills (McMullen et al.,
2017). This suggests that having a well-connected representation of numerical relations is not only
related to faster and more efficient problem solving, but also to a more complex understanding of
arithmetic relations, which can be foundational to algebraic thinking (McMullen et al., 2017; Nunes et
al., 2016).
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 3
Designing tools for developing adaptive number knowledge in the context of regular classroom
teaching can be challenging. In order for students to develop their own flexible procedures they
should be provided opportunities for extensive deliberate practice with various combinations of
numbers and operations (Baroody, 2003; Lehtinen et al., 2017; Verschaffel et al., 2009). This requires
extra resources from the teacher in developing and using individualized training methods. Indeed, the
few existing intervention studies which are not limited to teaching a few strategies but aim at
supporting students’ discovery of flexible and adaptive problem solving procedures are case studies
or have a highly controlled experimental design using small samples. For example, case studies by
Heirdsfield and collegues (Heirdsfield & Cooper, 2004; Heirdsfield, 2011) suggest that using external
representations like the number line or the number square can be useful in promoting students’
engagement in discovering their own strategies when performing mental arithmetic. Additionally,
there is evidence that reflecting on ones’ solutions, comparing and contrasting different solution
methods or re-solving the same problem using an alternative procedure can promote students’
flexibility both in mental arithmetic and in equation solving (Blöte et al., 2000; Rittle-Johnson & Star,
2009; Star & Seifert, 2006).
The present study tested the effectiveness of the NNG game-based learning environment in
strengthening primary school students’ adaptive number knowledge and related arithmetic skills.
Using a game-based format allows extensive opportunities for practice in an engaging open-ended
context that does not pre-define ‘optimal’ solutions but prompts students to reflect on the efficiency
of various alternatives. Through careful design, discovery learning and exploration are organic
components of game-based learning (Ke, 2009; Squire, 2008) and including appropriate rules and
challenges can promote reflection (Ke, 2008; Kiili, 2007). Additionally, due to technological
affordances, the training is scalable and can take into account individual differences (Rutherford et al.,
2014). This format made it possible to apply the training in naturalistic classroom settings on a large
scale, which was not possible in previous interventions with similar aims. It is expected that the NNG
would provide teachers with an ideal complementary tool for enriching their methods for developing
flexibility and adaptivity with arithmetic problem solving and provide a novel training that can be
flexibly used over three different grade levels
1.2. Developing adaptive number knowledge using game-based learning
Digital games can be powerful in presenting complex mathematical concepts as they can provide
an alternative media where interaction with and exploration of the content is inherent (Devlin, 2011;
Ke, 2009; Lowrie & Jorgensen, 2015). However, it is clear now that games as such do not represent a
magic bullet and they do not automatically make the learning experience more motivating or
effective (Wouters & Oostendorp, 2013; Wouters, van Nimwegen, van Oostendorp, & van der Spek,
2013; Young et al., 2012). As with most media, it is a question of a meaningful translation and
integration of the learning content into the game context which makes game-based learning effective.
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 4
Thus, the added value of game-based learning depends on the design process where the learning
content integrated and not just added on top of the game-based media. This allows students to
interact with the content in a novel and alternative way that would otherwise be unavailable or
difficult to achieve in the traditional classroom practice (Arnab et al., 2015; Devlin, 2011; Habgood &
Ainsworth, 2011).
In spite of these affordances of game-based learning, many interventions aim to simply develop
basic arithmetic skills or to strengthen already acquired skills (Cheung & Slavin, 2013; Li & Ma, 2010).
Empirically tested game-based learning environments which target the development of complex
mathematical skills and knowledge in primary school mathematics education are rare. Additionally,
some exiting examples can be limited by the domain, for example training multiplicative reasoning
(Bakker, van den Heuvel-Panhuizen, & Robitzsch, 2015) or the understanding of multiplicative
relations (Habgood & Ainsworth, 2011) and only few game-based learning environments aim for more
holistic mathematical learning outcomes such as flexibility and adaptivity with arithmetic problem
solving. Such examples include the game-based intervention by Pope and Mangram (2015) which
aimed at training students’ flexibility with numbers and operations and an understanding of
properties of numbers, or the large-scale intervention by van den Heuvel-Panhuizen, Kolovou, and
Robitzsch (2013) which aimed to promote the development of reasoning about relations between
quantities that are foundational to early algebra knowledge.
Although the learning aims of the NNG are quite close to the ones described by Pope and
Mangram (2015) and van den Heuvel-Panhuizen and collegues (2013), that is the flexible use and
recognition of numerical relations in arithmetic problem solving, the format of the NNG allows for a
different experience with number combinations and numerical relations. Additionally, unlike previous
game-based learning environments (e.g. Bakker et al., 2015; Habgood & Ainsworth, 2011) the NNG
aims to develop the understanding of numerical relations, without limiting the scope of the game to
one type of arithmetic operation (e.g. multiplicative or additive relations). Instead, the aim of the
game is to develop the recognition and use of various numerical relations within the system of natural
numbers as a whole. To our knowedge, the NNG represents the first attempt to develop elemantary
school students’ adaptive number knolwedge by strengthening their understanding of numerical
relations.
Based on empirical results and theoretical suggestions for developing adaptive number
knowledge, the game design should be able to (a) maintain students’ long-term engagement to
mentally execute and compare a large number of various equations in a meaningful way (Baroody,
2003; Threlfall, 2009; Verschaffel et al., 2009), (b) be flexible enough to promote students’ curiosity in
searching for alternative solutions (Rodríguez-Aflecht et al., 2018), (c) provide students with an
external representation that supports the formation of rich networks of numbers and operations
(Mulligan & Mitchelmore, 2013), and (d) promote reflection on the quality and efficiency of their
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 5
solution process (Heirdsfield, 2011; Rittle-Johnson & Star, 2009; Star & Seifert, 2006). As the learning
content is integrated within the core game mechanics (Devlin, 2011; Habgood & Ainsworth, 2011;
Salen & Zimmerman, 2003), progress in the NNG means performing mental calculations and
combining and comparing various alternative number-operation combinations. The hundred-square
was selected as an external representation as it can be flexibly used to work within the system of
natural numbers, decomposing and recombining numbers, and provides a clear visual representation
of numerical relations and the base-ten system (Laski, Ermakova, & Vasilyeva, 2014).
Figure 1. Example of a NNG map in the energy scoring mode (harbour at number 100, first target
material at number 44).
Within the game, different game modes were developed to engage players in searching for
alternative solutions and reflect on the efficiency of these different solutions. For example, on Figure
1 the game mode requires the player to minimize the sum of numbers used in their equations, so the
player has to look for key numbers which are useful to achieve this goal. One useful solution can be to
realize that both 90 and 88 are ideal to move forward using division with small numbers. The game is
open-ended, therefore there are no right or wrong answers but players need to select their
calculations considering the game rules and the position of islands (which define the available
numbers in a map) or the placement of starting and target numbers. Taking into account these
challenges, players need to mentally execute and compare several alternative routes when selecting
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 6
their calculations. It is this type of mental practice with various combinations of numbers and
operations which represents the core game action that is expected to develop students’ adaptive
number knowledge by strengthening the recognition and use of numerical characteristics and
relations during arithmetic problem solving (Brezovszky, Lehtinen, McMullen, Rodriguez, & Veermans,
2013; Brezovszky et al., 2015; McMullen et al., 2016, 2017).
Results of a pilot testing of the NNG showed that players executed around 180 calculations in an
hour of gameplay and used a large variety of number combinations (Brezovszky et al., 2013). Players
were likely to compare and contrast their solutions prior to execution and reflect on their solution
during gameplay. Additionally, results of a pilot intervention using a prototype of the NNG showed an
increase in sixth grade students’ adaptive number knowledge and math fluency after a seven-week
long training (Brezovszky et al., 2015). The present study scales up this training and explores the
effectiveness of the NNG in the development of adaptive number knowledge, math fluency, and pre-
algebra knowledge of primary school students in grades four to six.
2. Current study
The aim of the present study was to explore the effects of a 10-week long training of regular
mathematics teaching enriched by the NNG game-based learning environment on the development of
primary school students’ adaptive number knowledge, arithmetic fluency, and pre-algebra
knowledge. The effectiveness of the game-based learning environment was examined in an
ecologically valid setting where, in line with the Finnish educational context, teachers had the
freedom to decide about the practicalities of the gameplay. Furthermore, the study explored the
relationship between game performance and mathematical learning outcomes. Accordingly, the
present study asked the following research questions
2.1. How does training with the NNG affect the development of primary school students’
adaptive number knowledge, arithmetic fluency, and pre-algebra knowledge in different
grade levels?
An important aim of primary school mathematics education is to enhance the creative and flexible
use of mathematical knowledge by teaching alternative methods of problem solving (Nunes et al.,
2016; NCTM 2014). However, recent research shows that solely teaching strategies is insufficient
(Hickendorff, 2017). For students to develop more flexibility and adaptivity with arithmetic problem
solving they need to have a chance to practice with various number-operation combinations
(Baroody, 2003; Verschaffel et al., 2009). Research has shown advantageous numerical connections
can be noticed by those mathematical experts who have a dense and strong network of numerical
relations (Dowker, 1992). A well-connected representation of the natural number system enables
noticing of strategically important numbers and leads to a more frequent use of flexible strategies
(Heirdsfield & Cooper, 2004), which has been shown to be related to better general math abilities
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 7
(Star & Seifert, 2006). As well, adaptive number knowledge has been shown to be related to
arithmetic skills and knowledge (McMullen et al., 2016), and later pre-algebra knowledge (McMullen
et al., 2017).
In the NNG, students are encouraged to explore combinations of numbers and operations, look
for key numbers, and discover number patterns. Thus, it is expected that as a result of this repeated
practice students will develop their recognition and use of different numerical characteristics and
arithmetic relations, as indexed by their adaptive number knowledge, and also develop their
arithmetic fluency and pre-algebra knowledge. The NNG has a complex design with different game
features aimed at triggering different types of mathematical thinking. Thus, it can be anticipated that
students in the experimental classes, where regular teaching was enriched with the NNG play,
outperform students in the control classes but that the effects will be varying in different grade levels.
2.2. Does students’ performance on the NNG affect the development of mathematical learning
outcomes?
Studies in the domain of education and game-based learning are often criticized for failing to align
game goals and learning goals (Devlin, 2011; Young et al., 2012). In the NNG the educational content
is integrated within the core game mechanics (Habgood & Ainsworth, 2011; Lehtinen et al., 2015).
Thus, gaming is not delivered as a reward after students have engaged with the necessary
mathematical content, but players make meaningful progress interacting with the educational
content as they combine, compare, and strategically select the different calculations. Accordingly, it is
expected that the more practice players have with the mathematically relevant content of the NNG
the better their performance will be on the mathematical outcome measures.
3. Methods
3.1. Participants
Participants were 1,168 fourth to sixth grade primary school students (546 female) from four
urban and suburban areas in the southwest and middle of Finland. The mean age of the fourth graders
was 10.18, (SD = 0.42), Mage of fifth graders was 11.14, (SD = 0.38), and Mage of sixth graders was 12.20,
(SD = 0.45). Table 1 provides a description of the distribution of participants by grade in the
experimental and control groups.
Table 1
Number of Participants by Grade and Experimental Condition
Group
Grade
Total
4
5
6
Experimental
63
309
270
642
Control
72
297
157
526
Total
135
606
427
1,168
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 8
Participation was voluntary; informed consents from parents and assent of the students were
obtained before data gathering. All students in the experimental classes played the NNG as part of
their mathematical teaching and completed the pre- and post-tests as part of their regular school
work. However, data was only gathered from students who had the consent to participate in the
study. Ethical guidelines of the University of Turku were followed. All teachers in both the
experimental and control classes were qualified primary school teachers with a masters’ degree.
3.2. Procedure
The study was a large-scale cluster randomized trial. Randomization to experimental and control
conditions was done on the classroom level because classrooms are usually considered as ecologically
valid units of measurement in experimental designs in the field of education (Hedges & Rhoads, 2010;
Winn, 2003). The two-level hierarchical design with covariates (pre-test) was used in the power
analysis and design planning (Hedges & Rhoads, 2010). The parameters used in power analysis were
effect size (d = .35), power (> .8) correlations within (Rw2 = 0.5), and among (Rs2 = 0.8) clusters and
intraclass correlations (ICC = 0.1). The correlation estimates were the ones presented by Hedges and
Rhoads (2010) and the intraclass correlation used in the calculation was based on the values
presented in a comprehensive review of differences between schools in Nordic countries by Yang
Hansen, Gustafsson and Rosén (2014). Based on the power analysis, the minimum number of
classrooms needed were 12 for experimental and 12 for control condition. In order to make sub-
group comparison possible and better control the variation in ways how teachers used the NNG in
their classrooms, a substantially larger sample was used in the present study. There were in total 61
teachers that volunteered to participate and were randomly assigned into experimental (31 classes)
or control groups (30 classes). Two control classes were not able to participate because they had to
move to a temporary school building due to renovations.
The intervention started at the beginning of the spring semester when all participants completed
the pre-test measures. After the pre-test, the experimental group participated in the intervention
over a ten week period in which they used the NNG as part of their regular math classes. The game
was distributed to the experimental group on individual pen-drives and was played on PCs. Gameplay
was expected to be integrated into everyday math teaching, thus the intervention group did not
receive more training compared to the controls. For ethical reasons and in order to keep volunteer
teachers’ motivation during the implementation, after the post-test, the conditions were switched,
and the control group played the NNG for the rest of the semester (5 weeks). There was no pseudo-
treatment in the control classes. Control classes were instructed to continue their regular textbook-
based teaching. In all schools, in line with the national core curriculum (Finnish National Agency for
Education 2014), the local curriculum emphasized the creative use of alternative arithmetic problem
solving strategies.
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 9
Before data gathering, all teachers from both conditions were invited to participate in an
information session regarding the main features of the NNG and the general outline and aim of the
study. All teachers received written instructions including a link to a video guide on how to use the
game and were offered e-mail support in case of questions or technical difficulties. Teachers only
received general guidelines regarding the gaming sessions, since (a) the general design of the NNG is
open and flexible and was aimed to be suitable for various grade levels and (b) the present study
aimed to create an ecologically valid intervention in the Finnish context. Teachers were free to choose
if they wanted their students to play the game in pairs or individually, and in case of pair play, it was
the teacher’s decision how to select the pairs. For all classes playing in pairs teachers sent back the list
of pairs when the game log data was gathered during the post-test
1
.
The guidelines for teachers in the experimental group were to aim for around ten hours playing
time, with at least three playing sessions a week where a session is no shorter than 30 minutes.
Implementation fidelity was checked using the game log data. Based on the log data the average time
on task (effective gameplay) in the 29 experimental classes was 4 hours and 10 minutes, ranging
between average 3 hours and average 5 hours 30 minutes. There were two classes with average time
on task less than one hour. Because the attempt to have an ecologically valid design, these
differences in time on task were interpreted as a natural variation when this type of method is
introduced in the regular education. Therefore, all classes were included in the comparison of the
experimental and control groups.
3.3. Measures
Paper-and-pencil measures of adaptive number knowledge, arithmetic fluency, and pre-algebra
knowledge were administered by thirteen different trained testers (university master’s students or
researchers). Before data gathering, all testers took part in a training regarding test administration.
The testers used an automatic slide show including standardized timer and sound signals. The three
tests, took 45 minutes to complete and were administered during both time points in the following
order: arithmetic fluency, adaptive number knowledge, and pre-algebra knowledge
3.3.1. Adaptive number knowledge
The Arithmetic Production Task was used in order to measure participants’ adaptive number
knowledge. The task is a timed, paper-and-pencil instrument which aims to capture students’ ability
to recognize and use different numerical characteristics and relations during their arithmetic problem
solving (McMullen et al., 2017). Students are presented with four to five numbers and the four basic
arithmetic operations. Using these numbers and operations, the aim is to produce as many arithmetic
1
In the experimental group, repeated measures ANOVAs showed no significant interaction effect of time and mode of
play (individually or in pairs) for any of the math learning outcomes. Therefore, mode of play was not included in the
reporting of the results.
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 10
sentences that equal a target number as they can in 90 seconds. To increase the reliability of the task,
three more items were added to the post-test. Table 2 shows the items for the two time points. At
both times, the same example item was used to demonstrate the task. After explaining the
instructions, the example item had to be solved in the same amount of time as a regular item, but
students were free to ask clarification questions before and after solving it.
Table 2
Items of the Arithmetic Product Task for Pre- and Post-tests
Item
Pre-test
Post-test
Example
1,2,3,4
=6
1,2,3,4
=6
1
2,4,8,12,32
=16
2,3,9,15,36
=18
2
1,2,3,5,30
=59
1,2,6,14,42
=8
3
2,4,6,16,24
=12
2,4,6,16,48
=24
4
3,5,30,120,180
=12
1,2,3,30,36
=14
5
2,6,8,32,54
=18
6
3,4,5,6
=7
7
2,3,6,10,18
=38
Students’ solutions were transcribed and then automatically scored using Microsoft Excel
Macros written for this purpose. The scoring criteria were developed based on the results of
previously conducted studies using similar tasks (McMullen et al., 2016, 2017). In the present study,
the scoring criteria examined the quantity and complexity of students’ solutions on the Arithmetic
Production Task. For quantity, the total number of mathematically correct solutions matching the
instructions was counted (Correct). For complexity, the total number of multi-operational (Multi-op.)
solutions was counted. A solution was multi-operational if two or more different arithmetic
operations were used in order to reach the target number (i.e., 2 * 4 + 8 = 16 multi-operational, but
12 + 2 + 2 = 16 not). Final scores were made up of the total average of correct solutions and total
average of multi-operational solutions. Cronbach’s α reliability value for correct solutions was .70 for
pre-test and .86 for post-test and for multi-operational solutions .63 for pre-test and .80 for post-test
3.3.2. Arithmetic fluency
Basic arithmetic fluency was measured by the Woodcock-Johnson Math Fluency sub-test (WJ
III® Test of Achievement), which consists of two pages with a total of 160 items. Students have to
complete as many arithmetic problems (simple addition, subtraction and multiplication) as possible
during three minutes (for more details see Schrank, McGrew, & Woodcock, 2001).
3.3.3. Pre-algebra knowledge
The task measuring pre-algebra knowledge consisted of short-answer and multiple choice
questions on equation solving (i.e., 12 + ___ = 11 + 15). The task consisted of six multiple-choice items
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 11
during pre-test and six multiple-choice plus six fill-in items during post-test. At both time points
students had eight minutes to solve all items. Each correct item was worth one point, so the
maximum pre-test score could be six, while the maximum post-test score could be twelve. Cronbach’s
α reliability value for the pre-algebra knowledge test was .73 for pre-test and .88 for post-test. Due to
the different number of items during the two time points, the standardized sum scores of the pre-
algebra knowledge task were used for analyses.
3.3.4. Game performance
The number of maps completed was selected as an indicator of game performance in the
present study. As the educational content and game mechanics are integrated in the NNG interaction
with the game means also interaction with the relevant mathematical learning content. Additionally,
progress in the game (unlocking new levels) is only possible if students complete a set of maps within
a certain performance range. Thus, it can be hypothesized that the number of maps completed can be
used as a reliable estimate of students’ practice with different number combinations and numerical
relations.
A map is the basic unit of progress in the NNG, there are 64 maps in total, with 4 target
materials needing to be picked up and returned within each map. Each map has a start and an end
and provides a substantial amount and variation of mathematically relevant game activity in-between
depending also of the active scoring modes within a map. The layout of each map is different, and
every target material is placed on different numbers. Together with the two scoring modes (moves
and energy) this context provides a wide range of unique arithmetic combinations throughout
gameplay and ample opportunities for players to explore number patterns and establish arithmetic
connections. As players progress in the NNG maps get progressively harder and there are more and
more maps in the energy mode.
During the intervention, each student or pair of students received the NNG on an individual pen
drive. All game action was saved and stored in time-stamped text files on these pen drives. The pen
drives were collected at the time of the post-test. After the log data was copied, the pen drives were
returned to the students as a token of reward for participation. The game log data was summarized
and analysed using Microsoft Excel Macros written for this purpose.
3.4. Analyses
As the study was conducted in a naturalistic classroom setting and the unit of randomization was
classroom, a variance component analyses was conducted to explore classroom effects and assess the
need for multilevel analyses. While it is generally accepted that small ICC values indicate no need for
multilevel analyses, there is no agreement about exact guidelines regarding specific cut-off values.
Recommendations for considering multilevel methods can range from 0.10 (e.g., Lee, 2000) to 0.25
(e.g., Bowen & Guo, 2011), but the decision is largely dependent on the specific context and study
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 12
design. For the present study, the ICC values at the classroom level using post-test scores were 0.11
for arithmetic fluency, 0.11 for the number of correct solutions, 0.09 for multi-operational solutions
and 0.10 for pre-algebra knowledge.
In order to account for the possible effects of the hierarchical study design, intervention effects
were tested using the linear mixed model (LMM) procedure of SPSS version 24 (Heck, Thomas, &
Tabata, 2010; West, 2009). For each dependent variable (correct solutions, multi-operational
solutions, math fluency, or pre-algebra knowledge), two models were run and compared using the
likelihood-ratio test (Field, 2009; West, 2009). The first of the two models included only fixed effects
to explore the interaction of treatment and time. The second model was a random intercept model
including the same fixed effects and additionally random classroom effects. Thus four separate fixed
effects and four separate random effects models were run using the dependent variable
measurement occasion and treatment as factors. For each separate dependent variable, level1
variables included observations from the repeated measure nested within the students (level2), who
in turn were nested within the classes (level3). The time variable that refers to the measurement
occasion (pre- to post-test) was included at level1. The treatment variable that indicates if students
received training was added on level2 as a fixed covariate and ‘time * treatment’ was added as a
cross-level interaction term. In the random models, in addition to this fixed model structure, on level3
random intercepts (classroom membership) were estimated. Covariance structures were:
‘unstructured’ for level1 and ‘variance components’ for level3.
Based on the comparison of the fixed and random effects models, results from the model with
the best fit are reported for all dependent variables. Within the whole sample, compared to the
model including fixed effects only, the random intercept model including the classroom effects had
the best fit (p < .001) for all measured math learning outcomes. Similarly, in grades five and six
compared to the fixed effects only model, the random intercept model had the best fit for most of the
math learning outcomes (p < .001). The two exceptions were in grade six where for math fluency the
random intercept model had the best fit using the .05 cutoff value (p = .04) and for the multi-
operational solutions the random effects model did not have a significantly better fit than the fixed
effects model (p = .11). In grade four, adding random classroom effects to the model did not
significantly affect the model fit.
To answer the second research question, hierarchical linear regression was used to explore the
relationship of game performance and mathematical learning outcomes. Post-test scores of correct
solution, multi-operational solution, arithmetic fluency and pre-algebra knowledge were used
separately as dependent variables. For each analysis grade level was entered as a predictor in the first
step, followed by the matching pre-test score of the mathematical outcome used as a dependent
variable in the second step, and finally by game performance entered in the third step.
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 13
As the second research question examined the relationship of game performance (amount of
gameplay) and mathematical learning outcomes, using game log data, only students with sufficient
amount of gameplay were included (cf. Bakker et al., 2015; van den Heuvel-Panhuizen et al., 2013).
Thus from the experimental group, only students with at least one map completed were used in the
regression analyses. Based on this criterion 25 cases were excluded from the analysis, out of a total of
642 participants in the experimental group. On average, students in this group completed 27.02 maps
(SD = 12.65) with a range of 1-88 maps completed. The total number of maps completed does not
mean unique maps completed; students could repeat the same in order to achieve a better score.
4. Results and discussion
The linear mixed model analyses showed a significant interaction effect between experimental
condition and time-point for the number of correct solutions F(1,1053) = 11.52, p <.001, the number
of multi-operation solutions F(1,1073) = 5.93, p =.02, and arithmetic fluency F(1,1028) = 5.57, p = .02,
but not for pre-algebra knowledge, F(1,1065) = 1.90, p = .17. Overall, training with the NNG seemed to
be more effective than traditional instructional methods in developing students’ adaptive number
knowledge and math fluency, but not their pre-algebra knowledge. However, students’ prior
mathematical knowledge in grades four, five and six is very different and this across grade-level
analysis may conceal the more substantial impact of the NNG training within the grade levels. Thus,
the main examination of the intervention effects was made using separate analysis for each of the
three different grade levels.
4.1. Intervention effects by grade level
Tables 3, 4 and 5 show the raw average scores and the results of the linear mixed model analyses
of the experimental and control groups during pre- and post-test for grades four, five, and six,
respectively.
Table 3
Linear Mixed Model (Fixed Effects): Interaction Effect of Group and Time for Grade Four (n = 133)
Variables
Pre-test M (SD)
Post-test M (SD)
F(df)
p
Correct solutions
Exp.
2.08 (.78)
2.69 (.94)
8.11(1,117)
.01
Cont.
2.29 (.98)
2.48 (1.23)
Multi-op. solutions
Exp.
.51 (.43)
.81 (.59)
1.17(1,121)
.28
Cont.
.63 (.46)
.82 (.59)
Arithmetic fluency
Exp.
55.30 (16.61)
66.15 (14.77)
13.37(1,112)
.00
Cont.
65.75 (18.27)
70.58 (18.43)
Pre-algebra knowledge
Exp.
.54 (.32)
.28 (.22)
.38(1,120)
.54
Cont.
.66 (.30)
.44 (.30)
Note. Exp. = Experimental group; Cont. = Control group; Multi-op. = Multi-operational
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 14
Table 4
Linear Mixed Model (Fixed and Random Effects): Interaction Effect of Group and Time for Grade Five
(n = 599)
Variables
Pre-test M (SD)
Post-test M (SD)
F(df)
p
Correct solutions
Exp.
2.60 (1.04)
2.94 (1.20)
12.04(1,544)
.00
Cont.
2.83 (1.22)
2.91 (1.25)
Multi-op. solutions
Exp.
.77 (.61)
1.15 (.82)
6.75(1,555)
.01
Cont.
.86 (.72)
1.11 (.83)
Arithmetic fluency
Exp.
69.24 (18.38)
77.43 (19.76)
.07(1,530)
.80
Cont.
68.14 (17.00)
75.97 (18.57)
Pre-algebra knowledge
Exp.
.72 (.30)
.53 (.30)
.11(1,552)
.74
Cont.
.71 (.30)
.53 (.31)
Note. Exp. = Experimental group; Cont. = Control group; Multi-op. = Multi-operational
Table 5
Linear Mixed Model (Fixed and Random Effects): Interaction Effect of Group and Time for Grade Six (n
= 423)
Variables
Pre-test M (SD)
Post-test M (SD)
F(df)
p
Correct solutions
Exp.
3.03 (1.22)
3.33 (1.36)
.00(1,385)
.97
Cont.
2.90 (1.11)
3.11 (1.24)
Multi-op. solutions
Exp.
.94 (.67)
1.37 (.82)
.06(1,394)
.82
Cont.
.92 (.65)
1.36 (.80)
Arithmetic fluency
Exp.
75.66 (16.77)
84.94 (19.11)
3.00(1,381)
.08
Cont.
74.91 (16.43)
82.43 (20.23)
Pre-algebra knowledge
Exp.
.77 (.29)
.67 (.29)
6.68(1,389)
.01
Cont.
.81 (.27)
.62 (.32)
Note. Exp. = Experimental group; Cont. = Control group; Multi-op. = Multi-operational.
As results show, practice with the NNG had varying effects on different arithmetic skills in
different grade levels. In grade four, results showed a significant interaction of time and group for
correct solutions and arithmetic fluency, but no interaction effects were found for multi-operational
solutions or pre-algebra knowledge. In grade five, results show significant interaction effects of time
and group for correct solutions and multi-operational solutions, but no significant interaction effects
for math fluency and pre-algebra knowledge. In grade six, results show a significant interaction effect
for pre-algebra knowledge.
These results suggest that the NNG was able to support the development of different aspects of
arithmetic and mathematical development at different ages. In grade four, the game supported the
development of basic calculation fluency and the more basic aspect of adaptive number knowledge
(finding correct solutions). In grade five, where calculation fluency is already more established, there
was a positive effect of gameplay on both correct solutions and also in the more complex aspect of
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 15
adaptive number knowledge (finding multi-operational solutions). Effects for pre-algebra knowledge
were only found among sixth graders who otherwise did not benefit much from the game, the
settings of which might have been not challenging enough for them. Alternatively, it could be that
pre-algebra knowledge is a more complex skill that develops on top of other abilities, and practice
with the NNG is only beneficial for pre-algebra skills if students already have a relatively high level of
prior knowledge in arithmetic. Finding pre-algebra knowledge intervention effects only in grade six is
similar to the results of van den Heuvel-Panhuizen and colleagues (2013) who found the strongest
intervention effects in grade six – when compared to fourth and fifth graders – as a result of a game-
based training which aimed to train quantitative reasoning that is foundational to early algebra
knowledge.
In order to progress in the NNG, players need to mentally execute and compare many
calculations, look for strategically important numbers (i.e., numbers with many divisors, numbers
close to the target number), and continuously refine their strategies according to the different game
modes and challenges. This requires noticing and using numerical characteristics and relations in
order to arrive at efficient problem solving strategies in mental calculations (Brezovszky et al., 2015;
McMullen et al., 2017; Threlfall, 2009). This practice is in line with the idea that in order to develop
more flexibility and adaptivity with arithmetic problem solving, students need to have a chance to
practice with various number-operation combinations (Baroody, 2003; Verschaffel et al., 2009). It
seems that the playful practice in the NNG indeed enhances students’ awareness of numerical
relations, so they can notice more of these relations (i.e., correct solutions) and by having a richer
repertoire of number relations come up with more complex solutions (i.e., multi-operational
solutions). With more attention on calibrating gameplay for particular grades, it may be possible to
elicit even more pronounced effects. For example, offering younger students opportunities to
strengthen their understanding of the natural number base-ten system, while stimulating older
students to explore their understanding of arithmetic concepts such as the inverse nature of
multiplication and division more deeply.
The lack of methodologically sound interventions and large-scale randomized control studies is a
common finding of many reviews and meta-analyses in game-based learning (Hainey, Connolly, Boyle,
Wilson, & Razak, 2016; Wouters et al., 2013; Young et al., 2012). This is problematic because it can
distort the picture regarding the effectiveness of game-based learning as smaller and more controlled
quasi-experimental designs might inflate intervention effects, while effect sizes are generally very
small for large-scale randomized control trials (Cheung & Slavin, 2013; Wouters et al., 2013). In spite
of the possible small effects and loss of control in implementation, conducting large-scale randomized
interventions is important as they provide an opportunity to gain more ecological validity and realistic
estimate regarding the efficiency of game-based learning environments in the classroom setting
(Winn, 2003; Cook, & Payne, 2001).
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 16
4.2. Effects of game performance
The second research question of the present study explored the relationship of game
performance in the NNG and the development of students’ mathematical learning outcomes.
Hierarchical linear regression analyses were conducted in order to investigate the impact of game
performance on the improvement of the experimental group’s mathematical learning outcomes. As
Table 7 shows, after taking into account grade level and pre-test scores, the number of maps
completed still explains part of the variance for all outcome measures. Since the linear regression
cannot take into account the nested data structure, to confirm the significance level of beta values
linear mixed model analyses including pre-test scores, grade level and maps completed as fixed
effects and random classroom effects was performed. Results showed a significant main effect for the
number of maps completed p < .001.
Table 7
Hierarchical Linear Regression Analyses: Impact of Game Performance on the Mathematical Learning
Outcomes in the Experimental Group (n = 617)
Post-test
Correct
solutions
Multi-op.
solutions
Math Fluency
Pre-algebra
β
ΔR²
β
ΔR²
β
ΔR²
β
ΔR²
1
Grade-level
-.06
.03***
.02
.04***
.02
.09***
.19***
.12***
2
Pre-test
.68***
.45**
.53***
.32***
.78***
.57***
.41***
.19***
3
Game
performance
.09**
.01**
.20**
.03***
.09**
.01**
.21***
.04***
Total R2
.49
.39
.66
.36
Note 1. * p < .05, ** p < .01, *** p < .001. Note 2. Pre-test = Corresponding pre-test variable to the
post-test variable (correct solutions, multi-operational solutions, math fluency or pre-algebra). Note 3.
Multi-op. = Multi-operational
Results show that students who had more practice with the NNG also benefited more from the
gameplay. This suggests that the type of action necessary in order to make meaningful progress in the
NNG could be transferred outside the game environment as well. These results are in line with a
previous small-scale intervention study using the NNG (Brezovszky et al., 2015).
One issue with many game-based learning environments is that players engage with content or
features that are irrelevant from the perspective of the learning aims (Clark et al., 2011; Wouters &
Oostendorp, 2013). Even if students use the environment as intended, connecting the in-game
learning material with educational content outside of the game-based learning environment is often
problematic (Clark et al., 2011; Lajoie, 2005). In the NNG, at least eight paths from a starting point to a
target were taken within a completed map. But, considering that not all numbers are available and
that players need to adapt their strategies according to the active game mode (moves or energy), by
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 17
the time a map was completed a player most likely undertook a large number of mental calculations
(Brezovszky et al., 2013). In light of the results, it seems that extended practice with the NNG was able
to develop students’ recognition and use of numerical characteristics and relations.
4.3. Limitations
A few major limitations of the study are connected to the design decision which was to conduct an
ecologically valid large-scale intervention study. First, one of the major issues with this design is the
loss of control over the implementation details, which in the present study were only accounted for
by analyzing the game log data. It was an important aim of the present study to design an
intervention that is as close to everyday classroom practices as possible, especially given the large
autonomy Finnish teachers have. Thus, guidelines regarding the implementation of the NNG did not
specify strict details and teachers were free to decide, for example, the mode of play. This freedom
could result in a substantial amount of variation and can make the interpretation of the results
difficult. However, as students’ average time on task did not show a large variation across the
classrooms, it can be assumed that most students benefited from the training to a similar extent.
Connected to methodological decisions, it is important to mention that randomization on a
classroom level might have affected results. However, a classroom is a natural unit of analysis in
educational research and randomization by individual students is rarely done as it is almost
impossible for this type of study designs. Multilevel statistical analyses were conducted that takes into
account the nested nature of the data. Additionally, ICC scores were low and the amount of
participating classes was much higher than the necessary number suggested by the power analyses
which strengthens the generalizability of results despite the cluster randomized design.
Using regular instruction and no alternative intervention method (game-based or traditional) in
the control group affects the interpretation of the results in the present study. While the time period
for math instruction was the same for both groups (i.e. NNG play replaced a part of normal
mathematics lessons), the novelty of playing the NNG may have had an effect on student
performance on the post-tests. However, in a separate study of situational interest using the NNG,
results showed substantial variation in situational interest between students and across sessions
which was mainly explained by prior personal mathematics interest (Rodríguez-Aflecht et al., 2018).
This suggests that the novelty effect cannot explain group differences in the mathematical learning
outcomes of the current study. With regards to the type and quality of the regular teaching practice it
is important to add here that developing flexibility and adaptivity with arithmetic is an aim of the
Finnish National Core Curriculum and teachers are encouraged to use various representations, games,
discovery learning, and discussion, as well as plenty of group work in their everyday math classroom
practice (Kupari, 2008). For more comprehensive conclusions, future studies should compare
gameplay with the NNG with other game-based learning environments and more elaborated training
methods of flexibility with arithmetic problem solving without digital games.
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 18
A number of issues with the instruments used in the present study should also be addressed in
future studies. First, the relatively low pre-test reliability of the Arithmetic Production Task raises
concerns. These low values could be explained by the low number of items used during pre-test
and/or the novelty of the task type. Increasing the number of items in future studies, as was done on
the post-test in the present study, is recommended. With regards to the measure of game
performance, it has to be acknowledged that the number of maps completed is a crude indicator and
there might be more subtle differential patterns underlying this measure (i.e., performance within a
map, replay trials of the same map, more qualitative aspects of the problem solving strategies used
etc.). As the NNG logs different aspects of the game data, future studies could explore this question in
more detail.
5. Conclusions
NNG is, to our knowledge, the first game-based learning environment directly focused on
enhancing adaptive arithmetic knowledge and skills, which have been difficult to support in
traditional classrooms teaching. This suggests that the NNG could be a flexible tool to develop
complex mathematical skills and knowledge in a naturalistic classroom setting. Adaptive number
knowledge has been suggested to underlie proficiency with whole-number arithmetic problem solving
strategies (McMullen et al., 2017), a core feature of arithmetic development (Nunes et al., 2016;
Verschaffel et al., 2009). By promoting students’ adaptive number knowledge, the NNG is a valuable
pedagogical tool for supporting students’ development of flexible and adaptive arithmetic problem
solving. This type of support may have long-lasting value in mathematical development, for example
with learning algebra. Future studies that focus on integrating gameplay and more typical classroom
activities (Clark, Tanner-Smith, & Killingsworth, 2016; Wouters & Oostendorp, 2013) may improve
both the embedding of the NNG in the curriculum and its positive outcomes for students.
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 19
References
Arnab, S., Lim, T., Carvalho, M. B., Bellotti, F., de Freitas, S., Louchart, S., … De Gloria, A. (2015). Mapping learning and
game mechanics for serious games analysis. British Journal of Educational Technology, 46(2), 391–411.
https://doi.org/10.1111/bjet.12113
Bakker, M., van den Heuvel-Panhuizen, M., & Robitzsch, A. (2015). Effects of playing mathematics computer games on
primary school students’ multiplicative reasoning ability. Contemporary Educational Psychology, 40, 55–71.
https://doi.org/10.1016/j.cedpsych.2014.09.001
Baroody, A. J. (2003). The development of adaptive expertise and flexibility: The integration of conceptual and procedural
knowledge. In The Development of Arithmetic Concepts and Skills: Constructive Adaptive Expertise (pp. 1–33).
Lawrence Erlbaum Associates Publishers. https://doi.org/10.4324/9781410607218
Blöte, A. W., Klein, A. S., & Beishuizen, M. (2000). Mental computation and conceptual understanding. Learning and
Instruction, 10(3), 221–247. https://doi.org/10.1016/S0959-4752(99)00028-6
Blöte, A. W., Van der Burg, E., & Klein, A. S. (2001). Students’ flexibility in solving two-digit addition and subtraction
problems: Instruction effects. Journal of Educational Psychology, 93(3), 627–638. https://doi.org/10.1037/0022-
0663.93.3.627
Bowen, N. K., & Guo, S. (2011). Structural equation modeling. Oxford University
Presshttps://doi.org/10.1093/acprof:oso/9780195367621.001.0001.
Brezovszky, B., Lehtinen, E., McMullen, J., Rodriguez, G., & Veermans, K. (2013). Training flexible and adaptive arithmetic
problem solving skills through
Brezovszky, B., Lehtinen, E., McMullen, J., Rodriguez, G., & Veermans, K. (2013). Training flexible and adaptive arithmetic
problem solving skills through exploration with numbers: The development of NumberNavigation game. In 7th
European Conference on Games Based Learning, ECGBL 2013 (Vol. 2, pp. 626–634).
Brezovszky, B., Rodríguez-Aflecht, G., McMullen, J., Veermans, K., Pongsakdi, N., Hannula-Sormunen, M. M., & Lehtinen, E.
(2015). Developing adaptive number knowledge with the number navigation game-based learning environment.
In J. Torbeyns, E. Lehtinen, & J. Elen (Eds.), Describing and Studying Domain-Specific Serious Games (pp. 155–
170). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-20276-1_10
Cook, T. D., & Payne, M. R. (2001). Objecting to the objections to using random assignment in educational research. In F.
Mosteller, & R. F. Boruch (Eds.), Evidence matters (pp. 150-177). Washington, DC: Brookings Institution Press.
Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing
mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113.
https://doi.org/10.1016/j.edurev.2013.01.001
Clark, D. B., Nelson, B. C., Chang, H. Y., Martinez-Garza, M., Slack, K., & D’Angelo, C. M. (2011). Exploring Newtonian
mechanics in a conceptually-integrated digital game: Comparison of learning and affective outcomes for students
in Taiwan and the United States. Computers and Education, 57(3), 2178–2195.
https://doi.org/10.1016/j.compedu.2011.05.007
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital Games, Design, and Learning: A Systematic Review
and Meta-Analysis. Review of Educational Research, 86(1), 79–122. https://doi.org/10.3102/0034654315582065
Devlin, K. (2011). Mathematics education for a new era: Video games as a medium for learning. A K Peters.
Dowker, A. (1992). Estimation Strategies of Computational. Journal for Research in Mathematics Education, 23(1), 45–55.
Field, A. (2009). Discovering statistics using SPSS. Sage publications.
Finnish National Agency for Education (2014). Core curriculum for basic Education 2014.
http://www.oph.fi/english/curricula_and_qualifications/basic_education/curricula_2014
Habgood, M. P. J., & Ainsworth, S. E. (2011). Motivating Children to Learn Effectively: Exploring the Value of Intrinsic
Integration in Educational Games. Journal of the Learning Sciences, 20(2), 169–206.
https://doi.org/10.1080/10508406.2010.508029
Hainey, T., Connolly, T. M., Boyle, E. A., Wilson, A., & Razak, A. (2016). A systematic literature review of games-based
learning empirical evidence in primary education. Computers & Education, 102, 202–223.
https://doi.org/10.1016/j.compedu.2016.09.001
Heck, R. H., Thomas, S. L. T., & Tabata, L. N. (2010). Multilevel and Longitudinal Modeling with IBM SPSS. Routledge. New
York.
Hedges, L. V., & Rhoads, C. (2010). Statistical power analysis. In International Encyclopedia of Education (pp. 436–443).
Elsevier. https://doi.org/10.1016/B978-0-08-044894-7.01356-7
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 20
Heinze, A., Star, J. R., & Verschaffel, L. (2009). Flexible and adaptive use of strategies and representations in mathematics
education. ZDM - International Journal on Mathematics Education, 41, 535–540. https://doi.org/10.1007/s11858-
009-0214-4
Heirdsfield, A. M., & Cooper, T. J. (2004). Factors affecting the process of proficient mental addition and subtraction: Case
studies of flexible and inflexible computers. Journal of Mathematical Behavior, 23(4), 443–463.
https://doi.org/10.1016/j.jmathb.2004.09.005
Heirdsfield, A. M (2011). Teaching mental computation strategies in early mathematics. Young Children, 66(2), pp. 96-102.
Hickendorff, M. (2017). Dutch sixth graders’ use of shortcut strategies in solving multidigit arithmetic problems. European
Journal of Psychology of Education, 1–18. https://doi.org/10.1007/s10212-017-0357-6
Ke, F. (2008). A case study of computer gaming for math: Engaged learning from gameplay? Computers and Education,
51(4), 1609–1620. https://doi.org/10.1016/j.compedu.2008.03.003
Ke, F. (2009). A Qualitative Meta-Analysis of Computer Games as Learning Tools. Handbook of Research on Effective
Electronic Gaming in Education, 1–32. https://doi.org/10.4018/978-1-59904-808-6.ch001
Kiili, K. (2007). Foundation for problem-based gaming. British Journal of Educational Technology, 38(3), 394–404.
https://doi.org/10.1111/j.1467-8535.2007.00704.x
Kupari, P. (2008). Mathematics education in Finnish comprehensive school: characteristics contributing to student success.
In Proceedings of the XI International Congress in Mathematics Education.
Lajoie, S. P. (2005). Extending the scaffolding metaphor. Instructional Science, 33(5–6), 541–557.
https://doi.org/10.1007/s11251-005-1279-2
Laski, E. V., Ermakova, A., & Vasilyeva, M. (2014). Early use of decomposition for addition and its relation to base-10
knowledge. Journal of Applied Developmental Psychology, 35(5), 444–454.
https://doi.org/10.1016/j.appdev.2014.07.002
Lee, V. E. (2000). Using Hierarchical Linear Modeling to Study Social Contexts: The Case of School Effects, Educational
Psychologist, 35(2), 125-141, doi: 10.1207/s15326985ep3502_6
Lehtinen, E., Brezovszky, B., Rodríguez-Aflecht, G., Lehtinen, H., Hannula-Sormunen, M. M., McMullen, J., … Jaakkola, T.
(2015). Number Navigation Game (NNG): Design Principles and Game Description. In J. Torbeyns, E. Lehtinen, & J.
Elen (Eds.), Describing and Studying Domain-Specific Serious Games (pp. 45–61). Cham: Springer International
Publishing. https://doi.org/10.1007/978-3-319-20276-1_4
Lehtinen, E., Hannula-Sormunen, M., McMullen, J., & Gruber, H. (2017). Cultivating mathematical skills: from drill-and-
practice to deliberate practice. ZDM - Mathematics Education, 49(4), 625–636. https://doi.org/10.1007/s11858-
017-0856-6
Li, Q., & Ma, X. (2010). A Meta-analysis of the Effects of Computer Technology on School Students’ Mathematics Learning.
Educational Psychology Review, 22(3), 215–243. https://doi.org/10.1007/s10648-010-9125-8
Lowrie, T., & Jorgensen, R. (2015). Digital Games and Learning: What’s New Is Already Old? (pp. 1–9). Springer, Dordrecht.
https://doi.org/10.1007/978-94-017-9517-3_1
McMullen, J., Brezovszky, B., Rodríguez-Aflecht, G., Pongsakdi, N., Hannula-Sormunen, M. M., & Lehtinen, E. (2016).
Adaptive number knowledge: Exploring the foundations of adaptivity with whole-number arithmetic. Learning
and Individual Differences, 47, 172–181. https://doi.org/10.1016/j.lindif.2016.02.007
McMullen, J., Brezovszky, B., Hannula-Sormunen, M. M., Veermans, K., Rodríguez-Aflecht, G., Pongsakdi, N., & Lehtinen, E.
(2017). Adaptive number knowledge and its relation to arithmetic and pre-algebra knowledge. Learning and
Instruction, 49, 178–187. https://doi.org/10.1016/j.learninstruc.2017.02.001
Mulligan, J. T., & Mitchelmore, M. C. (2013). Early awareness of mathematical pattern and structure. In L. D. English & J. T.
Mulligan (Eds.), Reconceptualising early mathematics learning (pp. 29-45). New York: Springer.
https://doi.org/10.1007/978-94-007-6440-8_3
National Council of Teachers of Mathematics. (2014). Principles to actions: Ensuring mathematical success for all. Reston.
Nunes, T., Dorneles, B. V., Lin, P.-J., & Rathgeb-Schnierer, E. (2016). Teaching and Learning About Whole Numbers in
Primary School. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-45113-8
Pope, H., & Mangram, C. (2015). Wuzzit Trouble: The Influence of a Digital Math Game on Student Number Sense.
International Journal of Serious Games, 2(4). https://doi.org/10.17083/ijsg.v2i4.88
Rittle-Johnson, B., & Star, J. R. (2009). Compared With What? The Effects of Different Comparisons on Conceptual
Knowledge and Procedural Flexibility for Equation Solving. Journal of Educational Psychology, 101(3), 529–544.
https://doi.org/10.1037/a0014224
Rodríguez-Aflecht, G., Jaakkola, T., Pongsakdi, N., Hannula-Sormunen, M., Brezovszky, B., & Lehtinen, E. (2018). The
EFFECTS OF A MATHEMATICS GAME-BASED LEARNING ENVIRONMENT 21
development of situational interest during a digital mathematics game. Journal of Computer Assisted Learning.
https://doi.org/10.1111/jcal.12239
Rutherford, T., Farkas, G., Duncan, G., Burchinal, M., Kibrick, M., Graham, J., … Martinez, M. E. (2014). A Randomized Trial
of an Elementary School Mathematics Software Intervention: Spatial-Temporal Math. Journal of Research on
Educational Effectiveness, 7(4), 358–383. https://doi.org/10.1080/19345747.2013.856978
Salen, K., & Zimmerman, E. (2003). Rules of Play: Game Design Fundamentals. The MIT Press.
Schneider, M., Rittle-Johnson, B., & Star, J. R. (2011). Relations among conceptual knowledge, procedural knowledge, and
procedural flexibility in two samples differing in prior knowledge. Developmental Psychology, 47(6), 1525–1538.
https://doi.org/10.1037/a0024997
Schrank, F. A., McGrew, K. S., & Woodcock, R. W. (2001). Technical Abstract (Woodcock-Johnson III Assessment Service
Bulletin No. 2). Itasca, IL: Riverside Publishing. Retrieved from
http://www.riverpub.com/clinical/pdf/WJIII_ASB2.pdf
Star, J. R., & Seifert, C. (2006). The development of flexibility in equation solving. Contemporary Educational Psychology,
31(3), 280–300. https://doi.org/10.1016/j.cedpsych.2005.08.001
Star, J. R., & Rittle-Johnson, B. (2008). Flexibility in problem solving: The case of equation solving. Learning and Instruction,
18(6), 565–579. https://doi.org/10.1016/j.learninstruc.2007.09.018
Star, J. R., & Newton, K. J. (2009). The nature and development of experts’ strategy flexibility for solving equations. ZDM -
International Journal on Mathematics Education, 41(5), 557–567. https://doi.org/10.1007/s11858-009-0185-5
Squire, K. (2008). Open-Ended Video Games: A Model for Developing Learning for the Interactive Age. In Salen K., The
ecology of games: Connecting youth, games, and learning (p. 167-198). Cambridge MA: The MIT Press.
Threlfall, J. (2002). Flexible Mental Calculation. Educational Studies in Mathematics, 50(1), 29–47.
https://doi.org/10.1023/A:1020572803437
Threlfall, J. (2009). Strategies and flexibility in mental calculation. ZDM, 41(5), 541–555. https://doi.org/10.1007/s11858-
009-0195-3
van den Heuvel-Panhuizen, M., Kolovou, A., & Robitzsch, A. (2013). Primary school students’ strategies in early algebra
problem solving supported by an online game. Educational Studies in Mathematics, 84(3), 281–307.
https://doi.org/10.1007/s10649-013-9483-5
Verschaffel, L., Luwel, K., Torbeyns, J., & Van Dooren, W. (2009). Conceptualizing, investigating, and enhancing adaptive
expertise in elementary mathematics education. European Journal of Psychology of Education, 24(3), 335–359.
https://doi.org/10.1007/BF03174765
West, B. T. (2009). Analysing longitudinal data with the linear mixed models procedure in SPSS. Eval Health Prof, 32(3),
207–228. https://doi.org/10.1177/0163278709338554
Winn, W. (2003). Research Methods and Types of Evidence for Research in Educational Technology. Educational
Psychology Review, 15(4), 367-373. https://doi.org/10.1023/A:1026131416764
Wouters, P., & Oostendorp, H. Van. (2013). Computers & Education A meta-analytic review of the role of instructional
support in game-based learning. Computers & Education, 60(1), 412–425.
https://doi.org/10.1016/j.compedu.2012.07.018
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and
motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265.
https://doi.org/10.1037/a0031311
Yang Hansen, K., Gustafsson, J.-E. , & Rosén, M. (2014). School performance difference and policy variations in Finland,
Norway and Sweden. In K. Yang Hansen, J.-E. Gustafsson, M. Rosén, S. Sulkunen, K. Nissinen, P. Kupari, R.F.
Ólafsson, J.K. Björnsson, L.S. Grønmo, L. Rønberg, J. Mejding, I.C. Borge, & A. Hole. Northern Lights on TIMMS and
PIRLS 2011. TemaNord 2014:528, 25-48.
Young, M. F., Slota, S., Cutter, B., R., Jalette, G., Mullin, G., … Yukhymenko, M. (2012). Our Princess Is in Another Castle: A
Review of Trends in Serious Gaming for Education. Review of Educational Research, 82(1), 61–89.
https://doi.org/10.3102/0034654312436980
