Computer Gaming and Interactive Simulations for Learning: A Meta-Analysis

Abstract

Substantial disagreement exists in the literature regarding which educational
technology results in the highest cognitive gain for learners. In an attempt to
resolve this dispute, we conducted a meta-analysis to decipher which teaching
method, games and interactive simulations or traditional, truly dominates and
under what circumstances. It was found that across people and situations,
games and interactive simulations are more dominant for cognitive gain
outcomes. However, consideration of specific moderator variables yielded a
more complex picture. For example, males showed no preference while
females showed a preference for the game and interactive simulation programs. Also, when students navigated through the programs themselves, there
was a significant preference for games and interactive simulations. However,
when teachers controlled the programs, no significant advantage was found.
Further, when the computer dictated the sequence of the program, results
favored those in the traditional teaching method over the games and interactive simulations. These findings are discussed in terms of their implications
for exiting theoretical positions as well as future empirical research.

Full text

J. EDUCATIONAL COMPUTING RESEARCH, Vol. 34(3) 229-243, 2006
COMPUTER GAMING AND INTERACTIVE
SIMULATIONS FOR LEARNING: A META-ANALYSIS
JENNIFER J. VOGEL
DAVID S. VOGEL
JAN CANNON-BOWERS
CLINT A. BOWERS
KATHRYN MUSE
MICHELLE WRIGHT
University of Central Florida
ABSTRACT
Substantial disagreement exists in the literature regarding which educational
technology results in the highest cognitive gain for learners. In an attempt to
resolve this dispute, we conducted a meta-analysis to decipher which teaching
method, games and interactive simulations or traditional, truly dominates and
under what circumstances. It was found that across people and situations,
games and interactive simulations are more dominant for cognitive gain
outcomes. However, consideration of specific moderator variables yielded a
more complex picture. For example, males showed no preference while
females showed a preference for the game and interactive simulation pro
-
grams. Also, when students navigated through the programs themselves, there
was a significant preference for games and interactive simulations. However,
when teachers controlled the programs, no significant advantage was found.
Further, when the computer dictated the sequence of the program, results
favored those in the traditional teaching method over the games and inter
-
active simulations. These findings are discussed in terms of their implications
for exiting theoretical positions as well as future empirical research.
INTRODUCTION
Understanding the specific nature of the relationship between educational gaming
and simulation computer programs and how they affect learning is important for
229
Ó 2006, Baywood Publishing Co., Inc.

several reasons. First, due to society’s reliance on computers, children and adults
alike are now being required to interact with computers in all settings, including
educational ones. Second, schools, universities, and even job training settings are
now finding that computer programs are effective in reducing educational and
training costs (Rifkin, 1994). There is also some evidence to suggest that using
these computer games or simulations may actually “teach” people more effec
-
tively than traditional methods (Cassidy, 2003; Jenkins, 2002). For these reasons,
it is imperative that research investigates this effectiveness to more properly
utilize these programs.
The role of computers in education has changed drastically over the last several
decades. These advances stem largely from the increased power, accessibility,
and graphical capabilities of current computers (Kirriemuir, 2002; Naps et al.,
1997; Rifkin, 1994). Initially, computers were used for business, mathematical
computations, and then for leisure games. However, several reviews of computer-
assisted-instruction (CAI) covering articles from the last four decades have shown
that computers can also be used for learning (Bangert-Drowns, Kulik, & Kulik,
1986; Kirriemuir, 2002; Ryan, 1991). While this viewpoint is generally accepted,
teachers rarely incorporate computers into their daily teaching routines (Murphy,
Blaha, VanDeGrift, Wolfman, & Zander, 2002; Soloway, 1998). Some reasons for
this may involve pragmatic issues such as funding and access (Murphy, Blaha,
VanDeGrift, Wolfman, & Zander, 2002; Soloway, 1998). However, one might
hypothesize that teachers do not use computers because they require more effort
by the teacher for little perceived cognitive gain. This is not intended to place
blame on teachers for this situation. Rather, it is the job of researchers to find a way
to help already overworked teachers by creating computer systems that reduce the
children’s time-on-task and increase their cognitive gains. Stated another way,
computer programs need to teach people better and faster. They must also teach in
accordance with the state mandated curriculum requirements to be directly useful
for teachers (Soloway, 1998).
The difficulties described above impose a special challenge for scientists and
software developers in this area. First, we must develop a research base so that
we know how best to create positive learning outcomes. We must then translate
those results into a product that is attractive to the learner while being practical
for the teacher. Obviously, software designers will not wait for a program of
scientific research before responding to this need in the market. Indeed, educa
-
tional software is a fast-growing genre. However, several of these products have
been developed using “seat-of-the-pants” approaches rather than validated scien
-
tific principles. We seek to contribute to this area by one type of computer-based
training that has only recently attracted the attention of scientists. Our goal is to
arrive at an impartial assessment of these data along with some guidance about
how the learning strategy can best be deployed.
We have chosen to focus on the effectiveness of games and interactive simu
-
lations. There are several factors that point to this type of computer-assisted
230 / VOGEL ET AL.

learning as an area worthy of study. First, there is the practical reality that this type
of software is easily available, and that development costs are being reduced
seemingly every day. Thus, there is an exciting opportunity for the educational
user. It should be noted that current theory supports the notion that playing games
allows the brain to work more efficiently and thus take in more cognitive material
than it would in a more traditional setting (Baltra, 1990; Pange, 2003; Perry &
Ballou, 1997). Further motivation is a main focus for teachers based on the
premise that motivated and interested students will learn more and will do it
faster (Lardinois, 1989, cited in Siemer & Angelides, 1995; Rieber, 1996; Romme,
2003). Game theory, and video game theory specifically, support the belief that
computer games are highly engaging, motivating, and interactive (Ju & Wagner,
1997; Kafai, 2001; Rieber, 1996).
This range of how a game or interactive simulation is defined varies widely
(Manninen, 2002). The Society for the Advancement of Games and Simulation
in Education and Training (Saunders, 1987, cited in Galvao, Martins, & Gomes,
2000) defines “simulation” as, “a working representation of reality . . . [that[ may
be an abstracted, simplified, or accelerated model of process.” While “game” is
defined as, “one or more players compete or cooperate for pay-off according to a
set of rules or ...asetting in which participants make choices, implement those
options and receive consequences of those decisions as an effort to achieve given
objective,” (VanSickle, 1978, cited in Galvao, Martins, & Gomes, 2000). Siemer
and Angelides (1995) define the hybrid gaming-situations as, “a sequential
decision-making exercise, with the basic function of providing an artificial but
realistic environment that enables players to experience the consequences of their
decisions through immediate response.” Whatever the nuances of each individual
definition, one common thread must be included. Interactivity is the key factor in
creating better educational outcomes (Stoney & Wild, 1998). It is based on this
premise that the definitions of games and interactive simulations were created for
this analysis. They are defined as follows. A computer game is defined as such by
the author, or inferred by the reader because the activity has goals,isinteractive,
and is rewarding (gives feedback). Interactive simulation activities must interact
with the user by offering the options to choose or define parameters of the
simulation then observe the newly created sequence rather than simply selecting
a prerecorded simulation.
As mentioned previously, numerous meta-analyses and review articles have
been published showing small but positive effect sizes supporting CAI over other
teaching methods (Bayrakter, 2001; Chambers, 2002; Christmann & Badgett,
2003; Cohen & Decanay, 1992; Fletcher-Flynn & Gravatt, 1995; Kulik, 1994;
Kulik & Kulik, 1986; Lowe, 2001). The primary focus of these reviews is on
CAI defined as any program that augments, teaches, or simulates the learning
environment used in the traditional classroom (Quyang, 1993). Such programs
include, but are not limited to drill-and-practice programs, multimedia classrooms,
web-based instruction, and previously created simulations (Murphy et al., 2002).
COMPUTER GAMING AND INTERACTIVE SIMULATIONS / 231

A few additional meta-analyses have examined simulations and games. Lee
(1999) focused on the comparison of pure and hybrid simulations finding that
hybrid simulations had a significant advantage over pure simulations in learning
outcomes. VanSickle (1986) found that there was a slight advantage in gaming
simulations to produce positive attitudes toward the subject matter being studied
compared to other teaching methods. Additionally, the analysis revealed that using
gaming simulations for learning resulted in higher cognitive gains when compared
to other teaching methods, but not to traditional, lecture methods. A more recent
review article about games (Randel, Morris, Wetzel, & Whitehill, 1992) covering
the years 1984 to 1991 reported that of the 67 articles included, 38 found no
differences between computer games and traditional teaching methods, 22 favored
games, an additional five with questionable control groups also favored games,
and only three favored traditional methods. Additionally, results of individual
studies from 1986 to the present are equivocal (Brewster, 1996; Costabile,
De Angeli, Roselli, Lanzilotti, & Plantamura, 2003; Kim, Kim, Min, Yang, &
Nam, 2002; Laffey, Espinosa, Moore, & Lodree, 2003; McGarvey, 1986; Rosas
et al., 2003). While some studies showed significant differences favoring games
or interactive simulations over traditional teaching methods (Laffey, Espinosa,
Moore, & Lodree, 2003; McGarvey, 1986), other studies found the opposite result
(Costabile, De Angeli, Roselli, Lanzilotti, & Plantamura, 2003; Kim et al., 2002).
Still others showed no significant differences between the two types of teaching
(Brewster, 1996; McGarvey, 1986; Rosas et al., 2003).
Based on the differences of results, it is difficult for researchers to determine
the true nature of the relationship between gaming and interactive simulations with
learning. The differences may have arisen due to various differences that exist
among the articles. Each of these studies focused on different skills to learn, used
the computers differently, and used different subject populations. All of these
differences potentially account for the conflicting study results. We believe that a
meta-analysis will more accurately synthesize the results of the existing studies
thus providing more information about the state-of-the-art in this area.
The main object of this analysis is to control for each of these issues and
make an accurate determination of how games and interactive simulations relate
to learning.
METHOD
Potential studies were selected from computerized databases (PsycInfo, ERIC,
ACM, and Google), dissertation abstracts, and back-searches from gathered
articles’ reference lists. In order to be included in the analysis, each study must
have identified cognitive gains or attitudinal changes as one of its main hypoth
-
eses. Also, it was required that each study report statistics assessing tradi
-
tional classroom teaching versus computer gaming or interactive simulation
teaching. Studies were assessed using three moderator variables: Type of Activity
232 / VOGEL ET AL.

[(1) Interactive simulation (user must interact with the simulation by either
choosing or defining parameters of the simulation then observe its execution),
(2) Game (Any computer game that is interactive and defined as such by the
author, or inferred by the reader because the activity has goals,isinteractive,
and is rewarding (gives feedback)), (3) Unknown/unspecified]; Population
[AGE – Age (1, 2, 3, 4, 5, 6, 7—preschool (less than five years of age), elementary
(grades K-5, ages 6-11), middle (grades 6-8, ages 11-14), high (grades 9-12, ages
14-18), college (undergraduate study, ages 18-24), adult (25 years of age and
older), unknown/unspecified), GNR—Gender (1, 2, 3, 4—male, female, both,
unspecified), U.S.—User (1, 2, 3, 4—individual, group, both, unspecified)]; and
Computer Characteristics [RL—Realism (1, 2, 3, 4—photo-realistic, high-quality
cartoon-like pictures, low-quality programmed pictures, unknown/unspecified),
LC—Learner Control (1, 2, 3, 4—game is controlled by the: student, teacher,
computer, unspecified)]. A total of 248 studies were evaluated for inclusion.
However, after review, only 32 studies actually met the requirements and were
used for the analysis. Two raters were used to assess each study in the analysis.
The actual reliability found was 84% with a Cohen’s Kappa of .74.
All statistics used in each study were converted to the effect size index r using
the following formulas:
F to r = sq. root of F/F+df (1)
t to r = sq. root of t2/t2 + df (2)
z to r = sq. root of z2/n (3)
x2 to r = sq. root of x2(1)/n1 + n2 (4)
The n in equation 3 represents the overall sample size in each study. The n used in
equation 4 represents the sum of sample sizes for the two groups compared. Next,
we computed the overall Confidence Interval (CI) of r. And finally, a dot plot
graph was made using the correlation coefficient estimates and CIs of each study.
RESULTS
Two effect sizes were compiled for the overall results. The data suggest that,
overall, significantly higher cognitive gains were observed in subjects utilizing
interactive simulations or games versus traditional teaching methods (z = 6.051,
p < .0001 (N = 8549)). The fail-safe number (Nfs), or the number of undiscovered
studies with opposing results needed to change this conclusion, was 1465. Thus,
this finding was reliable. A main effect for attitude was also found (z = 13.74,
p < .0001) (N = 2378), Nfs = 117) reliably suggesting that subjects’ attitudes
COMPUTER GAMING AND INTERACTIVE SIMULATIONS / 233

toward learning when using the computers were significantly better than those
utilizing traditional teaching methods.
Gender
When evenly distributed across genders or when gender was unreported in the
study, significant results for cognitive gains in the game and interactive simulation
group were found (z = 8.073, p < .0001 (N = 2347), Nfs = 288) and (z = 4.190,
p < .0001 (N = 6102), Nfs = 348) respectively. Consistent with this, females
showed significant cognitive gains favoring the interactive simulation and game
method (z = 2.583, p = .0049 (N = 80), Nfs = 3). There was an insufficient number
of studies using only males to allow for a reasonable conclusion. However, those
studies that reported statistics comparing males and females found no significant
differences (z = .9910, p = .1594 (N = 394), Nfs = 0). Again, due to the low
fail-safe number, these results should be considered with caution.
Learner Control
Programs that were designed to automatically navigate students through the
system based on techniques such as decision trees or artificial intelligence, were
less effective than traditional classroom education in creating cognitive gains
(z = –2.099, p = .018 (N = 94), Nfs = 0). However, there is not a sufficient sample
size to draw this conclusion with confidence. Studies that used programs where the
learner controlled their navigation through the system showed opposite results.
Significant cognitive gains in the game and interaction simulation groups were
observed compared to the traditional teaching methods (z = 7.038, p < .0001
(N = 3656), Nfs = 1233).
Type of Activity
The type of activity does not appear to be influential. Studies using interactive
simulations, games, or a method that involved both had highly significant results,
similar to the overall effect, in the directions of higher cognitive gains compared
to traditional teaching methods (z = 9.147, p < .0001 (N = 2179), Nfs = 963);
(z = 3.706, p = .0001 (N = 2165), Nfs = 24); and (z = 3.209, p = .0007 (N = 4205),
Nfs = 0) respectively. The low fail-safe numbers in the game and combination
groups indicate low reliability for these results.
Age
Age groups were combined in order to attain an acceptable level of power.
Preschool, elementary, middle, and high school children showed significant
results (z = 4.111, p < .0001 (N = 6138), Nfs = 86) favoring game and interactive
simulations. Similar results were obtained for College and Adult populations
(z = 7.434, p < .0001 (N = 2336), Nfs = 494).
234 / VOGEL ET AL.

Realism
Level of picture realism in the computer programs did not alter the results. All
levels showed strong interactive simulation and game preferences, similar to the
overall effects. Results are summarized in Table 1.
User
Both user types (individual and group) showed significant results favoring the
interactive simulation and game methods (z = 7.352, p < .0001, (N = 3413),
Nfs = 1048) and (z = 2.222, p = .0131 (N = 931), Nfs = 11) respectively.
DISCUSSION
Not surprisingly, the overall results yielded significantly higher cognitive gains
and better attitudes toward learning for subjects utilizing interactive games or
simulations compared to those using traditional teaching methods for instruction.
This conclusion is based on a number of studies making it extremely unlikely
to be due to chance. These increased cognitive gains and improved attitudes were
consistently found (Boyd & Murphrey, 2002; Cowen, 1993; Laffey, Espinosa,
Moore, & Lodree, 2003; Ronen & Eliahu, 2000) yielding a very significant effect
strength. Basically, this means that across all situations and variables, interactive
simulations or games will most likely instruct subjects with better cognitive
outcomes and attitudes toward learning when compared to traditional teaching
methods. It has been previously argued (Schramm, 1977 cited in Clark, 1994)
that games will show increased cognitive gains due to the increased attention
paid to the curriculum used rather than due to the mode of presentation to
the learner. It is thus noted that many of the studies directly reported that the
curriculums used in both the control and experimental groups was identical
COMPUTER GAMING AND INTERACTIVE SIMULATIONS / 235
Table 1. Cognitive Gains Moderated by Realism
z-Score p-Value N
Nfs
observed
Nfs
needed
Photo-realistic
High-quality cartoons
Low-quality pictures
Unrealistic (numbers,
lines, graphs)
4.105
3.992
3.425
5.447
<.0001*
<.0001
.0003*
<.0001*
842
474
1617
1148
120
17
100
104
50
25
50
55
*Reliable result.

(Brewster, 1996; Marcum-Dietrich & Ford, 2002; Ronen & Eliahu, 2000; Shute
& Glaser, 1990; Watkins, 1998) thus reducing the likelihood of this occurring
and adding further to the validity of these findings.
Gender
However, when this finding was broken down into several categories using
different moderator variables, other results were occasionally found. In the case
of gender, when studies used an even distribution of both genders or used
only females, the results mirrored those found in the overall results (Andrews,
Schwartz, & Helme, 1992; Farrell, Arnold, Pettifer, Adams, Graham, &
MacManamon, 2003; Reis, Riley, Lokman, & Baer, 2000). Further, studies
comparing males and females yielded no significant differences between the
two suggesting that they perform similar to each other under both teaching
situations (Akpan & Andres, 2000; Choi & Grennaro, 1987; Laffey, Espinosa,
Moore, & Lodree, 2003). There were an insufficient number of studies to evaluate
results for males alone, but we are unaware of a theoretical position that suggests
that males might be disadvantaged in this regard. Thus, it seems that the observed
benefits of games and simulations can be reasonably expected in both genders.
Learner Control
The vast majority of studies utilized interactive simulations or games that
required the subject to navigate through the computer program based upon their
own preferences. These results, not surprisingly, yielded an effect size similar
to the overall effect size. There is little data to draw meaningful conclusions
about other control options, although the existing studies certainly suggest that
other types of control might mitigate the game advantage. Potentially, this trans
-
lates to the idea that self-direction is necessary for increased learning outcomes
to occur. Clearly, this is an area that requires further study before we can provide
meaningful guidance to the development community.
Type of Activity
Two main types of activities using the computer were explored. Subjects using
interactive simulations or games both significantly outperformed their peers
instructed using traditional classroom methods. The results of the interactive
simulation programs had a large fail-safe number suggesting that in fact, inter
-
active simulations are truly beneficial. However, the analysis of the gaming
programs yielded a low fail-safe number and thus should be considered with
caution. Considerably more studies comparing game usage for learning to tradi
-
tional methods need to be conducted before these results can be considered reliable.
Age
Across all age groups, significant results were found in favor of interactive
simulations and games. In other words, regardless of age, subjects increased their
236 / VOGEL ET AL.

knowledge more when taught using the computer than when learning in the
traditional format. This finding is somewhat counterintuitive since it is a common
assumption that children, due to shorter attention-spans, higher interest in play
activities, and lower intrinsic motivation to learn, enjoy and thus learn better using
computer games and interactive simulations compared to adults (Kafai, 2001;
McGrenere, 1996; Rieber, 1996).
Realism
Significant results in all levels of picture realism were found favoring inter
-
active simulations or games. As most effects sizes met their fail-safe numbers,
these findings can be considered reliable. Only the high-quality cartoon pictures
failed to meet their fail-safe number, likely due to the small number of studies
involved in the analysis. These results indicate that subjects will learn more using
games or interactive simulations at any level. It is not necessary for programs
to contain a high level of fidelity in order to see results. However, when com-
paring those studies that used pictures (excluding those studies that used words,
lines, or graphs), a positive correlation in effect size is seen. Meaning, as the
realism of the program increases, the amount of knowledge gained during the
“teaching time” also increased.
User
Finally, studies were separated according to the user. Both user groups, indi-
vidual and group, reported significantly higher cognitive gains in the interactive
simulation and game groups versus the traditional teaching method groups. This
finding suggests that whether subjects work alone with a computer or with a group
of peers, they will learn more using the computer compared to listening to an
instructor. However, it is noteworthy, that the effect sizes of the groups were quite
different. Specifically, those who used the computers alone yielded a much higher
effect size than those using the computers with a group of peers. This suggests that
while an increase in cognitive gains can be observed whenever an interactive
simulation or game is used, those allowed to work alone will likely outperform
those working in groups.
Summary
The overall result of the meta-analysis, then, was that those using interactive
simulations or games report higher cognitive gains and better attitudes toward
learning compared to those using traditional teaching methods. This result agrees
with the current overall theory stating that interactive experiential activities that
increase motivation also show increased learning outcomes (Baltra, 1990;
Montgomery & Fogler, 1996, cited in Cassidy, 2003; Prensky, 2002). For the
most part, this conclusion seems robust to the several potential moderators that
COMPUTER GAMING AND INTERACTIVE SIMULATIONS / 237

we considered, but the research base is insufficient to draw this conclusion
with much confidence.
Further hindering our ability to draw accurate conclusions was the fact that
too many articles were unable to be used. Methodological and reporting flaws
are rampant in the unused articles. No control group was the most frequently
found methodological flaw in the literature (Bills, 1998; Garg, Norman, Spero,
& Maheshwari, 1999; Hakkarainen, Lipponen, Jarvela, & Niemivirta, 1999;
Jackson, 1997; Ju & Wagner, 1997; Yildiz & Atkins, 1996). Without this com
-
parison, it is impossible to conclude that the given intervention accounted for
the change in results. Comparisons to traditional teaching methods can also not
be made. Additional problems exist in the literature further reducing the number
of studies able to be used in the analysis. Multiple studies failed to include
any statistical data in their reports (Decortis & Rizzo, 2002; Haidet, Hunt, &
Coverdale, 2002; Hammond, McKendree, & Scott, 1996; Najjar, Thompson, &
Ockerman, 1999; Parker, Cheatham, & Milling, 2000). In absence of data, the
research is left unusable. Many of the studies also left out important demographic
details (Cowen, 1993; Kim et al., 2002; Ronen & Eliahu, 2000; Rosas et al., 2003;
Shifroni & Ginat, 1997) or did not describe the programs and activities they used
as interventions in sufficient depth for us to categorize them with confidence
(George & Sleeth, 1996; Jantz, Anderson, & Gould, 2002; Jollicoeur & Berger,
1988; Klassen & Willoughby, 2003; Predavec, 2001). Thus, the literature often
fails to provide the information necessary to determine if games and interactive
simulations are indeed helpful. We hope that this article will draw attention to this
emerging and important area of instruction, and will motivate studies that will
allow us to more finely analyze the effects of this teaching approach.
REFERENCES
*References marked with an asterisk indicate studies included in the meta-analysis.
*Akpan, J. P., & Andres, T. (2000). Using a computer simulation before dissection to help
students learn anatomy. Journal of Computers in Mathematics & Science Teaching,
19(3), 297-313.
*Andrews, P. V., Schwartz, J., & Helme, R. D. (1992). Students can learn medicine with
computers: Evaluation of an interactive computer learning package in geriatric
medicine. Medical Journal of Australia, 157, 693-695.
Baltra, A. (1990). Language learning through computer adventure games. Simulation and
Gaming, 21(4), 445-452.
Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C. C. (1986). Effectiveness of computer-based
education in secondary schools. Journal of Computer-Based Instruction, 12(3), 59-68.
*Baxter, J. H., & Preece, P. F. W. (1999). Interactive multimedia and concrete three-
dimensional modeling. Journal of Computer Assisted Learning, 15(4), 323-331.
Bayraktar, S. (2001-2002). A meta-analysis of the effectiveness of computer-assisted
instruction in science education. Journal of Research on Technology in Education,
34(2), 173-188.
238 / VOGEL ET AL.

Bills, C. G. (1998). Effects of structure and interactivity on internet-based instruction.
Dissertation Abstracts International Section A: Humanities & Social Sciences
University Microfilms International, 58(12-A), 4451.
*Blank, G. D., Roy, S., Sahasrabudhe, S., Pottenger, W., & Kessler, G. D. (2003). Adapting
multimedia for diverse student learning styles. JCSC, 18(3), 45-58.
*Bliss, J. P., Tidwell, P. D., & Guest, M. A. (1997). The effectiveness of virtual reality
for administering spatial training in firefighters. Presence: Teleoperators and Virtual
Environments, 6, 73-86.
Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational
goals: Handbook I, cognitive domain. New York: Longmans, Green.
*Bone, R., & Lintern, G. (1999). Rehearsal versus map study as preparation for a flight
navigation exercise. Human Factors, 41(3), 467-473.
*Boyd, B. L., & Murphrey, T. P. (2002). Evaluation of a computer-based, asynchronous
activity on student learning of leadership concepts. Journal of Agricultural Education,
43(1), 36-45.
*Brewster, J. (1996). Teaching abnormal psychology in a multimedia classroom. Teaching
of Psychology, 23(4), 249-252.
Cassidy, O. (2003). I try it—Utilizing multimedia simulation as a vehicle to aid experiential
learning. Available online at: http://www.cs.tcd.ie/Oliver.Cassidy/Meta/meta.pdf
Chambers, E. A. (2002). Efficacy of educational technology in elementary and secondary
classrooms: A meta-analysis of the research literature from 1992-2002. Unpublished
doctoral dissertation, Southern Illinois University, Carbondale.
*Choi, B., & Grennaro, E. (1987). The effectiveness of using computer simulated
experiments on junior high students’ understandings of the volume displacement
concept. Journal of Research in Science Teaching, 24(6), 539-552.
Christmann, E. P., & Badgett, J. L. (2003). A meta-analytic comparison of the effects
of computer-assisted instruction on elementary students’ academic achievement.
Information Technology in Childhood Education Annual, 91-104.
Clark, R. E. (1994). Media will never influence learning. ETR&D, 42(2), 1042-1629.
Cohen, P. A., & Decanay, L. S. (1992). Computer-based instruction and health professions
education: A meta-analysis of outcomes. Evaluation and the Health Professionals,
15(3), 259-281.
*Costabile, M., De Angeli, A., Roselli, T., Lanzilotti, R., & Plantamura, P. (2003).
Evaluating the educational impact of a tutoring hypermedia for children. Information
Technology in Childhood Education Annual, 15(1), 289-308.
*Cowen, M. B. (1993). Designing an instructional simulation for a program entry panel.
Simulation & Gaming, 24(4), 500-506.
Decortis, F., & Rizzo, A. (2002). New active tools for supporting narrative structures.
Personal and Ubiquitous Computing, 6(5-6), 416-429.
*Engum, S. A., Jeffries, P., & Fisher, L. (2003). Intravenous catheter training system:
Computer-based education versus traditional learning methods. American Journal
of Surgery, 186(1), 67-75.
*Farrell, M. J., Arnold, P., Pettifer, S., Adams, J., Graham, T., & MacManamon, M. (2003).
Transfer of route learning from virtual to real environments. Journal of Experimental
Psychology: Applied, 9(4), 219-227.
*Faulkner, D., Joiner, R., Littleton, K., Miell, D., & Thompson, L. (2000). The
mediating effect of task presentation on collaboration and children’s acquisition
COMPUTER GAMING AND INTERACTIVE SIMULATIONS / 239

of scientific reasoning. European Journal of Psychology of Education, 15(45),
417-430.
Fletcher-Flynn, C. M., & Gravatt, B. (1995). The efficacy of computer-assisted
instruction (CAI): A meta-analysis. Journal of Educational Computing Research,
12(3), 219-242.
*Frear, V., & Hirschbuhl, J. J. (1999). Does interactive multimedia promote achievement
and higher level thinking skills for today’s science students? British Journal of
Educational Technology, 30(4), 323-329.
Galvao, J. R., Martins, P. G., & Gomes, M. R. (2000). Modeling reality with simula
-
tion games for a cooperative learning. Proceedings of the 2000 Winter Simulation
Conference, 1692-1698.
Garg, A., Norman, G. R., Spero, L., & Maheshwari, P. (1999). Evaluation of technology-
enhanced education. Academic Medicine 75(10), 87-89.
George, G., & Sleeth, R. G. (1996). Technology-assisted instruction in business schools:
Measured effects on student attitudes, expectations, and performance. International
Journal of Instructional Media, 23(3), 6p.
Haidet, P., Hunt, D., & Coverdale, J. (2002). Learning by doing: Teaching critical appraisal
of randomized trials by performing an in-class randomized trial. Academic Medicine,
77(11), 1161-1162.
Hakkarainen, K., Lipponen, L., Jarvela, S., Niemivirta, M. (1999). The interaction
of motivational orientation and knowledge-seeking inquiry in computer-
supported collaborative learning. Journal of Educational Computing Research, 21(3),
263-281.
Hammond, N., McKendree, J., & Scott, P. (1996). The PSYCLE project: Developing
a psychology computer-based learning environment. Behavior Research Methods,
Instruments, & Computers, 28(2), 336-340.
*Hess, R. D., & McGarvey, L. J. (1987). School-relevant effects of educational use of
microcomputers in kindergarten classrooms and homes. Journal of Educational
Computing Research, 3, 269-287.
Jackson, D. F. (1997). Case studies of microcomputer and interactive video simulations in
middle school earth science teaching. Journal of Science Education and Technology,
6(2), 127-141.
Jantz, C., Anderson, J., & Gould, S. M. (2002). Using computer-based assessments to
evaluate interactive multimedia nutrition education among low-income predominantly
Hispanic participants. Journal of Nutrition Education and Behavior, 34(5), 252-260.
Jenkins, H. (2002, March). Game theory. Technology review.
Jollicoeur, K., & Berger, D. E. (1988). Implementing educational software and evaluating
its academic effectiveness: Part II. Educational Technology, 28(10), 13-19.
Ju, E., & Wagner, C. (1997). Personal computer adventure games: Their structure,
principles, and applicability for training. The Database for Advances in Information
Systems, 28(2), 78-92.
Kafai, Y. (2001). The educational potential of electronic games: From games-to-teach to
games-to-learn. Paper presented at the Cultural Policy Center, Los Angeles.
*Kekkonen-Moneta, S., & Moneta, G. B. (2002). E-Learning in Hong Kong: Com
-
paring learning outcomes in online multimedia and lecture versions of an introductory
computing course. Journal of Computers in Mathematics & Science Teaching, 21(4),
361-379.
240 / VOGEL ET AL.

*Kim, J. H., Kim, W. O., Min, K. T., Yang, J. Y., & Nam, Y. T. (2002). Learning by
computer simulation does not lead to better test performance on advanced cardiac
life support than textbook study. JEPM, 14, 395-400.
Kirriemuir, J. (2002). The relevance of video games and gaming consoles to the higher
and further education learning experience, JISC, 3(1). Available at:
http://scholar.google.com/scholar?hl=en&lr=&client=safari&q=cache:MHPs2N6Kb
RoJ:technologiaedu.us.es/bibliovir/pdf/301.pdf+
Klassen, K. J., & Willoughby, K. A. (2003). In-class simulation games: Assessing student
learning. Journal of Information Technology Education, 2, 1-13.
Kulik, C. C., & Kulik, J. A. (1986). Effectiveness of computer-based education in colleges.
AEDS Journal, 19(2-3), 81-108.
Kulik, J. A. (1994). Meta-analytic studies of findings on computer-based instruction. In
H. F. O’Neil, Jr. & E. L. Baker (Eds.), Technology assessment in education and
training (pp. 9-33). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
*Laffey, J. M., Espinosa, L., Moore, J., & Lodree, A. (2003). Supporting learning and
behavior of at-risk youth children: Computers in urban education. Journal of Research
on Technology in Education, 35(4), 423-439.
Lee, J. (1999). Effectiveness of computer-based instructional simulation: A meta-analysis.
International Journal of Instructional Media, 26(1), 71-85.
*Levine, T., & Donitsa-Schmidt, S. (1996). Classroom environment in computer-inte-
grated science classes: Effects of gender and computer ownership. Research in Science
and Technological Education, 14(2), 163-178.
Lowe, J. (2001-2002). Computer-based education: Is it panacea? Journal of Research on
Technology and Education, 34(2), 163-171.
Manninen, T. (2002). Contextual virtual interaction as part of ubiquitous game design and
development. Personal and Ubiquitous Computing, 6(5), 390-406.
*Marcum-Dietrich, N. I., & Ford, D. J. (2002). The place for the computer is in the
laboratory: An investigation of the effect of computer probeware on student learning.
Journal of Computers in Mathematics & Science Teaching, 21(4), 361-379.
*McGarvey, L. (1986). Microcomputer use in kindergarten and a home: Design of the
study effects of computer use on school readiness. (Report No. PS-015-905) (ERIC
Document Reproduction Service No. ED 272 275).
McGrenere, J. (1996). Design: Educational electronic multi-player games—A literature
review. Technical Report 96-12. Department of Computer Science, University of
British Columbia, Vancouver, BC.
Murphy, L., Blaha, K., VanDeGrift, T., Wolfman, S., & Zander, C. (2002). Active
and cooperative learning techniques for the computer science classroom. Journal of
Computing Sciences in Colleges, 18(2), 92-94.
Najjar, L. J., Thompson, C., & Ockerman, J. J. (1999). Using a wearable computer for
continuous learning and support. Mobile Networks and Applications, 4, 69-74.
Naps, T., Bergin, J., Jiménez-Peris, R., McNally, M. F., Patiño-Martínez, M., Proulx, V. K.,
& Tarhio, J. (1997). Using the www as the delivery mechanism for interactive,
visualization-based instructional modules. Report of the ITICSE ’97 Working Group on
Visualization, 13-26.
Niemiec, R. P., Sikorski, C., & Walberg, H. J. (1996). Learner-control effects: A review
of reviews and a meta-analysis. Journal of Educational Computing Research, 15(2),
157-174.
COMPUTER GAMING AND INTERACTIVE SIMULATIONS / 241

*North, M. M., Sessum, J., & Zakalev, A. (2003). Immersive visualization tool for
pedagogical practices of computer science concepts: A pilot study. JCSC, 19(3),
207-215.
*Office of Social and Economic Data Analysis (2003, January). Analysis of 2002 MAP
Results for EMINTS Students. Retrieved from
http://emints.more.net/evaluations/reports/map2002.pdf
Pange, J. (2003). Teaching probabilities and statistics to preschool children. Information
Technology in Childhood Education Annual, 163-173.
Papert, S. (1995). Games to be played—games to be made. In Y. B. Kafai (Ed.), Minds in
play: Computer game design as a context for children’s learning (pp. ix-xii). Hillsdale,
NJ: Erlbaum.
Parker, B. C., Cheatham, T. J., & Milling (2002). Using technology to teach technology.
The Journal of Computing in Small Colleges, 17(4), 133-141.
Perry, E. L., & Ballou, D. J. (1997). The role of work, play, and fun in microcomputer
software training. ACM SIGMIS Database, 28(2), 93-112.
Predavec, M. (2001). Evaluation of e-rat, a computer-based rat dissection, in terms of
student learning outcomes. Journal of Biological Education, 35(2).
Prensky, M. (2002). The motivation of gameplay or, the real 21st century learning
revolution. On the Horizon, 10(1).
Quyang, R. (1993). A meta-analysis: Effectiveness of computer-assisted instruction at the
level of elementary education (K-6). Unpublished doctoral dissertation, Indiana
University of Pennsylvania.
Randel, J. M., Morris, B. A., Wetzel, C. D., & Whitehill, B. V. (1991). The effectiveness
of games for educational purposes: A review of recent research. Simulation and
Gaming, 23(3), 261-276.
*Reis, J., Riley, W., Lokman, L., & Baer, J. (2000). Interactive multimedia preventive
alcohol education: A technology application in higher education. Journal of Drug
Education, 30(4), 399-421.
Rieber, L. P. (1996). Seriously considering play: Designing interactive learning environ
-
ments based on the blending of microworlds, simulations, and games. ETR&D, 44(2),
43-58.
Rifkin, A (1994). eText: An interactive environment for learning parallel programming.
ACM SIGCSE Bulletin, 26(1).
Romme, A. G. L. (2003). Learning outcomes of microworlds for management education.
Management Learning, 34(1), 51-61.
*Ronen, M., & Eliahu, M. (2000). Simulation—A bridge between theory and reality: The
case of electric circuits. Journal of Computer Assisted Learning, 16(1), 14-26.
*Rosas, R., Nussbaum, M., Cumsille, P., Marianov, V., Correa, M., Flores, P., Grau, V.,
Lagos, F. L., Ximena, L. V., Rodriguez, P., & Salinas, M. (2003). Beyond Nintendo:
Design and assessment of educational video games for first and second grade students.
Computers & Education, 40(1), 71-95.
Ryan, A. W. (1991). Meta-analysis of achievement effects of microcomputer applications
in elementary schools. Educational Administration Quarterly, 27(2), 161-184.
*Schwid, H. A., Rooke, G. A., Ross, B. K., & Sivaragan, M. (1999). Use of com
-
puterized advanced cardiac life support simulator improves retention of advanced
cardiac life support guidelines better than a textbook review. Critical Care Medicine,
27, 821-824.
242 / VOGEL ET AL.

Shifroni, E., & Ginat, D. (1997). Simulation game for teaching communication protocols.
ACM SIGCSE Bulletin, Proceedings of the Twenty-Eighth SIGCSE Technical Sym
-
posium on Computer Science Education, 29(1), 184-188.
*Shute, V. J., & Glaser, R. (1990). A large-scale evaluation of an intelligent discovery
world: SMITHTOWN. Interactive Learning Environments, 1, 51-77.
Siemer, J., & Angelides, M. C. (1995). Evaluating intelligent tutoring with gaming-
simulations. Proceedings of the 1995 Winter Simulation Conference, 1376-1382.
Soloway, E. (1998). No one is making money in educational software. Communication
of the ACM, 41(2), 11-15.
SRI, International. (2002). E-desk: A review of recent evidence on the effectiveness of
discrete educational software (DHHS Contract # 282-00-008-Task 3). Washington,
DC: Murphy, R. F., et al.
*Stasko, J., Badre, A., & Lewis, C. (1993). Do algorithm animations assist learning?
An empirical study and analysis. ACM, Proceedings of the Conference on Human
Factors in Computing Systems, USA, 61-66.
Stoney, S., & Wild, M. (1998). Motivation and interface design: Maximizing learning
opportunities. Journal of Computer Assisted Learning, 14, 40-50.
TEEM. (2002). Report on the educational use of games. Little Shelford, Cambridge:
McFarlane, A., Sparrowhawk, A., & Heald, Y. Retrieved March 5, 2004 from
http://www.teem.org.uk/publications/teem_gamesined_full.pdf
*Tompson, G. H., & Das, P. (2000). Improving students’ self-efficacy in strategic
management: The relative impact of cases and simulations. Simulation & Gaming,
31(1), 22-41.
VanSickle, R. L. (1986). A quantitative review of research on instructional simulation
gaming: A twenty-year perspective. Theory and Research in Social Education, 14(3),
245-264.
Watkins, G. L. (1998). Achievement and attitudes with CD-ROM instruction. College
Student Journal, 32(2), 293-301.
*Weir, C. G., McManns, I. C., & Kiely, B. (1991). Evaluation of the teaching of statistical
concepts by interactive experience with Monte Carlo simulations. British Journal
of Educational Psychology, 61(2), 240-247.
Yildiz, R., & Atkins, M. (1996). The cognitive impact of multimedia simulations on
14-year-old students. British Journal of Educational Technology, 27(2), 106-115.
Direct reprint requests to:
Jennifer J. Vogel
Director of Research, CREATE
School of Film and Digital Media
CREATE Division
12461 Research Parkway
Suite 500
Orlando, FL 32826-3121
e-mail: jjvogel0511@yahoo.com
COMPUTER GAMING AND INTERACTIVE SIMULATIONS / 243