No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Computer-Based Integrated Learning Systems In The Elementary and Middle Grades: A Critical Review and Synthesis of Evaluation Reports
Abstract
Currently, schools are investing substantial funds in integrated learning systems (I.L.S.'s)—networked comprehensive basic skills software from a single vendor. Although rational arguments can be made for the effectiveness of I.L.S.'s, districts want—and vendors are supplying—empirical evidence for decisionmaking. This article reanalyzes results reported in thirty evaluations of I.L.S.'s by using a common “effect size” statistic and correcting, where possible, for deficiencies in the original designs and reports. Some studies (including the most widely cited) substantially over-report I.L.S. effectiveness. On average, I.L.S.'s show a moderately positive effect on student achievement. However, the poor quality of most evaluations and the likely bias in what does get reported at all provide too weak a platform for district purchasing decisions.
... During the past few decades, as computers have become more and more prevalent in classrooms (Alliance for Childhood, 2000;Jones & Paolucci, 1998;Megendollar, 2000), a massive amount of research and subsequent controversy has accumulated (Becker, 1992;Clark, 1994;Haffner, 2000;Hativa, 1988;Healy, 1999;Kearsley, 1998;Kulik, 1994;Wenglinsky, 1998), about various aspects of CBI. One of the controversies of interest entails the use of D&P CBI activities for mathematics instruction. ...
... Though D&P software has been studied in the classroom for decades, research on the effectiveness of D&P activities in CBI is inconsistent. An overwhelming accumulation of reviews spanning more than 3 decades of D&P CBI activities have indicated advantages when contrasted with traditional instruction (Alliance for Childhood, 2000;Bahr & Rieth, 1991;Becker, 1992;Berger, Belzer & Voss, 1994;Christensen & Gerber, 1990;Hativa, 1988;Kulik, 1994). However, the benefits are far from indisputable. ...
... According to Cuban (as cited in Alliance for Childhood, 2000), D&P has been indicated to be the only type of CBI to make a positive impact on academics. Some reviews of D&P CBI indicate nominal advantages over traditional instruction (Becker 1992;Berger et al., 1994;Roblyer, Castine, & King 1988). Other researchers have indicated that using instructional technology for D&P of basic skills can be highly effective according to a long history of use and a large body of data (Kulik, 1994;Kulik & Kulik, 1991;Underwood et al., 1996). ...
... Grade Number of studies Effect sizes Kulik and Kulik (1991) K-12 18 +0.25 Becker (1992) K-8 10 +0.18 Ouyang (1993) K-6 20 +0.16 Fletcher-Finn and Gravatt (1995) K-12 23 +0.12 Soe et al. (2000) K-12 17 +0.13 Blok et al. (2002) K-3 42 +0.19 ...
... Research on the effectiveness of various forms of educational technology applications such as computer-assisted instruction (CAI) for improving learning outcomes has been abundant since the 1980s. Several major meta-analyses of the impact of educational technology on reading have also been conducted in the past two decades (Becker, 1992;Blok, Oostdam, Otter, & Overmatt, 2002;Fletcher-Finn & Gravatt, 1995;Kulik & Kulik, 1991;Kulik, 2003;Ouyang, 1993;Soe, Koki, & Chang, 2000). Overall, all came to a similar conclusion, that educational technology generally produced small to moderate effects on reading outcomes with effect sizes ranging from +0.06 to +0.43 (see Table 1). ...
... For example, Blok et al. (2002) examined 42 studies from 1990 onward and found an overall effect size of +0.l9 in support of educational technology for K-3 students. Their conclusion was consistent with the findings of earlier reviews by Becker (1992), Fletcher-Finn and Gravatt (1995), and Ouyang (1993. Of particular relevance to our review are the two meta-analyses by Kulik and Kulik (1991) and Soe et al. (2000), which had a focus on K-12 classrooms. ...
The purpose of this review is to learn from rigorous evaluations of alternative technology applications how features of using technology programs and characteristics of their evaluations affect reading outcomes for students in grades K-12. The review applies consistent inclusion standards to focus on studies that met high methodological standards. A total of 84 qualifying studies based on over 60,000 K-12 participants were included in the final analysis. Consistent with previous reviews of similar focus, the findings suggest that educational technology applications generally produced a positive, though small, effect (ES = +0.16) in comparison to traditional methods. There were differential impacts of various types of educational technology applications. In particular, the types of supplementary computer-assisted instruction programs that have dominated the classroom use of educational technology in the past few decades were not found to produce educationally meaningful effects in reading for K-12 students (ES = +0.11), and the higher the methodological quality of the studies, the lower the effect size. In contrast, innovative technology applications and integrated literacy interventions with the support of extensive professional development showed more promising evidence. Although many more rigorous, especially randomized, studies of newer applications are needed, what unifies the methods found in this review to have great promise is the use of technologies in close connection with teachers’ efforts.
... Several studies included in the Li and Ma (2011) review did not establish initial equivalence (e.g., Funkhouser, 2003;Wodarz, 1994;Zumwalt, 2001). In his review, Becker (1992) found that among the seven known studies of the WICAT integrated learning system, only one provided some evidence on the comparability of comparison populations and provided data showing changes in achievement for the same students in both experimental and control groups. Studies with large pretest differences also pose threats to validity, even if statistical controls are used. ...
... Cherry-picking is a strategy used by some developers or vendors to pick favorite findings to support their cause. When analyzing the effectiveness of Integrated Learning Systems (ILS), Becker (1992) included 11 Computer Curriculum Corporation (CCC) evaluation studies in his review. Four of the 11 studies were carried out by the vendor. ...
... Effect sizes provided by the vendor were suspiciously large, ranging from +0.60 to +1.60. Upon closer examination, Becker (1992) found that the evaluators used an unusual procedure to exclude students in the experimental group, those who showed a sharp decline in scores at posttest, claiming that these scores were atypical portraits of their abilities. However, the evaluators did not exclude those who had a large gain, arguing that the large gain might have been caused by the program. ...
The present review examines research on the effects of educational technology applications on mathematics achievement in K-12 classrooms. Unlike previous reviews, this review applies consistent inclusion standards to focus on studies that met high methodological standards. In addition, methodological and substantive features of the studies are investigated to examine the relationship between educational technology applications and study features. A total of 74 qualified studies were included in our final analysis with a total sample size of 56,886 K-12 students: 45 elementary studies (N = 31,555) and 29 secondary studies (N = 25,331). Consistent with the more recent reviews, the findings suggest that educational technology applications generally produced a positive, though modest, effect (ES = +0.15) in comparison to traditional methods. However, the effects may vary by educational technology type. Among the three types of educational technology applications, supplemental CAI had the largest effect with an effect size of +0.18. The other two interventions, computer-management learning and comprehensive programs, had a much smaller effect size, +0.08 and +0.07, respectively. Differential impacts by various study and methodological features are also discussed.
... Sixty thousand dollars will buy twenty-six student stations, a computer that functions as the server, a printer, and the coursework (Estep et al. 1999). Becker (1992) argues that perhaps monies could be better spent on mandating smaller classes, raising teachers' salaries to attract more educated people into the teaching profession, purchasing printed material, providing inservice training for handling classroom managements problems, or training in the use of other innovative instructional programs. ...
... School districts that have purchased an ILS are reluctant to put some students on the systems and then deny others the same opportunity for the sake of developing experimental and control groups (Estep et al. 1999). Becker (1992) states the poor quality of most evaluations and the likely bias of reported information provides too weak a platform for district purchasing decisions. (Schacter, 1999). ...
... Teachers' roles in implementing ILS instruction and integrating ILS activities with classroom activities is a critical factor in determining the effectiveness of ILS programs with regard to students' achievement . Becker (1992) performed a meta-analysis of 32 evaluative studies of ILS programs and found that a critical ingredient for successful student outcome was the integration of teacher-led classroom activities with computerbased activities. Becker found that teachers who were the most effective implementers of ILS instruction incorporated the ILS instruction with classroom instruction. ...
The purpose of this mixed methods study was to ascertain using multiple tools and methods, the impact of teachers' interaction with primary students on mathematics performance using the Successmaker Integrated Learning System to determine effective teacher practices. The sample of nineteen primary teachers from an elementary school in southwest Florida included five first grade teachers, seven second grade teachers, and seven third grade teachers. Data analysis included total years of teaching experience, years of experience in Florida schools, and years of experience at each grade. Both the total years of teaching and the years of teaching at the research site were correlated to the Student Gains Reports in mathematics provided by the Successmaker Management System. A Pearson Product Moment correlation coefficient was computed to determine whether and to what degree a statistical relationship existed between the variables of years of teaching experience and student gains. Total yea
rs of teaching experience and total years at the research site revealed a significant correlation of 0.77 (p
... Both SuccessMaker, and Tomorrows Promise have these features. Becker (1992) makes the point that the curriculum content and the features available in particular Integrated Learning Systems should not be regarded as static. The value of an ILS is significantly determined by the extent to which the curriculum content, and frequently the system itself, are evolving in the light of technological development and the learning performance of students. ...
... The effect sizes computed from data supplied by NCS LEARN ranged from +0.6 to +1.6. Becker (1992) argued that the procedure adopted by NCS LEARN of dropping cases that showed a sharp decline in performance should have been matched by the elimination of cases showing a sharp improvement. However even after adjustment the effect sizes were only reduced in one of the 4 studies and then only to +0.35. ...
This Executive Review is a brief summary of a report of research conducted for the Department of
Education, Training and Youth Affairs between 1997 and 2001.
The results of the study suggest that the systematic use of computer-based learning systems in the form
of Integrated Learning Systems (ILS) in primary schools can produce substantial growth in number
achievement for most children. For low achieving children improvements in reading were also noted.
These results are particularly impressive given that the curriculum base of the systems is more
appropriate for North American children.
... According to Bailey (1993) and Becker (1992), there are a variety of perceived advantages to utilizing ILSs. First, there is a centralized management system that can carry a student's records through a variety of grade levels. ...
... First, this study adds to the knowledge regarding the value of computer-based instruction on a general level and with respect to the ALEKSa specific form of ILS. There are a variety of studies that have explored the implications of computer-based instruction and the utilization of ILSs at a breadth of grade levels (Kulik & Kulik, 1991;Becker, 1992). However, there are only a limited amount of studies that have focused specifically on the results of the usage of ALEKS program (Carter, 2004;Love, 2004). ...
... Several major meta-analyses of the impact of education technology on reading have been conducted in the past two decades (Becker, 1992;Blok, Oostdam, Otter, & Overmatt, 2002;Fletcher-Finn & Gravatt, 1995; C. L. C. Kulik & J. A. Kulik, 1991;J. A. Kulik, 2003;Ouyang, 1993;Soe, Koki, & Chang, 2000). ...
... For example, Blok, Oostedam, Otter, & Overmatt (2002) examined 42 studies from 1990 onward and found an overall effect size of +0.l9 in support of education technology for K-3 students. Their conclusion was consistent with the findings of earlier reviews by Becker (1992), Fletcher-Finn & Gravatt (1995), Kulik & Kulik (1991); Ouyang (1993); and Soe, Koki, & Chang (2000). ...
Best Evidence Encyclopedia is a free web site created by the Johns Hopkins University School of Education's Center for Data-Driven Reform in Education (CDDRE) under funding from the Institute of Education Sciences, U.S. Department of Education.
... In a summary of studies that were designed to assess the effectiveness of the Josten's integrated learning system (ILS), Becker (1992) observed only one was determined to have been effective (Sacramento Unified School District, 1989;Zollman, Oldham, and Wyrick, 1989). However he also reported that the studies were poorly designed. ...
... A more careful examination of what it takes for students to learn well from computer-based instruction ought to be the focus of future studies as well. For example, as recommended by Becker (1992), it would be fruitful to analyze student outcomes in the context of individualized and whole class instruction that is tailored to the nature of the class, the curriculum content, and the competencies being taught. ...
The purpose of this study was to determine whether the number of hours spent using the Josten?s integrated learning system (ILS) increased the academic achievement of students (N = 1,425) identified as at-risk for academic failure. Results of multiple regression analyses revealed that ILS hours were predictive of both reading and mathematics achievement even when gender, socioeconomic status, race/ethnicity, teacher longevity, teacher experience, and previous achievement were controlled. In addition, race and the length of teachers? experience were significantly related to reading achievement.
Curvilinearity was also assessed to determine whether a performance peak is reached as a result of an optimal number of hours of computer use, but results were non-significant. As indicated by the results, the ILS program may be a factor in promoting students? reading and mathematics achievement. Recommendations for future studies and the instructional use of ILS are discussed.
... Both SuccessMaker, and Tomorrows Promise have these features. Becker (1992) makes the point that the curriculum content and the features available in particular Integrated Learning Systems should not be regarded as static. The value of an ILS is significantly determined by the extent to which the curriculum content, and frequently the system itself, are evolving in the light of technological development and the learning performance of students. ...
... The effect sizes computed from data supplied by NCS LEARN ranged from +0.6 to +1.6. Becker (1992) argued that the procedure adopted by NCS LEARN of dropping cases that showed a sharp decline in performance should have been matched by the elimination of cases showing a sharp improvement. However even after adjustment the effect sizes were only reduced in one of the 4 studies and then only to +0.35. ...
Over recent years there has been intense community debate over educational standards in Australian schools. This report addresses the possible contribution of a type of emerging technology called "integrated learning systems" to developing numeracy and literacy in primary school children. The study began in 1997 and trialled the use of two integrated learning systems which are computer based systems known as Tomorrows Promise and SuccessMaker The systems are currently in use in the US, and are designed to manage the delivery of curriculum materials to individual students. They are capable of providing feedback to both the student and the teacher through diagnostic assessments. . These systems were tested in developing number and language concepts with grade 3, 4 and 5 children in a number of ACT schools.The study included control groups for each school who had access to more traditional literacy and numeracy programs. The results of the study suggest that substantial growth in number achievement, and for low achieving children in reading, can result from the systemic use of computer-based learning systems (Integrated Learning Systems) in primary schools, even when the curriculum base of the systems is more appropriate for North American children. After working with ILS for three years, children gained on average six months in mathematics compared with children working with the normal school curriculum. In general, children, teachers and parents were positive about the use of Integrated Learning Systems.
... Furthermore, the results of research on the impact of educational technology on reading coincide in showing positive but moderate effects compared to traditional methods (Becker, 1992;Fletcher-Flinn and Gravatt, 1995;Soe et al., 2000). The use of reading instruction programs that use ICT resources as an educational supplement, which were the most frequently used programs in past decades, did not have a significant effect on reading performance (Dynarski et al., 2007;Campuzano et al., 2009). ...
The digital transformation of educational systems requires an evaluation of the effects of the integration of technologies in teaching-learning processes. From a pedagogical approach, Information and Communication Technologies (ICT) are defined, on the one hand, as the set of technologies that contain, store and disseminate information (e.g., e-books, videos, or databases) and, on the other hand, those technologies designed for short-term communication (e.g., social networks and smartphones). Academic achievement is one of the most widely used variables to try to understand how information and communication technologies affect student learning outcomes. Several international studies have shown little improvement in performance attributed to the use of ICT, although other reviews have shown positive results in relation to certain curricular areas. However, in general, the research is inconclusive and more studies are needed on this complex relationship. A systematic review was carried out using the Education Resources Information Center (ERIC) educational database as a documentary source, and research articles on academic performance and ICT use were selected (n = 100). As a result, there was evidence of improved performance in educational practices enriched with ICT. Mathematics and science are the areas of greatest interest to researchers, and it was observed that the educational systems most oriented toward competitiveness and educational selectivity are the most productive in this field. The discrepancies between the “macro-studies” of international organizations and the “micro-studies” analyzed in this review are discussed.
... Cherry picking only those results or reporting results of only those subgroups that show positive outcomes is a risk, if the developers or vendors have supported the studies financially or conducted the studies themselves. For example, Becker (1992) re-analyzed the effectiveness of several vendor-conducted studies, finding cherry picking of results; and while recalculating the results, he found a drop in effect size from 0.80 to 0.35, which is much closer to the average of the other reviews. ...
Computer-assisted interventions (CAI) on basic number skills have been studied over the last 70 years. The technical development from large, room-sized, mainframe computers to handhelds is revising the school pedagogy in a similar fashion to what school books once did. Computers provide a tool for delivering instruction, but still the contents of CAI have followed on the ruling pedagogical trends of the time. The basic models of repetitive practice to gain arithmetic fluency and problem-solving–oriented discovery learning can be found from CAI on numerical skills. The increasing knowledge about educational neuroscience has not changed these models but turned the attention to the details in the types of numerical stimuli used in training. During these 70 years, there has been a prolific increase in the amount and quality of studies on CAI, but a declining trend in the effectiveness. The charm of novelty is transforming into a daily tool for learning.
... Kulik and Kulik (1991) 1966-19869 +0.39 Ryan (1991 1984-1989 8 +0. 30 Becker (1992) 1977-198911 +0.27 Khalili & Shashaani (1994) 1988-199218 +0.52 Fletcher-Flinn and Gravatt (1995 1987-1992 24 +0.32 Christmann, Baegett, & Lucking (1997) 1984-1994 13 +0.18 Liao (1998) 1986-19975 +0.13 Christmann & Baegett (2003 1966-2001 12 +0.34 ...
This meta-analysis examined the effect of computer-assisted instruction (CAI) on mathematics achievement in mainland China. Using rigorous inclusion criteria, we identified 36 qualifying studies, covering 13,438 students in grades 1-12. They produced a moderate overall effect (ES=+0.38) in comparison to traditional instruction. Of the various types of CAI, Cooperative-CAI and Inquiry-CAI had the largest effects. This finding seemed to support the logic behind the recent curriculum reform in mainland China, namely that CAI produced better results when combined with new teaching models. Even Supplemental-CAI and Transmission-CAI were more efficient than traditional transmission teaching. The impact of other substantive and methodological study features on the variation of effect sizes by moderator analysis are also discussed.
... Additionally, although CAI can be part of a core curriculum, serious games are usually self-contained and supplement the core curriculum through individualized drills, adaptive practice, and assessment (Cheung & Slavin, 2013). Finally, when serious games record students' progress and supply reports, these systems are integrated learning systems (ILS; Becker, 1992). In the opening vignette, Ms. Lesning is grappling with how to best use a serious game, which is embedded in an ILS, as a supplement for reading instruction. ...
Computer games and computer‐assisted instruction have become prevalent in reading classrooms. However, research‐based guidance for teacher use of such resources has not progressed at the same rate. In lieu of research, myths circulate about the utility of these resources, particularly when students use computer games independently. The authors synthesize research regarding computer games for young readers and particularly focus on issues of classroom implementation. This article is structured as a series of myths regarding expectations and best practices with computer games. The authors then present research findings to refute the myths and conclude with practical recommendations for teachers. Final recommendations emphasize the need for young readers to interact more, with adults and peers, in order for the learning that occurs in the computer environment to transfer to authentic reading activities. The authors encourage teachers to draw on pedagogical and content knowledge to be critical consumers of computer resources.
... A number of metaanalyses have evaluated the students' learning effectiveness in multiple domain subjects, while some reviews have considered only single domain subject such as mathematics (Cheung & Slavin, 2013;Rakes, Valentine, McGatha, & Ronau, 2010; Li & Ma, 2010;Slavin & Lake, 2008) and language (Archer et al., 2014). Furthermore, a majority of meta-analyses have been conducted on multiple grade levels such as elementary to college (Liao, 1999(Liao, , 20071998;Kulik & Kulik, 1991;Rakes et al., 2010;Ahmad & Lily, 1994;Fletcher-Finn & Gravatt, 1995;Li & Ma, 2010;Sung, Chang, & Liu, 2016a), elementary to secondary or middle school (Archer et al., 2014;Becker, 1992;Cheung & Slavin, 2013). Table 1 summarises the meta-analyses of the impact of technology on learning effectiveness of students, extracted for elementary students. ...
The existing studies suggest that if technology is interwoven comprehensively into pedagogy, it can act as a powerful tool for effective learning of the elementary students. This study conducted the meta-analysis by integrating the quantitative findings of 122 peer-reviewed academic papers that measured the impact of technology on learning effectiveness of elementary students. The results confirmed that the technology has a medium effect on learning effectiveness of elementary students. Further, this study analysed the effect sizes of moderating variables such as domain subject, application type, intervention duration, and learning environment. Finally, the impact of technology at different levels of moderating variables has been discussed and the implications for theory and practice are provided.
... Researchers have considered the benefits of computer use for mathematics learning for decades. Computers can provide developmentally appropriate experiences for children (Clements, 2002), as well as motivate (Becker, 1992;De Smedt et al., 2013) and activate (Chambers & Sprecher, 1980) children. Furthermore, they provide immediate and continuous feedback as well as repetitive practice, all of which are found to be important for children with weak skills (Hasselbring, 1986). ...
Weaknesses in early number skills have been found to be a risk factor for later difficulties in mathematical performance. Nevertheless, only a few intervention studies with young children have been published. In this study, the responsiveness to early support in kindergarteners with most severe difficulties was examined with two different computer programs. Two intervention groups were matched by age, visuo-spatial, and phonological working memory, as well as early number skills. After a short and intensive computerized intervention, the results indicated significant intervention effects for verbal counting Wilcoxon ES (r) = 0.46, and dot counting fluency, r = 0.52, when practiced with GraphoGame Math, as well as for basic arithmetic, r = 0.63, when practiced with Number Race. The findings suggest that a targeted computerized practice can produce specific training effects in kindergarteners most at-risk for mathematics difficulties. The results are discussed with regard to practical implications for educational game development.
... • [8] revisa os "integrated learning systems" e mostra que os resultados são positivos, mas pequenos, e critica os desenhos metodológicos dos experimentos reportados. ...
Esta revisão sistemática visa a entender qual o ganho que o computador promove na ação pedagógica para alunos do Ensino Fundamental e Médio, tendo como base publicações das últimas três décadas, indexadas no banco de dados do “Education Research Information Center” (ERIC). Obtivemos como resultado um total de 109 artigos considerados relevantes para esta pesquisa; estes foram classificados em artigos experimentais positivos, negativos e neutros. Consideramos como positivos, artigos que apontam algum resultado favorável ao uso educacional dos computadores; como negativo, o oposto a estes e ainda, como neutros, aqueles que não informam, em seu resumo, o resultado do experimento estudado. Concluímos ainda que há poucas evidências experimentais publicadas em revistas internacionais que suportem a crença de que o computador proporciona
ganhos na Educação Fundamental e Média. Na revisão das metaanálises, concluímos que elas indicam resultados mais otimistas, para o uso de computadores na educação, que os resultados experimentais permitiriam deduzir e que muitas apresentam problemas metodológicos.
... Whilst it does provide students with feedback on the correctness of their responses, a process that is desirable for effective learning (Sivin-Kachala et aI., 1997), its worksheet nature makes it susceptible to the same pedagogical flaws that were identified by Erlwanger (1975) in the Individually Prescribed Instructional (IPI) packages that proliferated in the US in the 1970s (Baturo, Cooper, & McRobbie, 1998). Whilst there is very little evidence of the ILS's improving student learning (e.g., Becker, 1992), they are nevertheless reasonably popular in many schools in Queensland. The exploration of the reasons for this is the focus of the study described in this paper. ...
Eight classes (Years 4 to 7) from a Queensland primary school trialed an integrated learning system (ILS) as a means of re mediating students' mathematics learning problems. At the end of the trial, the teachers were asked whether they would recommend the system to other schools. Endorsement appeared to be related to the computer knowledge of the teachers and concomitant experiences of the students, the extent of integration of the ILS sessions in classroom teaching, and the pedagogical beliefs of the teachers. Over the last two decades, Australian educational authorities have striven to ensure that schools are preparing their students for the challenges and demands of the modem computer age. Queensland aims to have, by 2001, at least four computers in every classroom, and teachers competent in the use of technology. However, the proliferation of computers in schools has not been accompanied by an integration of technology into classroom instruction (Bracey, 1994; Fuglestad, 1996; Hadfield, Maddux, & Love, 1997; Jones, 1998), a condition that appears to stem from teacher anxiety (Kristiansen, 1992; Turner, 1995), a clash in pedagogical beliefs (Chu & Spires, 1991), lack of adequate in service (Castner, 1998), and problems associated with computers (Pacey, 1992; Yelland, 1997). One of the by-products of the growth of information technology in education has been the computer-based integrated learning system (ILS) which includes extensive courseware plus management software. An ILS has three essential components, namely, substantial course content, aggregated learner record system, and a management system which "will update student records, interpret learner responses to the task in hand and provide performance feedback to the learner and teacher" (Underwood, Cavendish, Dowling, Fogelman, & Lawson, 1996, p. 33). This paper reports on a study to determine reactions of teachers at one school to the core numeracy courses in an ILS.
... During the late 1980s the US educational software marketplace was dominated by integrated learning systems, extensive software resources developed over many years, providing a managerial support system and often based on a behaviourist or quasi-behaviourist view of learning. Over time, much research on this topic had moved on from attempting to measure learning outcomes (Bentley, 1991;Becker 1992a;Fischer, 1996) to considering the ways in which these systems might be improved and made more effective (Becker, 1992b;Maddux and Willis, 1992;White, 1992;Hativa and Becker, 1994). ...
Acknowledgements The project team wishes to acknowledge the support of Becta (the British Educational Communications and Technology Agency) for initiating and funding this project on behalf of the Department for Education and Skills (DfES), and the ongoing advice, encouragement and support which we have received, in particular from Malcolm Hunt, Head of Evidence and Research, Becta, and from Andrew Jones and Michael Harris, Education Officers, Becta.
... The growth of ITS and the accumulation of evaluation research justify a meta-analysis of the effectiveness of ITS on students' mathematical learning for the following three reasons. First, several reviews of the impact of ITS on reading already exist (Becker, 1992;Blok, Oostedam, Otter, & Overmaat, 2002;Kulik, 2003). Most recently, Cheung and Slavin (2012) reviewed the effects of educational technology on K-12 students' reading achievement, relative to traditional instructional methods. ...
In this study, we meta-analyzed empirical research of the effectiveness of intelligent tutoring systems (ITS) on K–12 students’ mathematical learning. A total of 26 reports containing 34 independent samples met study inclusion criteria. The reports appeared between 1997 and 2010. The majority of included studies compared the effectiveness of ITS with that of regular classroom instruction. A few studies compared ITS with human tutoring or homework practices. Among the major findings are (a) overall, ITS had no negative and perhaps a small positive effect on K–12 students’ mathematical learning, as indicated by the average effect sizes ranging from g = 0.01 to g = 0.09, and (b) on the basis of the few studies that compared ITS with homework or human tutoring, the effectiveness of ITS appeared to be small to modest. Moderator analyses revealed 2 findings of practical importance. First, the effects of ITS appeared to be greater when the interventions lasted for less than a school year than when they lasted for 1 school year or longer. Second, the effectiveness of ITS for helping students drawn from the general population was greater than for helping low achievers. This finding draws attentions to the issue of whether computerized learning might contribute to the achievement gap between students with different achievement levels and aptitudes. (PsycINFO Database Record (c) 2013 APA, all rights reserved)
... Such ILS programs automatically load one of an extensive sequence of exercises into each students computer. Evaluations of these systems show a moderate effect on basic skills (Becker, 1992;Kelman, 1990); however, one must consider other aspects of ILS, especially diminished teacher and student control. Also, there are possibilities that children will be less motivated to perform academic work following drill (Clements & Nastasi, 1985) and that their creativity may be harmed (Haugland, 1992). ...
... While there are those who tout the educational benefits of computer-based instructional technologies (Barth, 1990;Crosby & Stelovsky, 1995;Fletcher-Flinn & Gravatt, 1995;Friend & Cole, 1990;Greenfield, 1984;Johnston, 1995;Liu & Reed, 1994;Sloan, 1997;Vockell & Brown, 1992), there are others who reject this proposition (Clark, 1983(Clark, , 1991(Clark, , 1994Kay, 1996;Lookatch, 1995Lookatch, , 1996Lookatch, , 1997Mergendoller, 1996;Oppenheimer, 1997;Pepi & Scheurman, 1996;Russell, 1999). In addition, there has been much criticism regarding the quality of many of the studies that have indicated an advantage of CBI over traditional forms of instruction (Becker, 1992;Berson, 1996;Clark, 1983Clark, , 1994Cuban & Kirkpatrick, 1998;Lookatch, 1995Lookatch, , 1996Reeves, 1993Reeves, , 1998 Precisely what these factors are and to what degree they influence, facilitate or dictate how humans learn, however, remain sources of contention among scholars and researchers from a variety of educational disciplines (Brown, 1997a(Brown, , 1997b(Brown, , 1997cHiemstra & Brockett, 1994;Merrill 1994;Steinberg, 1989). ...
This is a report on research regarding the screen layout of Web-based training (WBT) programs, conducted with an eye toward providing evidence-based guidance for the design and development of WBT interfaces. Specifically, the study investigated the relative instructional benefits of two general types of WBT screen design, full-page and partial-page, in terms of both learner performance and learner satisfaction. The main hypotheses of the study were that the full-page design option would yield significantly better outcomes in both categories of interest.The study employed a mixed-method design, generating both quantitative and qualitative data. The main phase of the study was experimental, following a factorial design to explore the relationships between a single treatment variable (WBT screen design) in two treatment conditions (partial-page WBT design and full-page WBT design) and two dependent variables (learner performance and learner satisfaction). Both a full-page and
a partial-page version of the same Web-based tutorial were created, and 129 self-selected undergraduate students who reported having little or no experience with the tutorial subject matter were randomly assigned into the two treatment groups. Performance data were collected as scores on the tutorial's 18-item, multiple choice final exam, and satisfaction data were collected via a 10-item satisfaction survey. In addition, 59 of the study participants were randomly selected to participate in post-study session interviews.The results of the study yielded no significant difference between the two treatment groups for either learner performance or learner satisfaction; thus, making it impossible to reject the null hypothesis for either of the two primary research questions. The conclusion of this study was that the presence or absence of scrolling alone is not a significant factor either in how well a person performs in a WBT program or how satisfied they are with the learning experience
. However, while analysis of the post-study session interview data supported this conclusion, the fact that a large majority of the interviewees stated a preference for the full-page, non-scrolling WBT interface design suggests that some elements inherent in the full-page design might warrant further consideration and/or study.
... According to Trotter (1990) large gains have been reported on standardized test scores in studies examining student learning outcomes with integrated learning systems (ILS). But Becker (1992) completed a meta-analysis of nearly 100 studies and concluded that standardized tests provide little conclusive evidence of ILS impact on achievement. However, The Teachers and technology: ...
... These results are consistent with those in Becker's (1992) review on ILS effectiveness. Becker's report reviewed results from 32 early studies of ILS effectiveness in basic skills instruction. ...
... In 1990, many educators expressed support for these systems (Mageau, 1990), but by 1994 others suggested that they have little positive effect on leaming and may be harmful to certain students (Thomburg, 1994). A review of ILS research by Becker (1992) indicated that after reviewing more than 30 studies related to integrated leaming systems, most of the studies were of an unsatisfactory quality due to methodological problems. Overall, the effects identified by the studies are widely variable and quite modest. ...
ACKNOWLEDGMENTS This dissertation would not have been possible without the encouragement and support from numerous people. I wish thank my committee members, Terence Ahem, Arturo Olivarez, Nancy Maushak, Robert Price, and Sandi Cooper for their time, willingness to help, and expertise. I will be forever grateful to my family for providing me with the opportunity to pursue and enhance my education, for without their love and support this dissertation would not have been possible. TABLE OF CONTENTS ACKNOWLEDGMENTS,ii ABSTRACT,vii USTOFTABLES,viii LISTOFHGURES,ix CHAPTER I. INTRODUCTION,1 Statement of the Problem ,5 Research Questions ,7 Research Hypotheses ,8 Rationale,9
Technology advancements in education have fuelled worldwide digital technology agendas, providing students with skills necessary for success in the twenty-first century, including digital literacy and the ability to communicate effectively, think critically, work effectively in teams, and generate original ideas. With just about 7 years left, achieving SDG4 has become the hallmark of many educational systems. Objective 4.7 calls for opportunities to ensure all learners acquire the knowledge and skills to promote sustainable development. Adapting different technological tools has become prominent in our quest for maximizing meaningful student engagement and active participation in the teaching-learning process. This chapter provides a position orientation of computer-assisted instruction (CAI) in mathematics education with cognizance of its benefits, challenges, and opportunities. We begin with a position on CAI’s meaning, objectives, features, and mode of delivery. Theoretically, we discuss some technology integration frameworks used in a CAI environment and contribute to the debate on using CAI in mathematics. Through a review of meta-analysis studies, we offer evidence of the positive effect of CAI on students’ mathematics learning outcomes.
Most of the published research in educational gaming has had methodological limitations. Process data critical to understanding under what conditions games can promote learning are typically not collected, and unreliable student and teacher self-reports are often the primary data source used when assessing the educational efficacy of many games. To address these and other methodological issues, the authors have developed a research software tool, OpenVULab1, which can record screen activity during game2 play in classroom settings remotely and unobtrusively, together with a synchronized audio track of player discussion. This chapter describes the structure, operation, and affordances of the tool and reports on the results of a field trial designed to evaluate its utility. In this trial, 42 college students were studied using OpenVULab as they played a coursework-related web-based learning game. The chapter concludes with an analysis of the trial outcomes, showing how they concretely demonstrate the methodological advantages that the use of OpenVULab offers researchers.
Στις μέρες μας, τα Οπτικοακουστικά Μέσα διαδραματίζουν σημαντικό ρόλο στην Εκπαίδευση Ενηλίκων, και είναι ο λόγος για τον επαναπροσδιορισμό τους στη Μεθοδολογία της Διδασκαλίας. Τα Οπτικοακουστικά Μέσα είναι κομμάτι του τριπτύχου Τεχνολογίες Πληροφορίας και Επικοινωνίας (ΤΠΕ), τα οποία ενσωματώνουν πληροφοριακά και επικοινωνιακά συστήματα, καθώς και νέα τεχνολογικά συστήματα. Η συγκεκριμένη θεωρητική μελέτη επιχειρεί να συνοψίσει, μέσα από βιβλιογραφική επισκόπηση, τους ποικίλους παράγοντες που φαίνεται να έχει η χρήση των Οπτικοακουστικών Μέσων στην Εκπαίδευση Ενηλίκων. Τα Οπτικοακουστικά Μέσα, όπως αντανακλάται στη βιβλιογραφία, φαίνεται να προϋποθέτουν έναν νέο τρόπο προσέγγισης για μια αποτελεσματική διδασκαλία, η οποία απαιτεί πολλές νέες δεξιότητες και ικανότητες.
This study adds resources to the current research on computer-aided second-language (L2) learning by focusing on a relatively under-explored area of online oral assessment using video recording. It fills the gap of existing literature by looking into the area through the lens of students’ perception. Drawing on data collected through questionnaires and interviews of students enrolled in Chinese and Japanese language programs at Curtin University, this study found that video-recorded oral assessments facilitated L2 learning by improving learners’ motivation and encouraging self-reflection. The study also sheds light on the importance of technical and teamwork guidance in maximising learning outcomes. Data variations between beginners and advanced learners who participated in this study gave weight to the argument that a ‘one-size-fits-all’ approach is not effective in online language learning, so facilitators may need to carefully evaluate the level of learners’ language competence when they are incorporating online components into their programs.
This chapter examines selected recent studies of the design and use of digital curriculum programmes and dynamic mathematical tools in school mathematics. The examples chosen bring out diversity both in the types of digital resources which are being adopted for teaching school mathematics and in the ways in which these are being taken up in instructional activity. These examples also show how any particular resource can be used in very different ways, and in ways quite different from those espoused by its advocates or intended by the designer. Digitised versions of traditional textbooks are cautiously innovative while individualised learning designs promote more ‘personalised’ instruction. Use of dynamic digital mathematical tools can support exploratory patterns of mathematical activity, underpinned by feedback from students’ interaction with these tools.
There is no questioning that the way people live, interact, communicate, and conduct business is undergoing a profound, rapid change. This change is often referred to as the "digital revolution," which is the advancement of technology from analog, electronic and mechanical tools to the digital tools available today. Moreover, technology has begun to change education, affecting how students acquire the skill sets needed to prepare for college and a career and how educators integrate digital technological instructional strategies to teach. Numerous studies have been published discussing the barriers of integrating technology, the estimated amount of investment that is needed in order to fully support educational technology, and, of course, the effectiveness of technology in the classroom. As such, this article presents a critical review of the transitions that technology integration has made over the years; the amount of resources and funding that has been allocated to immerse school with technology; and the conflicting results presented on effectiveness of using is technology in education. Through synthesis of selected themes, we found a plethora of technological instructional strategies being used to integrate technology into K-12 classrooms. Also, though there have been large investments made to integrate technology into K-12 classrooms to equip students with the skills needed to prepare for college and a career, the practical use of this investment has not been impressive. Lastly, several meta-analyses showed promising results of effectiveness of technology in the classroom. However, several inherent methodological and study design issues dampen the amount of variance that technology accounts for.
There is no questioning that the way people live, interact, communicate, and conduct business is
undergoing a profound, rapid change. This change is often referred to as the “digital revolution,”
which is the advancement of technology from analog, electronic and mechanical tools to the digital
tools available today. Moreover, technology has begun to change education, affecting how
students acquire the skill sets needed to prepare for college and a career and how educators integrate
digital technological instructional strategies to teach. Numerous studies have been published
discussing the barriers of integrating technology, the estimated amount of investment that is
needed in order to fully support educational technology, and, of course, the effectiveness of technology
in the classroom. As such, this article presents a critical review of the transitions that
technology integration has made over the years; the amount of resources and funding that has
been allocated to immerse school with technology; and the conflicting results presented on the effectiveness
of using is technology in education. Through synthesis of selected themes, we found a
plethora of technological instructional strategies being used to integrate technology into K-12
classrooms. Also, though there have been large investments made to integrate technology into K-
12 classrooms to equip students with the skills needed to prepare for college and a career, the
practical use of this investment has not been impressive. Lastly, several meta-analyses showed
promising results of the effectiveness of technology in the classroom. However, several inherent
methodological and study design issues dampen the amount of variance that technology accounts
for.
Technology integration in teaching is one of vital medium to provide more engaging and interactive learning process in class. Integration of technology in English class is inevitable to achieve National education goals based on Malaysian Education Blueprint (2013-2015). Thus, the purpose of the study is to investigate teacher " s perception on contribution of technology use in English Language classes. Theresearchwas conducted in Kemena District in Bintulu Division, Sarawak. There were 80 English teachers in Kemena District involved as respondents for data collectionbased on questionnaire methods. The result of the study revealed that English teachers in Kemena District have positive attitude towards the instructional technology integration in their language classroom. However, instead of this positive attitude, the usage of the technologies is below the targeted rates because it was restricted by a few factors: lack of time and lack of technological means in the school. Teachers found that modern technology motivates the learners and improve their English skills. Besides, it also helps the teachers to have good control of their teaching in the classroom. For traditional technology, it is beneficial for teachers because it provides good control of the classroom and it encourages meaningful learning. So, both technologies basically were good for class controls. Copy Right, IJAR, 2016,. All rights reserved.
The purpose of this research was to determine if providing teachers with a wider variety of computer-based resources to integrate with the instructional activities for their students would have a positive effect on student achievement and attitudes. In this study, two variations of Integrated Learning System (ILS) software were available to teachers and students. One type of software ("foundations") was designed to provide students with an individualized learning environment whereby students could complete computer-based reading activities at their own pace, receive feedback from the computer, and move on to the next set of activities. The other type of software ("explorations") was designed as a more open-ended environment, where teachers selected specific activities for students (or groups of students) to complete. Results of the study indicate that the students of teachers who used either the explorations software or a combination of the foundations and explorations software in their instructional activities performed significantly better on standardized achievement measures than students of teachers who used the foundations software. In addition, students who used the explorations software had more positive attitudes toward reading than students who used the foundations software.
This study examined the effects of a cross-age tutoring program on computer literacy learning of second graders. The tutoring program consisted of 3 training sessions, 10 weekly 30-min tutoring sessions, and 3 support conferences. Seventeen fourth-grade students tutored seventeen second graders on three computer programs: a keyboarding program, a problem-solving program, and a word processing program. Observations of the second-grade tutees were conducted before and after the tutoring program to determine the effects of the tutoring program on their computer literacy learning. The results indicated that prior to the tutoring, five second graders could play a game, and three of them could use a simulation program. The observations following the program indicated that all 17 tutees could use and interact with the programs presented, a game, and a simulation program.
The purpose of this review was to examine the literature on the effectiveness of computers in social studies instruction and learning. Studies exploring computer activities including simulations, drill and practice, educational games, tutorials, database management, word processing and writing, and graphing were included. Assessment of the efficiency and effectiveness of computer technology in the social studies remains in its infancy with a limited research base. The author explores required shifts in the pedagogical functioning of social studies educators and associated future research directions for the integration of computers into the social studies curriculum.
This study examined the experiences of computer coordinators who have worked in elementary schools that have had computers for at least eight years and represents part of a longitudinal study conducted over four years. Using a replication of ethnographic methods that successfully guided the earlier wave of the study, the article considers the microprocesses of interaction among computer coordinators and classroom teachers, and how those patterns are related to the integration of computers in elementary schools. Findings reveal that, whereas school administrators and classroom teachers rely on computer coordinators to promote the use and integration of computers in their schools, these specialists are limited in their endeavors because of various structural and social conditions of their work. The examination of work relations among computer coordinators and classroom teachers may have critical implications for the direction of computer implementation in elementary schools.
This article reports a study designed to assess the effects of computer integration on students in terms of academic achievement and attitudes toward academic subjects and computers. Computer integration was perceived as a complex instructional system in which student learning is impacted by teachers, students, school administrative and technical support, computer hardware and software resources, and the computer lab and classroom settings. Eight teachers of four subjects at one high school participated in this study for seven months. A quasi-experimental design was used. Student achievement was assessed by teacher-made tests. Attitude was assessed as part of the student survey questionnaire using the instrument developed by Bear, Richards, and Lancaster (1987). In addition, three students from each integration class were interviewed. It was found that there was no significant effect of computer integration on achievement, and although positive attitude toward computers was high both before and after the integration period, there was no significant change in student attitude toward computers after the computer integration. Generally, students’ perceived using computers as having a positive effect on their learning. The methodological and instructional implications of the findings are discussed.
This descriptive study identifies and describes how exemplary technology-using teachers in south western Ohio are using technology in their elementary classrooms. Qualitative and quantitative data collection strategies were used. Data gathered from technology coordinators were used to create a questionnaire that was sent to teachers identified as exemplary. The picture that emerges from this study is a classroom that is changing from the traditional classroom to a classroom where a wide variety of teaching techniques are being used. Students use technology predominantly in a tool or constructivist mode. Discrepancies between teacher’s beliefs and practices were examined. Staff-development data were also analyzed.
The purpose of this research was to determine if providing teachers with a wider variety of computer-based resources to integrate with the instructional activities for their students would have a positive effect on student achievement and attitudes. In this study, two variations of Integrated Learning System (ILS) software were available to teachers and students. One type of software (“foundations”) was designed to provide students with an individualized learning environment whereby students could complete computer-based reading activities at their own pace, receive feedback from the computer, and move on to the next set of activities. The other type of software (“explorations”) was designed as a more open-ended environment, where teachers selected specific activities for students (or groups of students) to complete. Results of the study indicate that the students of teachers who used either the explorations software or a combination of the foundations and explorations software in their instructional activities performed significantly better on standardized achievement measures than students of teachers who used the foundations software. In addition, students who used the explorations software had more positive attitudes toward reading than students who used the foundations software.
Over the last few years, public educational policies have had an emphasis on the need to equip schools with computers and change teaching practices, to match the needs of information society. This paper analyzes the SAEB research data to check basic school students from the 4th to the 8th grades and 3rd year high school students' performance, as well as to study the relation between their performance and the use of computers. The results show us that, for all grades and for all social-economic classes, heavy use of computers is related to lower school performance. For 4th grade students from poor social backgrounds, even the moderate use of computers is associated to worse performance in Portuguese and mathematics. These results point out to the need to reflect on the role of computers in teaching, especially when it comes to poor students, for whom the use of computers is associated to a worse school performance.
Studies of Integrated Learning Systems have endorsed their potential for enhancing pupils' learning, but they have also highlighted the importance of the context and method of use in influencing the nature and quality of the learning outcomes. This paper describes an evaluation of a new version of this type of software which is explicitly designed to make links with several external factors, in particular with another software application for supporting investigative work in practical science. The evaluation adopts a situated approach which explores the contributions to the learning process made by the teacher,the student as well as the software itself. The results amplify the value of small group working, the role of the teacher and the aspects of software design which help to secure the benefits of the system.
In this rejoinder to Mazoué (J Comput High Educ, 2012) article, “the deconstructed campus,” we react to his arguments regarding the replacement of face-to-face teaching on college campuses with computer-supported approaches, including on-line learning, intelligent cognitive tutors, and open-ended learning environments where, rather than being confined to standard degree programs, students act increasingly as “free agents” in constructing and selecting learning experiences that interest them. While agreeing that such changes are inevitable and potentially beneficial in improving teaching and learning in higher education for the twenty-first century, we take issue with Mazoué’s strident criticism and dismissal of the contributions of human teachers to student development via coaching, modeling, and selected uses of didactic instruction. More fundamentally, we caution against embracing technology solutions lacking credible evidence of effectiveness for learning and implementation feasibility. We argue instead that instructional design processes, not media, need to form the anchor for course and experiential learning design, both older and cutting edge forms.
Recently, studies have been initiated to evaluate computer-based integrated learning systems (ILSs). Among those characterized by good scientific design, few have found the ILS to have positive effects on student achievement or attitudes. These findings might lead one to believe that the ILS has little impact on valued student outcomes. However, in a large national study partially reported in this chapter, it was shown that there may be another explanation for such findings, namely that in most schools ILSs are not being implemented effectively. Analyses of effects of level of implementation on outcome measures revealed that with weak implementation there is little if any positive effect of the ILS and sometimes even a negative effect, whereas with strong implementation, the ILS had a positive effect on student outcomes. The authors argue that evaluation of an ILS before its implementation fully meets the vendor's implementation standards is premature at best and may seriously underestimate the potential effectiveness of the ILS. The authors suggest why school implementation of ILSs may not be optimal, and outline implications of these findings for future ILS research and evaluation.
As políticas públicas de educação têm dado bastante ênfase, ao longo dos últimos anos, à necessidade de informatizar as escolas e modificar práticas de ensino devido ao advento da sociedade de informação. Este artigo usa as pesquisas do saeb para verificar o desempenho de alunos de 4ª e 8ª série do ensino fundamental e da 3ª série do ensino médio e a relação deste desempenho com o uso de computador. Os resultados demonstram que para os alunos de todas as séries e para todas as classes sociais o uso intenso do computador diminui o desempenho escolar. Para alunos da 4ª série, das classes sociais mais pobres, mesmo o uso moderado do computador piora o desempenho nos exames de português e matemática. Esses resultados indicam claramente que é preciso repensar o papel do computador no ensino, sobretudo para os alunos mais pobres, para quem o uso do computador está surpreendentemente associado a uma piora nas suas notas.
This review examines three sources of evidence about the effects of school instructional uses of computers on student achievement: national surveys of utilization practices, research reviews of the effectiveness of computer‐assisted instruction (CAI), and recent experimental studies of CAI. The review concludes that existing evidence of computer effectiveness is scanty, and existing studies provide little guidance for schools to decide how to use computers for instruction. A new research initiative is described that combines the benefits of survey research and field experimental research to produce more conclusive findings.
American school officials almost invariably compare local and national achievement through one of the following norm-referenced tests: (1) the California Achievement Test; (2) the Stanford Achievement Test; (3) the Metropolitan Achievement Test; (4) the Science Research Associates Test; (5) the Comprehensive Test of Basic Skills; and (6) the Iowa Test of Basic Skills. The vast majority of districts has shown steady improvement in norm-referenced test scores over the past 15 years despite the fact that other indicators of achievement do not reflect gains in American education. By early 1988 all 50 states were testing above the publisher's national norm, a phenomenon dubbed the "Lake Wobegon" effect, in reference to the mythical town where all children are above average. The survey conducted for this report in 1989 found 48 of the 50 states again scoring above the national norm, while 90% of elementary schools and 80% of secondary schools exceeded the national norm. The causes of this phenomenon (including possible cheating, deceptive testing practices, and misleading reporting methods) are discussed, and steps to avoid cheating are reviewed. A final chapter considers the effects of these inflated scores. Three appendices present: the test results from the 50 states; a survey of test security practices; and other indicators of state achievement. (SLD)