Theodore W. Frick

Indiana University Bloomington | IUB · School of Education

The theory of totally integrated education (TIE) predicts that mental structures formed by learners are expected to be stronger when “knowing that one,” “knowing how,” and “knowing that” are integrated with learner emotions and intentions. Such whole, completely connected mental structures are expected to be less vulnerable to forgetting. TIE theor...

If you were an investor, and you could choose between a company that is not likely to use effective business strategies and a company that is three to five times more likely to use them, where would you put your money? Likewise, if you could use methods of education that were three to five times more likely to help students succeed in their learnin...

In this naturalistic design-research study, we tracked 172,417 learning journeys of students who were interacting with an online resource, the Indiana University Plagiarism Tutorials and Tests (IPTAT) at https://plagiarism.iu.edu. IPTAT was designed using First Principles of Instruction (FPI; Merrill in Educ Technol Res Dev 50:43–59, 2002, https://...

Extant chatbots such as ChatGPT and Bard are currently able to converse with humans in natural language, demonstrating impressive linguistic responses. Or so it seems. I critically examine artificial intelligence systems such as these chatbots through examples of dialogue. When taking a systems view of AI, there is a vast and unique human culture i...

Extant chatbots such as ChatGPT and Bard are currently able to converse with humans in natural language, demonstrating impressive linguistic responses. Or so it seems. I critically examine artificial intelligence systems such as these chatbots through examples of dialogue. When taking a systems view of AI, there is a vast and unique human culture i...

The online Indiana University Plagiarism Tutorials and Tests have been available since 2002, now 20 years. We started out with online teaching and learning, and now it is providing big data for instructional-design research. In the past 355 weeks (since 2016), more than 1,025,000 students have achieved the main learning goal by passing a difficult...

Analysis of Patterns in Time (APT) is a well-established methodology which is an alternative to traditional quantitative and qualitative research methods. The fundamental unit of analysis in APT is a temporal map, which is a record of observed events which are classified by categories of occurrence. Temporal maps ca be queried later for estimating...

Slides used in the presentation I made to the 4th Annual Learning Analytics Summit. I cover about 50 years of my quest to do learning analytics. And I do this rather quickly, in about 25 minutes (35 slides). The blue underline links in the slides go to various resources that you can explore in further depth.

Innovative Learning Analytics for Evaluating Instruction covers the application of a forward-thinking research methodology that uses big data to evaluate the effectiveness of online instruction. Analysis of Patterns in Time (APT) is a practical analytic approach that finds meaningful patterns in massive data sets, capturing temporal maps of student...

Education provides guided and intended learning across various human disciplines. The result of disciplined inquiry about education is distinct from the process of education itself. If adequate, educational research should result in knowledge about education—that is, educology. Educology is needed to improve education, in contrast to trial-and-erro...

In Restructuring Education Through Technology, I incorporated systems thinking to identify seven types of relationships in educational systems: teacher-student, student-content, student-context, teacher-content, teacher-context, content-context, and education system-environment relationships (Frick 1991). I now revisit these education system relati...

Content in education is typically conceived as subject matter, often divided into disciplines such as mathematics, English, history, science, geography, and so forth. Content is often further conceived as being embedded in media such as textbooks, handouts, movies, computers, posters, and bulletin boards that are used in the context of classrooms i...

Educology is “knowledge of education.” Since knowledge is “recorded signs of knowing” and education is “intended and guided learning,” educology is therefore “recorded signs of knowing about intended and guided learning.” Knowledge of education systems is also part of educology. Distinctions are made in educology among terms that include education,...

The Theory of Totally Integrated Education (TIE) predicts that mental structures formed by learners are expected to be stronger when ‘knowing that one’, ‘knowing how’, and ‘knowing that’ are integrated with learner emotions and intentions. Such whole, completely connected mental structures are expected to be less vulnerable to forgetting. TIE theor...

We briefly tell our story about the Indiana University Plagiarism Tutorials and Tests (IPTAT) from the original design and development in 2002 through 2016. Widespread cheating on the Certification Test in 2012–2013 required us to redesign the test. The changes resulted in a structure that offered billions and trillions of test combinations for und...

We illustrate a very recent research study that demonstrates the value of Massive Open Online Courses (MOOCs) as vehicles for research. We describe the development of the Indiana University Plagiarism Tutorials and Tests (IPTAT). Our new design has been guided by First Principles of Instruction: authentic problems, activation, demonstration, applic...

If teachers follow First Principles of Instruction in guiding student learning, those students are likely to form stronger mental structures organized by their emotions. Student emotions and sensations should naturally arise during active experience of authentic, real-world tasks. Students will learn to think in ways that are consistent with those...

Imagine that a strategy to improve the quality of medical practice was to punish doctors whose patients are more likely to die, by paying them less than other doctors or possibly firing them. Doctors who specialize in oncology, cardiology, and geriatric care would be punished because of higher patient mortality rates. Who would want to become an on...

In this chapter we describe Analysis of Patterns in Time (APT) and how it can be used to analyze gameplay choices to provide evidence of a play-learner’s understanding of concepts modeled in a game. APT is an empirical approach to observing and coding phenomena as mutually exclusive and exhaustive categories within classifications. These data form...

Imagine that a strategy to improve the quality of medical practice was to remove doctors whose patients are more likely to die. For example, doctors who specialize in oncology, cardiology, and geriatric care would be soon be eliminated because of their higher patient mortality rates. This strategy would further result in an overabundance of pediatr...

The purpose of this project was to document the redesign of an existing doctoral reading course for an online environment. Potential methods for actualizing the proposed course structure in an online environment, including technology tools and interactions are discussed. The design process began within the framework of the Four-Component Instructio...

Educators increasingly view the high level of engagement and experiential learning offered by games as a means to promote learning. However, as with any designed learning experience, player experiences should provide an accurate representation of content to be learned. In this study, the authors investigated the DIFFUSION SIMULATION GAME (DSG) to a...

The Theory of Totally Integrated Education (TIE) predicts that mental structures formed by learners are expected to be stronger when ‘knowing that one’, ‘knowing how’, and ‘knowing that’ are integrated with learner emotions and intentions. Such whole, completely connected mental structures are expected to be less vulnerable to forgetting. TIE theor...

Conducting an iterative usability testing, a set of prompts used as a form of instructional support was developed in order to facilitate the comprehension of the diffusion of innovations theory (Rogers, 2003) in a simulation game called the Diffusion Simulation Game (DSG) (Molenda & Rice, 1979). The six subjects who participated in the study stated...

The purpose of this study was to document the redesign of an existing doctoral reading course for online environment. This design process includes contributions from van Merriënboer's 4C/ID model and Frick's Totally Integrated Education (TIE) theory, and the redevelopment of course objectives to meet strategic goals within the doctoral program.

This survey was carried out on behalf of the Association for Educational Communications and Technology (AECT) by a team of doctoral students and faculty in instructional systems technology at a large Midwestern university. The 2001 survey aims, like its predecessors, to determine the needs of AECT’s members and to identify the Association’s weaknes...

Self-directed e-learning (SDEL) refers to electronic learning environments where there are often no peer learners or instructors regularly available. Past studies suggest that lack of time and lack of motivation are primary causes of learner attrition in online settings. However, little is known about what influences motivational change during SDEL...

Extant research studies have found that autonomy support has a positive impact on the perceived competence and intrinsic motivation of students. However, few studies have investigated how autonomy supportive classrooms can be implemented. Montessori education is established upon the philosophy of helping each child attain self-mastery and independe...

In this paper, we present a design case describing the creation of a new online version of the Diffusion Simulation Game (DSG). The DSG is a serious game that teaches change management strategies aligned with Rogers’ (2003) diffusion of innovations theory. The goal of the game is to promote the understanding of strategies that result in the adoptio...

In this quasi-experimental case study, we compared five sections of a basic undergraduate technology course. Within an asynchronous web forum, pre-service teachers wrote short critiques of websites designed by their classmates. This peer feedback was provided anonymously by students in two classes (n = 35) whereas providers and recipients of peer f...

Recent research has touted the benefits of learner-centered instruction, problem-based learning, and a focus on complex learning. Instructors often struggle to put these goals into practice as well as to measure the effectiveness of these new teaching strategies in terms of mastery of course objectives. Enter the course evaluation, often a standard...

While student global ratings of college courses historically predict learning achievement, the majority of recent U.S. college graduates lack proficiency in desired skills. Teaching and Learning Quality (TALQ), a new course evaluation instrument, was developed from extant instructional theory that promotes student learning. A survey of 193 students...

Numerous instructional design models have been proposed over the past several decades. Instead of focusing on the design process (means), this study investigated how learners perceived the quality of instruction they experienced (ends). An electronic survey instrument containing nine apriori scales was developed. Students responded from 89 differen...

Technology skills instruction is an important component of educational technology courses, which has been shown to raise pre-service teachers' computer self-efficacy. Computer self-efficacy, in turn, is positively related to their self-efficacy for technology integration. Studies of undergraduate technology skills instruction found that classroom i...

This chapter surveys methods, techniques, practices, and challenging issues in user-centered design and development (UCDD). The traditional instructional systems design (ISD) approach has been criticized for its bureaucratic and linear nature and its slow process. Two alternatives to that approach are discussed here: rapid prototyping and participa...

This paper presents several brief papers about the process of systemic change. These are: (1) Step-Up-To-Excellence: A Protocol for Navigating Whole-System Change in School Districts by Francis M. Duffy; (2) The Guidance System for Transforming Education by Charles M. Reigeluth; (3) The Schlechty Center For Leadership In School Reform by Monica Sol...

While there are many distance education studies of student satisfaction or perceptions of CMC, studies on residential student perceptions of CMC are rare. A paper survey was administered to 105 residential graduate and undergraduate students at a midwestern U.S. university. Results indicated that the majority of students preferred face-to-face disc...

Our general strategy of designing, developing, and maintaining over 6,000 webpages on the Indiana University School of Education website was to: •Use an inquiry-based approach to design — user needs assessment, rapid prototyping, and usability testing. •Keep content in XML format, separate from its appearance on the web. •Have web designers at the...

How did we build a site that has grown to more than 6,000 Web pages and 50 million hits per year - with a few part -time people, with no budget for the first five years, and a minimal budget the next five? We will describe an efficient and effective design process fundamental to our strategy that is essentially a method of disciplined inquiry. Our...

Instructional illustrations are widely used in textbooks and have been shown to have the potential to aid learning. However, illustrations that are not understood as their designers intend them to he may waste resources at best and interfere with learning at worst. Learners may recognize images but not understand illustrations when their meanings a...

The Diffusion Simulation Game (DSG) was created to provide graduate students in the Instructional Systems Technology (IST) Department at Indiana University with an opportunity to play the role of a change agent charged with bringing about innovations in teaching methods at a fictional junior high school. Originally created by an IST faculty member...

After more than four decades, development of artificially intelligent tutoring systems has been constrained by two interrelated problems: knowledge representation and natural language understanding. G. S. Maccia's epistemology of intelligent natural systems implies that computer systems will need to develop qualitative intelligence before these pro...

Administrators at a large midwestern university recognized that their World Wide Web site was rapidly becoming an important factor in recruiting new students. They also expected this Web site to serve many different types of information needs for existing students, faculty, staff, and alumni. an interdisciplinary team of faculty, graduate students,...

The purpose of this experimental study was to investigate the comparative perceptibility of hypertext navigation buttons in three configurations: buttons with both pictorial symbols and text labels, with text labels only, and with pictorial symbols only. An instructional HyperCard stack was created in three versions, each differing only in the type...

Item response theory (IRT) has most often been used in research on computerized adaptive testing (CAT). Depending on the model used, IRT requires between 200 and 1,000 examinees for estimating item parameters. Thus, it is not practical for instructional designers to develop their own CAT based on the IRT model. Frick improved Wald's sequential prob...

Expert systems can be used to aid decision making. A computerized adaptive test is one kind of expert system, though not commonly recognized as such. A new approach, termed EXSPRT, was devised that combines uncertain inference in expert systems with sequential probability ratio test stopping rules. Two versions of EXSPRT were developed, one with ra...

Education is viewed more broadly than schools only. An educational system consists of four basic kinds of components: teachers, students, content and contexts. Six types of diadic relationships exist among these four components, as well as those between an educational system and its environment. To restructure an educational system is to change one...

Three extant methods of adapting the length of computer-based mastery tests are described and compared: 1) the sequential probability ratio test (SPRT), 2) Bayesian use of the beta distribution, and 3) adaptive mastery testing based on item response theory (IRT). The utility of the SPRT has been empirically demonstrated by Frick [1]. Research has a...

One of the potential advantages of computer-based instruction (CBI) is individualization of instruction. However, this goal has not been fully realized in practice, due largely to limitations of natural language understanding and to combinatorial explosion. It is nonetheless possible to develop CBI programs which can adapt to students, depending on...

Expert systems can be used to aid decisionmaking. A computerized adaptive test is one kind of expert system, although not commonly recognized as such. A new approach, termed EXSPRT, was devised that combines expert systems reasoning and sequential probability ratio test stopping rules. Two versions of EXSPRT were developed, one with random selectio...

Analysis of patterns in time (APT) is a method for gathering information about observable phenomena such thatprobabilities of temporal patterns of events can be estimated empirically. If appropriate sampling strategies are employed, temporal patterns can be predicted from APT results. As an example of the fruitfulness of APT, it was discovered in a...

Expert systems can be used to aid decision making. A computerized adaptive test (CAT) is one kind of expert system, although it is not commonly recognized as such. A new approach, termed EXSPRT, was devised that combines expert systems reasoning and sequential probability ratio test stopping rules. EXSPRT-R uses random selection of test items, wher...

The sequential probability ratio test (SPRT), developed by Abraham Wald, is one statistical model available for making mastery decisions during computer-based criterion referenced tests. The predictive validity of the SPRT was empirically investigated with two different and relatively large item pools with heterogeneous item parameters. Graduate st...

Computers are being used increasingly to educate handicapped children, yet despite their instructional potential they are being used relatively infrequently to train preservice teachers. Recently faculty and staff at Indiana University have developed a Special Teacher Education and Evaluation Laboratory (STEEL) to use technology as an integral part...

Observer disagreement is important insofar as it limits the reliabilities of observational records. This discussion evolves around methods and conditions under which observer agreement can be measured as to minimize such an occurrence. Observers should be trained to nearly perfect agreement with a criterion or expert coder on unambiguous examples o...

How can one determine efficiently if an informational website or an e-learning product is working well? Relatively small numbers of the target audience are needed to improve a product during formative evaluation and usability testing as part of product development and revision cycles. However, during summative evaluation, how many subjects are need...

Blended instruction is becoming more commonplace in higher education. Students not only attend classes, meeting face-to-face with each other and their instructors, but they also can communicate electronically outside of class meetings using course management tools such as WebCT, BlackBoard, Angel, and the like. There has been a considerable amount...

Abstract Traditional usability testing uses a probability model to determine the number of subjects needed, but this model poses considerable limitations. The purpose of this study is to offer an alternative approach employing the Sequential Probability Ratio Test (SPRT). Fifty-onepeople representing a campus community participated in a usability t...

Map & Analyze Patterns & Structures Across Time (MAPSAT) is a new set of mapping tools that are appropriate for analyses of system dynamics and structure in education. MAPSAT consists of two complementary methodologies: Analysis of Patterns in Time (APT) and Analysis of Patterns in Configuration (APC). In APT, a researcher creates temporal maps by...

The No Child Left Behind Act of 2001 is likely to fuel the need for systemic change in education. I propose to develop a prototype of a Web-based simulation, called SimEd, to help teachers, students, parents, administrators and school boards to: 1) understand the nature of systemic change in education; and 2) know what to look for in terms of conse...

In today's political climate there is considerable talk about improving education by getting rid of poor teachers as a means of improving quality of our schools. The method of determining effective and ineffective teachers that is being considered seriously in U.S. states and by the federal government is called a value-­‐ added model (VAM) 1 . The...

This article describes the application of instructional design theories and game design principles in the creation of a paper prototype for a simulation game on technology integration in education (SimTIE-Math). The study employs a design and development research design in which artifacts that resulted from that design process are examined, includi...