Comparative Analysis of Online Learning versus Classroom Learning

Abstract

Compares the learning outcomes of an Internet-delivered course to the educational outcomes of the same course delivered in the classroom. The higher level of cognitive learning achieved by the online group was attributed primarily to the self-selected nature of the online students. Also associated with the higher learning outcomes were the instructional design of the online course that applied adult motivation and learning theories. (AEF)

Full text

CHAPTER ELEVEN
A COMPARATIVE ANALYSIS OF PRE–LICENSING
INSURANCE ONLINE LEARNING WITH
TRADITIONAL CLASSROOM LEARNING
Terrence R. Redding & Jack Rotzien
The information age and the knowledge explosion are driving an
increasing demand for education. The advent of the World Wide
Web and ease of access to the Internet permits the Internet to be
used as a delivery mode for distance education courses. However,
studies which compare the results of educational content delivered
over the Internet to that delivered in the classroom, have been
inconclusive. Many report no significant difference (Russell,
1999).
Easing educational content delivery also introduces problems
associated with instructional design to optimize learning online.
One criticism of Internet based courses is that they are poorly
constructed being little more then speaker notes and reference
materials published on the Internet. One report that reviewed
hundreds of these various writings published over the last decade,
finds that the overall quality of the research is questionable and
thereby renders many of the findings inconclusive (The Institute
for Higher Education Policy, 1999). The report cautions that
policymakers and education leaders have “a lot to learn” about
how distance education can enhance learning.

A Comparative Analysis
2
PURPOSE
The purpose of this study is to compare the educational outcomes
of an Internet (online) delivered course to the educational
outcomes of the same course delivered in the classroom. The
course was designed by OnLine Training Institute (OLT).
A wide array of instructional methodologies, delivery modes,
and strategies have been evaluated. Herbert Walberg and his
colleagues reviewed over 3,000 studies and summarized the effect
of these methods on student performance (Borg & Meredith,
1989). A subset of their findings, those which may be associated
with the instructional design used at OLT, are presented in Table
11.1.
The first column reports the interventions/strategies that were
tested. The second column is the effect size (ES). The ES statistic
is a quantitative way to describe how the average student who
received a strategy performed as compared to an average student
who did not receive the strategy. Researchers consider effect sizes
larger than .33 to have practical significance. A negative effect size
means the students receiving the strategy did not perform as well
as the average student who did not receive the strategy (Borg &
Meredith, 1989). An effect size of zero would mean that there were
no significant differences between the performance of the average
student receiving a strategy and students who did not receive the
strategy. The third column presents percentile equivalents to aid in
interpreting the ES. For example, the percentile of 88 for
reinforcement means that the average student after receiving the
intervention of reinforcement did as well as on an achievement
measure as a non-reinforced student who scored at the 88th
percentile.
Effect size will be used in this study to compare the online
students’ grade point average to the other non-online student
groups grade point average.
Table 11.1. Instructional Strategy Effects on Student
Learning Outcomes

A Comparative Analysis
3
Effect
Method
Size
Percentile
1.
Reinforcement
1.17
88
2.
Cues and feedback
.97
84
3.
Graded homework
.79
79
4.
Cooperative learning
.76
78
5.
Class morale
.60
73
6.
Personalized instruction
.57
72
7.
Home interventions
.50
69
8.
Adaptive instruction
.45
67
9.
Tutoring
.40
66
10.
Instructional time
.38
65
11.
Home environment
.37
64
12.
Higher-order questions
.34
63
13.
Individualized instruction
.32
63
14.
Individualized mathematics
.32
63
15.
Teacher expectations
.28
61
16.
Assigned homework
.28
61
17.
Computer-assisted instruction
.24
59
18.
Peer group
.24
59
19.
Sequenced lessons
.24
59
20.
Advanced organizers
.23
59
21.
Homogeneous groups
.10
54
22.
Class size
.09
54
23.
Programmed instruction
-.03
49
Source: Data from Herbert Walberg, “Improving the
Productivity of America’s Schools,” Educational
Leadership, 41, no. 8 (1984): 24. (Borg & Meredith, 1989)
THE ONLINE COURSE—SELF–DIRECTED LEARNING DESIGN
One Internet based educational institute bases its instructional
design for online delivery on notions associated with self-directed
learning, adult learning and motivation theory, and Bloom’s
Taxonomy (Bloom & Krathwohl, 1956). The instructional design
focuses on knowledge, comprehension, adult motivation as
described by Redding (1995), (Redding, Rugolo, & Eisenman,

A Comparative Analysis
4
1999), (Redding, Caudell & Lucius, 1999) and Spear and Mocker
(1984). Recognizing self-directed learners may desire to control
when and where they study, the online course selected for study is
designed as an asynchronous guided learning course delivered in a
virtual classroom which includes automated e-mail to the
instructor, and online conferencing between the instructor and
students.
The course is designed to prepare individuals who are seeking
to become professional insurance agents in the area of life, health,
and annuities. The study site is the state of Florida, which tightly
regulates insurance agent preparation and licensing. Individuals
seeking to become life, health, and annuities agents must complete
a mandatory 40 hour pre-licensing course. The course must be
approved by the state, be taught by an approved instructor, be
offered by an approved educational provider, and be taught from a
state mandated study manual. Before an individual is permitted to
take the state insurance agent examination, they must first
complete the mandatory course, and pass an end of course
examination.
The courses being compared share the following attributes:
• Same author.
• Same instructor.
• Same course outline.
• Same printed study manual.
• Same end of course examination.
• End of course examination administered under the same test conditions.
The courses being compared had the following differences.
• Different educational grouping. Grouped by market.
• Different instructional design. Classroom—lecture vs. SDL online.
• Different delivery mode. Classroom vs. Internet.
The course author and instructor is Jack Rotzien, CLU. Rotzien
is the Director of Insurance Training at OnLine Training Institute.

A Comparative Analysis
5
He has been directly involved in designing and delivering
insurance training, pre-licensing courses, and continuing
professional education courses in Florida for the past 24 years. He
is a founding member of OnLine Training, Inc. His commitment to
students and his strong desire to aid them in being successful has
been a hallmark of all his instructional endeavors. He is the past
founder and president of Insurance Systems of Florida, which
certified over 18,000 pre-licensing students before the school was
sold in 1991. Rotzien entered the insurance business with “The
John Hancock” in 1975. He later became a unit manager and
recruiter. During this period Rotzien started a Life and Health
licensing school under the sponsorship of the Palm Beach Chapter
of CLU, ChFC. Rotzien still actively teaches pre-licensing courses
for Palm Beach Community College. He has been active in the
Palm Beach Association of Life Underwriters. He is a former
officer and board member of the Palm Coast Association of Health
Underwriters. He is the past President of the Palm Beach County
Chapter of Society of Financial Service Professionals.
The online instructional design was crafted by OnLine Training
Institute as specified above. Additionally, the course is limited by
design to place emphasis on knowledge acquisition and
comprehension in cognizance with Bloom’s Taxonomy (Bloom
and Krathwohl, 1956). Special emphasis is placed on
reinforcement, clues, feedback, and reducing specific identifiable
barriers to learning. Of interest is the programmed instruction
aspect of the online course. Table 11.1 indicates the ES can be
negative for programmed instruction. This is offset by the other
elements of the instructional design, ready access to the instructor
through automated e-mail, scheduled online chat, FAX, and 1-800
phone access as required. The tone of the course is written in an
active voice with the intended audience envisioned as an adult
who, through current circumstances, must learn the material in
order to gain access to a new career field.
STUDY POPULATION

A Comparative Analysis
6
Pre-licensing training for the life insurance industry can be thought
of as four separate markets. These markets are “private” in house
classes, “commercial” classes run statewide, “community college”
classes, and the emerging “online” classes. The researchers used
these markets to separate the study population into comparative
groups from which samples were drawn. The selection of these
groups permit a comparison of the Internet based course group to
each of the other market groups.
The private classes are normally conducted by commercial
schools on a negotiated contract basis. These classes are arranged
around the work schedule of employees. They will tend to be held
in banks or brokerage houses. The student’s capacities are similar
making the instructing job simpler. The employer pays for the
class, and students are often compensated while attending.
The commercial schools are run by a few large statewide
companies that conduct open enrollment classes on a regular
schedule . The schools have a very diverse student base, both in
age and education. The classroom situation is hard for the
instructor to balance. Most of these students are changing careers
who tend to be well motivated. The schools are nearly all
conducted on 4 day, 2 weekend schedules.
The Community College markets: The weekend class student
base is the same as the commercial schools. The colleges are the
only outlet for evening, open enrollment, classes that tend to have
a majority of female agency office personnel usually married, who
want to avoid weekend classes and have the least motivation to
become licensed. They also attract people who are employed on
weekends. They usually are held from 6–10 p.m. for 10 sessions.
The OnLine Classes attract students with widely varied
backgrounds. The computer literate person is usually more affluent
than average. The typical student is a high activity type, not
wanting to give up weekends, or have inconvenient hours or
location. This group also contains the student who does not want to
be tied down to a schedule, but may also have undisciplined study
habits. Some will lack the discipline to regiment their time of study
or length of study. The online student population includes students

A Comparative Analysis
7
who might normally have selected any one of the other groups
with which to study.
Most students have accolades for the methods used online as it
is more focused, requiring more continuous concentration than
classroom attendance, while also permitting the student to choose
when, where, and for how long they study. Of the total number
who enrolled in the online pre-licensing course, only one chose to
transfer to Community College—classroom group, based on a
reluctance to study online. One must interact with the computer,
which is unforgiving in its diligence to keep students focused on
the content being presented. Therefore, the online course appeals
to those who prefer to make steady progress in their studies, and
who do not enjoy sitting and listening to material, not directly
applicable to their study goals, or for extended periods. The online
course relies upon its internal instructional design, individual self-
motivation, and not a regimen for the discipline needed to learn.
During the final four hours of each of the insurance pre-
licensing courses being compared, three end-of-course proctored
examinations are administered. Each of the examinations,
annuities, life, and health are designed to measure the level of
comprehensive knowledge acquired by the student. The online
students are required to come in to the classroom and complete the
end of course examinations under the same conditions as the
classroom students. A grade point average is computed for each
examination. Students are required to achieve a cumulative score
of 70 percentile before they are permitted to sit for the state
licensing examination.
METHODOLOGY AND FINDINGS
The data collected was reduced to descriptive statistics, as depicted
in the following four tables, and then further reduced to
comparative effect size, depicted in the final table, in order to
facilitate drawing inference.

A Comparative Analysis
8
The insurance pre-licensing course covers three discrete topical
areas, annuities, life, and health. Each is tested separately
representing the knowledge required to effectively conduct a line
of business in a specific area. While most students obtain a license
in all three areas, some students don’t. This explains why there is a
variation in the N in the descriptive statistics of each group.
The effect size is computed using the cumulative Average
Mean of each comparative group.
Table 11.2. Private Class Financial Institution Managers–Classroom
Annuity
Life
Health
Average
Mean
63.53
92.13
84.63
78.35
Mode
60
96
86
#N/A
Median
60
92
86
79.33
SD
20.49
4.81
10.09
13.17
Skew
-0.05
0.15
-0.82
-1.04
Min
28
84
66
44
Max
96
100
96
97.33
Var
419.76
23.18
101.71
173.45
N
17
16
16
17
Table 11.3. Commercial Schools–Classroom
Annuity
Life
Health
Average
Mean
84.67
79.01
84.40
81.62
Mode
100
76
92
92.67
Median
88
80
88
83
SD
13.60
13.78
11.45
10.33
Skew
-1.31
-0.62
-1.08
-0.58
Min
41
42
50
57.33
Max
100
100
100
98
Var
184.86
189.90
131.08
106.70
N
45
73
70
73
Table 11.4. Community Colleges–Classroom
Annuity
Life
Health
Average

A Comparative Analysis
9
Mean
81.52
84.29
83.80
82.80
Mode
96
96
92
#N/A
Median
84
90
88
85.67
SD
15.92
12.71
12.32
12.91
Skew
-0.70
-0.47
-1.05
-0.61
Min
48
62
54
58
Max
100
100
100
98.67
Var
253.37
161.51
151.75
166.70
N
21
21
20
21
Table 11.5. Community Colleges–OnLine
Annuity
Life
Health
Average
Mean
92.27
92.08
92.11
92.37
Mode
100
98
92
94.66
Median
95.5
96
94
94.66
SD
8.50
9.35
9.16
8.02
Skew
-1.37
-1.91
-1.67
-1.71
Min
68
64
68
70
Max
100
100
100
100
Var
68.21
78.81
72.93
56.90
N
22
23
18
23
Table 11.6. Effect Size–OnLine Compared to Other Groups
Groups compared
Effect Size
OnLine vs. Private
1.36
OnLine vs. Commercial
1.04
OnLine vs. Community College–Classroom
.93
Source: Student records and data collected by Rotzien (1999) to
support this research.
The Mean was reported by topic, Annuity, Life, Health, and
also as a cumulative Average Mean for each group. The higher the
Mean, the better the instructional outcome. The standard deviation
(SD) provides an indication of instructional quality and
consistency of cognitive learning. The smaller the SD, the more
consistent the instructional outcome. The Min and Max provide an

A Comparative Analysis
10
indication of range which is a reflection on the homogeneity of the
group by topic. The Variance (Var) is an indication, like SD, of
the instructional quality and consistency. The N reflects the
number of students in each sample, by topic, for which we have
descriptive statistics. The N under Average, indicates the number
of Mean scores used to compute the cumulative Average Mean
GPA score for a particular group.
DISCUSSION
Distance education via the Internet is becoming increasingly
common. A wide array of instructional formats are used online.
This study seeks to compare the outcomes associated with an
instructional design specifically developed by OnLine Training
Institute (OLT) based on notions associated with self-directed
learning, adult learning theory and adult motivation. OLT’s course
is an adaptation of an existing classroom based pre-licensing
course so that it can be taught online via the Internet. This study
compares the results of the online course to the results of the in the
classroom course. The results were measured using end of course
examinations producing a grade point average (GPA) for each
topic taught. Groups being compared were taught by the same
instructor, using the same course syllabus, study manual, and
evaluated using the same end of course examination for each topic
taught. Online the students interact directly with the instructional
content, paragraph by paragraph, section by section, and chapter by
chapter. In the classroom the instructional method was primarily
lecture with some group interaction between the instructor and the
students.
The first group, Private Classes, achieved the lowest over all
GPA. To some degree this is a reflection on their lack of interest in
the annuity market and possibly the insurance field. Their primary
business is banking or investments, and they offer other investment
products. Generally, they are individuals who have secure
positions, and are engaged in a pre-insurance licensing course

A Comparative Analysis
11
primarily because they have been directed to do so, in order to be
able to sell certain forms of life and accident insurance. The
insurance field is not a core component of their industry. Their
effort to learn this material lacks the same level of motivation that
characterizes someone who is making a career change, and who’s
income will be derived primarily from insurance sales. Thus, their
lower GPA is understandable. It is of interest to note their Life SD
is 4.81, compared to 20.49 and 10.09 for Annuity and Health. The
lower standard deviation for Life reflects their focus, and increased
emphasis by students and the instructor in the area of Life.
Typically, these classes have homogeneous student populations
and are easier to instruct from the instructor’s point of view.
The second group, Commercial Schools, includes for profit
training companies that conduct pre-licensing insurance courses
statewide. Classes are tailored to specific student groups as
required. This tailoring is reflected in the lower N for Annuity
because not all students who go through the commercial schools
require certification for that line of business. Students who enroll
with commercial schools range from out of work individuals
seeking to enter the new career field of insurance, to individuals
who are fully employed in the financial industry. The wide range
of scores (Min and Max) reflect this variance in the sample.
The third group, Community Colleges, conducts pre-licensing
courses less frequently. Typically they offer courses during the
week one month, and then on weekends the following month. The
result is they have two very different groups going through their
training and thus their student population is less homogeneous. The
evening classes suffer from fatigue because the students are trying
to learn material after a full day of work. The students in the
evening class are typically fully employed, however. The weekend
classes have students that are fresh and ready to learn, but often
unemployed.
The fourth group, Community College–online, is a self-
selected group. At the time of this report, this option had only been
available through one community college for three month. Three
classes of two, six, and 15 students constitute the online group. It

A Comparative Analysis
12
can also be thought of as a crossover group because students who
would typically have enrolled with one of the other three groups,
chose to enroll in the online group once they became aware of its
availability. The online group typically achieved the highest GPA
Mean in each topic, and the highest cumulative GPA Average
Mean. The SD and Variance for this group indicate a consistent
high quality learning of content occurs. The online group is the
most successful at cognitive learning as measured by the end of
course examinations.
These findings are somewhat surprising given the diverse
nature of the students: literally representing new entries to the
field, administrative personnel, bankers, brokers, and other
individuals presently involved in the financial industry. Also, the
instructional design and course delivery in the classroom has
matured over an extended period of time (12 years). One should
expect them to be optimal. While the online course is relatively
new, less than one year, having been revised and improved just a
few times since having been placed online.
Effect Size
Often tests of statistical significance are inappropriate for making
inference about the practical significance of research. One
approach is to calculate effect size (ES) as an effective means of
performing statistical power analysis. As mentioned above, ES
above .33 is considered useful. Using the data displayed in Table
11.1, and considering the ES depicted in Table 11.6 one could
conclude that an average student in the Commercial group would
score about 84% in the online class. The improvement for the
average student from the Private group could be even better.
Student Comments
Students queried concerning why they choose to take the pre-
licensing course online uniformly reported it satisfied their desire
to control when and where they accessed the course. Some
indicated it addressed their need to be home with children (single
parents), others indicated it addressed their concern about physical

A Comparative Analysis
13
access in that they are handicapped, while still others expressed
concern about physical safety citing that the online course meant
that they did not have to venture out at night nor walk late at night
across an empty parking lot to reach their car.
Instructor Comments
Online students are more confident in their ability to learn the
material required on their own. Given the nature of the online
course vs. the in the classroom experience the instructor came to
believe online is more precise, time efficient, and effective then the
traditional in the classroom experience. The instructor also felt that
online instruction is not for everyone. Many simply would not be
able to learn online and should be in a classroom. The greatest
advantage of the online course is that the student controls the pace
of the course.
OnLine Training Institute Comments
OLT is able to measure the number of times and amount of time a
students spends online using the course–or dwells on a single
question or page. Because of the interactive nature of the online
course, the instructional group can know what is being typed,
clicked, or selected each time a student makes a choice and moves
through the course. Online we can know more about the student
choices and cognitive thought processes then the instructor in the
classroom can know. This information is reviewed monthly in an
effort to improve the instructional design.
The instructor in the classroom can be sure of little more than
the student is present–unless the instructor actively interacts with
each student. When the instructor poses a question–only one
student can respond. However, when the instructional design team
poses a question online–we know that each student will respond
before they move forward through the course. Properly designed,
using a student centered approach, online training can be an
effective way of ensuring cognitive learning occurs.
This particular online course has served as a proof of concept
model for OLT’s instructional design.

A Comparative Analysis
14
Discussion—Distributive Level of Effort Model
The online pre-licensing course took literally years to design,
revise and release. The course has only been available for a few
months online. But, it produces consistent cognitive learning
outcomes–well above the 12 year old in-the-classroom model.
The 12 year old in the classroom instruction should be near
perfection–and is taught by a true sage. The sage, who is also the
author and instructor of record for the online course, reports that
his in the classroom course benefited greatly from the detailed
effort he made to construct the online course. Each word, phrase,
and fact was carefully considered as it was selected for inclusion in
the course. The sequence elements were taught, laying a firm
foundation of knowledge and then comprehension, was considered
and then tested and then revised as the design for the course was
finalized.
A similar effort for classroom instructional design might well
have a similar impact on all instruction offered in the classroom.
The difference in cognitive outcomes may well be explained as
much by the level of effort and collective attention of the
instructional design team as it is by the self-selected nature of the
students.
From a distributed level of effort model, shown below, it can
be argued that a sage on the stage, working with 10 students, must
distribute his effort across 10 students. Thus the instructor to
student ratio is 1:10. The student can be thought of as receiving 10
percent of the instructor’s time.
A five person design team (mostly women, by the way),
however, focuses their effort on the single student who sits at the
computer working through the online course. They have to design
for his every potential choice (again - the online students are in fact
mostly women). Thus the instructor to student ratio is 5:1. In the
online model the student gets a comparable 500% of an instructor’s
time. The instructor’s time and responses are also structured,
formal, and never ad-hoc.

A Comparative Analysis
15
From an instructional design point of view - one could say that
the online instructional design has an advantage over the classroom
instruction with an aspect ratio of 50:1.
1:10 5:1
50:1
Figure 11.1. The distributive level of effort model shows the relationship between the
traditional instructional design effort of one instructor to many students,
and the online model of a design team to one student.
Distributive Level of Effort Model
Traditional
OnLine
Student
Instructor
One to Many
Many to One

A Comparative Analysis
16
This is part of the reason the online course is so successful.
CONCLUSION/RECOMMENDATIONS
The online course is an effective way of conducting pre-licensing
insurance training. The implication is that the instructional design
employed by OLT is effective for designing Internet based courses
delivered to adult students seeking to acquire the knowledge and
comprehension associated with licensure. The higher GPA of the
online students indicates online instruction was more effective then
classroom instruction. However, given the self-selected nature of
the online sample, more research is needed before definitive
conclusions can be made. Other limitations of the study include:
the small sample size for the online group, and an inability to
control for reading level, work/home environment, ethnicity, age,
or educational background of any of the participants.
The results of the study do provide support for the conclusion
that online instruction for individuals entering the insurance field
can be effective, and can be more effective then traditional
classroom delivered instruction for those students confident in their
ability to learn online.
Recommendations
Based on the results of this study, the following recommendations
are made:
• Additional studies should be conducted – following a more robust experimental
design in which students are randomly assigned to either the online or the classroom
pre-licensing course.
• This study should be replicated comparing other courses, developed using the OLT
instructional design methodology for other fields of employment which require
licensure, to determine if this approach to online instruction is transferable to other
professions or disciplines.
REFERENCES

A Comparative Analysis
17
Bloom, B.S., & K rathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of
educational goals, by a committee of college and university examiners. Handbook I: Cognitive
Domain. New York, Longmans, Green, 1956.
Borg, W.R., & Gall, M.D. (1989). Educational research: An introduction. Longman, New York and
London.
Redding, T.R. (1995). Reordering Maslow’s needs hierarchy based on self-directed learning
considerations. In H.B. Long & Associates, New dimensions in self-directed learning (pp.165-
180). Public Managers Center (Educational Leadership and Policy Studies Department):
College of Education, University of Oklahoma.
Redding, T.R., Caudell, & Lucius, S. (1999). Comparing self-directed learning to an empowerment
organizational motivation model: Self-directed learning as a way to Zapp! Employees. In H.B.
Long & Associates, Contemporary ideas and practices in self-directed learning (pp.225-238).
Public Managers Center (Educational Leadership and Policy Studies Department): College of
Education, University of Oklahoma.
Redding, T.R., Eisenman, G., & Rugolo, J.J. (1999). Training in technology for late adopters:
Learning in retirement, Computers for Seniors. In H.B. Long & Associates, Contemporary
ideas and practices in self-directed learning (pp.239-251). Public Managers Center
(Educational Leadership and Policy Studies Department): College of Education, University of
Oklahoma.
Russell, T.L. (1999), The no significant difference phenomenon. Instructional Telecommunications,
North Carolina State University.
Spear, G.E. (1988). Beyond the organizing circumstances: A search for methodology for the study of
self-directed learning. In H.B. Long & Associates (Ed.), Self-directed learning: Application
and theory (pp.199-222). Athens, GA: University of Georgia.
The Institute for Higher Education Policy (1999). What’s the difference? A review of contemporary
research on the effectiveness of distance learning in higher education. The Institute for Higher
Education Policy, Washington, DC.

A Comparative Analysis
18