Content uploaded by Starr Roxanne Hiltz
Author content
All content in this area was uploaded by Starr Roxanne Hiltz
Content may be subject to copyright.
PRODUCTIVITY ENHANCEMENT FROM
COMPUTER-MEDIATED COMMUNICATION:
A SYSTEMS CONTINGENCY APPROACH
A systems contingency approach to social impacts
of
computing predicts that
productivity enhancement will be a function
of
a complex interaction
of
social and technical systems.
STARR ROXANNE HILT2
New office technology can increase productivity, but
often this potential is not realized. This article de-
scribes the conditions under which one new type of
interactive computer system for information workers is
or is not perceived as enhancing the productivity of its
users. The study supports the assertion that availability,
use, and beneficial impacts of new computer systems
do not necessarily form a causal chain. Available sys-
tems may or may not be used, and, if they are used,
they may or may not enhance productivity of informa-
tion workers
[l,
2, 6,
21,
26, 401.
Computer-mediated communication systems
(CMCSs) use a computer to create, store, process, and
distribute text files and databases. They include elec-
tronic mail, computerized conferencing systems, and
office support systems (text processing and managerial
decision. support systems with group communication
components). Although most often studied as a mode of
communication, they are also a form of management
information system when used in the office
[19,
22, 32,
34, 391. Pilot studies indicate that CMCSs could in-
crease the productivity of “knowledge workers,” who
now compose the majority of the labor force in the
United States and other post-industrial societies [5, 14,
20, 431.
A recently completed study of new users of four
CMCSs I:171 was designed to identify the determinants
of acceptance or rejection of CMCS as a communication
mode and potential productivity-enhancing tool. It in-
It 1988 AC!I1 0001.0782/88/1200-1438 $1.50
eluded new users of three computer conferencing sys-
tems that are designed for topic-oriented group discus-
sions, and one computer-based message system. In all,
three interrelated aspects of acceptance of CMCS were
examined: subjective satisfaction, amount of use, and
perceived benefits or outcomes. This article summa-
rizes the results relating to the question of prclductivity
enhancement. The measurement of the three dimen-
sions of acceptance (satisfaction, use, and benefits) and
their interrelation is treated in more detail in [IS].
Information Worker Productivity
Management communications is the largest single cost
factor for U.S. businesses today, exceeding the costs for
direct labor and capital, and exceeding net profits by a
factor of three to five [41]. CMCSs have the potential to
reduce communication delays and costs. As Culnan
and Bair [5] emphasize: “Communication is the critical
process in organizations because organizations consist
of people who spend the majority of their time commu-
nicating.” They define organizational productivity
somewhat differently from the way we do, as “organi-
zational processes completed on schedule,” a narrower
definition which emphasizes efficiency. Using only an
efficiency criterion for productivity focuses an assess-
ment of potential benefits attributable to CMCS on
measuring time and money savings from media substi-
tution [35]. In our study efficient communication (more
speed, less cost] is only one component of prod.uctivity
enhancement. The major component is effectiveness,
the quality of decisions and output by professional and
1438 Communications of the ACM December 1988 Volume 31 Number 12
Articles
managerial personnel. Thus we agree with Strassman
[40,
p. 1471 that “information technology itself may not
save any time . . . . What counts.. is whether the deci-
sions reached are the right ones.” A crucial aspect of
improving decision quality is obtaining information
from outside the immediate organization: “It is charac-
teristic of unproductive organizational designs to foster
passing information back and forth among the internal
organizational workers” [JO, p. 271.
The subjects of this study are information workers
(managers and professionals and some supporting cleri-
cal personnel) engaged in various task groups within an
between their employing organizations. An objective
measure of whether the new technology made them
more effective in realizing the goals of their organi-
zations [7, p.
78-791
would be desirable. However,
the quality of their decisions and recommendations,
and the quality and efficiency of the intra- and inter-
organizational coordination among task groups cannot
be measured objectively across a variety of organi-
zational contexts
[lo,
381.
We therefore had to settle for
users’ reports of changes in their own behavior.
Our measures of contributions to productivity grow
out of the information science literature on research
productivity. Pelz and Andrews [31] found that fre-
quency of contact with colleagues is positively corre-
lated with performance levels. Johansen, DeGrasse, and
Wilson [ZO] developed questionnaires that measured
productivity increases by focusing on the kinds of
changes in communication patterns that Pelz and
Andrews identified as productivity-related. A prior
study
[18]
used some of these items to construct a fac-
tor score to measure productivity enhancement among
seven groups of scientific and technical workers using
EIES (the Electronic Information Exchange System) at
New Jersey Institute of Technology. The best predictors
of productivity increases were the number of people
“met” online, the total number of people with whom a
user was communicating online, and the amount of
time spent online.
But how generalizable were these results? Could
these findings be applied to other types of users and
other systems? Measures used in earlier studies of
CMCS and productivity were replicated for this
multi-system study with a variety of types of users.
The research reported here is the first to systemati-
cally employ identical measures of the same variables
in a single longitudinal study that includes a variety
of CMCS with different types of users.
THEORETICAL FRAMEWORK
Among the theoretical and empirical approaches to
studying the acceptance and diffusion of technology
and its impacts on society, three major emphases were
identified: technological determinism; individual differ-
ences; and human relations. The “package” or “systems
contingency” approach is a synthesis of these ap-
proaches. The major schools of thought and research
emphasize different sets of variables in examining the
impacts of information systems as well as different ide-
d
ological perspectives and values. We have simplified
the approaches by focusing on differences in the types
or level of variables that are expected to have the most
influence on whether or not benefits are perceived
from CMCSs.
In technological
determinism,
characteristics of the sys-
tem or technology determine user behavior. Rob Kling,
in his review of theoretical perspectives [23], identifies
the “systems rationalists” as those who tend to believe
that efficiently and effectively designed computer sys-
tems will produce efficient and effective user behavior.
Moshowitz [gg] has a parallel category, the “techni-
cists,” who “define the success or failure of particular
computer applications in terms of systems design and
implementation,” while Goodwin
[ll]
speaks in terms
of “functionality and usability” as the determinants
of system use and effectiveness. Technological and
rational economic determinants at the system level
include the functions of a particular CMCS, the charac-
teristics of the interface, and the cost in time and
money of using the new system compared to its alterna-
tives. To the extent that the assumptions of technologi-
cal determinism are correct, the particular system used
would account for much of the variance in the experi-
ences users have online and in their perception of pro-
ductivity impacts.
The
individual differences
or “people determined”
[27]
approach to predicting how people will respond when
confronted with a new technology emphasizes charac-
teristics of the individual such as attitudes and attri-
butes [including “personality type”), expectations, be-
liefs, skills, and capabilities [47]. Attitudes consist of an
affective dimension involving emotions (“Computers
are fun”) and a cognitive dimension based on beliefs
(“Using this system will increase my efficiency”). In
some investigations, attitudes have been shown to
predict behavior
[24],
whereas in others, there seem
to be attitude-behavior inconsistencies (e.g., [ST]). The
strength of an attitude-behavior relationship seems to
increase when specific attitudes are correlated with
specific behaviors, as compared to general attitudes cor-
related with specific behaviors [12]. In this study, pre-
use expectations about the specific system were ex-
pected to correlate more strongly with subsequent use
of and reactions to that system than were attitudes and
beliefs about computers in general.
The human relations approach “focuses primarily on
organizational members as individuals working within
a group setting” [33]. The small groups an individual is
part of are seen as the most powerful determinants of
behavior. From this perspective, characteristics of the
group and its task will be the most important determi-
nants of the success of an information system imple-
mentation. Participation in the decision to use CMCSs,
user training and support, the nature of existing ties
among group members, and the style of leadership or
group management (electronic or otherwise) are seen as
crucial determinants of the acceptance and impacts of a
new communications technology.
We expected all of the sets of variables to account for
December 1988 Volume 31 Number 12 Communications
of
the ACM 1439
Articles
variations in perceived productivity impacts of CMCS,
but their effect is not additive; rather, we expect effects
of the technology to be contingent upon the implemen-
tation context. Thus, the theoretical approach we took
can be equated with what Kling [23] calls the “package”
approach to the social impacts of computing. Markus
[pi’] speaks of “interaction theory” in terms of the inter-
action of system and context of use, emphasizing inter-
action with the political structure of the whole organi-
zation, whereas our study was limited to interaction
with the individual and group level variables within
the implementation context.
In
s,ystems contingency fheory, there is
no
such thing as “one best way” to design
CMCSs because “What’s best
for an
orgmization, or a subunit within if, is a
highly complex function
of
many
variables.”
Another name for this approach is “systems contin-
gency” theory, which is based on General Systems the-
ory [3, 9, 25, 301. Productivity impacts are hypothesized
to be contingent upon the characteristics of the higher-
level systems within which the technology is used (Fig-
ure
1).
Each subsystem within a larger system can be
analyzed as a unit of behavior in its own right, or as a
subunit of behavior interacting with other subunits,
The behavior of one subunit depends on its environ-
mental relationship to other units or subunits that have
some control over the consequences it desires
(421.
In
this stu.dy, the human-computer system is composed at
the lowest level of an individual who has certain skills
and attitudes, and a CMCS with particular attributes
related to functionality and interface. This system may
be used to communicate among the members of a
group, which usually has a specific task or application
for the system and that has or develops a social struc-
i
l-
FIGURE 1. Systems Contingency Approach: Human-Computer
Interaction is Nested within Larger Action Systems
1440
Communications
of
the ACM
ture with the emergence of one or more “leaders.” Each
user group is in turn nested within a larger organi-
zational context. Since the groups in this study were
located within a variety of organizations, the organi-
zational climate within which the groups were func-
tioning varied.
In systems contingency theory, there is no such thing
as “one best way” to design organizational communica-
tion systems in general or CMCSs in particular, because
“what’s best for an organization, or a subunit within it,
is a highly complex function of many variables” [8]. In
terms of the relative power of technological versus so-
cial determinants, social context (characteris-tics of the
individuals, the user groups, and the larger organi-
zational and social structure in which the technology
is embedded) might be a more powerful predictor of
acceptance than characteristics of the systems them-
selves [13].’
METHOD
Plan of the Study
During the course of the study of scientific and techni-
cal users of EIES [IQ]. the findings of 36 published stud-
ies of CMCS were synthesized. This inventory of vari-
ables, measures, and hypotheses was reorganized to
reflect our three theoretical orientations and used to
develop the variables, measures, and hypotheses for
this study [16, 221. Whenever possible, the actual items
used to measure variables were replicated from pre-
vious studies.
To adequately test causal assertions, at least two sets
of data had to be collected. A baseline questionnaire
measured independent variables that describe individ-
ual attributes and expectations, and preexisting rela-
tionships with members of online groups. Dependent
variables were measured approximately four months
later during the follow-up questionnaire, which in-
cluded questions about technological barriers and expe-
riences with the system that may have negatively af-
fected acceptance during the preceding four months.
A summary of the variables used in our questionnaires
and initial hypotheses appears in Appendix A.
The Sample
The data for this study include pre-use and follow-
up questionnaires plus system usage time for :new users
of four CMCS. which differed in terms of functionality,
interface, and user populations (see Table I). Pre-use
December 1988 Volume 31 Number
72
Articles
TABLE I. Characteristics of the Systems and their Users
A. SELECTED SYSTEM FEATURES
FEATURE
Conferencing
Messages
Directory
Integrated text editor
Survey subsystem
Overall complexity
Linear Linear Branching None
yes yes no yes
yes yes no no
yes yes no yes
yes no yes no
High Medium Medium Low
6. CHARACTERISTICS OF USERS AT PRE-USE, BY SYSTEM
(Percentages of Responses)
VARIABLE=
N Responses
Computer experience (novices)
Used CMC before
Group member
Identify leader
Motive: Just curious
Incentive: Required
Paying self
Home terminal
Communications frequency c 3 mos
Female
Age: 40+
Grad degree
Employed in business
Position: Exec or manager
How important (1 or 2)
Task importance (1 or 2)
Expect useful (1 or 2)
Expected time (1 + hrs/wk)
Note: All differences significant at least at 0.05 level
‘See Appendix A for a description of each variable.
208 84 71 150
35 8 9 13
22 32 40 21
77 47 14 46
95 68 78 62
23 44 69 35
12 6 1 18
7 7 63 2
56 30 88 10
60 31 31 6
17 20 9 27
60 16 24 34
62 45 46 18
46 30 54 96
57 22 40 80
48 29 23 50
21 51 19 29
34 36 24 41
71 29 52 33
questionnaires were distributed and some data ob-
tained for
348
new users of EIES and for
234
new users
of the Swedish COM system at its “QZ” (Stockholm
University) installation. The COM questionnaires were
translated into Swedish for those respondents not fluent
in English. Both are not-for-profit, academic-based com-
puterized conferencing systems that include linear con-
ferencing structures and message systems. Both are also
fairly small in terms of total membership, with under
2,000 total users at the time of the study. Directories
include descriptions of members and conferences avail-
able for communication. COM’s interface, however, is
quite different from that of EIES. EIES also includes
many more specialized subsystems, such as systems for
generating and disseminating reports, online surveys,
and databases.
In addition, 197 users of a commercial, publicly
accessible conferencing system (“PUBLICON”) and
156
users of a commercial electronic mail system
(“INTMAIL”), both in the United States, have been in-
cluded. INTMAIL includes the “standard” kind of in-
basket and out-basket mail handling facilities of most
commercial message systems. It has no conferencing
capabilities, and the entire “user manual” is a folder,
compared to the much more voluminous documenta-
tion for the conferencing systems. Both of these systems
operate on networks with tens of thousands of users.
There are no overall directories included in the sys terns.
Follow-up questionnaires were mailed to each par-
ticipant after four months of use. Items measuring
productivity-related variables were included only on
a “long” version of the follow-up. For EIES and COM.
we had data on cumulative time online before sending
the follow-ups. Those identified as “dropouts” (less than
four hours total online) were sent only a short follow-
up with a checklist of reasons for their limited use.
Response rates for the follow-up varied from a low of
40
percent for COM to a high of
56
percent for EIES.
(See [15] for a more detailed discussion of sampling and
response rate problems.)
Because of different sample sizes and selection meth-
ods, and differences in response rates, the combined
sample was not evenly divided (see Table II). About
one-third of the total all-systems sample, and almost
one-half of the long follow-ups (rather than one-fourth)
are from EIES users, a point that should be remembered
when looking at the “all systems” data. For correlation
coefficients based on one measure from the pre-use and
one from the follow-up, the number of responses
for specific systems other than EIES is often too small
December 1988 Volume 31 Number 12 Communications of the ACM 1441
Articles
TABLE II. Significant Differences in Characteristics of Users, by System (Percentage of Responses)
Number of Responses
Computer Experience:
(novices) (%)
Ever Used
Group Member (%)
Leader
Motive: Just Curious
Incentive: Required
Paying for Self
Home Terminal
Communications Frequency c 3 mos
Fema.le
Age: +40
Education:
Gralduate Degree
Business
Position-Executive or Manager
How IImportant (1 or 2)
Task Importance (1 or 2)
Expected Time (1+ hrs/wk)
Expectations (Mean)
208
35 8
9
13
22 32 40 21
77 47 14 46
95 68 78 62
23 44 69 35
12 6 1 18
7 7 63 2
56 30 88 10
60 31 31 6
17 20 9 27
60 16 24 34
62 45 46 18
46 30 54 96
57 22 40 80
48 29 23 50
21 51 19 29
71 29 52 33
4.9 4.7 4.2 5.1
a4 71 150
Key To Variables: See Appendix A for wording of most items. The following are not
included in Appendix A: Ever Used = Ever used a CMCS
before? Leader = Does this group have an official leader, manager, or moderator?
to provide adequate statistical power,’ so we may
have failed to observe some relationships for COM,
PUBLICON, and INTMAIL that would have produced
significant results if the samples had been larger.
Differences among Users of the Four Systems
Differences among the four systems go beyond software
(see Table I). A fundamental difference is that all of the
users of INTMAIL worked for the same organization;
the acronym “INTMAIL” was chosen to indicate that
their CMCS was being used as an internal mail system.
Users of the other three CMCSs were scattered among
many different organizations, and were employed pri-
marily for inter- rather than intra-organizational com-
munication. INTMAIL users were most likely to have
felt “required” to use the system as a condition of their
employment. Since they were using INTMAIL to sup-
port their everyday internal corporate communications,
they reported the highest importance ratings for com-
munication.
A second basic difference is cultural. Most of the
COM users were Swedish, while most of the users of
the other systems were American.
Another important difference is whether or not users
joined the system as members of a specific task group.
Only on EIES were there a number of task-oriented
groups on the same system. Among those in groups,
--
ZTheoretically. responses could have been weighted so that each system
would contribute one-fourth of the variance on the “all systems” data. How-
ever. the proportion of responses by system varies greatly for different items.
dependin:: upon whether one is looking at sterns chosen from the pre-use.
short follow-up. or long follow-up: as well as whether the items deal wth
questions on the group or task. which were answered only by respondents
who belonged lo a specific group and had a specific task to accomplish. Thus.
it is not possible to assign a single weighting factor for respondents in this
study.
95 percent of EIES respondents could identify a group
leader or moderator, but only
66
to 75 percent of group
members on other systems could do so.
PUBLICON users are distinguished by the fact that
very few belonged to a task-oriented group; on the con-
trary, most wandered onto the system because they
were “just curious” and were probably using the system
for entertainment or exploration. Unlike most of the
users of other systems, they were paying for their on-
line time themselves. This system was included be-
cause it is one of the most widely used commercially
available conferencing systems currently on lthe mar-
ket; copies of the software have been installed in many
organizations.
Given this diversity of user populations and applica-
tions, a variable must be extremely strong to overcome
the other factors and differences and thereby produce
consistent effects across the four systems. When find-
ings are inconsistent for different systems, we cannot
determine which of the many differences in software
and user population is responsible. However, when
findings are consistent across the systems, we have
strong evidence for the generalizability of the finding.
Measuring and Identifying Productivity Impact
Factors
The long version of the post-use questionnaire included
a number of questions designed to measure impacts of
system use. A principal components factor analysis em-
ploying SPSSX varimax rotation was used to Ireduce
these items to a smaller number of underlying dimen-
sions. Table III displays the items used to measure the
dependent variable and their correlations with the two
underlying factors identified. To establish the reliability
of the factor structure across systems, the analysis was
1442 Communications
of
the ACM December 1988 Volume 31 Number I2
Articles
TABLE III. Productivity Impact Factors
Factor Names and Loadinss
Quantity2 L 0 85 0.24
Quality2 0.82 0.34
Useful2 0.81 0.33
Reach2 0.64 0.31
Short-term 0.27 0.80
Long-term 0.25 0.91
Leads 0.47 0.61
Stock of ideas 0.33 0.52
Variance accounted for 60.9 13.8
Cumulative variance 60.9 74.7
Eioenvalue 4.87 1.11
KEY: The variables correspond to semantic-differential and Lickert-
type items on the follow-up questionnaire.
Quantity2: System increased quantity of work (1 = definitely yes,
7 = definitely not. Mean = 4.5, SD = 1.8)
Ouality2: System increased quality of work (1 = definitely yes,
7 = definitely not. Mean = 4.5, SD = 1.8)
Usefula: “How useful have you found the system to be for your
work?” (1 = very useful, 7 = not useful at all. Mean = 4.0,
SD = 1.9)
ReachP: Easier to reach people (1 = strongly agree, 5 = strongly
disagree. Mean = 2.7, SD = 1.2)
Short-term: Contributed to short-term career advancement (1 =
strongly agree, 5 = strongly disagree. Mean = 3.3, SD = 1.2)
Long-term: Contributed to long-term career advancement (1 =
strongly agree, 5 = strongly disagree. Mean = 3.1, SD = 1.2)
Leads: Provided leads. references, other useful information (1 =
strongly agree, 5 = strongly disagree. Mean = 2.7, SD = 1.3)
Stock of ideas: Increased stock of ideas (1 = strongly agree,
5 = strongly disagree. Mean = 2.6, SD = 1 .l)
replicated separately for EIES (the system with the larg-
est number of responses) and the other three systems
combined. The factor structure was the same, and the
coefficients were nearly identical when the sample was
split in this manner. (See [IS] for the table of results of
the replicated factor analysis.) The factor scores for
each individual served as the dependent variables in
this analysis.
Productivity is composed primarily of items relating
to whether system use increased the efficiency, the
quality, the usefulness and the ease of communicating
with people. Obtaining information or ideas shows a
somewhat weaker correlation with the productivity
factor. High scores indicate a lack of perceived produc-
tivity improvements of this nature.
Career is related most strongly to the items pertain-
ing to long-term and short-term contributions to career
advancement. Also highly correlated is whether the
system provided leads or other useful information or
increased the “stock of ideas.” High scores indicate a
perception that the system did not have any of these
desirable impacts on the professional career. From
here, we will focus mainly on productivity as a depen-
dent variable, since it is more important for organi-
zational impacts than are advantages leading to individ-
ual career advancement.
Measuring the Independent Variables
Since a number of the independent variables are multi-
dimensional, several questions were used to measure
these variables, and factor analysis was used to reduce
the set of items to a smaller number of underlying di-
mensions. An appendix available from the author in-
cludes the details of the composition of the variables
simply listed as “factors” in Appendix A.
General attitudes toward computers were measured
using ten semantic differential scales, selected from 37
items originally devised by Zoltan [48]. The three gen-
eral attitudinal factors identified include feelings about
computers being dull and dreary to use, unreliable and
inefficient, and difficult and demanding.
High scores on the factor media frustration (commu-
nications frustration) indicate disagreement with state-
ments about the telephone, meetings, and mails being a
waste of time. These items were newly devised for this
study.
The questions used to construct an index of pre-use
expectations about prospective productivity-related
outcomes of system use are included in Appendix A
and the resulting scores, by system, are shown in
Table IV. Expectations are moderately positive, on the
average. INTMAIL users had the highest expectations,
while PUBLICON users had the lowest expectations of
productivity enhancement.
In measuring attitudes toward the group, competi-
tiveness clearly separated out as a different dimension
from four other pre-use questions about the group, that
dealt with liking and trusting the members. These
items and the items on leadership skill (which formed a
single factor; see Appendix B) were also newly devised
for this study.
The 14 items measuring subjective satisfaction with
the CMCS system were used in previous studies [14,
451. Interface, satisfaction with the system interface,
focuses on items indicating that dialogue with the sys-
tem is understandable, easy to learn, and courteous.
The performance factor has to do with feeling that one
is saving time and being productive while actually
US-
ing the system. Unexpressive emphasizes feeling un-
able to express one’s views and unable to get an
impression of others. Mode problems concentrates on
items about feeling distracted by the mechanics of the
system, constrained by it, and overloaded with informa-
tion. (See Appendix B for these items and the factor
loadings.)
RESULTS
Means and standard deviations for the items constitut-
ing the productivity factor are included in Table III. For
TABLE IV. The Expectations Index, by System
“* _
.” ANALYSIS OF VARIANCE
n *,;z:; ‘, *
~&;stern ’ - Yeti. ‘. Standard
Devistion
EIES 4.9 1.0
COM 4.7 1.1
PUBLICON 4.2 1.2
INTMAIL 5.1 1.2
F
=
14.06;
p = 0.000; Index Mean = 4.7; Std Dev = 1.2.
Note: See Appendix A for items included in the expectations index.
December 1988 Volume 31 Number 12 Communications
of
the ACM
1443
Articles
most items, answers are slightly skewed toward posi-
tive responses, but a sizable proportion of users failed to
perceive productivity enhancements of various types.
For example, on the item asking about increased quan-
tity of work completed, 29 percent agreed, 26 percent
were unsure, and 45 percent disagreed. Thus, there is
cons:iderable variance in the extent to which productiv-
ity enhancements are perceived as a result of CMCS
use. ‘The productivity-related outcomes that are most
widely reported are increases in one’s “stock of ideas”
and leads to useful information, and in the ease of
reaching people. It should be noted that the items used
to measure productivity increases at post-use contain
three questions that were repeated from the pre-use
expe’ctations index, but five of the questions are differ-
ent and more specific.
System Differences
Table V shows some statistically significant differences
among the four systems in the extent to which users
report benefits. When no other variables are controlled,
PUBLICON clearly is perceived as the least productive,
and INTMAIL as the most productive. By contrast, indi-
vidual career advancement is perceived to have been
most likely as a result of EIES use and least likely for
INTMAIL. Although these differences are statistically
significant, they do not account for a large proportion of
the variance (eta squared). We will return to the rela-
tive importance of differences among systems after first
1ooki:ng at the other correlates of the productivity fac-
tor, which will be the focus for most of the subsequent
analyses in this paper.
TABLE V. Perceived Productivity-Related Impacts
Mean Factor Scores by System
Analysis of Variance
EIES 0.01 -0.22
COM -0.23 -0.05
PUBLICON
0.31 0.14
INTMAIL
-0.42 0.35
F 9.4 6.7
P 0.001 0.001
Eta 0.28 0.24
Eta
Sq
0.08 0.06
Factor descriptions:
Prod~crive-productivity impacts of system use, including increases in
quantity and quality of work. High scores are NEGATIVE; users disa-
gree Ithat these impacts have occurred. See Table III for question
wording.
Career-High scores indicate that the system is NOT perceived as
contributing to long-term or short-term career advancement.
Correlates of the Productivity Factor
Pearson correlation coefficients were computed for the
relationship between productivity and each indepen-
dent variable included in this study. To facilitate com-
prehension of the pattern of the mass of coefficients,
only those significant at least at the
0.10
level are in-
cluded in the display of results in Table VI, giving it a
graph:ical element: looking at it can indicate at a glance
where the correlations are clustered. To further sim-
plify the presentation of results, the number of re-
sponses is not shown in this table. (It can be approxi-
mated for any cell from the response information in
Table II.)
Expectations predict subsequent evaluations, creating
a “self-fulfilling prophecy” [28]. The strongest correlates
of perceived productivity improvements after four
months of system use are expectations about these
same variables at pre-use. Motivation for using the sys-
tem, in the form of having a specific project rather than
just being curious about how such systems work, is also
related for all four systems. As predicted, these correla-
tions with expectations about the specific system are
much stronger than the correlations for general atti-
tudes toward computers.
A second set of correlates relate to the group and
task: the importance of communicating with those on-
line, the importance of the online task, liking the group,
frustration with alternative communication modes, and
leadership skills. The relationship between leadership
skills and perceived productivity improvements held
for all systems, though it was not statistically significant
for two of them.
A third strongly related set of variables has to do
with the online communications: number of communi-
cation partners (no one), the strength of ties that had
emerged (get know and friends2), and the perceived
worth of the actual communications received. The cor-
relations for friends2, the number of people online at
four months who are considered personal friends at
that time, replicates as statistically significant for all
systems except QZCOM.
The coefficients for the subjective satisfac:tion factors
demand special examination. An extremely high corre-
lation between the performance and the productivity
factor suggests that they are redundant measures of
very similar concepts. Thus, performance will be ex-
cluded as a candidate variable in subsequen.t regression
analyses.
Satisfaction with the system interface (interface) and
feelings that it is difficult to express emotional and so-
cial communications via this medium (unexpressive)
are significantly but not strongly correlated with per-
ceived productivity enhancement. Of the subjective
satisfaction factors, the
most
strongly correlated mea-
sure that is not redundant is mode problems, percep-
tions of problems with this mode of communication,
such as feeling distracted by the mechanics, con-
strained in the types of communications one can en-
gage in, and overloaded with information. As would be
expected, the higher the perception of such mode prob-
lems, the lower the perception of productivity enhance-
ment. However, even the correlation for mode prob-
lems is moderate
(0.21
overall), and is not replicated
independently for three of the four systems.
It is also worth noting other variables that are not
very strongly related to perceived productiv.ity impacts,
even though one might expect them to be. Total hours
online is positively related, but the correlation is a rela-
tively weak 0.17 for the combined sample, and is not
1444 Communications of the ACM December 1988 Volume 31 Number 12
TABLE VI. Correlates of the Productivity Factor
Attitude Factors
Dull
Unreliable
Difficult
Media Frustration
Expectations about System
Expected Time
Motive
Expect Easy
Expect Friendly
Not Frustrating
Expect Time Saving
Expect Productive
Expect Efficiency
Expect Quality
Expect Useful
Expectations
How Enter
Incentive
Individual Characteristics
Computer Experience
Sex
Age
Education
Position
Typing
Hours Work
Group Variables
Group Member
Group Name
Size of Group
Number Known
Friends
Communication Frequency
Like Group
Compete
How Satisfied
How Important
Leadership Variables
Leadership
Leader Skill
Task Characteristics
Task Importance
Enjoy Task
Experiences with System
Terminal Access
Documentation
Cost to Reach
Cost to Use
Complicated
Phone Problems
Packet Problems
Bad Experiences
Not Like
No One
No Interest
Use and Satisfaction
Time4
Interface
Performance
Unexpressive
Mode Problems
Number Others
Friends2
Get Know
0.24
0.21
l
*0.32 “0.20 “0.39 “0.37
“-0.16
l
*-0.35
‘-0.10
-0.09
“-0.20
“-0.33
“-0.33
“0.47
“0.49
l
*0.51
“*-0.54
“‘-0.15
‘-0.36 “-0.42
-0.30
l
*-0.42 -0.24
l
-0.35
‘-0.27
l
-0.28
-0.25
“-0.49
l
*-0.46
l
*0.74
“0.68
“0.79
“-0.76
-0.15
-0.13
l
-0.22
l
*-0.31
“-0.40
l
*0.34
l
*0.53
l
*0.51
“-0.52
l
-0.32
“0.66
l =0.49
l
*0.61
l
*-0.58
‘0.39
l
-0.18
l -0.39
““0.40
“0.29
“-0.36
l
0.18 “-0.42
l
-0.31 ‘0.18 ‘0.26
l
-0.22 *0.28
l
-0.10
l
-0.10
l
*-0.28
“-0.43
“-0.49 -0.15 0.25
l
*-0.13
“-0.16
“-0.20
“-0.17
‘0.16
l
*-0.26
“-0.30 ‘0.36
“0.64
l
=0.56
l
*0.22
0.13
“0.25
l
*0.37
l
*0.33 *0.48
0.40
“0.39
0.13
‘0.50
-0.47
l
*0.70
l
*0.65
l
*0.32
l
*0.12
‘-0.12 ‘-0.18
“0.33
l
*0.33
“0.37
l
*0.23
l
*0.19 “0.29
l
*0.56
-0.24
“-0.32
l
*0.36 “0.61
l
0.17
l
*0.47
l
*0.25
l
0.12
l
0.1 1
l
*-0.20
‘-0.11
**-0.14 ‘-0.15
l
*-0.17 “-0.29
l
*-0.27
l
*-0.33
“-0.18 *-0.18
l
0.16
“*-0.24
l
-0.21
“-0.37
-0.16
**-0.53
l
-0.14
l
“-0.26
l
*-0.17
“-0.17
l
*-0.58
‘0.10
“-0.21
l
*-0.14
l
*-0.23
l
*-0.20
l
-0.25
“-0.63
-0.25
**-0.14
‘“-0.69
**-0.55
**-0.48
l
*0.21
l
*-0.20
“-0.22
l
‘-0.23
l
*-0.43
l
*-0.55
0.14
-0.14
l
*-0.32
“-0.23
l
*-0.45
-0.23
Correlations listed only if p < .lO
* Significant at .05
‘* Significant at .Ol
See Appendix A for explanation of variable names
TABLE VII. Results of Simultaneous Regression Equations for Blocks of Variables (Beta Coefficients for Variables Significant at .O!i)
s\E&&#bl+ ’ ’ / ’ : j
18 Individual Characteristics
df
Attitude Facbors
Dull
Unreliable
Difficult
Media Frustration
Expectations about System
Expected Time
Motive
Easy
Friendly
Not Frustrating
Expectations Index
How Enter
Incentive
Characteristics
Computer IExperience
Sex
Age
Education
Position
Typing
Hours Work
Multiple R
Adjusted R2
13 Group and Task Variables
df
Size of Group
Number Known
Friends
Communirzation Freq.
Like Group
Compete
How Satisfied
How Important
Leader Skill
Leadership
Task Impor?ance
Enjoy Task
Multiple R
Adjusted R2
17 Experiences On-line
df
Terminal Access
Documentation
Phone Problems
Packet Problems
Cost to Reach
Cost to Use
Complicated
Bad Experiences
Not Like
No One
No Interest
Friends 2
Get Know
Interface
Unexpressive
Mode Problems
Time 4
System = EIES
System = COM
System = lntmail
Multiple R
Adjusted h:’
18,232 18,79 13,9 18, 65 18,17
-0.19
-0.39 0.05
-0.29
-0.29
-0.45 -0.29
0.60 0.48
0.92
0.31 0.05 0.63
13,63 13,49
0.33
0.54 0.55
0.15 0.12
17,300 17,99
0.23
-0:17
-0.20 -0.20
-0.66
0.17
-0.35
0.22
-0.22
0.49
17,lO 17,73 17,38
0.62 -0.41
0.47
-0.40
-0.71
-0.26
0.82
0.58
-0.88
-0.29
0.92
0.67
-0.15 -0.33
-0.28
-0.18 -0.26 0.44
-0.33
-0.44
0.52 0.62 0.84 0.55 0.64
0.25 0.27 0.22 0.14 0.15
Articles
significant for all four individual systems. Neither pre-
vious experience with computers nor typing skills has
any significant relationship with eventual productivity
impacts.
Multiple Regression
Multiple regression enables us to determine the best
combination of variables for predicting whether or not
CMCSs are perceived as contributing to productivity
enhancement. When the variables are entered in sets
according to the levels laid out in the theoretical frame-
work for this study, multiple regression also enables us
to test for the relative size of effect of sets of related
variables,
individual and system experiences levels, were used. In
both cases, expectations are by far the strongest predic-
tor. The second strongest predictor is the distinction
between the mail system and the conferencing systems.
When the leadership skill factor is allowed to enter the
equation, (not shown in Table VIII), it also makes a
significant improvement. Thus, variables at the individ-
ual, group, and
system
level all play a part in determin-
ing perceptions of productivity increases from CMCSs.
TABLE VIII. Final Regression Equations for Productivity,
All Systems
A. Simultaneous Equation Without Leader Skill (df = 8,253)
However, regression procedures must be chosen and
interpreted with care when there are so many inde-
pendent variables. The results of a simultaneous regres-
sion with all 48 variables included would not be very
meaningful or useful. Our procedure is to first look at
equations for the three sets of variables, and then to use
the results to select candidate variables for more parsi-
monious simultaneous and stepwise equations. (A simi-
lar procedure is followed in Weber’s exploratory analy-
sis of the impacts of computer technology on jobs
[46].)
The advantage of the stepwise procedure using
trimmed sets of predictors is that it may give us better
insight into both the relative importance of predictors,
and the decomposition of technical “system” differ-
ences from differences among systems in user charac-
teristics.
Variable
Expectations”**
System = Intmail”’
System = Corn”’
Time 4”
No One**
Not Like”
Regression
Coefficient
-0.34
-0.70
-0.46
-0.00
-0.21
-0.22
Standardized
Regression
Coefficient
-0.42
-0.28
-0.21
-0.16
-0.16
-0.10
Get
Know -0.01 -0.07
System = EIES -0.14 -0.07
Constant 3.12
Multiple R = 0.64, Adjusted R = 0.40, F = 22.4,
p =
0.001
6. Stepwise Equation (Ail Systems) With Leader Skill (df = 4,101)
In these multiple regressions, “system” was treated in
a variety of ways. Separate analyses were conducted for
all systems individually (most of which are not in-
cluded here due to space limitations); some analyses
compared the three conferencing systems to the mail
system: system was also treated as a series of “dummy
variables” in the all-systems combined data.
Step Variable
Standardized
Regression Regression
Coefficient Coefficient
T Value
Most of the individual variable coefficients are not
statistically significant when all variables, divided into
three subsets, are used. Table VII displays those that
are, for the four systems combined and for each indi-
vidual system. Results are not shown for the group-
level variables for COM, PUBLICON, and INTMAIL be-
cause there are so few cases with group-level data that
the number of variables approached the number of de-
grees of freedom. However, the strongest group-level
variables for each system were noted, and are included
in some subsequent analyses. Comparing the adjusted
proportion of variance accounted for by the three equa-
tions, the individual-level predictors do the best job for
all systems except EIES. Most of this predictive power
is due to the strong correlations with the pre-use expec-
tations index.
1 Expectations’“* -0.35 -0.44 -5.4
2 Leader Skill’ 0.21 0.21
2.6
3
No One’ -0.24 -0.18 2.3
4 System = Intmail’ -0.42 -0.17 2.1
Constant 2.33 6.2
Step 4, R = 0.63, Adjusted R2 = 0.39, F = 16.3,
p =
0.001
* Significant at 0.05
** Significant at 0.01
*** Significant at 0.001
The second version of the equation displayed in Ta-
ble VIII uses stepwise regression and includes leader-
ship skill. When the variance due to differences in pre-
use expectations among users is first removed, both the
second and third strongest predictors are group or so-
cial-level variables. However, there is still a significant
difference between the mail system and other
systems.
Optimal Equations for Conferencing Systems versus the Mail
System
Table VIII shows the results of successive trimming of We have seen that the internal mail (INTMAIL) system
equations to eliminate variables which were not signifi- is different than the three conferencing systems, on the
cantly related to productivity, and whose removal did basis of qualitative differences in functionality, charac-
not substantially decrease explained variance. Two ver- teristics of its users, and results of multiple regression
sions of simultaneous equations, one with leadership equations which include “system” as a set of dummy
skill (for which data are missing in many cases), and variables. Our final analysis attempts to select the best
one which included significant variables only at the set of predictors for reported productivity enhance-
December 7988 Volume 31 Number 12 Communications of the ACM 1447
Arficles
ments for the three conferencing systems (combined), The importance of communication with other group
versus the mail system. The procedure was identical to members and the importance of the task also appear in
that of the final equations displayed in Table VIII: Suc- the trimmed equation for the mail system, but not for
cessive elimination of variables which were not signifi- the conferencing systems. There is also an intriguing
cantl;y related to productivity at the 0.05 level, and negative relationship for the mail system between the
whose removal did not substantially reduce the total number of other persons online who were friends be-
explained variance. fore system use began and productivity enhancement.
Initial expectations are a very strong predictor for This suggests that groups of friends within the same
both classes of systems (Table IX). However, the other organization who use electronic mail may employ it for
variables in the final equations differ. Leadership skill social versus task- or productivity-oriented communi-
is the second strongest predictor for the group-oriented cation. However, this relationship did not appear as
conferencing systems, and total time online at the end significant in the bivariate analysis for the mail system.
of four months also appears in the equation. It seems to emerge only when other variables such as
TABLE IX. Efficient Regression Equations for Productivity-Three Conferencing Systems vs. the Mail System
..’ Standardized
., ,
.,‘:, .Regression Regression Significance
of T
A. Three Conferencing Systems Combined (df = 5, 95)
Expectations -0.29
Leader Skill 0.19
No one -0.26
Time 4 -0.005
Mode Problems -0.19
(Constant) 2.11
6. Mail System (df = 6, 14)b
Competitive 0.94
Expectations -0.44
Hew Important -0.29
Hard -0.25
Friends 0.07
Task Importance 0.18
(Constant) 3.4
a R = 0.62, Adjusted RZ = 0.35, F = 11.7, p = 0.000
o R = 0.97. Adjusted R2 = 0.93, F = 42.6, p = 0.000
-0.36 -4.3 0.000
0.21 2.4 0.02
-0.19 -2.3 0.03
-0.17 -2.0 0.05
-0.16 -1.9 0.06
5.4 0.02
0.84 9.0 0.000
-0.57 -7.1 0.000
-0.57 -5.2 0.000
-0.38 -5.2 0.000
0.36 4.9 0.000
0.33 4.2 0.000
6.1 0.000
Results for the mail system must be interpreted with
caution because of the small number of cases. Cer-
tainly, the large proportion of variance explained for
the mail system is at least partially an artifact of small
sample size and probably would not be replicated for a
much iarger sample of mail system users. However, it is
notable that certain variables are relatively strong pre-
dictors for the mail system, but not for the three confer-
encing systems and vice versa. It is evident that confer-
encing and mail systems differ in their modes of use
and value.
The strongest predictor for the mail system is the
relationship between perception of a high degree of
compei.itiveness within the online group, and a lack of
perceived productivity enhancements. Evidently, per-
sonal competitiveness may play a strongly inhibiting
role when all users are members of the same organiza-
tion. But it is not as likely to be as important in inhibit-
ing inter-organizational use of and benefits from CMCS.
If the members of a group within the same organization
feel that they are competing against one another for
promotions or other rewards. then they probably will
not use a communications technology to cooperate with
and possibly improve the productivity of their “rivals.”
expectations and competitiveness are taken into ac-
count. Finally, the total time spent online during the
four months, between the baseline and follow-up sur-
veys, does not appear in the final equation for the mail
system, whereas it does appear for the conferencing
systems.
Productivity Enhancement by Time Online, by System
We were surprised not to find a stronger correlation
between the number of hours logged online and re-
ported productivity enhancement. Table X explores
this relationship for each of the three systems for which
we had sufficient data and begins to explain -the rela-
tively low Pearson correlation coefficients.
The relationship between time online and reported
productivity increases is not linear, but is more like a J
curve. For the two conferencing systems, only those
who spent considerable time online
(50
hours or more)
are likely to report substantial productivity e:nhance-
ment. When measured by eta, a correlation coefficient
which is suitable for nonlinear relationships. the corre-
lations are larger and more significant. If we had con-
tinued the study of conferencing system users for a
longer period of time. such as the
18
to
24
months used
1440 Communications
of
the ACM December 1988 Volume 31 Number 12
for the previous study [14], we would have had a much
larger number of users logging hundreds of hours, and
probably would have observed a larger linear correla-
tion coefficient.
For the INTMAIL system, by contrast, productivity
enhancement is reported even at low levels of use, but
perceived productivity enhancement does not signifi-
cantly increase for the mail system with higher levels
of use.
of reaching people. There is a great deal of variation
among individuals, groups, and systems in whether or
not productivity benefits are perceived.
Non-findings are almost as important as the observed
strong correlations in understanding the nature of de-
terminants of perceived productivity improvements.
Time online is only moderately related, and productiv-
ity is not strongly related to subjective satisfaction,
except for the largely redundant system performance
factor. For instance, it is not very strongly related to
satisfaction with the interface of a system. Productivity
improvements, thus, are clearly quite distinct from
other dimensions of acceptance of CMCSs.
The strongest correlates of productivity improve-
ments for all four systems are pre-use expectations
about whether the system would increase productivity.
Other strong determinants relate to the group context:
leadership skill is important, and strong competitive
feelings may hamper productivity.
Four process variables play a role in determining pos-
itive productivity outcomes. One is the perceived value
of the items contributed by the other group members.
Another is time spent online, which is positively re-
lated to perceived productivity impacts for the confer-
encing systems (but not the mail system). A third is
whether or not “mode problems” were encountered,
and the fourth, is how many new people users got to
know while online.
System software differences do appear to make a sig-
nificant impact on whether or not there will be produc-
tivity increases, but system enters a stepwise regression
equation in only the fourth place. However, since sys-
tem software differences were confounded with differ-
ences in the group and organizational contexts in this
study, it is impossible to isolate the effects of software
design.
The best equations for predicting productivity in-
creases are markedly different for the conferencing sys-
tems versus the mail system. This appears to be the
main impact of software differences: given four basi-
cally well designed but quite different CMCSs, the so-
cial context and software differences will interact to
affect the most productive applications of the system.
For instance, in the internal mail system, those who
were competing with other online group members were
not as likely to perceive productivity improvements
from its use. On the other hand, lack of competitive-
ness in the user groups does not show up in the equa-
tions for the three conferencing systems. Similarly,
even low level use of the internal mail system tended
to be seen as productivity enhancing, but conferencing
system users were less likely to perceive payoffs at low
levels of use. This may be due to the limited function-
ality and greater simplicity of the mail system. Greater
ease of use of the simpler system, combined with the
“mail” metaphor, which suggests substitution for other
forms of written communication, may enable users to
experience benefits more quickly. While initially easier
to understand, the mail metaphor may also limit users
in what they will attempt to accomplish in using the
system
[4].
Conferencing and mail systems differ in
their modes
of
use and value.
These findings support the point of view that the
different functionality of mail systems versus confer-
encing systems means that they will be used for differ-
ent kinds of tasks, and that their relationship to produc-
tivity enhancement will occur through different kinds
of efficiency versus effectiveness payoffs. Since the
electronic messages are used to replace telephone or
mail to individuals, payoffs in time saved can be per-
ceived even at low levels of use. Conferencing systems
that support complex, long-term group discussions, on
the other hand, seem to demand substantial invest-
ments of user time before significant enhancements to
group productivity are perceived.
SUMMARY
From the systems rationalist perspective, productivity
increases are the ultimate payoff of CMCSs for groups,
organizations, and society as a whole. The dependent
variable of primary interest in this study is a factor
(productivity) composed mainly of perceived improve-
ments in the quantity and quality of work, the overall
usefulness of the system, and improvements in the ease
TABLE X. Level of Use by PRODUCTIVE Factor, by SYSTEM
Analysis of Variance
LEVEL ALL EIES PUBLICON INTMAIL
l-3 Hours
Mean
N 0.20
38 0.77 -0.32
14 16
4-9 Hours
Mean
N
1 O-49 Hours
Mean
N
50+ Hours
Mean
N
ANOVA:
0.05 0.23 0.35 -0.26
94 30 28 26
0.08 0.10 0.22 -0.53
143 83 46 7
-0.48 -0.64 -0.18 -0.57
36 21 9 3
4.3
5.9
2.4 0.19
0.005 0.001 0.07 0.90
0.20 0.35 0.27 0.11
0.04 0.12 0.07 0.01
P
ETA
ETA SQ
December 1988 Volume 31 Number 12 Communications of the ACM 1449
Articles
With the more complex conferencing systems, there organizational use. Ideally, applications for which ex-
tends to be a longer learning period during which users ternal validations of subjectively perceived enhance-
disco,ver the potential value and applications of ments in productivity could be obtained could be
a var:iety of features that can be useful for supporting selected. One or more control groups that do not
long-term collaborative group tasks [19]. The lack of use a CMCS in the interim could be used to separate
correlation between use and perceived benefits for effects of system use from the effects of repeated mea-
the internal mail system may also be related to the sures of similar constructs. Finally, though regular
fact t:hat its users were most likely to report that sys- use versus “dropout” behavior could be established in
tem use was “required” of them by their employer: the four month time frame used in this study, it is
they would continue to use it even if they did not be- recommended that data collection focused on pro-
lieve that it was more effective than alternative modes ductivity impacts continue for at least six months
of communication. after system implementation.
Many questions are unanswered by this study. If ex-
pectations are the best predictor of perceived outcomes
months later, what factors influence expectations? (See
[36]
for an exploration of this question.) Also, even
though most of the questions used to measure pre-use
expectations about the systems and post-use percep-
tions of productivity increases were different, might the
high correlation be a methodological artifact of the sta-
bility of attitudes, rather than evidence that those
who (are most receptive to CMCSs use them most
effect.ively?
A field experiment is recommended as the next logi-
cal step in exploring the determinants of productivity
impacts of CMCSs. Pre-use attitudes that appear to
strongly predict subsequent acceptance could be used
to screen potential users and groups and to make sure
that different experimental groups are comparable at
the outset. Hopefully, the selection of groups and appli-
cations could avoid confounding the type of system
(mail versus conferencing) with intra- versus inter-
We began with a theoretical orientation that pre-
dicted relatively strong roles for social determinants of
the impacts of CMCSs (on both the individual and
group level), and a relatively weak role for attributes of
the hardware and software that make up the computer
system itself. Whether or not CMCS use results in pro-
ductivity enhancement was hypothesized to be contin-
gent upon characteristics of the users, chara.cteristics of
the groups and tasks for which system use was applied,
and characteristics of system software and system ac-
cess. Our findings support this point of view. Interac-
tive computer systems are not just technological inno-
vations, a clear “cause” that will have predictable
“effects” in all cases. They are implemented. in a social
context, and form part of a socio-technical system. Out-
comes such as increased productivity are potentials
that may or may not be realized for specific organiza-
tions or applications of the technology, and which are
unlikely to occur if new users do not have positive
expectations and attitudes.
APPENDIX A
INDEPENDENT VARIABLES: NAMES AND CORRESPONDING QUESTIONS
The following list of variables includes a brief notation indicating the predicted relationship to measures of acceptance of CMCS,
including perceptions of enhanced productivity. For example:
Atfitudes toward computers: A “+” indicates a hypothesized positive relationship for this set of variables. For instance,
ipeople
with
negative attitudes toward computers in general at pre-use were expected to use the system less and be less likely to report
beneficial outcomes such as productivity enhancement.
Experience using computers: A “0” indicates that no significant relationship was predicted.
Attitudes toward alternative media: A “-” indicates a hypothesized negative relationship; the less satisfied users are with
preexisting communication modes such as telephones and face-to-face meetings, the more likely they are to use, like, and benefit
from CMCSs.
For nominal-level measures, asterisks indicate the category for Which productivity enhancement was expected to be most likely
to occur. If no symbol follows a type of variable, no prediction has been made, either because of conflicting findings or a lack of
evidence from previous studies.
These directional hypotheses for single variables are simplistic and can be misleading. Our primary interest is in the larger sets
of variables relating to experiences with the system, group and individual characteristics, and the way outcomes are contingent
upon combined effects of these sets of variables.
-
1. CHARACTERISTICS OF INDIVIDUALS (Measured at Baseline)
AlTlTlJDlNAL VARIABLES
Attitudes toward computers (+)
Dull factor
Unreliable factor
Difficult factor
Attitudes toward alternative media (-)
Media frustration factor
1450 Communications
of
the ACM December 1988 Volume 31 Number 12
Articles
Expectations about system (+)
Expected time
Motive
Expect easy
Expect friendly
Not frustrating
Expect time saving
Expect productive
Expect efficiency
Expect quality
Expect useful
Expectations
How enter
How much time in the average week do you foresee yourself using the system?
Which of the following best describes your motivation to use the system?
(1) I was just curious about how such systems work, and wanted to try one.
(2)
l
I intend to use it on a specific project.
Please indicate your expectations and feelings about how it will be to use this system.
(1) Hard to learn to (7) Easy to learn.
(1) Impersonal to (7) Friendly.
(1) Frustrating to (7) Not frustrating.
(1) Time Wasting to (7) Time saving.
(1) Unproductive to (7) Productive.
Do you expect that use of the system will increase the efficiency of your work (the quantity of
work that you can complete in a given time)? (l-7 scale)
(1) Definitely yes.
(4) Unsure.
(7) Definitely no.
Do you expect that use of the system will increase the quality of your work? (l-7 scale)
(1) Definitely yes.
(4) Unsure.
(7) Definitely not.
Overall, how useful do you expect the system to be for your work?
(1) Very Useful.
(7) Not useful at all.
Index of mean (time saving, productive, efficiency [reversed], quality [reversed], useful
[reversed]). Chronbach’s alpha .86.
Do you anticipate entering the material into the system YOURSELF or having someone else do
it for you?
(1) Type it myself.
(2) Have it typed.
(3) Both will occur.
(responses 2 and 3 reversed for scoring).
Pressure to Use the System
Incentive Which statement best describes your incentive for using the system?
(1) I am required to use it.
(2) I have been requested to use it.
(3) I am free to use it as I wish.
Individual Skills and Characteristics
Computer experience (0) Which of the following best describes your previous experience with computer systems?
(1) I am a novice-this will be my FIRST USE of a computer system.
(2) I have OCCASIONALLY used computer terminals and systems before.
(3) I have FREQUENTLY used computer systems.
(4) Use of computers is central to my PROFESSIONAL work.
Sex (1) Female * (2) Male
Age t-1 What is your age?
Education (0) Please indicate the amount of formal education you have completed: (9 categories ranging from
(1) grammar school or less to (9) doctorate).
Position (0)
Typing (0)
Would you classify your position as primarily:
(1) Management or administration.
(2) Senior executive.
(3) Professional or technical.
(4) Secretarial/clerical.
(5) Other support staff.
How would you describe your typing skills?
(1) None.
(2) Hunt and peck.
(3) Casual (rough draft with errors).
(4) Good (can do 25 w.p.m. error free).
(5) Excellent (can do 40 w.p.m. error free).
Hours work (+) About how many hours do you work each week, on the average?
December 1988 Volume 31 Number 12 Communications
of
the ACM 1451
Articles
Like group (+)
Complete (-)
How satisfied (-)
How important (+)
Leader skill (+)
Task Characteristics
Task importance (+)
2. GROUP AND TASK CHARACTERISTICS
Group member
Size of group (+)
Number known (+)
Friends (+)
Are you joining the system as a member of a “group”?
(1) No
(2) Yes *
How many people are in this on-line group?
Of all the people with whom you can communicate on the system, about how many do you
already know, in the sense that you have communicated or worked with them previously?
Of all the users on the system whom you know, how many do you consider to be personal
friends?
Communication
frequency (+) Before using the system. how frequently did you communicate with those in your grolJp who are
distantly located?
(1) Daily.
(2) Several times a week.
(3) About once a week.
(4) About twice a month.
(5) About once a month.
(6) About once every 3 months.
(7) Less than once every 3 months.
(8) Never.
factor
factor
How satisfied are you with these communications with distant colleagues?
(1) Very satisfied.
(4) Neutral.
(7) Very dissatisfied.
How important is it for you to communicate with people whom you expect to be on-line?
(1) Very important to (7) Not Important.
Factor (measured at follow-up-see appendix).
Compared with the other tasks that now compete for your time, how important to YOLI is this
on-line task?
Enjoy task (+) (1) Very important to (7) Very unimportant.
Compared with the other tasks that now compete for your time, how much do you enjoy working
on this on-line task?
(1) Very much.
(4) Neutral.
(7) Very little.
-
- 3. EXPERIENCES WITH THE SYSTEM (Variables Measured at Follow-Up)
Name QUESTIONNAIRE ITEM
Experiences with System (+)
Please use a check mark to indicate whether each of the following factors has been very important (l), somewhat important (2).
or not important at all (3) in limiting your use of the system.
Terminal access:
Documentation:
Cost to reach:
Cost to use:
Complicated:
Phone problems:
Packet problems:
Bad experiences:
Not like:
No one:
No interest:
We and Satisfaction
Time4 (+)
Interface (+)
Performance (+)
Inconvenient access to a terminal.
Documentation looked inadequate or difficult.
Cost of reaching the system.
Cost of using tee system.
The system is too complicated.
Trouble with telephone connection.
Trouble with packet-switched network.
Had some bad experiences (system crashed or did not seem to work correctly).
I do not like using a computer system like this.
There is no one on this system with whom I wish to communicate a great deal.
I am not very interested in the subjects being discussed.
Cumulative hours on-line at four months (monitor data).
Factor measuring satisfaction with system interface.
Factor measuring perception of system performance.
1452 Communications of the ACM December 1988 Volume 31 Number 12
Articles
Unexpressive (-)
Mode problems (+)
Group Variables
Number others (+)
Friends2 (+)
Get know (+)
Factor indicating dissatisfaction with CMCS for social-emotional communication.
Factor measuring problems encountered with CMCS as a mode of communication-higher scores
indicate fewer problems.
With approximately how many different people are you currently exchanging regular communica-
tions on the system?
Of all the users on the system whom you know, how many do you consider to be personal
friends?
How many people with whom you now communicate have you gotten to know on the system?
APPENDIX B
LEADERSHIP SKILL AND SUBJECTIVE SATISFACTION FACTORS
LEADERSHIP SKILL FACTOR LOADINGS
Task Skill 0.83
Social Skill 0.84
Overall 0.93
Authoritarian 0.03
Note: Questions are seven-point semantic differential scales:
Task Skill-“task oriented skills” (1 = excellent, 7 = poor).
Social Skill-social skills related to maintaining group cohesive-
ness (1 = excellent, 7 = poor).
Overall-overall leadership performance (1 = excellent, 7 = poor).
Authoritarian-leader is self-oriented (authoritarian), group-oriented
(egalitarian) (1 = authoritarian, 7 = egalitarian).
POST-USE SYSTEM SATISFACTION FACTORS
FACTOR NAMES AND LOADING
ITEM INTERFACE PERFORMANCE UNEXPRESSIVE MODEPROBLEMS
Overall
Stimulating2
Understandable2
Courteous
Hard2
Impersonal2
Frustrating2
Waste2
Unproductive2
Express
Impression
Distracted
Constrained
Overloaded
Variance accounted for
Cumulative variance
Eigenvalue
-0.42
-0.29
-0.77
-0.83
0.70
0.56
-
0.67
0.30
0.26
-0.06
-0.10
0.50
0.12
0.09
40.5
40.5
5.67
-0.51
-0.48
-0.25
-0.35
0.09
0.32
0.23
0.70
0.83
-0.05
-0.13
0.15
0.07
0.15
12.0
52.5
1.68
0.36
0.41
0.14
0.19
-0.03
-0.25
-0.03 0
-0.17
0.68
-
0.60
-
-0.06
-0.30
0.17
8.5
61 .O
1.19
-0.10
0.06
-0.07
-0.06
0.28
0.09
0.33
0.27
0.24
-0.04
-0.01
0.62
-
0.58
-
0.52
-
7.9
68.9
1.10
Note: Items include seven-point semantic differential scales or five-point Likert-type scales in a section headed “Reactions to the System.”
Overall: “Overall, the system is _m (1 = Extremely good, 7 = Extremely bad)
‘I find using the system to be”:
Stimulating2: 1 = Stimulating, 7 = Boring.
UnderstandableP: The language of the system (system interface) is (1 = Understandable, 7 = Confusing)
Courteous: 1 = Courteous, 7 = Unfriendly.
HardP: 1 = Hard to learn, 7 = Easy to learn.
Impersonal2: 1 = Impersonal, 7 = Friendly.
FrustratingP: 1 = Frustrating, 7 = Not frustrating.
WasteP: I = Time wasting, 7 = Time saving.
UnproductiveP: 1 = Unproductive, 7 = Productive.
Thinking back over your experiences so far with the system, how frequently have you felt:
Distracted: Distracted by the mechanics of the system (1 = Always, 5 = Never).
Constrained: Constrained in the types of contributions you could make (1 = Always, 5 = Never).
Overloaded: Overloaded with information (1 = Always, 5 = Never).
Express: Able to express your views (1 = Always, 5 = Never).
Impression: Able to get an impression of personal contact (1 = Always, 7 = Never).
December 1988 Volume 31 Number 12 Communications
of
the ACM 1453
Articles
Acknowledgments. Work on this project has been par-
tially supported by a grant from the National Science
Foundation (DCR 8121865). Elaine Kerr assisted with
the (questionnaire design and supervised the data
col-
lection for two of the four systems. Kenneth Johnson
advised and assisted with the data analysis. Murray
Turoff contributed to interpretations relating to differ-
ences in software features and their impacts.
REFE:RENCES
1. Attewell. P.. and Rule, J. Computing and organizations: What we
know and what we don’t know. Commun. ACM 27. 12 (Dec. 1984),
118441191.
2. Eeroudi. j.J., Olson, M.H.. and Ives. B. An empirical study of the
impact of user involvement on system usage and information satis-
faction. Confmu?~. ACM 29. 3 (Mar. 1986). 232-236.
3. Buckley. W.. Ed. Modern Systenls Research for the Behavioral Scientisf.
Aldine. Chicago. Ill.. 1968.
4. Carroll. JM.. and Thomas. J. Metaphor and the cognitive representa-
tion of computing systems. 1EEE Trans. Man.. Syst., Cybern. aud SMC-
12 (Mar.-Apr. 1962). 107-116.
5. Culnan. M.J.. and Bair. J.H. Human communication needs and orga-
nizational productivity: The potential impact of office automation.
\ Amer. Society for hrfo. Sci. 34. 3 (May 1983). 215-221.
6. Driscoll. j. Progress in office systems: Another viewpoint. In A.E.
Cawkell. Ed.. Handbook of Iqformafio~~ Technology and Office Sysfems.
North Holland. Amsterdam, 1986. 815-822.
7. EXzioni. A. A Contparative Aualvsis of Complex Omanizations. Free
8.
9.
10.
11
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
Press. New York.‘l961.
Farace. R.V.. Mange. P.R.. and Russell. H.M. Comnlunicating and
Orgamzing. Addison-Wesley, Reading, Mass.. 1977.
Galbraith. J. Desrgning Complex Organizatrons. Addison-Wesley. Read-
ing, Mass.. 1973.
Goodman, 0,s.. and Pennings. J, (Eds.). New Perspecftves on Organi-
zatiomzl Effectiveness. Jossey Bass. San Francisco. 1977.
Goodwin. N.C. Functionality and usability. Conrmutr. ACM 30. 3
[Mar. 1987). 229-233.
Hebelein, T.A., and Black. J.S. Attitudinal specificity and the predic-
tion of behavior in a field setting. J. Personality aud Social Psych. 33, 4
(Apr. 1976). 474-479.
Hiltz. S.R. Viewing computing systems within a social context. In
Mason. R.E.A., Ed..
Information
Processing 83: Proceedings of the IFIP
!?th World Computer Congress. Paris. (Sept. 19-23. 1983). North Hol-
land, Amsterdam. 647-655.
Hiltz. S.R. Online Communities: A Case Study of the Office of the
Futurr. Ablex. Norwood. N.J.. 1984.
Hiltz. S.R.. and Johnson, K. Measuring acceptance of computer-
lnediated communication systems. Paper presented at the lnterna-
iional Communication Association, Toronto. (May 1987). Revision
accepted for publication by I. Amer. Sot. Info. Sci. (in press).
Hiltz. S.R.. and Kerr, S.B. Studa of Computer-Mediated Communica-
!m?z Systenrs: A Synthesis
of
the Findings. Final Report to the National
Science Foundation. New Jersey Institute of Technology, 1981.
Hiltz. S.R.. Kerr, E.B.. and Johnson, K. Deternlinants of Acceptance of
IComputer-Mediated Communicatio?l Systems. Res. Rep. 22. Computer-
zed Conferencing and Communications Center. N.J.I.T.. Newark,
X.J.. 1985.
Hiltz. S.R.. and Turoff. M. The evolution of user behavior in a com-
7uterized conferencing system. Commun. ACM 24. 11 (Nov. 1981),
739-751.
l-liltz. S.R.. and Turoff. M. Structuring computer-mediated commu-
nication systems to avoid information overload. Commun. ACM 28, 7
(July 1985), 680-689.
yohansen. R.. DeCrasse. R. Jr., and Wilson, T. Group Communication
Through Computers, Vol. 5: Effects on Working Patterns. Report R-41.
institute for the Future. Menlo, Park, Calif.. 1978.
Keen. P. Information systems and organizational change. Conrnfurr.
.ACM 24, 1 (Jan. 1981). 24-33.
Kerr, E.B.. and Hiltz. S.R. Computer-Mediated Commumcdtion Systems:
Status and Evaluation. Academic Press, N.Y.. 1982.
Kling. R. Social analyses of computing: Theoretical perspectives in
recent empirical research. Comput. Sure. 12, 1 (Mar. 1980). 61-110.
Lapierre. R.T. Attitudes versus action. Social Forces. 13, 2 (1934),
230-237.
Lawrence, P.. and Lorsch, J, Organization and Environment. Harvard
Graduate School of Business Administration, Boston, 1967.
Lucas. H.C., Jr. Why Information Systems Fail. Columbia U. Press,
New York. 1975.
27. Markus. M.L. Power, politics, and MIS implementat:on. Commun.
ACM 26. 6 (June 1983). 430-444.
28. Merton, R.K. Social Theory and Social Structure, Free Press. New York.
1968.
29. Mowshowitz. A. On approaches to the study of social issues in
computing. Commun. ACM 24, 3 (Mar. 1981). 146-155.
30. Parsons. T. The Social Sustem. Free Press. New York. 1951.
31
32.
33.
34.
35.
Pelz, D.C.. and Andre&. F.M. Scientists in Organizations: Productive
Climate
for
Research and Development. lohn Wiley. New York. 1966.
Quarterman, J.S., and Hoskins. J.C. Notable computer networks.
Commun. ACM 29, 10 [Oct. 1986), 932-971.
Rice. R.E. Applying the human relations perspective to the study of
new media. Computers and Socxty 75, 4 (Winter 1988). 32-37.
Rice. R.E.. and Associates. The New Media: Communi’cation, Research,
and Techfrology. Sage. Beverly Hills. Calif.. 1984.
Rice. R.E., and Bair. J.H. New organizational media and productiv-
ity. In Rice, R.E. and Associates, The New Media. Sage, Beverly Hills,
1984.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
Rice. R.E.. and Torobin, I. “Expectations about the Impacts of Elec-
tronic Messaging: The role of media usage, task completing network
links, and organizational status.” Paper presented al. International
Communication Association 36th annual conference, Chicago, IL.
1986.
Schuman, H.. and Johnson, P. Attitudes and behavior. Review of
Sociology 2 (1976). 161-207.
S&bell. C.B. Office productivity: A miwxconomic framework for
empirical research. Office: Technology and People I, 11962). 91-106.
Steinfield. C.W. Computer-mediated communication systems. In
M.E. Williams. Ed.. Ann. Rev. Info. Sci. and Tech. 21 :1966). 167-202.
Strassman. P.A. Information Payoff: The Transformation of Work in the
Electronic Age. Macmillan. New York. 1985.
Strassman. P.A. A direct connection to productivity. Computerworld
22. OlA (Jan. 6.1988) 17-22.
Tos. H.L., and Hamner, W.C. Organizational Behauiop and Manage-
ment: A Contingency Approach. St. Clair Press, Chicago. 1974.
Uhlig. R.P.. Farber. D.J., and Bair. J.H. The Office of Ihe Future: Com-
mumcation and Computers. North Holland, Amsterdam. I 979.
Van de Ven. A.H.. and Walker, G. The dynamics of interorganiza-
tion coordination. Admin. Sci. Quarterly 29 (1964). 5!%621.
Vallee. J.. Johansen. R., Lipinski, H.. Spangler. K.. and Wilson, T.
Group Communication Through Conlputcrs: Vool. 4 SocLal, Managerial,
and Economic Issues. Rep. R-40. Institute for the Future, Menlo Park,
Calif.. 1974.
Weber. R. Computer technology and jobs: An impact assessment
model. Commun. ACM 31, 1 (Jan. 1988). 68-77.
Zmud. R.W. Individual differences and MIS sncces!:: A review of the
empirical literature. Manag. Sci. 25. 10 (Oct. 1979). 966-979.
Zolton. E. How acceptable are computers to professional persons? In
Proceedings of the Conference on Human Factors in Computer Systems.
(Gaithersburg, Md.. Mar. 15-17. 1982) ACM. New York. 1982. pp.
74-77.
CR Categories and Subject Descriptors: H.4.3 [Information Systems
Applications]: Communications Applications; K.4.3 [Computers and
Society]: Organizational Impacts
General Terms: Experimentation, Human Factors. Performance
Additional Key Words and Phrases: Computer-mediated communi-
cation
ABOUT THE AUTHOR:
STARR ROXANNE HILTZ
is a professor of Computer and
Information Science at New Jersey Institute
of Technology. She
is currently working on the development and evaluation of
computer-mediated communications software tailored to a di-
versity of objectives, including applications to higher educa-
tion. Author’s present address: Starr Roxanne Hiltz, Computer-
ized Conferencing and Communications Center, New Jersey
Institute of Technology, Newark, NJ
07702.
Permission to copy without fee all or part of this material is granted
provided that the copies are not made or distributed for direct commer-
cial advantage, the ACM copyright notice and the title of the publication
and its date appear, and notice is given that copying is by permission of
the Association for Computing Machinery. To copy otherwise. or to
republish, requires a fee and/or specific permission.
1454 Communications of the ACM December 1988 Volume 31 Number 12
