PsyScope: A new graphic interactive environment for designing psychology experiments

Full text

Behavior
Research
Methods,
instruments,
and
Computers
1993,
25
(2),
257-2
71
10.
REAL-TIME
DATA
COLLECTION
Chaired
by
William
Maki,
North
Dakota
State
University
PsyScope:
An
interactive
graphic
system
for
designing
and
controlling
experiments
in
the
psychology
laboratory
using
Macintosh computers
JONATHAN
COHEN,
BRIAN
MAcWHINNEY,
MATI’HEW
FLATF,
and
JEFFERSON
PROVOST
Carnegie
Mellon
University,
Pittsburgh,
Pennsylvania
PsyScope is
an
integrated
environment
for
designing
and
running
psychology
experiments
on
Macintosh
computers.
The
primary
goal
of
PsyScope
isto
give
both
psychology
students
andtrained
researchers
a tool
that
allows
them
to
design
experiments
without
the
need
for
programming.
PsyScope
relies
on the
interactive
graphic environment
provided
by
Macintosh
computers
to
ac
-
complish
this
goal. The
standard
components of a
psychology
experiment—groups,
blocks,
trials,
and
factors—are
all
represented graphically,
and
experiments
are
constructed
by
working
with
these
elements
in
interactive
windows
and
dialogs. In
this
article,
we
describe the
overall
or
-
ganization
of
the
program, provide
an
example
of
how
a simple
experiment
can
be
constructed
within
its
graphic
environment,
and
discuss
some
of
its
technical
features
(such as
its
under
-
lying
scripting
language,
timing
characteristics,
etc.).
PsyScope
is
available
for
noncommercial
purposes
free of
charge
and
unsupported
to
the
general
research
community. Information about
how
to
obtain
the
program
and
its
documentation
is
provided.
The laboratory microcomputer
has
become a crucial
tool for conducting
psychology
experiments. In recent
years,
a number of software programs have been
devel
-
oped on the
Macintosh
that
rely
on
its
user-friendly
graphic
interface
to assist in the
implementation
of
psy
-
chology
experiments
on
microcomputers. However, for
the most part, these have been
limited
in range
and/or
power.
For
example,
all
of
them
lack a
programming,
or scripting, language, and few
of
them
have the
capabil
-
ity
to
add external devices for more accurate
timing
of
PsyScope
was conceived
and
designed
by
Jonathan
Cohen
and Brian
MacWhinney
and
implemented
by
Matthew
Flatt and Jefferson
Provost.
Contributions to its development were also
made
by
Jay Gowdy, Eric
Sedlar, Darius Clynes,
Eric
Selberg, and Robert Findler. Support for
its development was provided
by
NIMH
grants
(MH
00673
and MH
47073)
to
thefirst
author,
an
NIH
grant (RD
17790)
to the
second author,
and a FIPSE grant
(P1
16B2)
to
both
the first and second authors. In
addition,
support was
provided
by
the laboratories
of
Elizabeth Bates,
Maggie Bruck,
Cathy
Harris,
Mark Johnson, Jay McClelland, Mark
Seidenberg,
and
Jonathan
Vaughan, who together
have
formed the
PsyScope Development Consortium.
Correspondence
should be
ad
-
dressed
via electronic
mail
to either Jonathan Cohen (jc5e
+
@andrew.
cmu.edu)or
Brian
MacWhinney
(brian+@andrew. cmu.edu)or viaU.S.
mail
to either author at the Department
of
Psychology,
Carnegie
Mel
-
lon University, Pittsburgh,
PA
15213.
input and
stimuli.
There
are powerful systems that run
on
otherplatforms.
For
example,
Walter Schneiderat
the
University
of
Pittsburgh
has
developed
the most
sophisti
-
cated
of
these—a
system
called
the
Microcomputer
Ex
-
perimental Laboratory
(MEL;
Schneider,
1988).
However,
this runs
only
on
IBM-compatible
PCs
and lacks the
ad
-
vantages of
an
easy-to-use
graphic interface.
Vaughan
(1992)
has
articulated
the
call for the development of
an
easy-to-use,
intuitive
environment in
which
psychologists—
neophytes and experts alike—can develop experiments
without the need for
sophisticated programming
skills.
We
have
developed
a
program
called PsyScope that
is
intended to address many
of
these limitations. PsyScope
was developed at
Carnegie
Mellon University
by
Jona
-
than Cohen,
Brian
MacWhinney,
Matthew
Flatt,
and
Jefferson
Provost.
The
primary
goals
of PsyScope are to
eliminate
the
need for
programming
skills
on the part of
users and to provide
them
with a
set
of tools that
cor
-
responds
directly
to the
conceptual
building
blocks
of
ex
-
perimental
design:
groups,
blocks, trials, events
(stimuli
and responses),
and
factors.
The
program
runs
on
Macin
-
tosh computers and
provides
a
graphic interface
that
al
-
lows the user to design
an
experiment
starting with the
scientific question in
mind,
rather than
being
distracted
257
Copyright
1993
Psychonomic Society,
Inc.

258
COHEN,
MACWHINNEY,
FLATT,
AND
PROVOST
by
having
to
program
it.
This
allows
students to focus
on
understanding the
principles
of
experimental
psychol
-
ogy, rather
than
the
mechanics
of
experimentconstruction
and computer
programming.
It
also
allows
experienced
researchers
to
implement
experiments
intuitively, using
tools
that correspond to the
constructs
with which they
are
familiar.
Although
PsyScope
currently
has
many
strengths,
it
would
be misleading
to
suggest
that
in
its
current
form
it
is
a complete solution to
all
problems
in experimental
psychology. The goals
of
this article are
to
provide
an
introduction
to
PsyScope—including
the
philosophy
under
-
lying its
design
and the overall organization of the
program—and to
summarize
some
of
its
strengths and
weaknesses.
The
Philosophy
Behind
PsyScope
The basic philosophy
underlying
PsyScope is
that
im
-
provements inthe methodologyof
science
inevitably
lead
to empirical and theoretical
advances.
In the case
of
experimental psychology,
the
introduction
of
micro
-
computer technologyhas
been
both aboon and a
burden.
It has been a
boon
in that it has allowed us far
greater
precision
in experimental control
and
measurement.
But
it has
become
a
burden
in
that
ithas forced many a
bud
-
ding psychologist to spend more time on
programming
than
on
experimental
psychology. We believe that
psy
-
chologists should be
spending
their time thinking and
doing
psychology
rather
than learning
how
to become
computer
programmers.
Even
those
students and
re
-
searchers
who are accomplished programmers need
to
be
able to move
quickly
between the conceptualization
of
a
new
experiment
and its
implementation.
Students
or researchers
who
are
not
expert programmers
often
stumble
on
this
hurdle.
Inthisrespect,
nonprogram
-
mer
psychologists
are much
like
chemists
without Bunsen
burners
or geologists without rock hammers or
com
-
passes.
The
goals
of
PsyScope
are to eliminate the
need
for programming skills
on
the
part
of
users
and to
pro
-
vide
them
with a
set
of
tools
that
correspond
directly
to
the
conceptual
building
blocks
of
experimental
design:
groups, blocks,
trials,
events,
and factors.
The program
runs
on
Macintoshcomputers
and
provides
a graphic
inter
-
face that
allows
the
user
to design an
experiment
starting
with the
scientific
question in mind, to
construct
a
full
factorial
elaboration
of
thisexperiment, and
to
pursue
the
articulation
oftheexperimentby
designing
theexact
stim
-
ulus
types
andthe sequence
of
events
for
each cell in the
factorial design.
The
Overall
Organization
of
PsyScope
While the goal
of
PsyScope is
to
eliminatethe
need
for
programming,
we recognize that this
goal
may not be
realistically,
or at leastnot
immediately,
attainable.
There
is always a tension betweenthe
ease
of use
of
structured
graphic
enviromnents
and
the power of ageneral-purpose
programming
language.
To
make
PsyScope
as
flexible
and
as
generally applicable
as
possible
and
to address theneeds
of more
sophisticated
users as
well
as
nonprogrammers,
we have built
PsyScope
around
a
general-purpose
script
-
ing
language.
This language, called
PsyScript,
recognizes
all
of
the
experimental design
structures
that
are
supported
in
the
graphic
environment:
groups, blocks, trials,
events,
and factors. In fact,
when
an
experiment
is constructed
using graphic
tools,
PsyScope
actually writes and stores
it as a script. Scripts
are
text
files
and
can
be
edited
directly in PsyScope
or
using any
standard
text
editor.
Furthermore,
the
graphic
and
scripting
environments are
fully interactive. That is, any
structure created
in the
graphic environment is
immediately
added
to the script,
and
anything
entered
in
the script that has a
graphic
rep
-
resentation appears immediately in
graphic
form, This
interactivity
has
two
valuable
uses:
(1)
beginning
scripters
can use this as a
learning
tool, by
observing
the effects
that
adding
or
modifying
graphic
elements has
on
the
script,
and (2)
experienced
scripters
can
take
advantage
of
the
convenience
of
thegraphic environmentto
construct
thebasic outline
of
an
experiment, eliminating
repetitive
typing,
andthen move
to
the script
for
the
“detail
work”
needed to customize the
experiment.
PsyScope
is also
interactive in the
sense
that
individual trials
can
be run
as
they
are
constructed,
singly or in
blocks,
so
that
the
user
can get immediate feedback about how the trial will
appear when the
experiment
is run. In
addition
to the
graphic
environment
and
scripting
language,
there
are a
number
of consoles
and
utilities
that
permit
the editing
of
files
(including
log and data
files)
andthat provide
con
-
trol
over
the running
of
the
experiment.
Inthe
following
section, we
provide
anexample of
how
to
build
an
experiment
using
PsyScope’s
graphic
tools.
In
later
sections, we
describe
the scripting language and
some
of the
utilities
and
technical
features of PsyScope.
A
BRIEF
EXAMPLE OF
BUILDING
AN
EXPERIMENT
IN
PSYSCOPE’S
GRAPIUC
ENVIRONMENT
When
building
anew
experiment
inthe graphic
environ
-
ment, the
user
begins by using a palette
of
tools and
objects in the
design
window
to
link
together
icons
that
represent
thecomponents
of
theexperiment. Imagine that
we
want to
construct
a simple reaction
time
experiment
to
study the effects
of
distance (from the fixation point)
and size
of
the
target
on word
naming
latency.
We
will
call this the Acuity
Experiment,
andit will
have
two
fac
-
tors,
Position and
Size.
We
begin
by
opening up the
de
-
sign window and linking a factor table object to the
ex
-
periment
(see
Figure
1).
Double-clicking
on any
of
the
icons
in the design
win
-
dow opens up dialog windows that allow
further
elabora
-
tion
of
the design.
For
example, double-clicking on the
factortable
icon
opens
the factor table forthis experiment
(see Figure 2).
Of
course, thetable doesnot yet have any
factors or levels.
These
have to be added.
Our
goal in
this
experiment
is to
construct
a
two-factor experiment
to
measure
the
effects
of
distance
(from
fixation)
andword
size on
naming
latency. Our first factor is the position
of
the word, which will have
five
levels. To
create
this

HYPOTHESIS
TESTING
259
~l::J—
~
Subject
Info
Acuity
[Hperiment
X?
Variables ~
Show
Events
~
Show
Lists
[)
Tools
kN
0
mi
Objects
~..fIAcuity Experiment
~E
~
~
~
;~
\
.~
l;’~j~zJ
Eve
nt~
~iI~
~
~
~___ ~I
~*
IJ~
Figure
1.
The
design
window.
Experiments
are
constructed in
the
design
win
-
dow by
linking
graphic
objects to the experiment. In this example, a
factor
table
object
is being linked to the experiment object.
factor, we click
on
the new factor
button
and
then
name
it
“Position.”
The
factor
appears in the
table,
and the
new
level
button
is
activated so
that
levels
can
be
defined
for
the
factor. Figure
3
shows
how
the factor table
ap
-
pears
once
the
five
levels
for
the
Position
factor
havebeen
created andthe
Size
factor and
its
three
levels
have been
added.
Factors and
levels
in a table can
be
rearranged
easily
by
clicking
and dragging. This
allows
the
user
to
cus
-
tomize
the display
and/or
to
change
the
order in which
trials
are
run
(see
Figure
4).
Once
the
factor
structure
has
been established,
the
next
step
is to establish the
struc
-
ture of
the
trials
that
will
instantiate
each of the
cells
in
the
design. We begin by establishing the overall
struc
-
ture of the trials for
all
of
the cells in the design. To
do
this, we can drag the
mouse
to select
all
of
the cells, or
simply
type
1-A
.
Clicking
on
the Open
Cells
button or
double-clicking
on
the
selected
cells then opens the
Tem
-
plate
window.
This window
contains
apalette
of
tools
and
objects
for
creating
and
linking
the events that will
make
up
a
prototypical
trial. An event is
simply
a
period
of
time
during
which
something
occurs.
The events
that
make
up
[~~TFactor
j
[
hm..
~
~ ~
(Open
Ceiisj
ThbIe:
1
[~iTab1e
in~J
[i>?i.?*. ~4?~!:d
S
Within~
Random
No
Factors.
Pd~Jk~
—1~
Figure 2. The factor table window before any factors
have
been
created.

260
COHEN,
MACWHINNEY, FLATT, AND
PROVOST
rNew
Factor
I
[
New
Le’~i~
Table:
1
‘v
rDelete
Facto~J
~
i’.~i~
}
(_~
Tablei~i~)
[I.?I.?~.
T~bi.?)
Open
Ceijil
S
Within~
Random
Small Middle
~?
Large —
Position
LeftFar
11
1
LeftNear
1
1
1
Center
1
1
1
Ri~htNear
1
1
1
RightFar
1
11
~JK1I
Figure
3. The factortablewindowonce the
Position
and
Sizefactors
and
their levels
have
been
created.
The
numbers
indicate
the
weight
of
eachcell in
the
table,whieh
is how many
times
that cell will be
chosen
on each pass through the
table.
a trial in the
Acuity
Experiment
are the Fixation event
(duringwhich a fixation
point
ispresented), the
Stimulus
event (which
occurs
at the end of
the
Fixationevent), the
Response
event (during which
the
subject
will
respond),
and
the
frI
event
(the
intertrial
interval).
These
arecreated
and
represented
graphically
along
a time
line
in the
Tem
-
plate
window
(see
Figure
5).
Each type of event is represented inthe palette with
its
own
icon. The
set
of
events standardly available in
PsyScope is listed in
Table
1.
In
addition,
new types of
events
can be
created
and added to PsyScope as code
resources. However,
this
requires
programming
exper
-
tise
(see
below).
Each event is
associated
with a
set
of
attributes,
which
are used to specify
exactly
how
that
event will occur. The
start
time and
duration
of the event can be
set
directly
from the Template
window
by
selecting the
event
and
using thepop-up
menus
atthe bottom
of
the
window
(see
Figure 6).
The other attributes
for
the event are
set
in the
Attrib
-
utes
dialog, which is
opened
by
double-clicking
on
the
event in the Template
window.
Each
attribute
has
apop-
LeitFar
Middle
1
Small
—
Large
—
LeftNear
Middle
1
Small
Large
—
1
Center
Middle
Small
Large
—
1
RightNear
Middle
I
Small
I
Large
1
RightFar
Middle
1
Small
1
Large
1
New
Factor]
[
~ ~
Table:
1
‘v
~ ~ ~~
[
Set
Table
Info
1
~ ~
[Open
Cells
)
•
I4ithin,
Random
__________________
Trials:
Position
Size
}
Figure 4. The factor table after the Size factor
has
been dragged to
be
a
row
and
the
Small
level
of
the
Position
factor
hasbeen
dragged
below
the
Middle
level.

HYPOTHESIS
TESTING
261
up
menu
that
is used to assign
its
value
(see
Figure 7).
In
the
Acuity
Experiment,
we
want
the
value
of
the
Size
and Position
attributes
for
the
Stimulus
event to be
deter
-
mined
by
the factorial design of the experiment.
To
do
this,
we
return
to
the
factor table.
Beginning
with
the
Size
factor,
we select
the
cells
for
the
first
level
by
clicking
on
“Small”
(see
Figure 8). The Template window and
Attributes
dialogs
are
all
linked
to
the
factor
table
window
so
that
when a
subset
of cells is
selected
in the factor
ta
-
ble,
any
changes
made in the Template
window
or
At
-
tributes
dialogs
will
apply only to
those
cellsof thetable.
That
is,
the characteristics of
the
trial can
now
be
cus
-
tomized
for
each
cell, to suit the
factorial
design of the
experiment.
Any
feature
of
the
trial can
be
changed,
and
new ones
can
be added, for
any
single cell or
set
ofcells
in the factor
table.
Having
selected
thecells for the
Small
level
ofthe
Size
factor, we
now
go
to the
Attributes
dialog
for the
Stimu
-
lus event and
set
the
Size
attribute to
some
small
size,
say,
9
point.
To
do
this, we choose
“Set
To”
from the
pop-up
menu
for the
Size
attribute. This
opens
a
dialog
that
allows
us to
set
the
size
interactively
(see
Figure 9a);
alternatively, we can
use
a
shortcut
to
enter
the value as
text
(see
Figure
9b). We repeat this series of
steps
to
as
-
sign
the
values
for
the
Middle
and
Large levels
of
the
Size
factor
and
then
do
the
samefor the
Position
factor, in this
case setting the value of
the
Position
attribute.
Now
that
the basicstructure oftheexperimenthas been
set
up
and the values have been assigned
for
the levels
of each
factor,
we
need to
specify
the actual stimuli that
will
be presented. In
this
experiment, we will
simply
choose words
randomly
from a list.
To
do this, we select
all
of the cells in the
factor
table
and then
go
to the
At-
tributes
dialog
for the
Stimulus
event.
There,
we choose
“Vary
By” from
the
Stimulus
attributepop-up
menu
and
“List”
from
the
“Vary
By”
submenu
(see Figure
10).
Since
no
list exists
yet,
this
will
open the
List
dialog,
which
allows
us to
create
a new one (see
Figure
11).
When we are
done
and
have closed the List dialog,
the
Attributes
dialog
will
reflect
the
fact
that
the stimulus for
the
Stimulus
event
will
be
chosen, trial
to
trial,
from
the
list
(see
Figure
12).
After
a few more dialogs to specify
how
responses
will
be
recorded,
to
set
up
some
instruc
-
tions and a rest period,
and
to block the
trials,
we have
completed
the
design
of
the
experiment. Figure
13
shows
how
it
will
appear in
the
design
window.
We can
now
run the
experiment,
by
choosing
Run from
the Run menu,
clicking
on
the
Run button in the console
window, or
typing
1 -R.
The
responses for each trial
will
be
recorded
in a
standard
tab-delimited text
file.
We can
also preview
trials
at
any
point during thedesign
process
by
clicking
on
the
Preview
button in
the
Template
win
-
dow
(to
preview a
single
trial) or in the Block
dialog
(to
preview a block of
trials).
GENERAL
CAPABILITIES
AND
SPECIFICATIONS
Psy
Script:
The
Psy
Scope
Scripting
Language
While
the
stated
goal
of
PsyScope is to allow users to
build experiments
without
programming,
we
recognize
that
there
is
always a
tension
between the ease of use of
structured
graphic
environments
and the
power
provided
by
general-purpose
programming
languages. To make
PsyScope
as
flexible
and
as generally applicable as
pos
-
sibleand to address the
needs
of
more
sophisticated
users
Figure
5.
The Template
window
after the events for a prototypical
trial
have
been
created. The
Stimu
-
lus event is
selected.
Notice the menus at the bottom
of
thewindow for
setting
the
start
time and duration
for the event.

262
COHEN,
MACWHINNEY,
FLATT, AND
PROVOST
Time
Event Type
Table
1
Standard
PsyScope
Event
Types
Waits
for
a
timed
period.
A Text
~-
Paragraph
Presents
free
unformatted
text.
Presents
formatted
text.
Document
PICT
Pasteboard
Presents
the full
contents
of
a
text
file.
Presents
a
graphic
contained
in
a
file in
PICT
format.
Presents
any
combination
of
text
and/or
PICTs
simultaneously.
These
can
be
positioned
relative
to
one
another.
~~:>)
Sound
Bbox
Output
Presents
a
sound
contained
in a
file
in
SoundEdit
format.
The
entire
file
orjust
a
labeled
sound
can
be
presented.
Turns
one
of
the
three LED
lights
on
the
CMU
button
box
on
or
off,
or
changes the
status
of
one
of
the
8
TTL
output
lines.
Input
Records
a
single mouse
click,
a
single
key
press,
or a
single
response
from
the
CMU
button
box
(a
button
press
or
a
signal
from the voice
activated
relay).
Continuous
Keyboard
Input
Records
a
string
typed
at
the
keyboard.
as
well
as
those of
nonprogrammers,
we have
incorpo
-
rated
into
PsyScope
a
general-purpose
scripting language.
This language, called
PsyScript,
supports
all
of the
ex
-
perimental
design
structures
that appear in the graphic
interface: groups, blocks,
trials,
events,
and
factors.
In
fact, the script and graphic environment are fully
inter
-
active,
so that
any
element
that
is
created
or changed in
the
graphic environment is automatically
entered
in
the
script, andany element
that
is added to the script and
has
a graphic representation
immediately
appears in the
ap
-
propriate
windows
and/or dialogs of
the
graphic
environ
-
ment. Scripts are
text
files
and can be edited
directly
in
PsyScope or using any standard text editor.
The
text
editor.
PsyScope
has abuilt-in text
editor
for
editing
scripts
and other text files. The editor has a
large
numberof features,
including
formatting, full search and
replace
capabilities,
the
ability
to generate
word
and
line
counts, a
set
of
“quick-reference”
menus
for looking
up
and
inserting
any term
in
the
scripting
language, and an
“Evaluate”
function
for
evaluating expressions in the
script
(see
Figure
14).
Brief
description
of
PsyScript.
PsyScript is best
de
-
scribed
as
a
declarative
language.
It is used to define the
structure
of the
experiment
in
hierarchical
form:
An
ex
-
periment is composed
of
groups,
a group is composed of
blocks,
a
block
is
composed
of
trials
(templates),
and
trials
are
composed
of
events.
Each
of
these,
and
all
other
com
-
ponents of PsyScope,
are
specified
as
“entries”
in the
script. Each entry has
a
title and a set of attributes that
define
its
characteristics.
The entry for each component
in the
experiment
hierarchy
has
one attribute that
speci
-
fies the
components
that
belong to it
(i.e.,
are below it
in the
hierarchy).
Figure
15
shows
fragments
from two
parts
of
the
script
for
the
Acuity
Experiment.
Icon
Description

HYPOTHESIS
TESTING
There
is
abasic,
implicit
procedure
in
PsyScript,
which
is
the
instantiation
of
experimental
trials
from
the
state
-
ments in
the
script.
The
compiler
begins
with
the
top-level
entry in
the
script
(the
experiment
entry) and works
its
way downthehierarchy,
looking
to see
what
components
it is made up of (groups,
blocks,
templates,
events,
etc.)
until
all
of the events of
each
trial
in
the
experiment are
fully
defmed.
Any
undefmed
elements
are
assigned
prede
-
fined default
values
(except the
stimulus
for
each
event,
which must be defined). Variable expressions and
func
-
tion
calls
are
permitted
at any level, providing extensive
flexibility
inthe
definition
of
the
experiment.
In
particu
-
lar,
PsyScript provides
a
powerful
set
of list operations
(e.g.,
accessing
lists,
sublisting,
mapping between lists,
etc.)
that
assist
in
the
development
ofhighly
sophisticated
randomization
designs.
Script
Compilation
and Run
Modes
The script is compiled
when
the
experiment
(or a
sub
-
set
of
trials)
is
run. This
can
be done in oneof
two
modes:
either
one trial at a time
(“trial-by-trial
mode”)
or
all
at
Stimulus:
~.D________________
0
Stimu
lus
Event
Attributes
A
Name:
[~imulus
1
Event
tgpe:
I
Teat
“1
®
Stimulus
Attribs
0
Event
Attribs
Default
‘ri
Stgle:
T
Default
‘ri
!P,i~
Font
Default
Size
Tnefoult
wi
Face
j
Default
‘~
Color:
I
Default
‘v
I1~,
kA...t.—.
I
fl.~S..I.
— I
Ii~
263
Figure
6.
The
pop-up
menus
for setting the start
time
of
the
Stimulus
event.
Figure
7.
The
Attributes
dialog
for the
Stimulus
event, showing the
pop-up
menus used for
setting
the
value
of
each attribute.

264
COHEN,
MACWHINNEY,
FLATT, AND
PROVOST
LeftFar
~nl
Middle
——
L
1
Large
I
1
~mlflL.~
LeftNear
Middle
‘~
Large
L
1
I 1
—
Center
Middle
Large
—
RightNear
a7;!u
Middle
•
1
—
Large
1
RightFar
~mi~
Middle
•
•
1
Large
jc~cJi
Figure 8. Clickingon the heading for a particular level
of
a
factor
selects
all
of
the cells for that level.
once before
any
trials are
run
(“precompile
mode”).
When
PsyScope compiles a
trial,
it determines and
se
-
quences
all of the events for
that
trial and assigns the
values
for
all
of their attributes. These are written in a
memory structure,
which
is
then read
by
the
trial
manager
that
executes
the
trial in
real
time
(timing
is discussed in
the next
section).
Trial-by-trial
mode.
In thismode,
PsyScopecompiles
a single trial and
then
runs it before compiling the next.
The
primary
advantage
of
this mode
is
that features
of
the experiment can
change
as it
proceeds—for
example,
the duration of a
stimulus
can
change
from
one trial to
thenext
on
thebasis
of
previous reaction
times. The
dis
-
advantageof this
mode
is that the
compile
time
may
vary
from trial to
trial,
introducing
variability in the intertrial
interval.
Thiscan be
overcome
inone
of
two
ways:
either
by explicitly
including
an intertrial
interval
that is
longer
thanthe maximum
amount
of time it
will
take to
compile
B
G
Stimulus
Event
Httributes
A
Name: StImulus
I
Event
type:
I
Teøt
®
Stimulus
Rttflbs
0
Event
Ilttrlbs
Stimulus:
Default
“I
0’
Style:
Default
‘v
Font
Default
‘~
Set
To
•,‘~
.~I4
Face
Default
i’~
Color j
,~efauIt
:~
Ne’w
Fac~J
ri~iew
Level
1
[Delete
Factor)
IDeleteLevels)
[Open
Cells]
Table:
1
.,rI
Set
Table
Info
j
(p..?~.
•
Within,
Random
Position
1514’\
Trials:
A
0
Size
for
Stimulus
_____________
U
l~t~lic
i:i
lJnth?rlinI~
U
ll~uI?rt
c:1
E~i~~nd
.~1Ol1*?
Figure
9.
Two methods for entering attribute
values:
(a)
interactively or
(b)
directly
as text.

HYPOTHESIS
TESTING
265
Figure
10.
Choosing
“Vary
By” and
“List”
from the
Stimulus attribute
menu
causes
the
stim
-
ulus for
each
trial to
be
chosen
from
a
list.
a trial (this can be done by
setting
an
experiment
attrib
-
ute) or
by
precompiling all of
the
trials
before running
the experiment.
Precompile
mode.
In this mode, PsyScope reads the
entire
script and
constructs
all
ofthe trials of the
experi
-
ment before any trial is
run.
This mode
optimizes
per
-
formance
at the time the experiment is run (eliminating
any unspecified intertrial intervals), but this is at the
ex
-
pense
of
the
flexibility
offered
by
the
trial-by-trial
mode.
Previewing
and
precheckingtrials. In
addition
to the
two general run
modes
just
described, singletrials or
sets
oftrials can be compiled and run usingthe
Trial
Monitor
console
(see
Figure
16)
or by using the
Preview
buttons
in the Templatewindow and Block
dialog. These
provide
the user
with
immediate
feedback
about
how trials will
appear when the
experiment
is
run. The
Trial
Monitor
can also be used to
monitor
the events
of
a trial as it runs
(for
debugging
scripts) and to
precheck
the
experiment
(e.g.,
to see
how
long
it
takes to compile
trials,
to be sure
that
all
stimulus
files
are
correctly
referenced,
etc.).
Randomization
PsyScope supports a
wide
variety
of
randomization
de
-
signs, including
mixed versus
blocked,
full
factorial,
List:
Stimlus
List
Item
Order:
I
Sequential
‘ri
Items
1-i;;-]
Weiqht
A
Stimulus
~flCTOF1
Value Setting:
I
Set
To:
~ ~~it
Ua~ij1
~
A Name:
Stimulus
Stimulus:
Event
tgpe:
j
Teat
®
Stimulus
Attribs
—
I~T~V
.rT~
StUle:
Font:
Size:
Face:
Color:
•
___
Script
)~
NiuHiple
6roup:
flther:
Block:
U~IGU,I
Template:
I
Default
‘v
I
Default
‘vj
I —
—
item
1
1
HOUSE
.Q.
item
2
1
TABLE
item
3
1
BOOK
ltfl~
Figure
11.
The List
dialog,
once
a
few items
have
been
created.

266
COHEN,
MACWHINNEY,
FLATT,
AND PROVOST
I
Vary
By
‘v
II
List:
~
•
Style:
I
Default
‘v
Font
LDefauIt
wi
Size
I
Default
~:~!~i
Face
Default
‘v
Color:
Default
‘i”~
~Ii
•4....L...
I
fl..~C...••I• —
‘0
Figure
12.
The Attribute
dialog
for the Stimulus event,
indicating
that the
Stimulus
attribute
will be varied by the Stimulus List.
nested factorial, and
between-subject
(including
Latin
squares)
designs.
There are
also
facilities
for defining
groups
for
between-subject
designs. All of the
above
are
available
within
thegraphic
environment,
mostwitha
sin
-
gle
click of the
mouse.
More
complex designs
can also
be
developed
in the graphic environment, although this
requires more experience with
PsyScope.
In addition,
PsyScript can be used to
construct
designs
and
randomi
-
zations
of
arbitrary
complexity.
Data
Output
PsyScope provides
no
facilities for
statistical analysis.
There is
currently
a
wide
variety of
statistical
packages
available
for
the
Macintosh.
Rather
than
trying
to
dupli
-
Subject
Info
X?
Variables ~
Show
Events ~
Show
Lists
(
Notes
Tools
9~j~ts
Eve
ts
~
~)))[i~d
—~
Ic~1~
A
Name:
Istimulus
Event
type:
I
Teat
‘ri
®
Stimulus
Rttribs
0
Event
Attribs
Stimulus:
0
‘0
Figure
13.
The
design window showing the
complete
experiment.
Notice that the events and the list all
appear
as elements
of
the
experiment.

HYPOTHESIS
TESTING
267
_______~JI
•(~T~1F~f~1
Interactive
Mode
[
Reinitialize
~J
[~ái~j
[I:u1~Iunu~j
,/Interdctiue
r~1ode
Scroll
to
Changes
0.8.1.5
.‘Script
Enabled
iment”
11)1
d~t
<11 (1LIFId
l(
I~
Balance
Statistics
Count
Finds
Shift Left
Shift
Right
Comment
Lines
Uncoinment
Lines
Uppercase
st
Lowercase
EL
ockl
Instructions2 BIock2
l.t~s1
*?qITu?n
Script Keywords
Script
Functions
EHP
Keywords
Actions
ConditiOnS
~ ~
Figure
14.
The editor window,
showing
the Tools
menu.
Rcuity
Experiment:
Format
Factor’
loputDeuices:
riolisE
tEY
lime,’:
Ilocinitosh
Flags:
N0_SAUE....SCREEH
~>
EveritDefinitmor,s
DataFile:
~acu:ty.data
ScaleBlocks:
1
Fixation::
Cycles:
1
EveritType:
Text
Instruction,,:
acUity.inst
DataFields:
RESP0IISE_LRBEL
urat
ion.
500
St or
t
Re
I
Blocks:
Instructions
Blocki
Rest
tlock2
—
Stimulus:
*
>
BlockDefinitions
Stimulus::
EuentType:
Text
BIockl,.
Ouration:
500
Ternplate3:
“Toblel
Star’tRef:
0
after’
end
of
Fixationi
tlock2:
BlockOuration:
15
Templates:
Tablel
Instruct
loris::
Templates:
~lnstructions
Template”
Rest:
Templates:
lnstructions
Template
Figure
15.
Two
script
fragments
from the
Acuity
Experiment
script.
The one on the left
shows
the entries for
the
experiment
and
two
of
the
blocks
in the experiment
(corresponding
to
the
graphic
elements in the
design
window; see
Figure
13);
the
one on
the
right
shows the
entries
for
the Fixation and
Stimulus
events
(which
are
displayed
graphically
in the
design
and
Template
win
-
dows;
see
Figures
13
and
5,
respectively).

268
COHEN,
MACWHINNEY,
FLATT,
AND
PROVOST
Trial
Monitor
Figure
16.
The
Trial
Monitor
console
and the Run By
Index
dialog that can
be
opened
from
it.
cate these
efforts
within PsyScope, our
goal
has been to
make
available
as
much
information
about the experiment
aspossible
for
inclusion
in the data
file—in
standard text
format—and
thento allow the user to choose what
infor
-
mation to actually record.
Adata file
consists
of a list of datalines with a
specifi
-
able number of tab-delimited
columns.
A data
line
is
generated
during
a
trial
under
conditions
that
are
specified
by
the
user. Typically,
data
lines are scheduled to occur
in response
to
subject
input
(e.g.,
a
keypress
or a
mouse
click).
However, data
lines
can
be
generated
at any other
point
during
a
trial
(the
beginning
or end of
an
event,
when
a user-defined variable
reaches
some
value,
etc.). Each
data
line
contains
at
least
three
columns,
which record
the number of
the
trial,
the
name
of
the event, and the
condition
that
generated
it.
In
addition,
the usercan
choose
Format:
Factor
Mode:
Direct
(set
trial
counts)
®Run
I’
I
0
Practice
Total
to run:
1
Total
compiled:
0
Optimization:
Standard
ILDo
Triii~
]
L!!.~’~’~
IIY~1
J
(By
lndea...]
NeHt:
0
U
Check
UI
LI)ltd
SI
IEYI4
I~
L~5I
~t~erlI
s
~
Compile
all
trials
before
running
U
Auto
reset
eap
after
run
trials
0
Monitor
events
0
<~tt~p
________
Hun
By
Indea
________
Run
Trials
ILno
Triiii]
First:
~
[
Cance[]
Last:
I’
I
Last
ran:
—
Trials
already
compiled:
U
B
EHperiment
l~1i;iiii
1
Windows
Stirn
Ud~ition...
Stirn
Sizes...
Stirn
Font...
A
#>
MenuDefinitions
Menus::
Experiment
Pararns
Experiment::
InputDevicesMenultem
TimerMenultem
SettingsMenultem
UserLevelMenultem
DataFieldsMenultem
Paraxns::
IIStim
DuratiOns
“Stim Sizes~
~Stiin
Fonts
Stim
Duration::
500
Stim
Sizes::
9
12 18
Prorr~ts:
Short
Middle
Long
Stim
Font::
Helvetica
C
Stim
Sizes
___________________
~J
[~nc~i~]
[
Ele1auli~]
Short
7
MIddle
12
I
Long
118
Figure
17.
(A)
Scriptfragment
showing
the addition
of
Params as
a
custom menu, which has
three
items: Stim
Duration,
Stim
Sizes,
and
Stim
Font. (B) The
Params
menu.
(C)
The
dialog
opened
by choosing the
“Stim
Sizes”
item
from
the
Params
menu.

HYPOTHESIS
TESTING
269
to include columns for text
labels,
experimental
condi
-
tion name
(generated
from
the
factor
table),
and
the
time
of
occurrence
(relative
tothe beginning of
the
trial
or
some
event), as
well
as a
number
of
other
items.
A
header can
also
be added to
the
data
file,
which can
include
infor
-
mation
about
the
time and
date
of
the
run,
the
type
of
ma
-
chine
on
which it took
place,
and
subject
identification
information. Finally,
PsyScope
has
a built-in
facility
for
recording
information
about
each session
(such
as
sub
-
ject
and run
information
and
error
messages)
in
a
sepa
-
rate
log file. This
is
also
stored
as
a
text
file, which can
be
changed
at
any
time and
used
by
multiple
experiments.
Timing
PsyScopeprovides
a
minimum
of l6-msec
accuracy
(the
temporal
resolution
of the Macintosh operating
system)
inthe
timing
of
events
when
run
on
a
free-standing
Macin
-
tosh. However, it has the capability
to
provide
l-msec
accuracy when
used
with the CMU button
box
(see next
section) or another
external
timing
device.
An
extension
must be written to
use
an
external
timing device
(see
be
-
low);
once
this
is
written,
it
can
be
added to PsyScope
simply
by
dropping
it inthe
PsyScope
Extensions
folder.
The
timing
of
stimulus
presentation
andresponse
record
-
ing
are, ofcourse,
limited
by
the
input
and
output
devices
that are used.
CMU
Button Box
The CMU
button
box
is
a
device
that
has
been
developed
at
Carnegie
Mellon
University foruse in
microcomputer-
controlled
psychological
experiments.
It
can
be
usedwith
Macintosh
orIBM-PC-compatible
computers.
It
connects
to the
Macintosh
viathe modem or
printer
port and
pro
-
vides
a
millisecond
timer as well
as
three
colored
buttons
with
associated
LED lights, a
voice-activated
relay, four
additional
TTL input lines, and
eight
TTL output lines.
The
extension
for this device is built
into
PsyScope,
al
-
lowing PsyScope to
recognize
all
of its input and output
lines as
well
as
providing
PsyScope with l-msec
timing
accuracy. This
device
will be fully described in a
sepa
-
rate
paper
that
is currently
under
preparation.
Input
and
Output
Input.
PsyScope
canrecord
mouse
clicks
or
movement,
single or
multiple
keypresses,
and input from the CMU
button
box
(described
above).
All
input
signals
(including
mouse
and
keyboard
input)are
recorded
asynchronously,
so that
multiple
simultaneous
inputs
can
be
recorded
(within
the
temporal
resolution
provided
by
the
hardware).
Output.
PsyScope
can
present
1-
to
24-bit
visual
stim
-
uli to
any
standard display
that
is connected to a
Macin
-
tosh andto anyor
all
of
the
displays
in a multiple-display
system. Stimuli
presented
on
the screen are
linked
to the
vertical retrace
(VBL) signal
by
default, but this can be
defeated
by
setting
an
experiment
attribute.
Formatted
text
as well
as
graphics
stored in PICT format
(either
in a
file
or resource) can be
displayed.
PsyScope can play 8-bit
sounds stored
as
SND
resources or in
SoundEdit
format.
The
LED lights
and
output
lines of the CMU button
box
can
also
be
controlled.
PsyScope
uses
standard
Macin
-
tosh Toolbox
calls
to
construct
and
present
all
stimuli to
their
corresponding
output
devices.
Stimuli
are presented
asynchronously,
permitting
multiple stimuli
to
be
pre
-
sented
at
the
same time.
The
only
exception
to
this
is
refresh-synchronized
screen
displays, which require
PsyScope to
wait
for the
VBL
signal
(0-16
msec),
How
-
ever,
multiple
refresh-synchronized
stimuli
can
be
pre
-
sented
simultaneously
by
using
the
PasteBoard
event
type.
This
composes
the
stimuli
into
a
single
bitmapbefore
writ
-
ing to
the
screen,
waits
for
the
VBL
signal, and then
presents
the
entire
bitmap
at once.
In
addition
to the
standard
input and output devices,
PsyScope
can be
extended
to work with other
devices
by
writing
externals,
which
are
discussed
in
the
next
section.
Customizing
PsyScope
There
are
two
ways
in
which
PsyScope
can
be
extended:
by
using
the
scripting
language
to customize the interface
and
by
writing
externals.
Customizing
the
interface.
PsyScope provides built-
in
tools
for extending
the
interface
to
include
new
inter
-
active
elements,
such
as
menus
and dialogs.
These
can
be
used
to
conveniently
modify
experimental
parameters
at
the
time the
experiment
is run
(e.g.,
the
duration
of
an
interstimulus
interval)
or to
record
information
that
needs
to be
entered
at the
time
theexperiment is
run
(e.g.,
subject
identification,
choice
of conditions,
etc.).
These
are defined in the same
way
that the components of
an
experiment aredefined,
by
including new
menus
and
di
-
alogs
in entries inthe
script
and specifying
their
contents
or format in attributes
of
the entry. Figure
17
shows
a
script
fragment
that
adds
a
custom menuto the menubar.
Externals:
The
PsyScope
Extensionsfolder.
PsyScope
also
provides
the
ability—for
C
programmers—to
write
extensions
to
the
program
itself,
called
PSYXs.
Conven
-
tions for writing PSYXs
(similar
to HyperCard
XCMDs)
are documented inthe
PsyScope
manual. PSYXs can
be
written
to
control
new
input
and/or outputdevices, to
con
-
nect
new
timing
devices,
to
carry
out
specialized
func
-
tions
that
can be
called
from
the
scripting
language, and
to add new
dialog
or
graphic
elements to the interface.
PSYXs can
also
be
written
to
replace
the
compiler
that
reads the script and
creates
the memory
structure
for the
experiment,
permitting
the
design
of experimentsin a
lan
-
guage other than
PsyScript.
Once
a PSYX is
written,
it
is added to
PsyScope
simply by dropping it into the
PsyScope Extensions
folder.
No
recompiling
of
the
pro
-
gram
is
needed;
the
extension
will
become active
the
next
time
PsyScope
is started. PSYXs have
recently
been
writ
-
ten to send and
receive
Apple
Events (Apple’s standard
for
interapplication
communication
on
the
Macintosh)
and
for the
DigiDesign
16-bit sound
board,
and
extensions
are
currently under
development
for continuous microphone
recording
(straight
to
disk)
and
for
an
oscilloscope-based
device
that
provides visual
displays
with
1
-msec
temporal
resolution.

270
COHEN,
MACWHINNEY,
FLATT,
AND
PROVOST
Help and
Documentation
PsyScope
has
an
integrated
help
system.
Help can be
called from
the
script
or
the
graphic
environment
and
pro
-
vides
a search ability
and
index.
The
PsyScope manual
is
still
under
development.
It
cur
-
rently
consists
of
five
parts.
Parts
I
through
ifi
and
Part
V
are
complete.
These
are a general
introduction
and
tutorial
for
using
the
graphic
environment
(Part
I),
a
reference
manual
for the
graphic
environment,
which
includes a
full
description of
every
window
and
dialog
(Part
II),
a
ref
-
erence
manual
for
all
of
PsyScope’s
main
menus
and
gen
-
eral
utilities
(Part
ifi),
and the
PsyScript
reference manual
(Part
V).
Part IV (an introduction to
scripting
experi
-
ments)
and Part
VI
(the
reference manual
for
writing
PSYX5)
are in
intermediate
stages
of completion. The
manuals
contain
extensive
figures
showing
the
screens
and
graphic
elements
used in
PsyScope.
All documents are
in
Microsoft
Word
format.
There
is
also
a
growing
library
of
commented
example
scripts, which implement a variety of standard
psychol
-
ogy
experiments
(e.g.,
Stroop,
the
Sperling task, lexical
priming,
etc.).
These
are
available
to members of the
PsyScope Development Consortium
(see
below).
System
and
Hardware
Requirements
System.
Psy
Scope
requires
atleast2
MB
of
RAM and
about the same
amount
of
hard-disk space. This is
suffi
-
cient to construct and store a moderately complex
ex
-
perimental
design.
More
complex
designs
or experiments
involving
many
stimuli
may
require more memory and
disk space.
CPUs.
PsyScope works
on all
Macintoshes
including
the
MacPlus
and
PowerBooks.
Although
experimentscan
be run
on
any of these
machines,
slower CPUs
have
a
hard time
keeping
up
with the
computational
demands
im
-
posed
by
use
ofthe
graphic
environment
and
by
the
con
-
struction
of trials
at
run
time.
The graphic
environment
can be bypassed
by
working
strictly within
the scripting
language. However, the time it takes to
construct
trials
(particularly with complex
designs)
can impose long
precompile
times
(in
precompile
mode)
or
intertrial
in
-
tervals
(in
trial-by-trial
mode).
We have
found
that all-
around
performance is
acceptable
on
a Macintosh
Ilsi
or
better.
Input
and
output
devices.
As
noted above,
PsyScope
supports
the
use of
multiple
1-
to 24-bit
color
monitors,
8-bit sound in
SND
resource or
SoundEdit
formats, and
16-bit
sound when
used
with a
DigiDesign
board.
Psy
-
Scope recognizes input from the
keyboard,
the
mouse,
and the CMU button
box
(described above). An
exten
-
sion is
currently
being developed
for
oscilloscope
display
(permitting
1
-msec
resolution
for
simple,
visually
pre
-
sented stimuli).
Limitations
of
PsyScope
Twomajor
types
of
experiments
arecurrently
not
well
supported.
Psycholinguistic
experiments involving the
constructionof
sentences
of
arbitrary
length
from
lists
of
stimulican be built
through
PsyScript,
but
they currently
cannot be designed in
the
graphic
environment.
We
plan
to
eliminate
this
restriction
in the
next
major
version.
Sec
-
ond,
PsyScope
is not
well
adapted
for the
design of
com
-
plex
user
interfaces
involved in
studies
of
problem
solving
or
computer-assisted
instruction.
For
such interfaces, it
is
betterto
think
in
terms
of
using a
program
like
HyperCard.
At
present, there is
a
practical
limitation
to
the
size
of
visual
displays
that
can
be
presented
within a singlerefresh
using standard
Macintosh
screens.
The
severity
of
this
limitation is
determined
by the speed
of
the CPU and
RAM
on
which
PsyScope
is run.
This
limitation
is due
to the
use
of
the
standard
Macintosh
Toolbox
call for
copying
bitmaps
from
off-screen
buffers
into
display
memory. We are
currently
exploring
options
for
circum
-
venting this
limitation
(or
trading
it
off
againstothers,
such
as
imagecomplexity)
and
will incorporate these
into
future
versions of PsyScope.
As
noted
above,
the
temporal
resolution
of PsyScope
isdetermined
by
the
characteristics
of
the timing and
out
-
put devices that are
used.
Standard
Macintosh
systems
provide a
temporal
resolution of
16
msec. With the CMU
button
box,
event and
response
timing
can
be improved
to
1
msec.
Improvements
in the
temporal
resolution
of
visual
displays
beyond
16
msec require specialized
hard
-
ware,
for which
extensions
to
PsyScope
are currently
being
developed.
CONCLUSIONS
AND
SUMMARY
PsyScope offers an
extensive,
integrated
environment
for
experimental
design
on the
Macintosh.
It takes full
advantage of the
Macintosh’s
graphic
environment,
al
-
lowing novice and
experienced psychologists
aliketo
de
-
sign andimplement
psychology experiments
without any
need for programming
under
most
circumstances.
At
the
same time, it offers
a
powerful scripting
language
for
more specialized
applications.
Interactivity
between the
graphic and
scripting
environments
allows
novices
to
dis
-
cover
how
the
scripting
language
works,
and it allows
more
sophisticated
users
to
quickly
outline an
experiment
before
customizing the
script. Furthermore,
it provides
toolsfor experienced
programmers
to
extend
its
capabili
-
ties. Most
importantly,
however, PsyScope
is
structured
around
the
basic
concepts
of
experimental
design,
provid
-
ingan
environment
in whichthese canbe
easily
learned
by
students
andused
intuitively
by
experienced psychologists.
DISTRIBUTIONPOLICY
Open Distribution
Policy
PsyScope is
available
free via anonymous
FTP
on
poppy.psy.cmu.edu.
Included
are the
latest
majorversion
of the PsyScope
application,
the
help
file,
release
notes,
and the latest version
of
the PsyScope Manual.
The
pur
-
pose ofan open
distribution
policy
is to make PsyScope

HYPOTHESIS
TESTING
271
available
to
anyone
who
may
benefit from
its
use
and for
its
development
to
benefit
from
a
wide
array of users.
Toward this end, we
welcome
(even
encourage)
bug
re
-
ports
(to
psybug@serviceberry.psy.cmu.edu)
as
well
as
suggestions
on
how
the
program
might
be
improved.
However,
our
limited
resources
do
not
allow
us to
provide
user
support
for
PsyScope
to
the
general
community.
Sup
-
port is
available,
however,
to
users
who
are
willing
to
as
-
sume a share in
supporting
PsyScope’s
further
development.
Consortium
Membership
The PsyScope
Development
Consortium
is composed
ofa group ofexperimental
psychology laboratories
in the
U.S.
and Canada
that
contribute
to
the
maintenance and
development of PsyScope.
Members
of the Consortium
have
access
to
frequent
upgrades
to
the
program,
user
sup
-
port, and membership in an
electronic
mail distribution
list for
reporting
bugs
and
discussion
and
clarification
of
PsyScope
issues.
Bugs
reported
by
Consortium
members
receive highpriority,
especially
if they
interfere
with
the
running
of
experiments.
Nembers
also
have
a voice
in
directing future
PsyScope
development
and,
of
course,
the
reward of
contributing
to
a
product
that
is
furthering
their research.
To
become
a
member
of
the PsyScope
Development
Consortium, please
contact
either
Jonathan
Cohen
(jc5e+@andrew.crnu.edu)
or Brian MacWhinney
(brian+
@andrew
.
cmu
.edu).
REFERENCES
SCHNEIDER,
w.
(1988).
Micro
Experimental
Laboratory:
An
integrated
system
for
IBM
PC
compatibles.
Behavior
Research
Methods,
instru
-
ments,
&
Computers,
20,
206.217.
VAUGHAN,
J.
(1992).
The
dimensions
of
computing
Behavior
Research
Methods, Instruments,
&
Co,npuzers,
24, 109-115.