Content uploaded by Christopher Dryer
Author content
All content in this area was uploaded by Christopher Dryer
Content may be subject to copyright.
In H. Johnson, N. Lawrence, C. Roast (Eds), People and Computers XIII, Proc of HCI’98, Springer. pp 209 -219.
Representation Matters: The Effect of 3D Objects
and a Spatial Metaphor in a Graphical User
Interface
Wendy Ark D. Christopher Dryer Ted Selker Shumin Zhai
IBM Almaden Research Center
650 Harry Road
San Jose, CA 95120 USA
+1 408 927 1912
{wsark, dryer, selker, zhai}@almaden.ibm.com
ABSTRACT
As computer graphical user interfaces (GUIs) are loaded with increasingly greater numbers of objects,
researchers in HCI are forced to look for the next step in constructing user interface. In this paper, we
examine the effects of employing more “natural” representations in GUIs. In particular, we experimentally
assess the impact of object form (2D iconic versus 3D realistic) and layout (regular versus ecological) have
on target acquisition time. Results indicate that both form and layout significantly affect performance;
subjects located targets more quickly when using interfaces with 3D objects and ecological layouts than
they do with 2D objects and regular layouts. An interface with an ecological layout, realistic objects, or
both may be an improvement over traditional interfaces.
Keywords
3D interface, graphical interface, spatial metaphor, icon, ecological layout, regular layout
INTRODUCTION
What is the next step in the evolution of the graphical user interface? One possible step is the move from
predominantly iconigraphic, 2D representations to more realistic, 3D representations. The implicit
assumption underlying the increasingly popular attempts to develop compelling 3D environments is that
users will find it natural and intuitive to navigate virtual spaces (e.g., Reeves and Nass, 1996).
The specific advantages, however, of realistic 3D GUIs are unclear. Moreover, it is also unclear which
attributes about a realistic 3D GUI would make it more useful. Note that we do not use the term 3D in the
sense of a virtual reality display, but we are talking about a normal, everyday desktop computer screen.
Clearly, a 3D GUI should not merely be the traditional GUI enhanced with the depth dimension. In the
natural 3D world, there are many different factors that “codify” objects. Many of these factors are visually
salient attributes that have us to recall objects. If we transfer these factors onto the interface, the interface
is more likely to involve a non-regular placement, differing external shapes, enhanced color, landmarks,
connectivity, and potential semantic associations than is a 2D GUI. In general, 3D GUIs may provide
more dimensions to distinguish and identify objects. The more redundant dimensions available to the user,
the greater the chance of the user being able to choose an attribute to which to relate. As a result, users
might find and recall the objects more quickly in such a 3D GUI than in a 2D GUI. In this study, we
examine whether and which factors of a 3D GUI affect how quickly users can acquire target objects in a
laboratory experiment.
In H. Johnson, N. Lawrence, C. Roast (Eds), People and Computers XIII, Proc of HCI’98, Springer. pp 209 -219.
THEORETICAL BACKGROUND
Placement
There is an large amount of research stating the importance of location having a direct effect on the recall
of information. Mandler et al. (1977) and Hasher and Zacks (1979) indicated that people automatically
encode spatial information. When recalling where a file is located, people want to expend the least amount
of cognitive efforts as possible so they can focus on the tasks they want to accomplish once they retrieve
the file. If forming spatial associations is a pre-attentive process, designers should use this to their
advantage when creating interfaces.
Kaptelinin (1993) showed that experienced users do not need to read menu item names while working with
the system. As novices, they primarily rely on the names of the menu items to accomplish simple tasks.
After the learning period is over, the location of the item names seems to be encoded and the selection
actions start to become automatic. Moyes (1995) and Kaptelinin (1993) both agree that users focus on
local attributes initially (e.g. icon form) but over time switch to identifying attributes which are global, or
require the user to consider the interface as a whole (e.g. the icon’s relative position among all other icons
on the screen).
Hess et al. (1994) also noticed the effect spatial placement has on recalling information. In their study, they
showed that spatial separation of items on a screen aids memory for a task requiring that the items being
monitored be available separately in memory.
Individual object locations are not the only concern in this study. More importantly, this experiment will
focus on the effects of object layout, or how individual objects are placed in relation to each other. In
particular, we are interested in the effect an office-like metaphor will have on user performance. Ecology,
the study of the habits of living organisms and their interactions with their environment, can be applied to
the office metaphor. Ecological considerations are important as people draw important cues from their
immediate environment and develop knowledge of the space over time and through the experience of
interacting with it. Benyon and Höök (1997) refer to many various types of spatial metaphors for which
designers use to represent vast amounts of information.
Van der Veer (1989) suggests that adequate metaphors can facilitate the learning process. In turn, the
learning process facilitates automatic actions and; hence, the cognitive load has been reduced.
The idea of organizing information by spatial metaphors have been advocated by many (Bolt, 1979; Cole,
1982; Malone, 1983). Although such an idea seems to be in agreement with the general theories, such as
method of loci, some laboratory studies have not supported spatial interface advantage. In particular, Jones
and Dumais had subjects read news articles and file them based on a certain condition (name only, location
only, name and location, or name and location separate). The subjects were then given a passage from one
of the ten articles (per condition) and were given three guesses as to where they had previously filed it. The
performance of the subjects led to the conclusion that the location only condition did not have any
significant advantage over the name only condition. The results also indicated that in the location and name
combined condition, the article was more easily found. These results demonstrate that more semantic
information provided for the objects (name plus location) will aid in the retrieval process. It is important to
recognize that there are two cognitive processes involved in this study: the recognition task and the recall
task. Jones and Dumais take measurements from both of these tasks which requires the subjects to not only
recall the spatial location of where the article has been filed, but also from which article the passage was
contained. However, in this experiment, we focus only on the recall task.
Objects in an ecological layout not only have their individual locations, but also form connectivities that
make physical sense. For example, in the office metaphor, a monitor is on the desk and the hardware is
attached to the monitor and the keyboard and the mouse are attached to the hardware. We propose these
connections will facilitate target acquisition.
External Shapes
In H. Johnson, N. Lawrence, C. Roast (Eds), People and Computers XIII, Proc of HCI’98, Springer. pp 209 -219.
Another characteristic of realistic objects is the differing external shapes. The 2D iconic representation has
a “framing” effect. The rectangular box surrounding each of the icons gives them a uniformity which does
not help to differentiate the icons. Having different shapes is a desirable quality when a distinguishable
feature is needed quickly.
Color
Color is another important feature of a realistic object. There is a large amount of research about when
color is helpful and when it is distracting and which colors are better to use than others. A particularly
interesting review of comparing the usefulness of color against various achromatic codes (size, shape, etc.)
gave evidence that a color-coded target was more accurately identified than the codes monochrome, size,
shape, and brightness (Christ, 1975). However, if color is not the target, then color becomes a shortcoming
and a distraction on the screen.
Landmarks
Landmarks are inherent in any situation. The difficulty in developing a good test to study landmarks is the
fact that landmarks are a personal discrepancy. We expect that unchanging features on a GUI such as a
table or a bookshelf in the form of what can be called visual landmarks will improve a person’s memory for
where things are located. Groupings of any kind, be they caused by a rim or window around objects or by
spatial separation could be considered landmarks and might improve performance.
Landmarks may serve as external memory aids and we know that external aids to memory are often
employed when other, intervening, cognitive events might interfere with the processes of learning and
recall, when accuracy is at a premium, and when memory load is to be minimized to facilitate the allocation
of attention to other activities. It seems that, in general, individuals prefer to use external aids to memory
rather than rely upon their own internal memory. This suggests that the effort occasioned by the use of
external memory props is less demanding than the cognitive effort required to encode and retrieve
information from internal memory sources (Findlay et al., 1988). However, Jones and Dumais (1986) did
not find any significant data relating to landmarks.
Semantics
Rothkopf et al. (1982) improved upon the claims of Mandler et al. (1977) and Hasher and Zacks (1979) by
stating that location provides especially privileged cues and that not all content-correlated background
stimuli are equipotent cues in associative learning. Lansdale et al. (1987) also stresses the important factor
in the utility of a cue enricher which seems to be the ability of the subject to form a meaningful association
with the relevant document. The more semantic information provided related to the object, the more cues
a person has from which to extract a useful meaning. In 1996, Lansdale et al. discovered even though there
were many differences between cues subjects used when describing characteristics of an object they
recently saw, they were consistent with cues which they had previously used.
METHOD
To examine the relative contributions of different factors distinguishing two styles of interfaces (3D,
ecological and realistic vs. 2D, regular and iconic), we operationalized two independent factors. The first,
“layout,” varies from regular object placement with no landmarks and no connectivity (“regular”) to non-
regular placement with landmarks and connectivity (“ecological”). The second, “object representation,”
varied from similar shape, simple color, and shallow potential semantic associations (“2D iconic”) to
different shapes, complex color, and rich potential semantic associations (“3D realistic”).
In order to test the hypotheses described above, the interfaces developed were the best representations for a
traditional 2D iconic interface and a new 3D realistic interface. The interfaces were based on the interfaces
used in a pilot study (Selker et al, 1997).
Design
There were many issues which had to be taken into account when designing the interfaces used in the
experiments. We attempted to make each of the two styles of interface resemble the most typical features
of their own class. The 2D iconic representations were in 8-bit color and were surrounded by identical
rectangular boxes.
In H. Johnson, N. Lawrence, C. Roast (Eds), People and Computers XIII, Proc of HCI’98, Springer. pp 209 -219.
The regular layout consisted of rows and columns typical to a user interface. The ecological layout
portrayed an office metaphor in which the computer monitor and hard drive were located in the center of a
desk with a bookshelf on the desk to the left of the computer. The ecological layout contained the 3D
interface attributes previously described. The desk and the bookshelf served as visual landmarks while the
computer monitor had connectivity with the hard drive which were also connected to the keyboard.
In order to account for the size discrepancies between the 2D and the 3D icons and also the difference in
the placement of the icons, Fitts’ law (Fitts, 1954) was used to calculate a regression slope to normalize our
results.
The four experimental conditions were: 2D with regular placement (Fig. 1), 2D with spatial placement
(Fig. 2), 3D with regular placement (Fig.3) and 3D with spatial placement (Fig. 4).
Figure 1: 2D Regular Figure 2: 2D Ecological
Figure 3: 3D Regular Figure 4: 3D Ecological
The test was written in Macromedia’s Lingo and displayed using Director on an IBM desktop computer
with a standard mouse.
Subjects
Twelve subjects were used. Nine were male and three were female. All used computers on a daily basis.
In H. Johnson, N. Lawrence, C. Roast (Eds), People and Computers XIII, Proc of HCI’98, Springer. pp 209 -219.
Procedure
The experiment used a balanced, within subjects design. The subjects were asked to find an object as
quickly as they could and then select the object by using the mouse to click on the object or the object’s
label. The subjects were assigned by a Latin square equally into one of four groups where each group
consisted of a different order of the four conditions.
There were ten objects to locate per condition and three trials within each condition with the sequences
varying per trial. The subject repeated the same test on a separate day (within a twelve to thirty-six hour
time period).
The test consisted of an instruction screen which was followed by the name of an object to find. When the
subject was ready, they were asked to click on a “GO” button so as to position the cursor in the center of
the screen. Once the subject clicked on the object or its label, the test continued to the next object. The
refreshed scene forces the subject to get reoriented to the interface and does not give way to any advantages
for neighboring objects. Errors were recorded along with the time it took to find each object.
Also, after the second day of the test, the subjects were given pictures of the four conditions and were asked
to rate the pictures on a scale of 1 to 5 (1 easy; 5 hard) based on how hard it was to find the object they
were looking for in each condition.
RESULTS
We were interested in whether object representation (2D iconic or 3D realistic) and layout (regular or
ecological) would impact the time it takes users to search for and click on objects. To start, we examined
whether other differences among the four conditions might impact our participants' performance. In
particular, the distance to a target (amplitude) and the size of a target (width) can determine the time it
takes to acquire a target, as described by the well-studied Fitts' Law. A transformation of this ratio
(amplitude / width), called Fitts’ index of difficulty, has been shown to be linearly related to target
acquisition time.
To rule out the possibility that any differences in performance between experimental conditions were due to
differences in Fitts’ index of difficulty, we examined the mean of the indices across the ten targets for the
four conditions. These means were 1.68, 1.62, 1.69, and 1.63 for the 2D Regular, 2D Ecological, 3D
Regular, and 3D Ecological respectively. A descriptive analysis provides a measure of the comparability
of the groups. A 2 (2D versus 3D) by 2 (regular versus ecological) analysis of variance on the indices
revealed no significant main effects or interactions. The statistic for the explained variance is F(3, 36) =
.040, p nonsignificant. These analyses suggest that the four conditions are not meaningfully different in
their average Fitts’ indices of difficulty.
We also examined the variance of the ten targets across the four conditions. The standard deviations were
.41, .54, .78, .48 for the 2D Regular, 2D Ecological, 3D Regular, and 3D Ecological, respectively. A test of
heteroscedasticity revealed no significant differences in the distribution of indices across conditions;
Levene statistic (3, 36) = 1.91, p nonsignificant. This analysis suggests that the four conditions also share
similar variance in the Fitts’ indices of difficulty.
In H. Johnson, N. Lawrence, C. Roast (Eds), People and Computers XIII, Proc of HCI’98, Springer. pp 209 -219.
To examine the impact of object representation and layout of subject performance, we looked at the mean
completion times over the three trials and two days by each of the four conditions. These results are
illustrated below in Figure 5.
We analyzed these data with a 2 (object representation) X 2 (layout) X 2 (day) X 3 (trial) repeated
measures analysis of variance with the order of the experimental conditions as a four level between subjects
factor. The Fitts’ indices of difficulty were added as a covariate to control for any possible variance
accounted for by target size and distance.
This analysis revealed a main effects for object representation and layout; for object representation F(1,8) =
41.02, p < .001, and for layout F(1,8) = 45.60, p < .001. Subjects found objects more quickly when they
had 3D realistic representation than they did when they had 2D iconic representations. Similarly, subjects
found objects more quickly when the objects had an ecological layout than they did when they had a
regular layout. The interaction between object representation and layout was not significant; these factors
aided performance indepently. Subjects performed best when objects had a 3D realistic representation in
an ecological layout and worst when objects had a 2D iconic representation in a regular layout.
Other effects were also significant in this analysis. The main effect for trial was significant; F(2,16) =
234.96, p < .001. Subjects tended to perform better on latter trials than they did on earlier trials. The main
effect for day was also significant; F(1,8) = 11.99, p < .01. Subjects tended to perform better on the second
day than they did on the first. The interaction between day and trial was significant as well; F(2,16) =
58.69, p < .001. In general, the effect of trial was greater on the first day than it was on the second day.
The day factor also interacted significantly with the layout factor; F(1,8) = 18.94, p < .01. Generally
speaking, the day factor had a bigger impact for the ecological layout conditions than it did for the regular
layout conditions (Fig. 6).
Finally, the interaction among order, trial, object representation, and layout was significant; F(6,16) = 9.05,
p < .001. We can offer no meaningful interpretation of this result. All other main effects and interactions,
including the effect of the covariate, were not significant.
Figure 5
Day1 Day2
0
10
20
30
40
50
3D_Ecol
3D_reg
2D_Ecol
2D_reg
Mean Completion Time (sec.) with Standard Error Bars
In H. Johnson, N. Lawrence, C. Roast (Eds), People and Computers XIII, Proc of HCI’98, Springer. pp 209 -219.
Figure 6
DISCUSSION
The results indicate a user will search for and acquire objects more quickly if they are presented with a 3D
ecological, realistic interface rather than a 2D regular, iconic interface. Specifically, an ecological layout
(as opposed to a regular layout) and a 3D realistic representation of objects positively affected experimental
task performance.
Interestingly, the effects of ecological layout and 3D realistic representation are additive. That is, the
subjects’ performance was better when either the ecological layout or the 3D realistic representation was
present, and they performed best when both were present. Moreover, the interaction between these factors
was not significant; their contributions to the interface are independent.
These results have important implications for interface design. For tasks which require identifying and
learning the place of objects, 3D realistic interfaces along with ecological layouts will provide increased
performance time. Interface designers can draw upon either factor. In applications where a regular layout
is necessary, 3D realistic objects may still be useful. In applications where 2D icons are necessary, an
ecological layout could still improve the usability of the interface. In other words, these results suggest that
an interface need not be completely 3D to be an improvement over the traditional 2D iconic interface.
We would like to explore the other issues in the 2D versus 3D comparison. We hope independently
investigating shape, color, landmarks and connectivity will provide insights into factors that can support the
3D realistic ecological representation as a favored desktop interface.
ACKNOWLEDGMENTS
We thank Ian May for his help in designing the initial interfaces.
REFERENCES
1. Benyon, D. and Höök, K. Navigation in Information Spaces: supporting the individual. INTERACT ‘97,
Chapman & Hall, 39-45.
2. Bolt, R.A. Spatial Data Management System. Final Technical Report, U.S. Defense Advanced Research
Projects Agency, 1978.
3. Cole, I. Human Aspects of Office Filing: Implications for the Electronic Office. In Proceedings of the
Human Factors Society - 26th Annual Meeting, Seattle, WA.
Day1,
T1 Day1,
T2 Day1,
T3 Day2,
T1 Day2,
T2 Day2,
T3
0
10
20
30
40
50
60
70
3D_Ecol
3D_reg
2D_Ecol
2D_reg
Learning and Retention over Separate Days (completion time in sec.)
In H. Johnson, N. Lawrence, C. Roast (Eds), People and Computers XIII, Proc of HCI’98, Springer. pp 209 -219.
4. Christ, R.E. Review and analysis of color coding research for visual displays. Human Factors, 17, 6,
542-570.
5. Findlay, J.M., Davies, S.P., Kentridge, R, Lambert, A.J., and Kelly, J. Optimum Display Arrangements
for Presenting Visual Reminders. Proceedings of the HCI’88 Conference on People and Computers IV,
1988, British Informatics Society Ltd, 453-464.
6. Fitts, P.M., The information capacity of the human motor system in conntrolling the amplitude of
movement. Journal of Experimental Psychology, 1954, 47, 381-391.
7. Hasher, L., and Zacks, R.T. Automatic and effortful processes in memory. Journal of Experimental
Psyhology: General, 1979, 108, 356-388.
8. Hess, S.M., Detweiler, M.C., and Ellis R.D. The Effects of Display Layout on Monitoring and Updating
System States. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting, 1984,
1336-1340.
9. Kaptelinin, V. Item Recognition in Menu Selection: The Effect of Practice. In INTERCHI ’93 Adjunct
Proceedings, ACM Press, 183-184.
10. Lansdale, M.W., Scrivener, S.A.R., Woodcock, A. Developing practice with theory in HCI: applying
models of spatial cognition for the design of pictorial databases. Int. J. Human-Computer Studies, 1996,
44, 777-799.
11. Lansdale, M.W., Simpson, M. and Stroud, T.R.M. Comparing Words and Icons as Cue Enrichers in an
Information Retrieval Task. INTERACT ‘87, Elsevier Science Publishers B.V., 911-916.
12. Malone, T.W. How Do People Organize Their Desks? Implications for the Design of Office
Information Systems. ACM Transactions on Office Information Systems, January 1983, 1, 1, 99-112.
13. Mandler, J.M., Seegmiller, D., and Day, J. On the coding of spatial information. Memory & Cognition,
1977, 5, 10-16.
14. Moyes, J. Putting Icons in Context: The influence of Contextual Information on the Usability of Icons.
Ph.D. Thesis, University of Glasgow, 1995.
15. Reeves, B. and Nass, C. The Media Equation. Cambridge University Press: Cambridge, 1996.
16. Rothkopf, E.Z., Fisher, D.F. and Billington, M.J. Efects on Spatial Context During Acquisition on the
Recall of Attrivutive Information. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 1982, 8,2, 126-138.
17. Selker, T., May, I., and Zhai, S. Representation Matters: Spatial Interface Can Facilitate Target
Acquisition. Presentation at the 6th International World Wide Web Conference, April 11, 1997, Santa
Clara, CA.
18.Van Der Veer, G.C. Individual differences and the user interface. Ergonomics, 1989, 32, 11, 1431-
1449.
