Content uploaded by Yingxu Wang
Author content
All content in this area was uploaded by Yingxu Wang on Oct 06, 2015
Content may be subject to copyright.
Content uploaded by Yingxu Wang
Author content
All content in this area was uploaded by Yingxu Wang on Nov 17, 2014
Content may be subject to copyright.
36 International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
ABSTRACT
Eyes as the unique organ possess intensively direct connections to the brain and dynamically perceptual ac-
cessibility to the mind. This paper analyzes the cognitive mechanisms of eyes not only as the sensory of vision,
but also the browser of internal memory in thinking and perception. The browse function of eyes is created
by abstract conditioning of the eye’s tracking pathway for accessing internal memories, which enables eye
movements to function as the driver of the perceptive thinking engine of the brain. The dual mechanisms of
the eyes as both the external sensor of the brain and the internal browser of the mind are explained based
on evidences and cognitive experiences in cognitive informatics, neuropsychology, cognitive science, and
brain science. The nding on the experiment’s internal browsing mechanism of eyes reveals a crucial role of
eyes interacting with the brain for accessing internal memory and the cognitive knowledge base in thinking,
perception, attention, consciousness, learning, memorization, and inference.
Unveiling the Cognitive
Mechanisms of Eyes:
The Visual Sensor Vs. the
Perceptive Browser of the Brain
Yingxu Wang, International Institute of Cognitive Informatics and Cognitive Computing
(ICIC),Department of Electrical and Computer Engineering, Schulich School of Engineering,
University of Calgary, Calgary, Alberta, Canada
Keywords: Articial Intelligence, Cognitive Computing, Cognitive Informatics, Cognitive Mechanism of
Eyes, Cognitive Model, Eye Movement, Human Sensory, Internal Browser, Memory Access,
Perception, Thinking Engine, Vision
1. INTRODUCTION
Eyes are commonly recognized as the window
of the mind and eye movement is the primary
sign of life in neuropsychology and cognitive
science, because over 70% of the sensory
information to the brain is captured by the vi-
sion receptors of eyes (Marieb, 1992; Smith,
1993; Sternberg, 1998; Reisberg, 2001; Carter
et al., 2009; Wang, 2003b, 2005, 2009c, 2009d,
2012b, 2013; Wang et al., 2006). Among the five
primary sensors such as vision, hearing, smell,
taste, and touch, eyes transfer the largest portion
of sensory information greater than the sum of
others. Eyes are featured as the only sensory
that possesses both direct connections to the
central nervous system of the brain via three
neural pathways known as the sensory, motor,
and tracking pathways. The other sensors have
only part of them or are not directly connected
to the central nervous system rather than being
relayed by the pons before entering the brain
(Marieb, 1992; Coren et al., 1993; Woolsey et
al., 2008; Wang, 2003b, 2012b, 2013).
DOI: 10.4018/ijcini.2014010103
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014 37
The visual information captured by eyes
is represented in symbolic or semantic forms
in the brain after being processed (Hubel &
Wiesel, 1979; Pinel, 1997; Sternberg, 1998;
Wang, 2009c). Hubel and his colleagues dis-
covered in 1959 that the basic unit of vision is
a bar-like area known as hypercolumns (Hubel
& Wiesel, 1959, 1979) where an image frame is
represented by a set of 50 × 50 hypercolumns.
The size of a visual frame has been calibrated as
about 2,363 pixels according to the property of
the invariant resolution hat is inversely propor-
tional to the distance of visual objects (Wang,
2009e), although there are about 125 million
visual sensory nervous in the eye (Marieb, 1992;
Carter et al., 2009).
There are a number of fundamental ques-
tions yet to be answered in order to rationally
explain the brain identified in cognitive infor-
matics (Wang, 2002, 2003a, 2006, 2007a, 2012a,
2012d, 2014; Wang & Fariello, 2012; Wang
et al., 2009a, 2009b) and abstract intelligence
(2008, 2009a, 2010a) such as follows:
• How does the brain physiologically carry
out thinking and perception?
• How are thinking and perception controlled
and directed in the mind?
• Are all thinking mechanisms consciously
or intentionally controllable?
• What is the role of eye movement for brows-
ing and accessing the internal cognitive
knowledge base in thinking?
The cognitive mechanisms of eyes beyond
its conventionally recognized roles as for vision
sensory are a key for seeking answers to the
above list of fundamental questions about the
brain and the mind, because almost all answers
to them pinpoint to the eyes as both the visual
sensor and the perceptual browser of the mind
as the 6th sense of human brain.
Recent investigations into the cognitive
functions of eyes have led to the discovery of
the eye’s perceptual browser mechanism for
internal memory access in mental processes
(Wang, 2003b, 2012b; Wang & Wang 2006).
This finding is in line with the principles of
abstract intelligence and general real-time
systems where almost all system behaviors
are triggered by three types of external stimuli
known as trigger, timing, and interrupt events
(Wang, 2009a, 2009b; Wang et al., 2013). So
do the brain and the mental processes where
eye movements play a crucial cognitive role
as the perceptual driver in attention, memory
access, and thinking.
This paper presents the cognitive mecha-
nisms of eyes as both the visual sensory of the
brain and the perceptual browser of the mind. It
focuses on the internal and perceptual cognitive
mechanisms of eyes. A fundamental hypothesis
on the perceptual browsing mechanism of eyes
is introduced, which is supported by evidences
and experiments on eye movements, thinking,
and sleeping in neuropsychology and cognitive
informatics. In the remainder of this paper, the
neurophysiological and cognitive foundations
of eyes are described in Section 2 based on ana-
tomic structures of eyes, visual nervous, motor
control muscles, and the sensory receptors of
vision. On the basis of the anatomic and cogni-
tive models, the triple pathways of eyes known
as those of sensory, motor, and tracking are
analyzed. Cognitive mechanisms of eye brows-
ing functions are elaborated in Section 3 based
on the theory of abstract conditioning between
eye movements and internal memory access.
The cognitive functions of eye movements are
contrasted in the conscious and unconscious
modes. The internal browsing mechanism of
eyes via abstract conditioning explains a wide
range of cognitive roles of eye movements in
thinking, perception, attention, consciousness,
learning, memorization, and inference.
2. THE COGNITIVE MODEL
OF EYE MOVEMENT
The cognitive functions of eyes play a central
role in explaining the brain and human senses.
This section explores the neurophysiologic and
cognitive foundations of eyes. The functional
model of eye movement and the triple pathways
of eyes are analyzed based on the anatomic
model of eye control.
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
38 International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014
2.1. The Anatomic Model
of Eye Movements
The anatomic structure of eyes, the visual
nervous, and the sensory receptor of vision are
reviewed in (Marieb, 1992; Woolsey et al., 2008;
Carter et al., 2009). This subsection explains the
cognitive mechanisms of eye movements and
the formation of the reflexive conditionings of
the browser mechanism between eye movement
and internal memory access and perception.
The movement, focus, and shape-adjust-
ment of the eyeball are controlled by six-group
extrinsic muscles as illustrated in Figure 1.
The extrinsic muscles are known as the supe-
rior, inferior, lateral, medial rectus muscles,
as well as the superior and inferior oblique
muscles, as summarized in Table 1. The eye
motor muscle innervations are controlled by
the cranial nerves such as those of oculomotor
(III) as well as Trochlear (IV) and Trochlear
(IV) (Marieb, 1992).
It is noteworthy that the conventional model
for cranial nerve innervations of eye movements
as summarized in Table 1 are focused merely
on motor controls to eye muscles. An important
mechanism of eye movement tracking and status
registration in the feedback track of the cranial
nervous is overlooked in the naturally closed
control loop (Wang, 2003b, 2013).
Definition 1: Eye movement and status track-
ing is a cognitive mechanism where the
feedback information of eye movement
and their real-time status are unconsciously
Figure 1. Illustration of eye muscle control
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014 39
captured and maintained in the conscious
status memory in the cerebellum.
The information of eye movement and
status tracking is processed by the thalamus
and registered in the conscious status memory
in the cerebellum (Wang, 2012b, 2013). The
input-oriented tracking mechanism forms the
foundation for driving the internal percep-
tive engine via the thalamus in thinking and
memory access.
2.2. The Functional Model
of Eye Movements
There are four types of eye movements rec-
ognized in neurophysiology and cognitive
informatics known as saccades, tracking, gaze
(Marieb, 1992; Carter et al., 2009), and internal
browse (Wang, 2003b) as summarized in Table
2. The first three modes of eye movements are
conscious; while the internal browse mode is
subconscious or unconscious.
Definition 2: Saccade is a conscious mode of
eye movement that darts at a still object
in a visual frame where both eyes move
in the same direction.
Saccade is implemented by eye muscle
control signals from the oculomotor nuclei to
maintain a darting on a still object.
Table 1. Cranial nerve innervations to eye muscles
No. Category Muscle Eye movement Cranial nervous
1 Rectus Superior Move up Oculomotor (III)
2 Inferior Move down Oculomotor (III)
3 Medial Move inside Oculomotor (III)
4 Lateral Move outside Abducens (VI)
5 Oblique Superior Move upward outside Trochlear (IV)
6 Inferior Move downwards outside Oculomotor (III)
Table 2. The functional model of eye movements
No. Mode Description Innervations Category
1 Saccade Quickly small and jerky movement on a
still object in a visual frame
All rectus
and oblique
movements
External and conscious
2 Tracking Pursue a moving object in the visual
frame where both eyes move in the
same direction
(Tracking may also be implemented by
saccade with head moving)
All rectus
and oblique
movements
External and conscious
3 Gaze
(Vergence)
Gaze at an approaching or leaving
object where both eyes move in
opposite direction
Medial or lateral
rectus movement,
respectively
External and conscious
4 Internal
browse
Browse internal memories and access
to attention, thinking threads, and
perception processes particularly REM
during sleep
All rectus
and oblique
movements
Internal and
subconscious or
unconscious
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
40 International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014
Definition 3: Tracking is a conscious mode
of eye movement that smoothly pursues a
moving object in a visual frame in which
both eyes move in the same direction.
Tracing is implemented by eye muscle
control signals from the oculomotor nuclei to
maintain a smooth pursuit on a moving object.
Definition 4: Gaze is a conscious mode of eye
movement that forms vergence on a moving
object in a visual frame approaching closer
to or going away from the eye in which both
eyes move in opposite directions.
Gaze is implemented by eye muscle con-
trol signals from the oculomotor nuclei and
memories such as the SBM in the visual cortex
of the occipital lobe, the eye field in the STM
in frontal lobes, ABM in the parental lobe, and
CSM in the cerebellum.
The three modes aforementioned are basic
eye movements for embodying the external
sensory of vision that are identified in classical
literature (Marieb, 1992; Carter et al., 2009).
However, there is the forth mode of eye move-
ment called internal browse, which is identified
by Wang in 2003 (Wang, 2003b).
Definition 5: The internal browse is a subcon-
scious/unconscious mode of eye movement
that drives internal access to memories and
the cognitive knowledge base in order to
support thinking, perception, memoriza-
tion, and attentions in which the eyes move
in the same direction.
Internal browse of eyes may happen in
two situations: a) Subconscious during awake
when eyes are open, close, or blinking for
thinking, memory access, and attention switch-
ing, respectively; b) Unconscious during sleep
when eyes are closed such as in REM sleep for
internal memory browse (Aserinsky and Kleit-
man, 1953; Tsoukalas, 2012). The fourth mode
of eye movements will be formally analyzed
in Section 3.
2.3. Neuroinformation Pathways
of Eye Control and Tracking
The sensory and control pathways between the
eyes and the brain can be classified into three
categories: a) the sensory pathway; b) the mo-
tor pathway; and c) the tracking pathway. The
third pathway, the tracking pathway, for eye
position and status information as described
in Definition 1 is identified by Wang (2003b).
The triple mechanism of eye sensory and
control is the foundation to explain the cognitive
model of eye movements as shown in Figure
2. In Figure 2, the visual sensory pathways
are represented by normal solid arrows; the
eye motor control pathways are represented
by dash arrows to the output direction; and the
eye tracking information pathways are denoted
by dotted dash arrows to the input direction. In
the visual information pathway, the optic nerves
from both eyes reach the occipital lobes of the
cerebral cortex, which integrates the images
from each eye in order to form a binocular vision.
The sensory information of vision gener-
ated by the vision receptors is retained and
processed in various memories. According to
neuroinformatics (Bower, 1998; Dayan & Ab-
bott, 2001; Wang, 2007b, 2013, 2014; Wang &
Fariello, 2012), human memory can be classi-
fied into five types according to its functions
and contents known as the Long-Term Memory
(LTM), Short-Term Memory (STM), Sensory
Buffer Memory (SBM), Action Buffer Memory
(ABM), and Conscious Status Memory (CSM).
Definition 6: The Cognitive Memory Model
(CMM) is a logical partition of the human
memory system in five types according to
functions and contents of memories, i.e.:
CMM LTM
STM
SBM
(
||
||
||
||
)
ABM
CSM
(1)
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014 41
where ABM and CSM are identified in (Wang,
2013; Wang & Wang, 2006) in the brain that
logically model the memory system of humans
based on neurophysiological observations.
The LTM memory may be further classi-
fied into abstract memories (AM) and visual
memories (VM) according to the contents of
memory (Wang, 2013) in the occipital lobe
as follows:
LTM AM VM
CKB
EM PM
||
|| ( || )
= // including SM
(2)
where AM encompasses conceptual and se-
mantic memories embodied by a cognitive
knowledge base (CKB) (Wang, 2007a, 2009d)
and the semantic memory (SM); VM accommo-
dates the episodic memory (EM) and procedural
memory (PM) (Sternberg, 1998; Carter, et al,
2009). However, the final form of VM including
both EM and PM in LTM is unified in symbolic
and semantic forms according to Pinel (1997),
Sternberg (1998), and Wang (2009c).
In the sensory pathway of eyes, the image
frame information is retained in SBM in the
visual cortex of the occipital lobe with sup-
port of STM in the frontal lobe, and CSM in
the cerebellum (Marieb, 1992; Wang, 2013).
In the motor pathway of eyes, it is usually
perceived in literature that the control signals
to the eyes are buffered in the SBM in the
somatosensory cortex of the parietal lobe and
the ABM in the motor cortex of the parietal
lobe, respectively. However, it is noteworthy
that the major control to eyes is directly from
the thalamus rather than the motor cortex as
shown in Table 3. The motor control pathway
between eyes and thalami is supported by pairs
of sensory and control loops in SBM in the
somatosensory cortex of the parietal lobe and
ABM in the motor cortex of the parietal lobe.
In the tracking pathway of eyes, the
position and feedback status information is
directly transmitted into the central controller
of the thalamus and CSM in cerebellum via the
feedback tracks of the oculomotor, trochlear,
and abducens nervous as shown in Table 3.
The CSM maintains a tracking of the status of
eyes in the cerebellum, which will be used to
detect the differential of current and last status
of eyes in order to predict the trend and extent
of eye movement, as well as to establish the
relationship between the last eye status and the
captured vision information.
The direct and intensive interactions be-
tween eyes and thalami imply an extremely
important autonomous event-driven reaction
mechanism in which eyes plays an active role to
trigger and drive the thinking engine for proper
conscious, subconscious, or unconscious mental
activities. Such mental activities encompass
internal memory and CKB access, attention,
perception, and thinking, which are triggered
Figure 2. The triple pathways of eye sensory, control, and tracking
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
42 International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014
by the tracking information of eye movements
based on previously conditioned event-triggered
cognitive behaviors.
3. THE COGNITIVE
MECHANISM OF EYES
FOR INTERNAL MEMORY
ACCESS AND PERCEPTION
According to the observations on eye move-
ments as described in Section 2, a fundamental
finding on the cognitive mechanisms of eyes is
that the eyes are not only a sensor for vision, but
also a browser for internal memory and CKB.
The finding on the eye tracking pathway reveals
that the eyes are the driver of the perceptual
thinking engine of the brain. In other words,
the eyes work as a bi-functional organ: the vi-
sual sensor (photoreceptor) and the perceptual
browser of the mind. The following subsec-
tions explain the cognitive mechanisms of the
implied roles of eyes as the internal browser in
fundamental human cognitive abilities such as
memory access, attention, perception, thinking,
learning, and inference in the unconscious, sub-
conscious, and conscious modes, respectively.
3.1. Abstract Conditioning
between Eye Movements and
Internal Memory Access
It is recognized that the thalamus is the central
processing unit (CPU) of the brain as the think-
ing engine (Wang, 2012b; Wang & Wang, 2006).
The thalamus is stimulated by the event-driven
mechanism where no responding activity is
generated when there is no external event or
internal motivation. Therefore, the thalamus is
closely coupled with eyes, the only moveable
organ of the brain, to embody the event-driven
mechanism of perception and thinking.
Corollary 1: The brain is a hybrid event-and-
motivation-driven system. Although the
motivation-driven mechanism is active and
autonomous, the event-driven mechanism
is passive and reflexive. Therefore, the
brain is not fully autonomous even when
it is conscious.
Table 3. Cranial nerve system innervations
No. Cranial nerve Function
Sensory Motor Parasympathetic
I Olfactory Smell - -
II Optic Vision - -
III Oculomotor - Eyes Iris for visual focusing
IV Trochlear - Eyes -
V Trigeminal General
sensation
Face & chewing -
VI Abducens - Eyes -
VII Facial Taste Facial expression Tear glands
VIII Vestibulocochlear H ea r in g &
balance
- -
IX Glossopharyngeal Taste Tongue (swallowing) Parotid salivary gland
X Vagus Taste Thorax & abdomen Heart rate, br eat hin g, and
digestive systems
XI Accessory - N e ck & s pi n al
accessory
-
XII Hypoglossal - Tongue (speech) -
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014 43
Although an individual brain is capable
to do a great variety of things, it needs to
be driven by external stimuli and/or internal
motivations via the attention and conscious
system (Wang, 2012c; Wang et al., 2013). Eye
movements are conditioned as the event for
driving the thinking engine particularly when
the body is still and there is no other stimulus.
The abstract conditioning mechanism explains
that although the first movement of eyes in the
beginning of a day is initiated by the brain, all
succeeding consequences of the thinking engine
during the day are driven by consequences of
the succeeding events. Just like a person plays
a game where, although the first initiative for
the game is fully autonomous, once the game
is started, the player’s brain is driven by the
unpredictable sequence of events in the game.
The turning of cause-effect in this case is very
interesting for explaining the autonomous and
event-driven facets of human brains. Except the
initial action as autonomous where the play’s
brain is the cause and the game is the effect, all
other causes and effects in the consequences are
immediately inversed where the passive event-
driven mechanism dominates the player’s brain.
This is the theoretical foundation for explaining
the game and web surfing addictions, as well
as other forms of additions.
Definition 7: Abstract conditioning is a re-
flexive mental phenomenon that links a
cognitive process or mental function to
a pre-designated or trained process cor-
responding to a certain event or stimulus.
The abstract conditioning between eye
movements and attentions, memory access,
perception, and thinking is typical cases of
abstract conditioning.
Definition 8: The internal browser function
of eye movements is formed by abstract
conditioning that links the internal memory
accesses activities during perception and
thinking to subconscious eye movements
and their differentiations (changes of status
and traces).
As a result, subconscious eye movements
become a driving force of internal memory al-
locations and accesses; in parallel, conscious eye
movements forms the indication of an external
attention. A set of typical abstract conditionings
of eye movements to internal memory accesses
for thinking is summarized in Table 4, where
a certain eye movement corresponds to pre-
conditioned cognitive functions. The internal
browse of eyes can be categorized as upward
and downward browses according to its target
of memories in different cortical lobes. The
upward browse of eyes is usually conditioned
to access LTM for historical events and acquired
knowledge of LTM in the temporal lobe; while
the downward browse is usually conditioned
to access STM for current events and thinking
threads of STM in the frontal lobe.
The abstract conditioning theory for the
internal browse mechanisms of eyes reveals
that the feedback stimuli of eye movement in-
nervations controlled by the cranial nerves of
oculomotor (III), Trochlear (IV), and Trochlear
(IV) generates corresponding stimuli to the
thalamus, especially when eyes are closed
(Mode 5) or during sleep (Mode 6) according
to Table 4. These proprioceptor afferents form
a driving force for internal memory access
and browsing during attention, consciousness,
memory, perception, and thinking.
According to cognitive informatics,
neurophysiology, and the CMM model, all
memories in the brain are closely connected
to the thalamus as the thinking engine (Wilson
& Keil, 1999; Wang, 207b, 2013). However,
the access mechanism to specific memory via
content-sensitive addressing is subconsciously
driven by eye movements, which generates an
appropriate stimulus to the thalamus. This is
the cognitive principle for explaining that why
internal memory access is always accompanied
by subconscious eye movement.
3.2. Unconscious Eye
Movement in Memory Access
and LTM Establishment
The internal browse mechanism of eyes is
a necessary function for thinking, memory
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
44 International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014
access, and attention control where eyes are
open, blinking, or closed, even during sleep-
ing. The major form of eye movements is
subconscious or unconscious in the cases such
as autonomous cognitive processes of think-
ing, perception, learning, memory retrieval,
long-term memory establishment, and sleep.
Subconscious skills are also performed in those
of conscious eye movements such as saccades,
tracking (smooth pursuit), and gaze (vergence)
as described in Table 2.
The Rapid Eye Movement (REM) during
sleep is an instance of the sixth mode of con-
ditioned eye movements in Table 4. REM is a
key evidence of unconscious and subconscious
memory access. It was originally observed
by Loomis and his colleagues in mid-1930s
(Loomis et al., 1937). Aserinsky and Kleitman
studied REM in sleep in the early 1950s (Ase-
rinsky and Kleitman, 1953). According to the
phenomenon of eye movement, sleep is gener-
ally divided into two types known as REM and
non-REM (NREM). The latter can be further
divided into four stages, i.e., Stages I through IV.
REM and NREM occur in alternating cycles for
4 to 6 times during human sleeping in a night,
each lasting approximately 90 to 100 minutes.
In general, REM sleep accounts for 10-25%
of sleep time as shown in Figure 3. However,
NREM sleep takes 75-90% of sleep time where
about 3-5% in Stage I, 50-60% in Stage II,
and 10-20% in Stages III and IV, respectively
(Hobson, 2009; Tsoukalas, 2012).
REM sleep is characterized by three
physiologic signs: a) Rapid eye movements; b)
Muscle atonia; and c) Electroencephalograph
(EEG) desynchronization (Aserinsky and Kleit-
man, 1953; Smith, 1993; Sternberg, 1998). A
fundamental question on REM sleep is why eyes
move unconsciously during sleeping. It can be
rationally explained by the Mode 6 mechanism
of the conditioned internal browse as modeled
in Table 4.
Hypothesis 1: The cognitive need for eye
movement during REM sleep is for LTM
establishment.
The following cognitive experiment is
designed to prove Hypothesis 1.
Experiment 1: Eye movement during REM
sleep: It is observed that no LTM may be
established if REM sleep is prevented by
interruptions or interferences such as in
the cases of long-distance fly, drunk, and
over-time work during night.
Corollary 2: The 24-hour law of LTM estab-
lishment: LTM is established during night,
particularly during the REM sleep period.
Table 4. Typical abstract conditionings of eye movements for internal memory access
Category Mode Eye movement
(in the same
direction)
Memory access Cognitive function
Conditioned
internal
browse
1 Up LTM Memory recall
2 Down STM Deep and complex thinking
3 Blink STM or LTM Refresh or switch thinking threads
4 Daydreaming STM or LTM Random access (thinking)
5 Movement when
closed and awake
a) STM
b) LTM
a) Deep and complex thinking
b) LTM of CKB, semantic, visual,
and episode
6 REM during sleep STM-LTM LTM establishment
External
visual sense
7 Saccade / tracking
/ gaze
Sematosensory / the
occipital lobe / STM
Visual sensing and attention
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014 45
Therefore, without a quality REM sleep,
the LTM about the last 24 hours would be
incomplete and even empty.
Corollary 3: LTM establishment is eye-
movement-driven, which enables a con-
tent-sensitive addressing mechanism that
organizes the contents captured in STM
into CKB in LTM.
From a cognitive point of view, REM is an
indicator of dreaming no matter it is observed
consciously or not. REM is closely associated to
visual imagery and information reorganization
during LTM establishment. In other words, an
observed dream during REM sleep is merely
a special case when the dream is interrupted
by an external event when it is consciously
captured. Therefore, dream as an indicator of
LTM establishment is always on-going during
REM sleep.
3.3. Conscious Eye Movement
in Thinking and Perception
Human thinking and inference can be rep-
resented as concurrent perceptual threads in
STM that are mapped in to individual’s CKB
in LTM. The mapping links between STM
and LTM as a set of contingent indexes is a
temporal neural connection between the iden-
tifiers of corresponding concepts in STM and
their detailed objects and attributes in LTM.
The mapping and indirect addressing must be
carried out by a certain physiological organ that
is dynamic, animated, and with an intensive ac-
cessibility to the central nervous systems (Wang,
2012b, 2013; Wang et al., 2006), particularly
the thalamus in the center of the brain, STM
in the frontal lobe, LTM in the temporal lobe,
and the occipital lobe (visual memory). It also
links to SBM in the parietal lobe and ABM in
Figure 3. EEG image of REM sleep (Courtesy of Sandman on Wikipedia)
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
46 International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014
the frontal lobe when a sensory stimulus and/
or a motor action are involved.
The detection of both external stimuli and
internal motivations can be formally described
by the following models in human attention
system (Wang et al., 2013).
Definition 9: The stimulus capture mechanism,
SC, for external attention is a differential
operation on SBM where a change of status
is identified as a new stimulus S against
that of CSM, i.e.:
SC f d
dt SBM S
f SBM CSM S
s
s
: ( )
: ( )
→
= ⊕ →
(3)
where
⊕
represents the exclusive-or operator
on a discrete set of events registered in SBM
and CSM.
Definition 10: The motivation detection
mechanism, MD, for internal attention is
a differential operation on STM where a
change of status is identified as a motiva-
tion M against that of CSM, i.e.:
MD f d
dt STM M
f STM CSM M
m
m
: ( )
: ( )
→
= ⊕ →
(4)
It is noteworthy that the stimulus capture
and motivation detection processes as mod-
eled in Definitions 9 and 10 are dependent on
the comparison between SBM/CSM or STM/
CSM. The differential detection and multiple
memory access to both external stimuli and
internal motivations are mentally embodied by
eye movement. Attention as a perceptive engine
of the brain driven by eye movement dispatches
the mind on an external object, event, and/or
an internal thread of thought. The switch of
thinking threads among inputs, outputs, and
internal inference processes is also driven by
eye movement.
Recent studies in cognitive informatics
recognize perceptivity as the sixth sense that
serves the brain as the thinking engine and the
kernel of the natural intelligence. Perceptivity
implements self consciousness inside the ab-
stract memories of the brain via subconscious
eye movement. Almost all cognitive life func-
tions rely on perceptivity that virtually sees
acquired visual images stored in the memories
of the brain without any sensory input, and
abstractly accesses acquired knowledge and
information elicited by the movement of eyes.
Hypothesis 2: The eyes are the intentional
driver of the thinking engine embodied
by the thalamus.
This is the cognitive mechanism of eyes
in the conscious mode.
Corollary 4: Conscious thinking and perception
are driven by the internal browsing mode
of eye movement.
The following cognitive experiment is
designed to prove Corollary 4.
Experiment 2: Irreplaceable eye movement
during thinking: In any case, eyeballs move
rapidly during thinking. Even when eyes
are closed, eyeballs still move in order
to conduct memory access and internal
knowledge browsing.
Corollary 5: No thinking may be properly con-
ducted without the conditioned browsing
of the eyes to internal memories.
The following cognitive experiment is
designed to prove Corollary 5.
Experiment 3: Constrained eye movement
during thinking: If the eyeballs are forced
not to move, one cannot read and think
because necessary memory access means
for thinking is interfered or interrupted.
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014 47
Corollary 6: The cognitive mechanism of
time-sharing multiplex of the eyes is
implemented via eye blinking.
When eyes are open, they mainly function
for vision sensory consciously and subcon-
sciously. However, eyes function as the brows-
ing driver for memory access subconsciously
when they move during blinking. The follow-
ing cognitive experiment is designed to prove
Corollary 6.
Experiment 4: The cognitive mechanism of
eye blinking: The eye blink frequency, or
the ratio of closed eye browsing period,
indicates the intensity of thinking. During
thinking the frequency of the eye blinking
rate will be significantly higher when the
problem or topic is more complicated. In
an extreme case, one may need to close
the eyes from time to time when one is in
deep thinking such as in an examination.
3.4. Conscious Eye Movement in
Attention and Consciousness
According to the Unified Framework of Selec-
tive Attention Model (USAM) (Wang et al.,
2013), the brain is dominantly driven by external
stimuli as a parallel real-time multi-thread sys-
tem with a set of conscious and subconscious
life functions (Wang, 2009b, 2012a; Wang &
Wang, 2006; Wang et al., 2006). On the basis
of USAM and CMM, the role of eye movement
in attention and consciousness can be rationally
explained.
The autonomic switching of human atten-
tions is driven and dominated by eye movement
(Wang et al., 2013). So does consciousness
because attention is the front-end of it (Wang,
2012c). Therefore, eye movement is a primary
sign of life.
Corollary 7: Human mental process is triggered
by the primary sensory dominant by eye
movement detected by the attention and
consciousness systems of the brain.
Experiment 5: Eye movement is irreplaceable
for both conscious and unconscious activi-
ties. For instance, eyes of an individual are
continuous moving during walking, even
in the case that eyes are closed or covered.
Experiment 6: In medical practice, the com-
plete loss of eye movement or the control
of the pupil, is recognized as an evidence
of brain death, because it indicates the
cease of not only the subconscious visual
sensory, but also the conscious perceptual
browser of the mind.
Experiment 7: Lack of eye movement is com-
monly recognized as an indicator of human
fatigue due to decayed conscious attention.
Therefore, closed eyes or lack of eye move-
ment are treated as an overtired brain status
in cognitive engineering such as in those of
automobile driving or machine operation.
Eyes are the unique organ that not only
have such wide range of connections and ac-
cessibility to the brain, but also possesses such
dynamic power interacting with the mind. The
implied cognitive functions of eyes as the inter-
nal browser explain how eyes access internal
cognitive knowledge base in thinking and
perception, as well as other high-level mental
processes such as attention, consciousness,
learning, memorization, learning, and inference.
4. CONCLUSION
A set of cognitive theories on the mechanisms
of eye tracking pathway and the abstract condi-
tionings associating eye movements to internal
memory access has been introduced. This work
has revealed that eyes are not only a sensor for
vision, but also a browser for internal memory,
knowledge, perception, and thinking, which
embodies a crucial role for accessing the internal
memory and the cognitive knowledge base.
A set of findings have been presented on the
dual mechanisms of the eyes as both the external
sensor of the brain and the internal browser of the
mind. a) There are three pathways between eyes
and the brain known as the sensory, motor, and
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
48 International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014
tracking pathways; b) Conscious thinking and
perception are driven by the internal browsing
mode of eye movement. No thinking may be
properly conducted without internal browsing
of the eyes; c) Human mental process is trig-
gered by the primary sensory dominant by eye
movement detected by the attention and con-
sciousness systems of the brain; d) Long-term
memory establishment is eye-movement-driven
by the content-sensitive addressing mechanism;
and e) Long-term memory is established dur-
ing night, particularly during the REM sleep
period according to the 24-hour law of long-
term memory establishment; The reveal of the
internal perceptual mechanism of the eyes is
not only theoretically significant to identify the
physiological organ of the thinking engine of
the brain, but also practically useful to explain
a wide range of cognitive mechanism of the
brain and mind in thinking, perception, atten-
tion, consciousness, learning, memorization,
and inference.
ACKNOWLEDGMENT
This work is supported in part by a discovery
fund granted by the Natural Sciences and Engi-
neering Research Council of Canada (NSERC).
The author would like to thank the anonymous
reviewers for their valuable comments on the
previous version of this paper.
REFERENCES
Aserinsky, E., & Kleitman, N. (1953). Regularly oc-
curring periods of eye motility, and concomitant phe-
nomena, during sleep. [PubMed]. Science, 118(3062),
273–274. doi:10.1126/science.118.3062.273
PMID:13089671
Bower, J. M. (1998). Computational neuroscience.
New York, NY: Plenum Press; doi:10.1007/978-1-
4615-4831-7
Carter, R., Aldridge, S., Page, M., & Parker, S.
(2009). Mapping the mind. Berkeley, CA: Univ. of
California Press.
Coren, S. L. M. Ward, & Enns, J. T. (1993), Sensa-
tion and perception (4th ed.). Harcourt Brace College
Publishers.
Dayan, P., & Abbott, L. E. (2001). Theoretical neuro-
science: Computational and mathematical modeling
of neural systems. The MIT Press.
Hobson, J. A. (2009). REM sleep and dreaming:
Towards a theory of protoconsciousness. [PubMed].
National Review, 10(11), 803–813. PMID:19794431
Hubel, D., & Wiesel, T. N. (1959). Receptive
fields of single neurons in the Cat’s visual cortex.
[PubMed]. The Journal of Physiology, 148, 574–591.
PMID:14403679
Hubel, D., & Wiesel, T. N. (1979). Brain mechanisms
of vision. [PubMed]. Scientific American, 82, 84–97.
PMID:91195
Loomis, A. I., Harvey, E. N., & Hobart, G. A. I.
(1937). Cerebral states during sleep, as studied by
human brain potentials. Journal of Experimental
Psychology, 21(2), 127–144. doi:10.1037/h0057431
Marieb, E. N. (1992). Human anatomy and physiol-
ogy (2nd ed.). Redwood City, CA: The Benjamin/
Cummings Publishing Co., Inc.
Pinel, J. P. J. (1997). Biopsychology (3rd ed.). Need-
ham Heights, MA: Allyn and Bacon.
Reisberg, D. (2001). Cognition, exploring the science
of the mind (2nd ed.). W.W. Norton & Company, Inc.
Smith, R. E. (1993). Psychology. New York, NY:
West Publishing Co.
Sternberg, R. J. (1998). In search of the human mind
(2nd ed.). Orlando, FL: Harcourt Brace & Co.
Tsoukalas, I. (2012). The origin of REM Sleep: A
hypothesis. Dreaming, 22(4), 253–283. doi:10.1037/
a0030790
Wang, Y. (2002), Keynote: On cognitive informat-
ics. In Proc. IEEE 1st International Conference
on Cognitive Informatics (ICCI’02), Calgary,
Canada (pp. 34-42). IEEE CS Press. doi:10.1109/
COGINF.2002.1039280
Wang, Y. (2003a). On cognitive informatics. Brain
and Mind: A Transdisciplinary Journal of Neurosci-
ence and Neurophilosophy, 4(2), 151-167.
Wang, Y. (2003b). Keynote. On cognitive mecha-
nisms of the eyes: The sensor vs. the browser of
the brain. In Proceedings of the 2nd IEEE Inter-
national Conference on Cognitive Informatics
(ICCI’03). London, UK: IEEE CS Press. doi:10.1109/
COGINF.2003.1225984
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014 49
Wang, Y. (2005), Cognitive informatics and internal
signal processing in the brain. In Proc. 26th Annual
Meeting of the Canadian and Applied Industrial
Mathematics Society (CAIMS’05), Univ. of Mani-
toba, Winnipeg (pp. 20-21).
Wang, Y. (2006), Keynote: Cognitive informat-
ics - towards the future generation computers
that think and feel, Proc. 5th IEEE International
Conference on Cognitive Informatics (ICCI’06),
Beijing, China, IEEE CS Press, July, pp. 3-7.
doi:doi:10.1109/COGINF.2006.365666 doi:10.1109/
COGINF.2006.365666
Wang, Y. (2007a). The Theoretical Framework of
Cognitive Informatics. International Journal of
Cognitive Informatics and Natural Intelligence, 1(1),
1–27. doi:10.4018/jcini.2007010101
Wang, Y. (2007b). The OAR model of neural infor-
matics for internal knowledge representation in the
brain. International Journal of Cognitive Informatics
and Natural Intelligence, 1(3), 64–75. doi:10.4018/
jcini.2007070105
Wang, Y. (2008). On contemporary denotational
mathematics for computational intelligence. Transac-
tions of Computational Science, 2, 6–29.
Wang, Y. (2009a). On abstract intelligence: Toward a
unified theory of natural, artificial, machinable, and
computational intelligence. International Journal of
Software Science and Computational Intelligence,
1(1), 1–17. doi:10.4018/jssci.2009010101
Wang, Y. (2009b). On cognitive computing. Inter-
national Journal of Software Science and Com-
putational Intelligence, 1(3), 1–15. doi:10.4018/
jssci.2009070101
Wang, Y. (2009c). On visual semantic algebra
(VSA): A denotational mathematical structure for
modeling and manipulating visual objects and pat-
terns. International Journal of Software Science and
Computational Intelligence, 1(4), 1–15. doi:10.4018/
jssci.2009062501
Wang, Y. (2009d). Toward a formal knowledge
system theory and its cognitive informatics foun-
dations. Transactions of Computational Science,
Springer, 5, 1–19.
Wang, Y. (2009e). The cognitive informatics theory
and mathematical models of visual information
processing in the brain. International Journal of
Cognitive Informatics and Natural Intelligence, 3(3),
1–11. doi:10.4018/jcini.2009070101
Wang, Y. (2010a). Cognitive Robots: A Reference
Model towards Intelligent Authentication. IEEE
Robotics and Automation, 17(4), 54–62. doi:10.1109/
MRA.2010.938842
Wang, Y. (2012a). Toward a cognitive behavioral
reference model of artificial brains. Journal of Com-
putational and Theoretical Nanoscience, 9(2),
178–188. doi:10.1166/jctn.2012.2011
Wang, Y. (2012b). On abstract intelligence and brain
informatics: Mapping cognitive functions of the brain
onto its neural structures. International Journal of
Cognitive Informatics and Natural Intelligence, 6(4),
54–80. doi:10.4018/jcini.2012100103
Wang, Y. (2012c). The cognitive mechanisms and for-
mal models of consciousness. International Journal
of Cognitive Informatics and Natural Intelligence,
6(2), 23–40. doi:10.4018/jcini.2012040102
Wang, Y. (2012d). On denotational mathematics
foundations for the next generation of computers:
Cognitive computers for knowledge processing.
Journal of Advanced Mathematics and Applications,
1(1), 118–129. doi:10.1166/jama.2012.1009
Wang, Y. (2013). Neuroinformatics models of hu-
man memory: Mapping the cognitive functions of
memory onto neurophysiological structures of the
brain. International Journal of Cognitive Informatics
and Natural Intelligence, 7(1), 98–122. doi:10.4018/
jcini.2013010105
Wang, Y. (2014), Keynote: Basic theories for neuro-
informatics and neurocomputing. In Proceedings of
2014 International Conference on Neural Networks
and Fuzzy Systems (ICNN-FS’14), Venice, Italy(pp.
14-15).
Wang, Y., & Fariello, G. (2012). On neuroinfor-
matics: Mathematical models of neuroscience and
neurocomputing. Journal of Advanced Mathemat-
ics and Applications, 1(2), 206–217. doi:10.1166/
jama.2012.1015
Wang, Y., Kinsner, W., Anderson, J. A., Zhang, D.,
Yao, Y., & Sheu, P. et al. (2009b). A doctrine of
cognitive informatics. Fundamenta Informaticae,
90(3), 203–228.
Wang, Y., Kinsner, W., & Zhang, D. (2009a). Con-
temporary cybernetics and its faces of cognitive
informatics and computational intelligence. IEEE
Trans. on System, Man, and Cybernetics. Part B,
39(4), 823–833.
Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
50 International Journal of Cognitive Informatics and Natural Intelligence, 8(1), 36-50, January-March 2014
Wang, Y., Patel, S., & Patel, D. (2013). The cogni-
tive process and formal models of human attentions.
International Journal of Software Science and Com-
putational Intelligence, 5(1), 32–50. doi:10.4018/
ijssci.2013010103
Wang, Y., & Wang, Y. (2006). Cognitive informat-
ics models of the brain. IEEE Transactions on
Systems, Man and Cybernetics. Part C, Applica-
tions and Reviews, 36(2), 203–207. doi:10.1109/
TSMCC.2006.871151
Wang, Y., Wang, Y., Patel, S., & Patel, D. (2006). A
layered reference model of the brain (LRMB) [C]
[PubMed]. IEEE Transactions on Systems, Man,
and Cybernetics, 36(2), 124–133. doi:10.1109/
TSMCC.2006.871126 PMID:16602600
Wilson, R. A., & Keil, F. C. (Eds.). (1999). The MIT
encyclopedia of the cognitive sciences. Cambridge,
MA: The MIT Press.
Woolsey, T. A., Hanaway, J., & Gado, M. H. (2008).
The brain atlas: A visual guide to the human central
nervous system (3rd ed.). New, York, NY: Wiley.
Yingxu Wang is professor of cognitive informatics and software science, President of Interna-
tional Institute of Cognitive Informatics and Cognitive Computing (ICIC, www.ucalgary.ca/icic/),
Director of Laboratory for Cognitive Informatics and Cognitive Computing, and Laboratory for
Denotational Mathematics and Software Science at the University of Calgary. He is a Fellow
of WIF (UK), a Fellow of ICIC, a P.Eng of Canada, and a Senior Member of IEEE and ACM.
He received a PhD in Computer Science from the Nottingham Trent University, UK, and a BSc
in Electrical Engineering from Shanghai Tiedao University. He has industrial experience since
1972 and has been a full professor since 1994. He was a visiting professor on sabbatical leaves
at Oxford University (1995), Stanford University (2008), University of California, Berkeley
(2008), and MIT (2012), respectively. He is the founder and steering committee chair of the
annual IEEE International Conference on Cognitive Informatics and Cognitive Computing
(ICCI*CC). He is founding Editor-in-Chief of International Journal of Cognitive Informatics
and Natural Intelligence (IJCINI), founding Editor-in-Chief of International Journal of Software
Science and Computational Intelligence (IJSSCI), Associate Editor of IEEE Trans on System,
Man, and Cybernetics - Systems, and Editor-in-Chief of Journal of Advanced Mathematics and
Applications. Dr. Wang is the initiator of a few cutting-edge research fields such as cognitive
informatics (CI, the theoretical framework of CI, neuroinformatics, the logical model of the
brain (LMB), the layered reference model of the brain (LRMB), the cognitive model of brain
informatics (CMBI), the mathematical model of consciousness, and the cognitive learning engine
(CLE)); abstract intelligence; cognitive computing (cognitive computers, cognitive robots, cog-
nitive agents, and the cognitive Internet); denotational mathematics (concept algebra, semantic
algebra, behavioral process algebra, system algebra, inference algebra, granular algebra, and
visual semantic algebra); software science (unified mathematical models and laws of software,
cognitive complexity of software, automatic code generators, the coordinative work organiza-
tion theory, and built-in tests (BITs)); basic studies in cognitive linguistics (such as the cognitive
linguistic framework of languages, semantic algebra, formal semantics of languages, deductive
grammar of English, and the cognitive complexity of text comprehension). He has published
400+ peer reviewed papers and 28 books in cognitive informatics, cognitive computing, software
science, denotational mathematics, and computational intelligence. He is the recipient of dozens
international awards on academic leadership, outstanding contributions, research achievement,
best papers, and teaching in the last three decades.
