Ability-based Keyboards for Augmentative and Alternative Communication: Understanding How Individuals’ Movement Patterns Translate to More Efficient Keyboards: Methods to Generate Keyboards Tailored to User-specific Motor Abilities

Abstract

This study presents the evaluation of ability-based methods extended to keyboard generation for alternative communication in people with dexterity impairments due to motor disabilities. Our approach characterizes user-specific cursor control abilities from a multidirectional point-select task to configure letters on a virtual keyboard based on estimated time, distance, and direction of movement. These methods were evaluated in three individuals with motor disabilities against a generically optimized keyboard and the ubiquitous QWERTY keyboard. We highlight key observations relating to the heterogeneity of the manifestation of motor disabilities, perceived importance of communication technology, and quantitative improvements in communication performance when characterizing an individual's movement abilities to design personalized AAC interfaces.

Full text

Ability-based Keyboards for Augmentative and Alternative
Communication: Understanding How Individuals’ Movement
Paerns Translate to More Eicient Keyboards
Methods to Generate Keyboards Tailored to User-specific Motor Abilities
Claire L. Mitchell Gabriel J. Cler Susan K. Fager
Delsys, Inc. and Altec, Inc., Natick, Department of Speech and Hearing
Communication Center of Excellence,
MA, USA Sciences, University of Washington, Madonna Rehabilitation Hospital,
cmitchell@delsys.com Seattle, WA, USA Lincoln, NE, USA
gcler@uw.edu sfager@madonna.org
Paola Contessa Serge H. Roy Gianluca De Luca
Delsys, Inc. and Altec, Inc., Natick, Delsys, Inc. and Altec, Inc., Natick, Delsys, Inc. and Altec, Inc., Natick,
MA, USA MA, USA MA, USA
pcontessa@delsys.com sroy@delsys.com gdeluca@delsys.com
Joshua C. Kline Jennifer M. Vojtech
Delsys, Inc. and Altec, Inc., Natick, Delsys, Inc. and Altec, Inc., Natick,
MA, USA MA, USA
jkline@delsys.com jvojtech@delsys.com
ABSTRACT
This study presents the evaluation of ability-based methods ex-
tended to keyboard generation for alternative communication in
people with dexterity impairments due to motor disabilities. Our
approach characterizes user-specic cursor control abilities from
a multidirectional point-select task to congure letters on a vir-
tual keyboard based on estimated time, distance, and direction of
movement. These methods were evaluated in three individuals
with motor disabilities against a generically optimized keyboard
and the ubiquitous QWERTY keyboard. We highlight key observa-
tions relating to the heterogeneity of the manifestation of motor
disabilities, perceived importance of communication technology,
and quantitative improvements in communication performance
when characterizing an individual’s movement abilities to design
personalized AAC interfaces.
CCS CONCEPTS
• Human-centered computing →
Accessibility; Accessibility de-
sign and evaluation methods; Interaction design; Interaction design
process and methods; User interface design; Human computer in-
teraction (HCI); Interaction devices; Keyboards.
This work is licensed under a Creative Commons Attribution International
4.0 License.
KEYWORDS
Accessible computing, keyboard design, personalization, augmen-
tative and alternative communication
ACM Reference Format:
Claire L. Mitchell, Gabriel J. Cler, Susan K. Fager, Paola Contessa, Serge
H. Roy, Gianluca De Luca, Joshua C. Kline, and Jennifer M. Vojtech. 2022.
Ability-based Keyboards for Augmentative and Alternative Communica-
tion: Understanding How Individuals’ Movement Patterns Translate to
More Ecient Keyboards: Methods to Generate Keyboards Tailored to
User-specic Motor Abilities. In CHI Conference on Human Factors in Com-
puting Systems Extended Abstracts (CHI ’22 Extended Abstracts), April 29–
May 05, 2022, New Orleans, LA, USA. ACM, New York, NY, USA, 7 pages.
https://doi.org/10.1145/3491101.3519845
1 INTRODUCTION
An estimated 5 million Americans have speech impairments that re-
quire the use of augmentative and alternative communication (AAC)
technology to meet their daily communication needs. Many AAC
users rely on computers, tablets, or smartphones to supplement or
replace their oral speech [
1
,
6
,
12
]. However, some individuals with
concomitant motor disabilities—such as those with cerebral palsy,
amyotrophic lateral sclerosis, Parkinson’s disease, and traumatic
brain injury, among others—lack the manual dexterity necessary to
control mainstream AAC technology. Instead of multi-input meth-
ods (e.g., ten-nger typing on a keyboard), these individuals must
rely on alternative, single-input modalities such as eye-tracking,
head-tracking, and switch-scanning to eectively access virtual
interfaces.
CHI ’22 Extended Abstracts, April 29–May 05, 2022, New Orleans, LA, USA
Unfortunately, current alternative communication technologies
© 2022 Copyright held by the owner/author(s).
that incorporate single-input access and keyboard interfaces of-
ACM ISBN 978-1-4503-9156-6/22/04.
https://doi.org/10.1145/3491101.3519845
fer limited versatility and personalization for those with motor

CHI ’22 Extended Abstracts, April 29–May 05, 2022, New Orleans, LA, USA Claire Mitchell et al.
disabilities that result in severe dexterity impairments. For this pop-
ulation, text is primarily generated using a virtual keyboard. Most
interfaces utilize the standard QWERTY layout, which has been
described as “grossly inecient” for single-input use for AAC [
26
].
Moreover, these devices often require extensive setup and mainte-
nance by a caregiver, frequent recalibrations, and manual interface
customizations that burden both caregiver and AAC user, leaving
many individuals poorly served. Users may attempt to compensate
using their own residual motor capabilities or through substantial
reliance on caregiver support and troubleshooting [
7
]. These and
other factors contribute to the nearly one-third of people who aban-
don their prescribed AAC device in favor of less eective dysarthric
speech, gestures, among other communication methods [2, 13].
The communication potential of AAC interfaces has improved
in recent years with the inclusion of basic automation features such
as word prediction, abbreviation expansion, and the ability to save
frequently used words and phrases. However, highly ubiquitous
interface software such as Communicator 5 (Tobii Dynavox; Pitts-
burgh, PA, USA), Proloquo4Text (AssistiveWare; Amsterdam, the
Netherlands), or Verbally (Intuary; San Francisco, CA, USA) still
require time-consuming, manual support from a trained caregiver
to accommodate common changes to personalize a keyboard to
meet the needs of the user. The computational optimization of
a computer interface to t the abilities of the user can improve
performance [
9
,
10
], yet no existing AAC technology can automat-
ically tailor a single-input keyboard interface to an individual’s
unique motor abilities. Existing research has examined the eects
of conguring a keyboard to given population (e.g., individuals
with tremor) [
24
], yet this approach ignores any heterogeneity in
motor abilities within the population [
14
,
22
]. Prior work has also
examined the use of automated models for classifying keypresses
on virtual keyboards [
8
]; however, these models were constructed
for multi-input typing on a QWERTY keyboard. Thus, in this work,
we build on the principle of ability-based design to develop and
evaluate an AAC system that characterizes an individual’s cursor
control abilities to automatically personalize the layout of a key-
board interface for improved communication. We present three use
cases involving people with diverse motor disabilities who use this
AAC system to create messages, comparing performance to the
ubiquitous QWERTY keyboard and a computationally optimized
but non-personalized keyboard.
2 ABILITY-BASED KEYBOARD GENERATION
AND EVALUATION
This work expands on novel methods for ability-based keyboard
generation in individuals without dexterity impairments [
23
], as
well as prior work on generically optimizing a keyboard to improve
communication in individuals with dexterity impairments due to
motor disabilities [
4
]. Here, we examine the feasibility of ability-
based keyboard generation for those with motor disabilities. We
briey summarize (see [
23
]) steps to algorithmically characterize
movement and generate personalized keyboards below.
2.1 Characterizing User Cursor Control
Our methods for characterizing cursor control abilities expand
on Fitts’ Law [
16
,
17
] to estimate movement time and distance
relationships relative to a given target angle rather than the typical
approach of grouping time and distance data irrespective of angle.
Users are presented with a modied version of the multidirectional
Fitts’ Law-based point-select task, wherein a grid of hexagonal
keys are congured on a computer screen and users must navigate
to and select a highlighted key (or “target”). Targets are seeded to
capture two-dimensional (2D) movements across a range of possible
distances and directions. Each target is categorized into one of 16
pre-specied bins spanning (360/16
=
) 22.5 degrees by the empirical
angle of target selection relative to the previous target [
23
]. Within
each bin, a linear regression is performed to yield constants a and
b from Mackenzie’s Shannon formulation of Fitts’ Law (Equation
1) [16, 17]:
 
MT = a + b ∗ ID , wher e I D = log2
D + 1 (1)
W
Specically, the movement time (MT; sec) to select a given target
is calculated as the travel time of the cursor between those clicks;
distance (D; pixels) is calculated as the Euclidean distance between
those click locations. Each distance D is converted to an index of
diculty (ID; bits) with W representing a constant target width
(pixels).
2.2 Generating a Personalized Keyboard
Interface
Personalized keyboards are created by leveraging character-to-
character (digraph) transition occurrences of a selected corpus,
MT (calculated as in Equation 1 with respect to target width and
movement distance), and target selection angle. The user-specic
a and b constants derived from movement data are sampled rel-
ative to the angle of target selection, then applied to Fitts’ Law
(Equation 1) to estimate MT and solve the quadratic assignment
problem (QAP) via the Fast Approximate QAP Algorithm [
27
] of
the GraphMatch function in graspologic (Microsoft, Redmond, WA,
USA) [
3
]. Thus, each keyboard computationally prioritizes common
letter transitions to be in directions easier for the user.
3 SYSTEM EVALUATION
3.1 Overview
Text input performance was evaluated for each participant when
using a keyboard generated via our personalization methods versus
that when using QWERTY or a generic, computationally optimized
keyboard. To enable comparisons with QWERTY, the width of the
space key on the QWERTY keyboard was set equal to all other
keys and positioned to the right of the “M” key (see [23]), as prior
work has shown that text input performance does not signicantly
dier in this conguration [
23
]. Distinct from the personalized key-
boards, which are derived from user-specic a and b constants that
vary according to movement direction, the generically optimized
keyboard was generated using standard directionally static Fitts’
constants of a
=
0.127 sec and b
=
1/4.9 sec/bits across all target se-
lection angles [
21
,
30
]. Digraph transition occurrences for both the
personalized and generically optimized keyboards were calculated
from a standard corpus for evaluating text-entry techniques [
20
].
Computer access was standardized across participants via a vali-
dated access method [
11
] comprising an inertial measurement unit

Ability-based Keyboards for Augmentative and Alternative Communication: Understanding
How Individuals’ Movement Paerns Translate to More Eicient Keyboards CHI ’22 Extended Abstracts, April 29–May 05, 2022, New Orleans, LA, USA
Table 1: Demographics of participants with motor disabilities, including primary communication modalities.
ID Age/sex Diagnosis Communication Method(s) Characteristics
P1 24/F Cerebral Palsy Oral speech, eye tracking, or nose on
touchscreen Involuntary spasms of the lower facial
musculature
P2
P3 40/M
65/M Cerebral Palsy
Parkinson’s Disease (16
yrs post-diagnosis)
Oral speech, ngers on touchscreen
Oral speech Neck rigidity, limited ability to tilt head left
Prominent bilateral resting and action tremor,
diculty coordinating head movement and
eyeblinks
(IMU) for cursor movement (activated via head tilts) and surface
electromyographic (sEMG) sensor for cursor clicks (activated via
winks or blinks).
3.2 Experimental Protocol
Three participants with motor impairments (1 female, 2 male; 24–65
years) completed 1–2 sessions based on availability. All individuals
gave informed consent in compliance with the Madonna Rehabili-
tation Hospital Institutional Review Board (P1 and P2) or Western
Institutional Review Board (P3) through either written consent or
verbal consent witnessed by a communication partner as appropri-
ate. Of the diagnoses (see Table 1), two were congenital (cerebral
palsy; P1, P2) and one was acquired (Parkinson’s disease; P3). All
participants reported that they primarily rely on their dysarthric
speech for face-to-face communication. P1 also reported using eye-
tracking (e.g., for writing school assignments or e-mails) and occa-
sionally using their nose for touchscreen-based navigation.
For each participant, the inertial sensing component of the hy-
brid sEMG/IMU access method was secured to the center of the
forehead—with the y-axis of the IMU parallel to the transverse axes
of the head—and the EMG sensing component applied over the
orbicularis oculi of the preferred eye. Computer access thresholds
were calibrated by instructing each participant to comfortably tilt
their head to the left and right twice, up and down twice, and wink
or hard blink twice [
11
,
28
]. These data were used to tune the 2D
range of cursor movement, as described in detail in [28].
Participants then carried out a movement characterization task
within a 9
×
9 honeycomb grid on a computer monitor with a
1920
×
1080 resolution to capture movement trajectories across
a range of movement distance and directions when using the
sEMG/IMU access method. Within the task, participants were in-
structed to navigate to and select the highlighted target as quickly
and as accurately as possible [
31
]. Participants completed this move-
ment characterization task in 5-minute intervals until 30 minutes
had been reached. Participants were allowed ample break time be-
tween each interval to minimize fatigue. The task terminated early
(i.e., before the 30-minute threshold) if each angle bin contained
5 or more targets and exhibited at least a weak MT-ID correlation
(R2 > 0.09 [
5
]). After completing the task, the 16 angle-specic
Fitts’-based a and b constants were used to generate a personalized
keyboard.
Each participant was then presented with the generically opti-
mized keyboard and their personalized keyboard in a pseudoran-
domized order (i.e., generically optimized rst or personalized rst).
Based on their assigned order, participants spelled out a subset of
balanced phrases from the corpus used for digraph occurrence calcu-
lations [
20
] in alternating blocks. A nal block using the QWERTY
keyboard was performed to serve as a reference for communica-
tion performance (i.e., due to widespread familiarity in using the
QWERTY keyboard for communication) once participants were
familiar with the access method. Blocks comprised 10 minutes of
interaction with one interface—not including participant-initiated
breaks between sentences—followed by a survey to capture their
experiences on a 10-cm visual analog scale [
4
]. The survey included
questions about how fast it felt (where 0 cm is anchored as “Very
slow” and 10 cm as “Very fast”) to use the generically optimized,
personalized, and QWERTY keyboards, as well as how easily they
understand the keyboard layout (0 cm as “Not easily” and 10 cm
as “Very easily”) as an estimate of keyboard familiarity. After each
survey, participants could take a break (10–20 minutes, as neces-
sary) to minimize fatigue. At the end of the study, participants were
surveyed on which of the two new interfaces they preferred (0 cm
as “First Interface” and 10 cm as “Second Interface”).
3.3 Data Analysis
User control was examined in the movement characterization task
via a series of target-to-target path trajectory metrics. Metrics in-
cluded an estimate of target-to-target movement velocity, as well
as standard, path-specic metrics of movement error, path e-
ciency, target re-entry, and selections per target [
19
,
25
]. Keyboard
communication task data were analyzed to assess communication
performance via metrics of speed (words per minute, or WPM),
target selection accuracy (%), information transfer rate (ITR; bits
per minute) [
29
], and throughput (bits/min) [
18
]. Survey responses
were also tabulated to assess user eort and preferences and are
reported from 0 to 10 cm.
4 MOVEMENT CHARACTERIZATION
4.1 Target-to-target Path Trajectories
Participant target-to-target path trajectories resulting from our
movement characterization task successfully show variations in
2D cursor control. These dierences are exemplied in the path
trajectories shown for each participant in Figure 1 and are further
summarized via path trajectory metrics in Table 2.
Each participant traveled at substantially dierent movement ve-
locities when traveling between targets, wherein P3 navigated the
fastest, followed by P2 then P1. P3’s fast cursor movements are fur-
ther characterized by multiple target overshoots, which accounts for

CHI ’22 Extended Abstracts, April 29–May 05, 2022, New Orleans, LA, USA Claire Mitchell et al.
Figure 1: Example path trajectories for accurate (black circle) and inaccurate (black ‘x’) clicks for A) P1, B) P2, and C) P3.
Table 2: Target-to-target Path Trajectory Results
Metric a Participant
P1 P2 P3
Movement Velocity (pixels/sec) 42.8 ± 22.5 64.6 ±23.9 134.8 ±40.7
Movement Error (pixels) 138.0 ± 114.1 172.9 ± 131.4 149.6 ± 112.6
Path Eciency (%) 57.8 ± 26.4 72.9 ± 21.0 39.5 ± 23.9
Target Re-entry 1.1 ± 1.9 0.3 ± 1.0 2.5 ± 3.5
Selections per Target 1.3 ± 0.6 1.6 ± 1.2 1.2 ± 0.7
a Movement characterization data are reported as mean ± standard deviation.
the observed low mean path eciency. P2’s target-to-target move-
ments were highly ecient, indicating precise movement with low
target re-entries, yet large deviations from the target axis through a
high movement error, suggesting a preference toward independent
movements in x and y directions. Despite P2’s propensity to not
overshoot the target, they often required more than one selection
to accurately click the target. In addition to slow cursor movements,
P1’s path trajectory results also demonstrate a slight preference to-
ward coordinated 2D movements with minimal deviation from the
task axis (via movement error) and a moderate mean path eciency.
On average, P1 exhibited approximately one target overshoot and
more than one selection to accurately click the target.
Participants required an average of ve 5-minute intervals to
complete the movement characterization task (P1: 4, P2: 5, P3: 6). No-
tably, all participants exhibited large movement errors (138.0–172.9
pixels) compared to those in the literature for people with motor
disabilities when using computer pointing devices (30.4 pixels [
15
]);
this is likely due to dierences in access method (sEMG/IMU vs.
computer mouse) and experimental methodology (e.g., task instruc-
tions).
4.2 Relationship between Movement Time and
Direction
The movement characterization algorithms eectively captured
dierences in movement time relative to distance and direction.
On average, P3 required 10.2 sec (SD
=
7.9 sec) to travel between
targets, whereas P1 required 7.4 sec (SD
=
5.9 sec) and P2 required
5.3 sec (SD
=
3.5 sec). MT-ID relationships were moderate-to-strong
for each participant when accounting for movement direction (R
2
=
0.35 for P1, R
2 =
0.44 for P2, and R
2 =
0.25 for P3) [
5
]; resulting
Figure 2: Fitts’ constants a (left; sec) and b (middle; sec/bit)
and resulting keyboards (right) for each participant (rows)
relative to the generically optimized keyboard (bottom row).
values for Fitts’ constants a and b are shown for each participant
relative to movement direction (0 to 360 degrees) in Figure 2.

Ability-based Keyboards for Augmentative and Alternative Communication: Understanding
How Individuals’ Movement Paerns Translate to More Eicient Keyboards CHI ’22 Extended Abstracts, April 29–May 05, 2022, New Orleans, LA, USA
Figure 3: Communication performance via WPM (left), accuracy (middle), and ITR (right) across block for participants (rows)
when using generically-optimized (dashed line), personalized (solid line), and QWERTY (dotted line) keyboards.
Using these derived sets of a and b constants, a personalized
keyboard was then generated for each participant (see section 2.2)
to reect their movement abilities. The keyboard for P1 reects the
larger movement times required to navigate diagonally down and
to the left via organizing fewer keys in that region of the keyboard.
The keyboard for P2 is arranged in a cross-like pattern, which re-
ects P2’s large movement errors and preference for moving in
independent (1D) x/y directions (i.e., by allowing for more indepen-
dent x/y movements than would a compact, circular keyboard), and
is also demonstrated via reduced a and b values in each cardinal
direction in Figure 2. The keyboard for P3 reects reduced a and
b constants within the horizontal plane, as the keyboard is wider
than generically optimized keyboard; this conguration describes
a preference for P3 to move left and right (i.e., rather than up or
down). Contrastingly, the generically optimized keyboard shows
eect of directionally static constants its circular geometry.
5 KEYBOARD COMMUNICATION
Keyboard communication results for personalized, optimized, and
QWERTY keyboards are shown in Figure 3.
P1 completed two blocks each with generically optimized (pre-
sented rst) and personalized (presented second) keyboards, fol-
lowed by one block with the QWERTY keyboard. P1 demonstrated
superior communication performance when using their person-
alized keyboard compared to either the optimized or QWERTY
keyboards for WPM, ITR, and throughput. When surveyed about
each keyboard, P1 consistently reported that it felt faster to use their
personalized keyboard (7.4
±
1.0 of 10 cm) than either the optimized
(4.3
±
0.2 of 10 cm) or QWERTY (6.6 of 10 cm) keyboards. P1 addi-
tionally indicated the most familiarity with QWERTY (10 of 10 cm),
but that it was easier to understand the layout of their personalized
keyboard (7.9
±
1.1 of 10 cm) than the optimized keyboard (6.2
±
0.7
of 10 cm). P1 ultimately designated a strong preference for their
personalized keyboard over the generically optimized keyboard
(9.5 of 10 cm in favor of personalized).
P2 was presented with their personalized keyboard rst followed
by the generically optimized keyboard, completing blocks 1 and 2 in
one day and blocks 3 and 4 in a second day alongside the QWERTY
block. P2 showed superior performance when using their personal-
ized keyboard during the nal block for each day (i.e., blocks 2 and
4); this is demonstrated via higher WPM, throughput, and ITR on
the rst day and by higher accuracy, throughput, and ITR on the sec-
ond day. Communication performance during the nal exposure to
the personalized keyboard (block 4) exceeded that of the QWERTY
board through accuracy, throughput, and ITR. Subsequently, P2
reported that it felt slightly faster to use the personalized (4.8
±
0.2
of 10 cm) and optimized (4.8
±
0.2 of 10 cm) keyboards compared
to the QWERTY keyboard (4.7 of 10 cm); however, the speed-of-
use ratings were quantitatively similar across all three keyboards.
P2 also reported that it was easiest to understand the layout of
QWERTY (9.7 of 10 cm), followed by their personalized keyboard
(6.2
±
1.0 of 10 cm), then the optimized keyboard (5.9
±
1.1 of 10 cm).
Following this, P2 strongly preferred their personalized keyboard
over the generically optimized keyboard (10 of 10 cm in favor of
personalized).
P3 completed two blocks each for personalized (presented rst)
and generically optimized (presented second) keyboards, followed
by one block with QWERTY. In their nal exposure to the new
keyboards, P3 demonstrated better performance using their person-
alized board in terms of accuracy and ITR. P3’s nal communication
performance when using their personalized keyboard exceeded that
of QWERTY through accuracy and ITR. Survey results were only
collected after the second block for personalized and optimized key-
boards. Dissimilar from their text input performance, P3 reported
that it felt slightly faster to create messages using the optimized
keyboard (3.5 of 10 cm) compared to their personalized keyboard
(3.2 of 10 cm), and that both new keyboards felt faster to use than
QWERTY (2.2 of 10 cm). P3 additionally reported that it was as
easy to understand the layout of their personalized keyboard as of
QWERTY (9.0 of 10 cm). P3 found it most dicult to understand the
optimized keyboard layout (7.7 of 10 cm) and, accordingly, exhibited
a strong preference for their personalized keyboard (10 of 10 cm in
favor of personalized).

CHI ’22 Extended Abstracts, April 29–May 05, 2022, New Orleans, LA, USA
6 SIGNIFICANCE & FUTURE DIRECTIONS
The results presented here provide insight into the communication
benets that may be achieved through automatic, ability-based
generation of keyboards for individuals with motor disabilities. By
examining cursor movements during a 2D multidirectional point-
select task, this work has yielded an automated method to character-
ize user movement patterns and abilities that may be used to control
a computer cursor. Unique geometric organization of the person-
alized keyboards visibly reected the diverse movement patterns
observed in the movement characterization, highlighting the im-
mense heterogeneity of the manifestation of motor disabilities. The
personalized keyboards also showed benets across participants
via ITR—a metric that unies both speed and accuracy—but not
throughput. These ndings indicate clear communication benets
in some users while also emphasizing the importance of using mul-
tiple metrics to comprehensively examine text input performance.
Finally, this work highlights the importance of user perception in
AAC: even with variability in performance across keyboards, all
three participants reported a strong preference for their personal-
ized keyboard and, furthermore, that using their new, personalized
keyboard felt faster than QWERTY.
Given these promising results, we intend to expand testing
among a larger, diverse group of prospective users who require alter-
native communication technology due to developmental disabilities,
acquired neurogenic disorders, and/or degenerative neurological
conditions to further elucidate the utility of our methods. We also
aim to conduct a eld study to examine the ecological validity of
our keyboard generation algorithms when used to create a person-
alized keyboard based on a user’s preferred access method (e.g.,
eye- or head-tracking device, joystick) instead of the sEMG/IMU
method examined here, which was new to participants. Finally, it
is important to note that although ITR was higher using the per-
sonalized keyboard over QWERTY, overall text input performance
was of similar magnitude between the new keyboards (optimized,
personalized) and QWERTY. This, especially emphasized through
speed (WPM), is likely the result of overtrained experience with
QWERTY and undertrained experience with the new keyboards
(each used for
∼
1 hour). These results are in line with prior work
suggesting overtraining eects will likely be overcome after 4–5
hours of interaction [
4
], indicating potential increases in text input
performance using the personalized keyboards. Future work should
examine the longitudinal eects of training with the new keyboards
compared to QWERTY.
7 CONCLUSION
Current AAC interfaces require tedious, manual support from a
trained caregiver to personalize a keyboard to meet user needs. In
this work, we extended the use of ability-based methods to individ-
uals with heterogenous motor disabilities to determine the utility
of automatically personalizing a keyboard to user-specic motor
abilities. In doing so, we observed diverse movement capabilities
as individuals navigated through dierent geometric organizations
and shapes to create messages on virtual keyboards. Communica-
tion performance improved in all participants when using their
Claire Mitchell et al.
personalized keyboard compared to QWERTY or a generically op-
timized keyboard. These results highlight the feasibility of auto-
mated, ability-based techniques as a valuable tool for generating
individualized keyboards that accommodate the diverse popula-
tion of individuals who must rely on alternative technologies for
communication.
ACKNOWLEDGMENTS
This work was supported in part by the National Institutes of Health
under Grant R43 DC018437 (awarded to Altec, Inc.) and by the De
Luca Foundation.
REFERENCES
[1]
Caitlin Brandenburg, Linda Worrall, Amy D. Rodriguez, and David Copland. 2013.
Mobile computing technology and aphasia: An integrated review of accessibility
and potential uses. Aphasiology 27, 4 (April 2013), 444–461. DOI:https://doi.org/
10.1080/02687038.2013.772293
[2]
Merrett Brown, Amy Nguyen, Shareen Dhillon, Kathryn Cordero, Analisa Church,
Stephanie Firme, and Gavens Abigail. 2018. Key AAC Issues. Retrieved from
https://ussaac.org/speakup/articles/key-aac- issues/
[3]
Jaewon Chung, Benjamin D. Pedigo, Eric W. Bridgeford, Bijan K. Varjavand,
Hayden S. Helm, and Joshua T. Vogelstein. 2019. GraSPy: Graph Statistics in
Python. J. Mach. Learn. Res. 20, (2019), 1–7. Retrieved from https://neurodata.io/
graspy.
[4]
Gabriel J. Cler, Katharine R. Kolin, Jacob P. Noordzij, Jennifer M. Vojtech, Susan
K. Fager, and Cara E. Stepp. 2019. Optimized and predictive phonemic interfaces
for augmentative and alternative communication. J. Speech, Lang. Hear. Res. 62, 7
(2019), 2065–2081. DOI:https://doi.org/10.1044/2019_JSLHR-S- MSC18-18-0187
[5]
Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences (2nd
edition ed.).
[6]
José-Manuel Díaz-Bossini and Lourdes Moreno. 2014. Accessibility to Mobile
Interfaces for Older People. Procedia Comput. Sci. 27, (January 2014), 57–66.
DOI:https://doi.org/10.1016/J.PROCS.2014.02.008
[7]
Susan Koch Fager, Melanie Fried-Oken, Tom Jakobs, and David R. Beukelman.
2019. New and emerging access technologies for adults with complex com-
munication needs and severe motor impairments: State of the science. https:
//doi.org/10.1080/07434618.2018.1556730 35, 1 (January 2019), 13–25. DOI:https:
//doi.org/10.1080/07434618.2018.1556730
[8]
Leah Findlater and Jacob O. Wobbrock. 2012. Personalized Input: Improving
Ten-Finger Touchscreen Typing through Automatic Adaptation. (2012).
[9]
Krzysztof Z. Gajos, Daniel S. Weld, and Jacob O. Wobbrock. 2010. Automatically
generating personalized user interfaces with SUPPLE. Artif. Intell. 174, (2010),
910–950. DOI:https://doi.org/10.1016/j.artint.2010.05.005
[10]
Krzysztof Z. Gajos, Jacob O. Wobbrock, and Daniel S. Weld. 2008. Improving
the performance of motor-impaired users with automatically- generated, ability-
based interfaces. In Conference on Human Factors in Computing Systems - Pro-
ceedings, 1257–1266. DOI:https://doi.org/10.1145/1357054.1357250
[11]
Matti D. Groll, Surbhi Hablani, Jennifer M. Vojtech, and Cara E. Stepp. 2020.
Cursor Click Modality in an Accelerometer-Based Computer Access Device.
IEEE Trans. Neural Syst. Rehabil. Eng. 28, 7 (July 2020), 1566–1572. DOI:https:
//doi.org/10.1109/TNSRE.2020.2996820
[12]
Thomas Huijbregts and James R. Wallace. 2015. TalkingTiles: Supporting Person-
alization and Customization in an AAC App for Individuals with Aphasia. Proc.
2015 ACM Int. Conf. Interact. Tabletops Surfaces, ITS 2015 (November 2015), 63–72.
DOI:https://doi.org/10.1145/2817721.2817723
[13]
Alla A Johnson. 2008. Factors related to the rejection and/or abandonment of
AAC devices Recommended Citation. University of New Hampshire, Durham.
Retrieved from https://scholars.unh.edu/thesis
[14]
Marta Kazandjian and Karen Dikeman. 2012. Guillain-Barre Syndrome and Disor-
dered Swallowing. Perspect. Swallowing Swallowing Disord. 21, 4 (2012), 115–120.
DOI:https://doi.org/10.1044/sasd21.4.115
[15]
Simeon Keates, Faustina Hwang, Patrick Langdon, P. John Clarkson, and Pe-
ter Robinson. 2002. The use of cursor measures for motion-impaired computer
users. Univers. Access Inf. Soc. 2, (2002), 18–29. DOI:https://doi.org/10.1145/638249.
638274
[16]
I. Scott MacKenzie. 1989. A Note on the Information-Theoretic Basis for Fitts’
Law. J. Mot. Behav. 21, 3 (1989), 323–330. DOI:https://doi.org/10.1080/00222895.
1989.10735486
[17]
I. Scott MacKenzie. 1992. Fitts’ Law as a Research and Design Tool in Human-
Computer Interaction. Human-Computer Interact. 7, 1 (1992), 91–139. DOI:https:
//doi.org/10.1207/S15327051HCI0701_3
[18]
I. Scott MacKenzie and Poika Isokoski. 2008. Fitts’ throughput and the speed-
accuracy tradeo. Conf. Hum. Factors Comput. Syst. - Proc. (2008), 1633–1636.

Ability-based Keyboards for Augmentative and Alternative Communication: Understanding
How Individuals’ Movement Paerns Translate to More Eicient Keyboards
DOI:https://doi.org/10.1145/1357054.1357308
[19]
I. Scott Mackenzie, Tatu Kauppinen, and Miika Silfverberg. 2001. Accuracy Mea-
sures for Evaluating Computer Pointing Devices. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, ACM.
[20]
I. Scott MacKenzie and R. William Soukore. 2003. Phrase sets for evaluating text
entry techniques. In Extended Abstracts on Human Factors in Computing Systems,
ACM, 754–755. DOI:https://doi.org/10.1145/765891.765971
[21]
I. Scott MacKenzie and Shawn Xia Zhang. 1999. The design and evaluation of
a high-performance soft keyboard. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, ACM, 25–31. DOI:https://doi.org/10.1145/
302979.302983
[22]
Tanja Makkonen, Hanna Ruottinen, Riitta Puhto, Mika Helminen, and Johanna
Palmio. 2018. Speech deterioration in amyotrophic lateral sclerosis (ALS) after
manifestation of bulbar symptoms. Int. J. Lang. Commun. Disord. 53, 2 (2018),
385–392. DOI:https://doi.org/10.1111/1460-6984.12357
[23]
Claire L. Mitchell, Gabriel J. Cler, Susan K. Fager, Paola Contessa, Serge H. Roy,
Gianluca De Luca, Joshua C. Kline, and Jennifer M. Vojtech. 2022. Ability-based
Methods for Personalized Keyboard Generation. (2022). Retrieved from https:
//arxiv.org/abs/2201.04593
[24]
Sayan Sarcar, Jussi PP Jokinen, Antti Oulasvirta, Zhenxin Wang, Chaklam Sil-
pasuwanchai Asian Institute of Technology, Asian Institute of Technology, and
Xiangshi Ren. 2018. Ability-Based Optimization of Touchscreen Interactions.
IEEE Pervasive Comput. 17, 1 (2018), 15–26. Retrieved November 24, 2021 from
www.computer.org/pervasive
CHI ’22 Extended Abstracts, April 29–May 05, 2022, New Orleans, LA, USA
[25]
Richard C. Simpson. 2013. Computer Access for People with Disabilities: A Human
Factors Approach. CRC Press, United Kingdom.
[26]
H S Venkatagiri. 1995. Techniques for Enhancing Communication Productivity
in AAC: A Review of Research. Am. J. Speech-Language Pathol. 4, 4 (1995), 36–45.
DOI:https://doi.org/10.1044/1058-0360.0404.36
[27]
Joshua T Vogelstein, John M Conroy, Vince Lyzinski, Louis J Podrazik, Steven G
Kratzer, Eric T Harley, Donniell E Fishkind, R Jacob Vogelstein, and Carey E Priebe.
2015. Fast Approximate Quadratic Programming for Graph Matching. PLoS One
10, 4 (April 2015), 0121002. DOI:https://doi.org/10.1371/journal.pone.0121002
[28]
Jennifer M. Vojtech, Surbhi Hablani, Gabriel J. Cler, and Cara E. Stepp. 2020.
Integrated head-tilt and electromyographic cursor control. IEEE Trans. Neural
Syst. Rehabil. Eng. 28, 6 (2020), 1442–1451. DOI:https://doi.org/10.1109/tnsre.2020.
2987144
[29]
Jonathan R. Wolpaw, Niels Birbaumer, Dennis J. McFarland, Gert Pfurtscheller,
and Theresa M. Vaughan. 2002. Brain–computer interfaces for communication
and control. Clin. Neurophysiol. 113, 6 (June 2002), 767–791. DOI:https://doi.org/
10.1016/S1388-2457(02)00057- 3
[30]
Shumin Zhai, Michael Hunter, and Barton A. Smith. 2002. Performance Opti-
mization of Virtual Keyboards. Human-Computer Interact. 17, 2-3: Text Entry for
Mobile Computing (2002), 229–269. DOI:https://doi.org/10.1080/07370024.2002.
9667315
[31]
Shumin Zhai, Jing Kong, and Xiangshi Ren. 2004. Speed–accuracy tradeo in
Fitts’ law tasks—on the equivalency of actual and nominal pointing precision.
Int. J. Hum. Comput. Stud. 61, 6 (December 2004), 823–856. DOI:https://doi.org/
10.1016/J.IJHCS.2004.09.007