Assessing reliability of short and tick box forms of the ANU-ADRI: Convenient alternatives of a self-report Alzheimer's disease risk assessment

Abstract

Introduction
To assess the reliability of short versions of the Australian National University Alzheimer's Disease Risk Index (ANU-ADRI).

Methods
A short form of the ANU-ADRI (ANU-ADRI-SF) was developed by assessing risk and protective factors with single questions where possible and with short forms of sub-questionnaires where available. The tick box form of the ANU-ADRI (ANU-ADRI-TB) was developed with unique questions for each risk and protective factor for Alzheimer's disease. The short versions were evaluated in an independent community sample of 504 participants with a mean age of 45.01 (SD = 14.85, range = 18–81).

Results
The short versions demonstrated high reliabilities when compared with the ANU-ADRI. However, the proportion of misclassification was high for some risk factors and particularly for the ANU-ADRI-TB.

Discussion
The ANU-ADRI-SF may be considered if less reliable questions from the ANU-ADRI-SF can be replaced with more reliable questions from the ANU-ADRI for risk/protective factors with high misclassification.

Full text

This is an open access article which appeared in a journal published
by Elsevier. This article is free for everyone to access, download and
read.
Any restrictions on use, including any restrictions on further
reproduction and distribution, selling or licensing copies, or posting
to personal, institutional or third party websites are defined by the
user license specified on the article.
For more information regarding Elsevier's open access licenses
please visit:
http://www.elsevier.com/openaccesslicenses

Featured Article
Assessing reliability of short and tick box forms of the ANU-ADRI:
Convenient alternatives of a self-report Alzheimer’s disease risk
assessment
Sarang Kim*, Nicolas Cherbuin, Kaarin J. Anstey
Centre for Research on Ageing, Health and Wellbeing, Research School of Population Health, The Australian National University, Canberra, ACT, Australia
Abstract Introduction: To assess the reliability of short versions of the Australian National University Alz-
heimer’s Disease Risk Index (ANU-ADRI).
Methods: A short form of the ANU-ADRI (ANU-ADRI-SF) was developed by assessing risk and
protective factors with single questions where possible and with short forms of sub-questionnaires
where available. The tick box form of the ANU-ADRI (ANU-ADRI-TB) was developed with unique
questions for each risk and protective factor for Alzheimer’s disease. The short versions were eval-
uated in an independent community sample of 504 participants with a mean age of 45.01
(SD 514.85, range 518–81).
Results: The short versions demonstrated high reliabilities when compared with the ANU-ADRI.
However, the proportion of misclassification was high for some risk factors and particularly for
the ANU-ADRI-TB.
Discussion: The ANU-ADRI-SF may be considered if less reliable questions from the ANU-ADRI-
SF can be replaced with more reliable questions from the ANU-ADRI for risk/protective factors with
high misclassification.
Ó2016 The Authors. Published by Elsevier Inc. on behalf of the Alzheimer’s Association. This is an
open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/
4.0/).
Keywords: Risk assessment; Alzheimer’s disease; Short versions; Screening
1. Introduction
At present, there is no cure or effective treatment for de-
mentia [1–3] and existing pharmacological treatments do
not modify the course of the disease [4]. Prevention is there-
fore one of the key objectives of current dementia research
[5], and increased attention has been paid to identifying risk
and protective factors for dementia. It is also essential that
people understand and address their risk profile for dementia
as early as possible before the development of the patho-
logic processes which lead to unrecoverable neurodegener-
ation. Easily accessible methods of risk assessment are an
important tool for facilitating population-level dementia
risk awareness.
A questionnaire-based risk assessment tool, the Austra-
lian National University Alzheimer’s Disease Risk Index
(ANU-ADRI), was developed to assess the presence of 11
risk and 4 protective factors for Alzheimer’s disease (AD)
[6]. These risk and protective factors have reliable scientific
evidence and can be measured by self-report. The ANU-
ADRI was validated on three independent cohorts [7]
against the Cardiovascular Risk Factors, Aging and Demen-
tia index and has been proposed as the key risk assessment
tool in large-scale dementia prevention trials. ANU-ADRI
has already been used in the first online dementia risk reduc-
tion trial in Australia [8,9] and is currently available to the
general public through the ANU-ADRI website (http://
anuadri.anu.edu.au/). The website has attracted over
11,700 visitors since its launch early 2014.
The authors declare no conflicts of interest.
*Corresponding author. Tel.: 161 2 6125 0713; Fax: 161 2 6125 1558.
E-mail address: sarang.kim@anu.edu.au
http://dx.doi.org/10.1016/j.trci.2016.03.001
2352-8737/ Ó2016 The Authors. Published by Elsevier Inc. on behalf of the Alzheimer’s Association. This is an open access article under the CC BY-NC-ND
license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Alzheimer’s & Dementia: Translational Research & Clinical Interventions 2 (2016) 93-98
Author's Personal Copy

The original ANU-ADRI takes approximately 15–20 mi-
nutes to complete, although longer time of up to 1 hour has
been reported for older adults who were not familiar with the
computer or Internet. The length of the ANU-ADRI did not
affect the participation of a dementia prevention trial [9] or
completion of publicly available ANU-ADRI through the
website. However, a shorter version of the ANU-ADRI
may have a role on public websites and online applications,
enabling the ANU-ADRI to reach wider audiences in more
diverse settings and in turn, save time in assessing an indi-
vidual’s risk level. Given that the average general practice
(GP) consultation time is 14.6 minutes (95% CI, 14.1–
15.0) [10], the length of the time taken to complete the cur-
rent ANU-ADRI can be considered long. Hence, by
providing shorter versions of the ANU-ADRI, health profes-
sionals such as GPs for example may be able to assess a cli-
ent’s risk level and provide them with relevant advice in a
timely manner. The ANU-ADRI can also be used to screen
individuals to identify those at high risk of developing AD
who would benefit most from intervention programs. We
therefore developed and evaluated two brief alternatives (a
short form and a tick box form) to the original longer version
of the ANU-ADRI while aiming to preserve its content
coverage.
2. Methods
2.1. Participants and procedure
One thousand and seventy three people from the com-
munity were approached by Qualtrics, a survey company.
Five hundred and four of them completed the original
ANU-ADRI as well as one of the short versions to be vali-
dated. Sample size calculations were estimated using
G*Power (version 3.1.3). To detect a medium difference
(0.5) between two independent sample means, with a 5%
risk of type I error (a), 95% power, and two equal groups
(1:1), 105 persons in each group was required. Our sample
size was more than double the required size, and half of the
participants (group 1; n 5251) completed the original and
the short form of the ANU-ADRI. The other half (group 2;
n5253) completed the original and the tick box form of
the ANU-ADRI. Participants were randomly assigned to
one of two groups by the Qualtrics survey program. Inclu-
sion criteria were being 18 years, being proficient in En-
glish, having internet access, and having no psychiatric or
neurological diagnoses.
2.2. Measures and procedures
2.2.1. Short form
A short form of the ANU-ADRI (ANU-ADRI-SF; see
Supplementary A) which included the validated short
forms of questionnaires used in the original ANU-ADRI
(e.g. International Physical Activity Questionnaire short
form [11] and 10 item Community Epidemiological
Study-Depression scale (CESD-10) [12]) while keeping
single-item questions (e.g., gender, age, and smoking sta-
tus) was developed. The risk and protective factors
covered by more than one item were also simplified into
single-item questions by adding questions together. For
example, two questions on diabetes “Have you ever been
told by a doctor or other health professional that you
have diabetes?” and “Have you ever been told by a doctor
or other health professional that you have high sugar
levels in your blood or urine?” became a single question
“Have you ever been told by a doctor or other health pro-
fessional that you have diabetes or have high sugar levels
in your blood or urine?”. These single items intended to
reduce the number of questions while keeping the con-
tents. These include education, diabetes, cholesterol, trau-
matic brain injury, cognitive activity, and fish intake. The
short form was estimated to take approximately 5 minutes
to complete.
2.2.2. Tick box form
The tick box form of the ANU-ADRI (ANU-ADRI-TB;
see Supplementary B) was built with single-item questions
for each AD risk and protective factors and was estimated
to take approximately 2 minutes to complete. The ANU-
ADRI-TB was developed with an assumption that a single-
item scale may be as sensitive and reliable as multi-item
scales. This assumption was supported in previous research
such as in depression [13] and physical activity [14]. It was
found that a single-item interview for depressed mood pro-
vides a reliable and accurate screen that can be used in clin-
ical settings that permit direct patient interview.
One of the short versions followed by original long
version of the ANU-ADRI was administered on the same
day.
2.3. Analysis
To analyze the reliability of the short versions, intra class
correlation coefficients (ICC) were used between the orig-
inal scores and short versions’ scores. The relative measure
of risk points attributed to each risk and protective factor as-
sessed in the ANU-ADRI-SF was the same as that used in the
original ANU-ADRI which ranged between 0–6 for risk fac-
tors and 27 to 0 for protective factors. ANU-ADRI-TB,
however, only had binary variables (having risk/protective
factors vs not) and was compared against binary variables
of the original ANU-ADRI. Although the original ANU-
ADRI did not evaluate misclassifications, as the tick box
form can only be measured as binary outcomes (having
risk/protective factors vs not), we felt it was worth exam-
ining misclassification in addition to reliability tests. The
number of risk and protective factors misclassified in the
short compared to the long form were calculated using bi-
nary variables (having risk/protective factors vs not). Per-
centages of misclassification of each risk and protective
factor were also calculated to examine false-positive (being
S. Kim et al. / Alzheimer’s & Dementia: Translational Research & Clinical Interventions 2 (2016) 93-9894
Author's Personal Copy

identified as having a risk factor when this is not the case)
and false-negative misclassifications (not being identified
as having a risk factor when this is not the case). Reliability
and misclassification were not investigated for age, educa-
tion, and BMI as exactly the same questions were used to
assess these variables in all three versions. The data were
analyzed using SPSS 22.
3. Results
Table 1 shows the participants’ demographic characteris-
tics and dementia risk exposure. Two hundred and forty nine
(49.4%) of 504 participants were males. Participants were
aged between 18 and 81 (M 545.01, standard deviation
[SD] 514.85) years and were evenly spread across age
groups of 30, 30–39, 40–49, 50–59, and 60 years. The
mean age of the original validation sample was higher than
the present study. However, ANU-ADRI is currently avail-
able to the general public as we are not restricting user’s
age. The statistics from the ANU-ADRI website demon-
strated that the mean age of website users was 48.71 years
(SD 518.90). Therefore, the current sample was a better
representation of the users of the ANU-ADRI.
The samples from group 1 (short form) and group 2 (tick
box form) were not significantly different across most
characteristics. However, those who completed ANU-
ADRI-SF were significantly more likely to have diabetes
than those who completed ANU-ADRI-TB.
3.1. Reliability
ICC were computed to study the reliability of the short ver-
sions of the ANU-ADRI (see Table 2). ICCs for ANU-ADRI-
SF suggested moderate to strong agreement with coefficients
ranging from 0.772 to 0.992. All were statistically significant
(P,.001) except for cognitive activity where the coefficients
was small and not significant (ICC 5.031, P5.285). The
ANU-ADRI-TB also showed small to strong agreement for
most risk factors with correlations ranging from 0.444 to 1
except for cognitive activity demonstrating extremely small
and insignificant (ICC 5.024, P5.287) coefficient value.
Internal consistency was also examined for CESD-10,
measuring depression, and it showed a high level of internal
consistency (Cronbach’s alpha of 0.879), which was compa-
rable with the internal consistency of CESD-20 (Cronbach’s
alpha of 0.934).
3.2. Misclassification
Misclassification of someone having or not having risk
and/or protective factors was also investigated using binary
Table 1
Demographic characteristics of participants in the short form and tick box form group
Characteristics Short form Tick box form c
2
Pvalue
Age (M, SD) 44.96 (15.03) 45.06 (14.69) t 520.072 .943
Gender
Females 51.0% 50.2% 0.032 .858
Education (M, SD) 14.54 (3.51) 14.94 (4.02) t 521.216 .225
BMI
Overweight 29.5% 29.6% 0.069 .966
Obese 28.3% 27.3%
High cholesterol 23.1% 26.1% 0.603 .437
Diabetes 13.9% 7.9% 4.727 .030
TBI 10.8% 9.9% 0.104 .747
Depression 37.1% 35.2% 0.192 .661
Physical activity
Medium 30.7 % 33.6% 3.037 .219
High 47.0% 50.2%
Cognitive activity
Middle 3.2% 3.2% 0.204 .903
Highest 31.5% 29.6%
Social activity
Low 40.6% 48.6% 5.120 .163
Fish consumption
0.25–2 p p/wk 44.2% 49.4% 4.166 .244
2–4 p p/wk 12.0% 7.5%
.4 p p/wk 6.4% 8.3%
Alcohol consumption
Light to moderate 71.5% 73.2% 0.181 .913
Smoking
Current smoker 24.3% 24.9% 2.204 .332
Past smoker 26.3% 31.6%
Pesticide 10.5% 9.2% 0.246 .620
Note. Figures are based on the original ANU-ADRI. Bold text indicates significance.
S. Kim et al. / Alzheimer’s & Dementia: Translational Research & Clinical Interventions 2 (2016) 93-98 95
Author's Personal Copy

variables. Fig. 1 demonstrated that those who completed the
ANU-ADRI-TB tended to have higher rates of misclassifica-
tions than those who completed the ANU-ADRI-SF.
Proportions of misclassification on risk and protective
factors were examined (Table 3). The false-positive misclas-
sification represented cases where a risk or protective factor
for AD was identified as present when it was not. The false-
negative misclassification represented cases where a risk or
protective factor for AD was identified as absent when it was
not. The factor with the highest false-positive misclassifica-
tion rate was cognitive activity (66.1%), whereas fish con-
sumption (13.9%) and physical activity (13.1%) had the
highest false-negative misclassification rates for the ANU-
ADRI-SF. For the ANU-ADRI-TB, cognitive activity
(68.4%) followed by fish consumption (11.9%) and alcohol
consumption (11.5%) had the highest false-positive misclas-
sification rates, whereas social activity level (34.4%) and
physical activity level (28.1%) had the highest false-
negative misclassification rates. Overall, more risk and
protective factors were misclassified with ANU-ADRI-TB
than ANU-ADRI-SF. Both short versions overclassified
protective cognitive activity and underclassified social and
physical activity.
4. Discussion
The present study developed and evaluated two short ver-
sions of the ANU-ADRI as possible alternatives to the orig-
inal ANU-ADRI to be used in timely restricted settings. The
short form of the ANU-ADRI demonstrated moderate to
good reliability as the ANU-ADRI scores between short
and long versions were similar. This suggests that when
there are time constraints, the ANU-ADRI-SF may replace
the original ANU-ADRI.
The ANU-ADRI-TB on the other hand had lower reli-
ability and higher occurrence of misclassification of risk
and protective factors than the ANU-ADRI-SF. Risk and
protective factors that were simplified into single items or
used shorter versions of existing questionnaires were likely
to lead to misclassification. This suggests that factors that
require comprehensive measures cannot be replaced by a
single question. This appears to be especially true for factors
that require a wide range of actions and/or activities (e.g.,
cognitive engagement) as well as frequencies and/or quan-
tity (e.g., frequency and duration of weekly physical activ-
ity). Another possible explanation for low reliability could
be the ceiling effect and that the original version had a wider
range of options. For example, most adults may be engaging
in at least one of the activities listed for cognitive activities
every day or nearly every day. It is not surprising to find
high misclassifications. Caution should therefore be exer-
cised when using the short versions of the ANU-ADRI and
other dementia risk assessment scales that do not have
adequate measures of risk factors.
Translation of epidemiological research on risk factors
for dementia requires the development of widely accessible
assessment tools. The unique contribution of this study is to
develop and evaluate a brief, practical AD risk assessment
tool that can be used by clinicians or self-administered.
The strength of the study was that we tested short versions
of assessment tools against an established, valid tool that
had been applied in practical settings. In addition, the scores
were drawn from scientific evidence. However, there are
some limitations to the study. The validation of the scales
has not been examined as this would require following par-
ticipants from when they did not have AD to when they
develop AD in large populations over long periods of
time to determine how well they perform in predicting
AD decades later. The present study demonstrates the reli-
ability of short versions of the ANU-ADRI when compared
to the longer version. Lack of similar short versions of
assessment tools also made it impossible for us to evaluate
validity of the scales. Future research should investigate the
validity of the short versions with large population-based
samples. We also did not assess participants’ cognitive
functioning at the time of the assessment as this was not
the aim of the present study. However, this should be
considered in a future study to ensure that participants
who are involved do not have any cognitive dysfunction
at the time of assessment.
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
012345
Short form Tick box form
Fig. 1. Number of risk and protective factors misclassification.
Table 2
Inter-class correlation (ICC) coefficients between short forms and long form
Risk/Protective
factors
Short form Tick box form
ICC (95% confidence
interval)
ICC (95% confidence
interval)
Cholesterol 0.926 (0.905–0.942) 0.979 (0.973–0.983)
Diabetes 0.909 (0.882–0.930) 1
TBI 0.776 (0.568–0.878) 0.702 (0.509–0.819)
Depression 0.890 (0.858–0.914) 0.789 (0.725–0.838)
Physical activity 0.796 (0.737–0.841) 0.444 (0.197–0.605)
Cognitive activity 0.031 (20.084 to 0.147) 0.024 (20.065 to 0.118)
Social activity 0.937 (0.919–0.951) 0.822 (0.684–0.889)
Fish consumption 0.772 (0.631–0.849) 0.751 (0.667–0.812)
Alcohol
consumption
0.990 (0.987–0.992) 0.640 (0.536–0.721)
Smoking 0.992 (0.990–0.994) 0.996 (0.995–0.997)
Pesticide 0.977 (0.971–0.982) 0.826 (0.774–0.865)
Bold text indicates significance.
S. Kim et al. / Alzheimer’s & Dementia: Translational Research & Clinical Interventions 2 (2016) 93-9896
Author's Personal Copy

Overall, the present study showed that the shorter ver-
sions of the ANU-ADRI, especially the ANU-ADRI-SF
have limitations but are of practical use. Short versions
were least accurate in assessing lifestyle activities because
these involve more complex assessment algorithms (phys-
ical, cognitive, and social activity). However, the short
form is acceptable for assessing medical and demographic
risk factors. Therefore, the use of the short versions should
only be recommended as a second choice, when individuals
have limited time and need a quick indication of an AD risk
level or where lifestyle activity is not the focus of the assess-
ment. The original ANU-ADRI cannot be fully replaced by
the short versions and when using the short form, less reli-
able questions from the ANU-ADRI-SF should be replaced
with more reliable questions from the original ANU-ADRI
for those risk and protective factors with low reliability
and high misclassification. We recommend using the tick
box version of the ANU-ADRI only where there is no possi-
bility of using the short form. One future possible use of the
tick box form may be administration using a stepped
approach, with participants being asked to complete longer
questions according to the tick box version results.
Acknowledgments
The research was funded by the Dementia Collaborative
Research Centres (DCRC) Knowledge Translation Small
Grant. K.J.A. is funded by National Health and Medical
Research Council fellowship 1102694, and N.C. is funded
by an ARC future fellowship 120100227. The funders had
no role in this research. S.K. had full access to all the data
in the study and takes responsibility for the integrity of the
data and accuracy of the data analysis.
Supplementary data
Supplementary data related to this article can be found at
http://dx.doi.org/10.1016/j.trci.2016.03.001.
RESEARCH IN CONTEXT
1. Systematic review: We conducted a systematic re-
view of the literature to identify a brief, evidence
based, validated, risk assessment tool for Alz-
heimer’s disease. No other tool that assesses level
of risk for Alzheimer’s or dementia has been devel-
oped that is convenient and based on self-report.
2. Interpretation: Our study results show that short form
of the ANU-ADRI may be considered where the
longer original version is not practical or when there
are time constraints. However, the original ANU-
ADRI remains the preferred choice. The tick box
form of the ANU-ADRI is not recommended for use
unless there is no other option. ANU-ADRI-TB is too
brief and insufficient for measuring the complexity
of cognitive, physical, and social activities.
3. Future directions: Further research is required to
examine the usability of the ANU-ADRI-SF in
various settings and feedback on effectiveness of the
scale should be collected for future improvement.
References
[1] Chung S, Mehta K, Shumway M, Alvidrez J, Perez-Stable EJ. Risk
perception and preference for prevention of Alzheimer’s disease.
Value Health 2009;12:450–8.
[2] Polidori MC, Nelles G, Pientka L. Prevention of Dementia: Focus on
Lifestyle. Int J Alzheimers Dis 2010;2010.
[3] Rockwood K, Wallack M, Tallis R. The treatment of Alzheimer’s dis-
ease: Success short of cure. Lancet Neurol 2003;2:630–3.
[4] Rodda J, Carter J. Cholinesterase inhibitors and memantine for symp-
tomatic treatment of dementia. BMJ 2012;344:e2986.
Table 3
Percentage of misclassification
Risk/Protective factors
Short form Tick box form
Base rate
z
False positive*False negative
y
False positive*False negative
y
Cholesterol 2.8% 2% 1.2% 0.4% 23.4%
Diabetes 3.6% 0% 0% 0% 9.1%
TBI 4.0% 0% 4.3% 0.8% 6.3%
Depression 3.2% 6.0% 3.2% 11.9% 36.1%
Physical activity 4.4% 13.1% 2.0% 28.1% 81.1%
Cognitive activity 66.1% 0% 68.4% 0.4% 29.6%
Social activity 7.6% 8.8% 2.0% 34.4% 44.6%
Fish consumption 0.4% 13.9% 11.9% 1.6% 17.1%
Alcohol consumption 0.4% 0.8% 11.5% 2.4% 71.2%
Smoking 1.2% 0.4% 0.4% 0.8% 53.6%
Pesticide 0% 0.8% 0% 4.0% 9.7%
*Misclassification as having risk/protective factors when they do not.
y
Misclassification as not having risk/protective factors when they do.
z
Based on the original ANU-ADRI.
S. Kim et al. / Alzheimer’s & Dementia: Translational Research & Clinical Interventions 2 (2016) 93-98 97
Author's Personal Copy

[5] Mangialasche F, Kivipelto M, Solomon A, Fratiglioni L. Dementia
prevention: current epidemiological evidence and future perspective.
Alzheimers Res Ther 2012;4:6.
[6] Anstey KJ, Cherbuin N, Herath P. Development of a new method for
assessing global risk of Alzheimer’s disease for use in population
health approaches to prevention. Prev Sci 2013;14:411–21.
[7] Anstey KJ, Cherbuin N, Herath PM, Qiu C, Kuller LH, Lopez OL,
et al. A self-report risk index to predict occurrence of dementia in three
independent cohorts of older adults: the ANU-ADRI. PLoS One 2014;
9:e86141.
[8] Anstey KJ, Bahar-Fuchs A, Herath P, Rebok GW, Cherbuin N. A 12-
week multidomain intervention versus active control to reduce risk of
Alzheimer’s disease: study protocol for a randomized controlled trial.
Trials 2013;14:60.
[9] Anstey KJ, Bahar-Fuchs A, Herath P, Kim S, Burns R, Rebok GW,
et al. Body brain life: A randomized controlled trial of an online de-
mentia risk reduction intervention in middle-aged adults at risk of Alz-
heimer’s disease. Alzheimer’s & Dementia: Translational Research &
Clinical Interventions 2015;1:72–80.
[10] Sayer GP, Britt H, Horn F, Bhasale A, McGeechan K, Charles J, et al.
Measures of health and health care delivery in general practice in
Australia. Canberra: Australian Institute of Health and Welfare; 2000.
[11] Bertakis KD, Azari R, Helms LJ, Callahan EJ, Robbins JA. Gender
differences in the utilization of health care services. J Fam Pract
2000;49:147–52.
[12] Green CA, Pope CR. Gender, psychosocial factors and the use of med-
ical services: A longitudinal analysis. Soc Sci Med 1999;48:1363–72.
[13] Chochinov HM, Wilson KG, Enns M, Lander S. “Are you depressed?”
Screening for depression in the terminally ill. Am J Psychiatry 1997;
154:674–6.
[14] Jackson AW, Morrow JR Jr, Bowles HR, FitzGerald SJ, Blair SN.
Construct validity evidence for single-response items to estimate phys-
ical activity levels in large sample studies. Res Q Exerc Sport 2007;
78:24–31.
S. Kim et al. / Alzheimer’s & Dementia: Translational Research & Clinical Interventions 2 (2016) 93-9898
Author's Personal Copy