Content uploaded by Hubertus Vrijhoef
Author content
All content in this area was uploaded by Hubertus Vrijhoef on Feb 14, 2014
Content may be subject to copyright.
ORIGINAL RESEARCH ARTICLE
Valuation of EQ-5D-3L Health States in Singapore: Modeling
of Time Trade-Off Values for 80 Empirically Observed Health
States
Nan Luo •Pei Wang •Julian Thumboo •
Yee-Wei Lim •Hubertus J. M. Vrijhoef
!Springer International Publishing Switzerland 2014
Abstract
Objective The aim of this study was to establish an EQ-
5D-3L value set using the time trade-off (TTO) method to
elicit the health preferences of the general Singaporean
population.
Methods The values of 80 EQ-5D-3L health states were
elicited from a general Singaporean population sample
using a TTO method. In face-to-face interviews, partici-
pants were asked to value a block of ten health states.
Various linear regression models were examined to assess
for goodness of fit to the data, at both aggregate and
individual levels. Prediction precision was assessed in
terms of mean absolute error (MAE), and numbers of
prediction errors larger than 0.10 and 0.20. Prediction
consistency and bias were also assessed.
Results A total of 456 participants provided data for this
study. The N3 model without a constant estimated using
the aggregate data exhibited the best fit of the data, pre-
dicted values with the least bias, and generated logically
consistent values for all 243 EQ-5D-3L health states. The
MAE was 0.1137, and 35 of 80 predicted values had errors
less than 0.10 in absolute magnitude. Based on this model,
the utility values ranged from 0.854 for state 11121 to
-0.769 for state 33333.
Conclusions The EQ-5D-3L value set can be estimated
using the TTO method in the multi-cultural, multi-ethnic
Singapore. Although the estimation precision is not opti-
mal, the health-state preference values generated in this
study are useful to health service researchers in the country
before estimates with smaller errors are available.
Key Points for Decision Makers
It appears that severe health problems are more
undesirable than early death to Singaporeans.
It seems that Singaporeans are more afraid of having
major health problems than people in many other
countries.
The health-state preference values estimated in this
study can be used to assess health technologies and
programs in Singapore.
1 Introduction
Quality-adjusted life-years (QALYs) that combine both
quantity and quality of life is the outcome measure in cost-
utility analysis (CUA) [1]. To calculate QALYs, health-
state utility that indicates the relative value of health states
using a scale of 0 (being dead) to 1.0 (full health) should be
used as the quality-of-life weight [1]. Health-state utility
values elicited from the general population are typically
used in CUA when the purpose of the analysis is to inform
decision making in the allocation of healthcare resources
N. Luo (&)!P. Wang !Y.-W. Lim !H. J. M. Vrijhoef
Saw Swee Hock School of Public Health, National University of
Singapore and National University Health System, 16 Medical
Drive, Block MD3, Singapore 117597, Singapore
e-mail: nan_luo@nuhs.edu.sg
J. Thumboo
Department of Rheumatology and Immunology,
Singapore General Hospital, Singapore, Singapore
J. Thumboo
Department of Medicine, National University Health System,
Singapore, Singapore
PharmacoEconomics
DOI 10.1007/s40273-014-0142-1
[2]. Health-state utility values can be determined for a set
of health states using a valuation technique such as time
trade-off (TTO) [3] and standard gamble (SG) [1], and then
used repeatedly whenever needed. Such health states are
usually defined using a standardized multi-attribute
descriptive system such as the EQ-5D-3L [4], the health
utilities index (HUI) [5], and the Short-Form 6-dimensions
(SF-6D) [6]. The prevailing strategy for determining the
utility values for the health states defined by a descriptive
system is to measure the values for a subset of the health
states using the TTO or SG method and then model this
data to predict for all the health states.
The EQ-5D questionnaire consists of two parts: a
descriptive system and a visual analog scale (VAS). The
descriptive system consists of five domains: mobility, self-
care, usual activities, pain/discomfort, and anxiety/depres-
sion, with each domain being described as three levels: ‘no
problems’ (level 1), ‘some problems’ (level 2), and
‘extreme problems’ (level 3). Each health state described by
EQ-5D-3L can be coded into a five-digit number. For
example, a health state in which a person has no problems in
mobility and self-care, no pain or discomfort, but some
problems in usual activities and extreme anxiety or
depression can be coded as 11213. This classification sys-
tem defines a total of 243 (3
5
) unique health states and can
be used as a questionnaire for respondents to classify their
own health status. The first value set for the EQ-5D-3L
health states was derived from the general UK population in
the Measurement and Valuation of Health (MVH) study
using a TTO method [4]. Country-specific EQ-5D-3L value
sets were subsequently developed using a similar research
protocol in many other countries [7–16]. Studies showed
that utility values of EQ-5D-3L health states differed across
countries [10,13,16,17], suggesting that each country
should develop its own value set for using the EQ-5D-3L to
evaluate health technologies within its own jurisdiction.
In Singapore, the EQ-5D-3L questionnaire has been
psychometrically validated [18–22] and widely used in
clinical and epidemiological studies [23–26]. However, the
values of the EQ-5D-3L health states to Singaporeans are
not known. As a result, researchers using the EQ-5D-3L in
Singapore had to use the UK, US, or Japanese value set to
acquire utility scores [18–26]. The aim of this study was
therefore to derive a Singapore societal value set for EQ-
5D-3L health states using the TTO method.
2 Methods
2.1 Sampling and Recruitment
This study was approved by the National University of
Singapore Institutional Review Board (NUS-IRB). In the
study, we drew a nationally representative sample of the
non-institutionalized general Singaporean population. We
excluded the institutionalized population because our pur-
pose was to estimate the general population’s health pref-
erences that are most relevant to decision making from the
societal perspective [27]. A three-stage sampling method
was used to recruit 500 residents aged 21 years and above,
with quotas set to achieve similarity between the sample
and the general adult Singaporean population in age, gen-
der, and housing type (an indicator of socioeconomic sta-
tus). The two minority ethnic groups (i.e. Malays and
Indians) were oversampled and only the English-speaking
Singaporeans were recruited due to limited resources. The
sampling procedure was as follows. First, a total of 20
locations were randomly selected from a sampling frame
comprising over 9,000 residential locations covering
almost all public housing blocks in Singapore. Second, 25
households were recruited from each selected location
using a systematic sampling method. Third, one resident
from each consenting household was invited to the study. If
the household had more than one eligible resident, the first
one whom the interviewer approached was recruited.
Residents who were foreign workers or could not speak
English were excluded. All respondents were interviewed
face-to-face in their homes by a trained interviewer.
Respondents received a gift voucher worth S$20 on com-
pletion of the interview.
2.2 The Valuation Interview
The interview started with an assessment of the respon-
dent’s health status using the EQ-5D-3L questionnaire.
This was for familiarizing the respondent with the EQ-5D-
3L descriptive system. Before proceeding to valuation of
the EQ-5D-3L health states, the interviewer went through
an example TTO task with the respondent to make sure the
latter understand the questions. The interviewer then
showed ten EQ-5D-3L health states, one at a time in a
random order, to the respondent and conducted the valua-
tion tasks as described below. The respondent’s demo-
graphic characteristics and feedback on the interview were
collected after the completion of the valuation tasks.
In each valuation task, the TTO value of a health state
was elicited by asking a respondent to state preferences for
11 pairs of scenarios each comprising two options: (a) to
live in the health state for 10 years and then die; and (b) to
live in full health for xyears and then die. The xvalues
were integers ranging from 0 to 10, and therefore there
were 11 pairs of scenarios. As our main goal was to esti-
mate the average values of the health states, we did not
pinpoint the exact indifference points in the TTO tasks by
asking about preferences at any fractions of years. For each
pair of the scenarios, the response could be (a), (b), or
N. Luo et al.
equal preference (no preference). As previous studies
found that TTO values were affected by the starting point
of xvalues [28], two starting points, 0 and 10 years, were
used in the valuation tasks in our study. For each health
state, half of the respondents started with a question asking
them to compare 0 years of full health (i.e. immediate
death) with 10 years of life in the health state, and the
length of full health in subsequent questions was increased
to 10 years, by 1-year increments (i.e. the bottom-up
sequence); the other half of the respondents started by
comparing 10 years of full heath with 10 years of life in
the health state, and then the length of full health was
decreased to 0 years by 1-year increments (i.e. the top-
down sequence). Respondents’ preferences for all 11 pairs
of scenarios were inquired about, even though they might
have stated equal preference for one or more pairs of
scenarios. This was different from previous studies in
which the iteration procedure stopped as soon as indiffer-
ence or change of preference occurred.
If a respondent’s preference was (b) when the xvalue
was 0 (i.e. immediate death), another 11 pairs of scenarios
would be posed to the respondent for stating preferences,
each pair comprising two options: (a) to live in full health
for 10 years followed by 10 years in the health state, and
then die; and (b) to live in full health for yyears and then
die. The values for ywere integers ranging from 0 to 10,
and the values were used in the order from 10 to 0 in all
interviews. Again, respondents’ stated preferences for all
11 pairs of scenarios were inquired. Respondents could
choose to redo the current or a completed TTO task if they
wanted to change their answers to any questions before the
end of the interview. As in previous EQ-5D-3L valuation
studies [4,7], a time board and show cards were used as
visual aids to help respondents comprehend the different
life scenarios.
2.3 The Health States
A total of 80 EQ-5D-3L health states were included in the
study for valuation. Those included state 33333 and 79
states that we selected through reviewing primary EQ-5D-
3L data collected in past surveys of general and patient
populations. Using pre-defined definitions, 24, 32, and 24
of the selected health states fell into the category of mild,
moderate, and severe health states, respectively. Mild states
had no domain in level 3 and up to three domains in level 2,
severe states had at least two domains in level 3, and all
other states were considered moderate. The 80 health states
were stratified and randomly distributed into eight blocks,
with each block comprising three mild, four moderate, and
three severe states (Table 1). Arrangements were made
such that each block of health states were rated by similar
number of respondents.
We elected to include more health states than previous
studies using the MVH protocol (n=42) [4,7]. This was
based on the rationale that direct valuation of more health
states decreases the interpolation space for value estima-
tion. Two recent valuation studies valued approximately
100 EQ-5D-3L health states using the TTO method [8,29].
2.4 Statistical Analysis
2.4.1 Calculation of Time Trade-Off (TTO) Values
The TTO value of each health state was estimated for each
respondent who valued that health state as follows. If the
health state was considered better than death, the value was
given by X/10, where Xis the xvalue at which equal
preference was stated. If there were multiple xvalues,
Xwas the mean of the values; if there was no value at
which equal preference was stated, Xwas the mean of the
maximum value at which option (a) was preferred and the
minimum value at which option (b) was preferred. For
example, if at 0–5 years option (a) was preferred and at
6–10 years option (b) was preferred, the TTO value of the
health state would be [(5 ?6)/2]/10 =0.55. Similarly, the
TTO value of a health state considered as worse than death
was given by (Y-10)/10. Poor data quality was considered
if a respondent rated all ten health states as having the same
value or worse than death, or rated only two or fewer health
states. Data of poor quality was excluded from the mod-
eling analysis.
2.4.2 Modeling of TTO Values
Linear regression models using various functional forms
were tested with the TTO data. Consistent with previous
studies [4,7], model inputs were characteristics of the
health states only and disutility (i.e. 1 minus TTO value) of
the health states was modeled. The functional forms we
tested included the main-effects, N3, and D1 models. The
main-effects model contained ten dummy variables,
including M2 and M3 for mobility, S2 and S3 for self-care,
U2 and U3 for usual activities, P2 and P3 for pain/dis-
comfort, and A2 and A3 for anxiety/depression. The ten
dummy variables were used to the estimate the effects of
the individual health problems on valuation of health
states. Since each domain was described using three dif-
ferent levels, two dummy variables were needed. The N3
model was the main-effects model with the addition of a
dummy variable, N3. The N3 term took the value of 1 if
any domain of a health state is in level 3; otherwise the
value is 0 [4]. The N3 term was used to estimate the extra
effects of any extreme problems on the value of a health
state. Both the main-effects and N3 models were estimated
with and without a constant in our study. Without the
Valuation of EQ-5D-3L Health States in Singapore
constant, the models assumed no disutility for state 11111,
which is consistent with the expected utility theory and was
recommended [6]. The D1 model contained the D1, and I2,
I22, I3 and I32 terms in addition to the ten main effects [7].
The D1 term represented the number of domains at level 2
or level 3 minus 1; the I2 term was the number of domains
at level 2 minus 1; the I3 term was the number of domains
at level 3 minus 1; the I22 and the I32 terms were squared
I2 and I3 terms, respectively. The D1, I2, and I3 were
created to estimate whether the additional effects of various
levels of problems on health-state valuation were linear to
the number of problems in a health state. The addition of
I22 and I32 allowed the assessment of a non-linear rela-
tionship between the number of problems and the TTO
valuation. The D1 model did not contain the constant term
as the D1 term was a pseudo constant term. Hence, a total
of five different functional forms were tested: (1) main
effects with constant; (2) main effects without constant; (3)
main effects with constant and the N3 term; (4) main
effects with the N3 term only; and (5) main effects with D1
and related terms. Model specifications are summarized in
Table 2.
All functional forms were estimated using the TTO data
at both individual and aggregate levels. At individual level,
the assumption of independence for the ordinary least
square (OLS) estimator was violated since each respondent
valued multiple health states. For this reason, the random
effects (RE) model and the fixed effects (FE) estimators
were also used. The Lagrange Multiplier (LM) test [30]
was used to test the model assumption for OLS versus RE
or FE models, and the Hausman test [31] was used to
compare the appropriateness of the RE and FE models. At
aggregate level, the disutility of the 80 health states (cal-
culated as 1 minus the mean TTO value) was modeled
using the OLS estimator. All models were tested for het-
eroskedasticity using the Breusch–Pagan test [32] and
normality of residuals were tested using the Shaprio–Wilk
test [33].
2.4.3 Evaluation of Model Performance
Four criteria were used to determine which model had the
best performance and therefore should be selected to predict
the EQ-5D-3L values. These were, in order of high to low
priority, consistency, bias, precision, and parsimony. Con-
sistency required the predictions for measured and unmea-
sured EQ-5D-3L states to be consistent with the utility
theory. That means that more severe problems (e.g.
‘extreme pain/discomfort’) should be associated with dis-
utility not lower than less severe problems (e.g. ‘moderate
pain/discomfort’) and that worse health states (e.g. 11133)
should not have higher TTO values than better health states
(e.g. 11132). While assessment of prediction consistency
could be based on the estimated regression coefficients for
the main-effects and N3 models, it was not straightforward
for the D1 model because of its complex model specifica-
tion. We therefore compared D1 model predicted values for
all possible pairs of EQ-5D-3L health states. Prediction bias
referred to systematically higher or lower predictions than
observed values. We were particularly concerned about
prediction bias for very mild and very severe health states.
For assessing prediction bias, we examined the overall
agreement between predicted and observed mean TTO
values for the 80 health states using intraclass correlation
coefficient (ICC) [34]. We also used the Bland–Altman plot
[35] to visually assess the prediction bias in different seg-
ments of the utility scale. Prediction precision of the models
was evaluated in terms of mean absolute error (MAE), the
numbers of prediction errors greater than 0.10 and 0.20.
Lastly, if multiple models performed similarly in consis-
tency, bias, and precision, the model using the simplest
functional form would be preferred (i.e. model parsimony).
Table 1 EQ-5D-3L health states valued by block
Severity Block A Block B Block C Block D Block E Block F Block G Block H
Mild 21121 21122 12122 11221 21212 12212 12121 11222
22112 11112 12221 11211 12111 12211 22121 11122
22211 21112 11121 21221 21211 21111 11212 22111
Moderate 32221 23221 22212 22222 11232 21222 21131 12231
11132 21231 13221 23222 22221 21113 12223 12222
11123 11223 21312 21232 22311 11113 22223 11131
21223 23211 22321 12232 21322 21321 11213 21311
Severe 33323 32322 21233 22331 33222 13332 23322 33312
22323 23332 12332 23312 33311 13333 33321 11233
23232 33333 33331 23333 23311 23323 33332 32332
The first to fifth digit in each vector represents mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. The numbers 1, 2,
and 3 represent no problems (level 1), some or moderate problems (level 2), and extreme problems (level 3), respectively
N. Luo et al.
In both individual- and aggregate-level modeling ana-
lysis, sampling weights were applied to the data to reflect
the distributions of age, gender, and ethnicity of the general
adult Singaporean population in 2010 [36]. Analyses were
performed with SAS 9.2 (SAS Institute Inc., Cary, NC,
USA) and STATA 12.0 (StataCorp LP, College Station,
TX, USA.
3 Results
3.1 Respondents’ Characteristics
A total of 505 respondents were successfully interviewed,
representing a response rate of 46.8 %. After excluding
respondents whose TTO scores were either the same
(n=2) or negative for all ten health states they rated
(n=47), data from 456 respondents was used for model-
ing analysis. There were no significant differences in so-
ciodemographics between excluded and included
respondents, and characteristics of the included respon-
dents were similar to those of the general adult Singapo-
rean population except that by design there was a higher
proportion of Malays (13.3 %) and Indians (17.3 %) in our
well educated sample (Table 3).
3.2 Distribution of TTO Values
The 456 respondents generated a total of 4,538 TTO values
for the 80 health states. The distribution of those TTO
values was bimodal, with the mean being 0.09 (standard
deviation 0.809) [Fig. 1]. Among the 4,538 values, 1,363
(30.0 %) were -1.0, 138 (3.0 %) were 0, and 167 (3.7 %)
were 1.0. Overall, TTO values derived from tasks using the
starting point of 0 years (mean 0.036) was not statistically
lower than those from tasks starting the TTO questions
from 10 years (mean 0.152; p=0.2865). One or more
indifference points were indicated by 24 respondents in 47
of the 4,538 TTO tasks (1 %); among these, multiple
indifference responses were observed in 31 TTO tasks
completed by 16 respondents.
3.3 Modeling Analysis
Results on modeling of individual-level data are showed in
Table 4. Regression coefficients for the ten main effects in
each model were statistically significant and consistent
with the utility theory in that more severe health problems
are associated with higher disutility. For example, level 2
mobility problems (M2) and level 3 mobility problems
(M3) were associated with a disutility of 0.1135 and
Table 2 Modeling of time
trade-off values: model
specifications
Independent variable (coding) Functional form
Main-effects terms Modeling of individual-level data
M2 (1 if some problems in mobility; 0 otherwise) Y=constant ?main-effects terms
M3 (1 if confined to bed; 0 otherwise) Y=main-effects terms
S2 (1 if some problems in self-care; 0 otherwise) Y=constant ?main-effects terms ?N3
term
S3 (1 if extreme problems in self-care; 0 otherwise) Y=main-effects terms ?N3 term
U2 (1 if some problems in usual activities; 0 otherwise) Y=main-effects terms ?D1 terms
U3 (1 if extreme problems in usual activities; 0 otherwise)
P2 (1 if moderate pain/discomfort; 0 otherwise) Modeling of aggregate-level data
P3 (1 if extreme pain/discomfort; 0 otherwise) Y=constant ?main-effects terms
A2 (1 if moderately anxious/depressed; 0 otherwise) Y=main-effects terms
A3 (1 if extremely anxious/depressed; 0 otherwise) Y=constant ?main-effects terms ?N3
term
N3 term
N3 (1 if extreme problems in any domain; 0 otherwise) Y=main-effects terms ?N3 term
Y=main-effects terms ?D1 terms
D1 terms
D1 (the number of domains in some or extreme problems
minus 1)
I2 (the number of domains in some problems minus 1)
I22 (the square of I2)
I3 (the number of domains in extreme problems minus 1)
I32 (the square of I3)
Valuation of EQ-5D-3L Health States in Singapore
Table 3 Sociodemographic characteristics of included and excluded respondents
Characteristic n(%) p-Value Singapore population (%)
Included respondents
(n=456)
Excluded respondents
(n=49)
Citizenship
Citizen 409 (89.6) 46 (93.9) 0.325 86.4
Permanent residence 47 (10.4) 3 (6.1) 13.6
Gender
Male 228 (50.0) 19 (38.8) 0.199 49.2
Female 228 (50.0) 30 (61.2) 50.8
Ethnicity
Chinese 310 (67.9) 29 (59.2) 0.475 74.2
Malay 60 (13.3) 10 (20.4) 13.3
Indian 79 (17.3) 9 (18.4) 9.2
Others 7 (1.6) 1 (2.0) 3.3
Age (years)
20–29 78 (17.0) 4 (8.2) 0.325 17.7
30–39 88 (19.3) 7 (14.3) 20.8
40–49 89 (19.5) 13 (26.5) 21.5
50–59 106 (23.2) 13 (26.2) 19.8
60 or more 95 (21.0) 12 (24.5) 20.2
Education level
No formal education 8 (1.8) 1 (2.0) 0.328 19.6
Primary 45 (10.0) 5 (10.2) 12.1
Lower secondary 19 (4.0) 5 (10.2) 10.9
Secondary 121 (26.3) 21 (42.9) 24.6
Upper secondary 74 (16.4) 5 (10.2) 9.9
Polytechnic diploma 67 (14.4) 2 (4.1) 6.2
Other diploma 29 (6.4) 4 (8.2) 4.9
University 94 (20.8) 6 (12.2) 11.7
Working status
Working 297 (65.3) 27 (55.1) 0.412
Homemaker 68 (14.8) 5 (10.2)
Student 25 (5.5) 0 (0.0)
Full-time National Service 5 (1.1) 1 (2.0)
Unemployed 13 (2.9) 3 (6.1)
Retired 47 (10.4) 4 (8.2)
Chronic conditions
Yes 128 (28.3) 17 (34.7) 0.500
No 328 (71.7) 32 (65.3)
EQ-5D
Mobility
Some problems 78 (17.3) 5 (10.2) 0.189
Extreme problems 11 (2.4) 0 (0.0)
Self-care
Some problems 8 (1.8) 1 (2.0) 0.899
Extreme problems 7 (1.6) 0 (0.0)
Usual activities
Some problems 18 (4.0) 1 (2.0) 0.484
Extreme problems 10 (2.2) 1 (2.0) 0.939
N. Luo et al.
0.2898, respectively, in the RE model including the main
effects and the N3 term. As can be seen in Table 4, N3 and
D1 models performed better than main-effects models in
prediction bias measured by ICC and prediction precision
measured by MAE. The LM test suggested that the RE and
FE estimators were more appropriate than the OLS esti-
mator (F=3065.11; p\0.0001). The Hausman test
indicated that the RE estimator was not as efficient as the
FE estimator for the main effect model with a constant and
the N3 model with a constant (p\0.05 for both). Pre-
dictions of all models using the individual-level data did
not pass the Breusch–Pagan test for heteroskedasticity or
the Shapiro–Wilk test for normality of residuals
(p\0.001), suggesting certain degrees of model
misspecification.
Modeling results of the mean TTO scores for the 80
health states using the OLS estimator are displayed in
Table 5. Regression coefficients estimated for the main
effects, N3 and D1 terms were statistically significant and
in the ranges as we expected. Similar to findings in indi-
vidual-level modeling, the N3 and D1 models outper-
formed the main-effects models in prediction bias and
precision. However, different from models based on indi-
vidual-level data, all the models based on the aggregate
data passed the Breusch–Pagan test for heteroscedasticity
and the Shapiro–Wilk test for normality of residuals.
Bland–Altman plots revealed that the N3 model without a
constant did not generally suffer from prediction bias at
any segments of the utility scale, while all other models
predicted lower TTO values for health states with mild and/
or severe health problems (Fig. 2). The MAE of this model
was 0.1137, and 35 of 80 values it predicted had errors less
than 0.10 in absolute magnitude. Based on this model, the
utility value could be calculated using the formula below;
the highest and the lowest utility value was 0.854 (for state
11121) and -0.769 (for state 33333), respectively.
Fig. 1 Distribution of observed
time trade-off values
Table 3 continued
Characteristic n(%) p-Value Singapore population (%)
Included respondents
(n=456)
Excluded respondents
(n=49)
Pain/discomfort
Some problems 111 (24.6) 8 (16.3) 0.179
Extreme problems 19 (4.2) 1 (2.0) 0.451
Anxiety/depression
Some problems 29 (6.4) 4 (8.2) 0.614
Extreme problems 5 (1.1) 0 (0.0)
Valuation of EQ-5D-3L Health States in Singapore
Table 4 Parameter estimates and goodness-of-fit statistics for models using individual-level data
Variable FE RE
Main effects
with constant
N3 with
constant
Main effects
with constant
Main effects
without constant
N3 with
constant
N3 without
constant
D1
Constant 0.1626 0.1059 0.1544 0.0978
M2 0.0932 0.0856 0.1028 0.1352 0.0940 0.1135 0.2018
M3 0.2952 0.2759 0.3011 0.3204 0.2788 0.2898 0.5390
S2 0.1380 0.1619 0.1369 0.1588 0.1607 0.1751 0.2736
S3 0.3370 0.3049 0.3407 0.3487 0.3083 0.3119 0.5498
U2 0.2528 0.2058 0.2559 0.2840 0.2095 0.2249 0.3176
U3 0.4534 0.3163 0.4518 0.4673 0.3162 0.3202 0.5409
P2 0.1483 0.1306 0.1468 0.1757 0.1288 0.1458 0.2443
P3 0.3499 0.2294 0.3460 0.3768 0.2252 0.2392 0.4812
A2 0.1110 0.1323 0.1116 0.1336 0.1329 0.1472 0.2299
A3 0.3645 0.2694 0.3643 0.3884 0.2685 0.2794 0.5240
D1 -0.1148
I32 -0.0456
N3 0.2816 0.2825 0.2940
ICC 0.910 0.937 0.908 0.914 0.939 0.935 0.928
MAE 0.1334 0.1137 0.1326 0.1393 0.1125 0.1187 0.1288
No. (of 80) [0.1 42 37 43 37 38 43 40
No. (of 80) [0.2 18 15 17 21 13 14 19
FE fixed effect regression, RE random effect regression, ICC intraclass correlation coefficient, MAE mean absolute error
Table 5 Parameter estimates and goodness-of-fit statistics for OLS models using aggregate-level data
Variable Main effects with constant Main effects without constant N3 with constant N3 without constant D1
Constant 0.1335 0.0809
M2 0.1558 0.2017 0.1419 0.1678 0.2658
M3 0.3280 0.3549 0.2906 0.3040 0.6961
S2 0.1160 0.1489 0.1409 0.1615 0.2485
S3 0.3826 0.3944 0.3421 0.3465 0.6493
U2 0.2730 0.3184 0.2316 0.2555 0.3649
U3 0.4240 0.4549 0.3099 0.3209 0.6115
P2 0.1469 0.1857 0.1249 0.1462 0.2529
P3 0.3289 0.3694 0.2125 0.2291 0.5716
A2 0.1080 0.1439 0.1280 0.1501 0.2376
A3 0.3563 0.3942 0.2620 0.2784 0.5219
D1 -0.1310
I32 -0.0915
N3 0.2740 0.2905
ICC 0.914 0.919 0.938 0.941 0.938
MAE 0.1324 0.1318 0.1125 0.1137 0.1153
No. (of 80) [0.1 45 42 37 35 34
No. (of 80) [0.2 17 15 13 13 14
OLS ordinary least square, ICC intraclass correlation coefficient, MAE mean absolute error
N. Luo et al.
Fig. 2 Bland–Altman plots of actual and predicted values based on OLS regression at aggregated level. Xaxis is the average between actual and predicted values; Yaxis is the actual values
minus predicted values. The middle dotted lines represent mean differences between actual and predicted values; the upper and lower dotted lines represent ±1.96 standard deviation of the
mean differences. OLS ordinary least square
Valuation of EQ-5D-3L Health States in Singapore
Utility ¼1#disutility ¼1#0:1678 $M2#0:3040$
M3#0:1615 $S2#0:3465 $S3 #0:2555$
U2#0:3209 $U3#0:1462 $P2#0:2291$
P3#0:1501 $A2 #0:2784 $A3#0:2905 $N3:
4 Discussion
It is worth noting that the design of our study differed from
that of past EQ-5D-3L valuation studies in several ways.
First, health states considered worse than death were val-
ued using the lead-time TTO method [37], which allowed
health states better and worse than death to be measured in
the same timeframe (i.e. 10 years). In MVH and other EQ-
5D-3L valuation studies, states better and worse than death
were valued using different timeframes and scenarios. It
should be noted that the lead-time TTO allows valuation of
states better and worse than death using a uniform proce-
dure [37]. However, we did not use the lead-time TTO to
value states better than death because lead-time TTO
resulted in low values for states better than death in a
previous study [38]. Another difference between lead-time
and classic TTO procedures is data censoring. The negative
value in our study and the MVH study was censored at
-1.0 and -39, respectively. In the MVH study, all nega-
tive values were arbitrarily compressed to the range of
-1.0 to 0; in contrast, we did not perform such data
transformation. Therefore, the estimated values in our
study and previous studies were not really comparable.
Second, we measured approximately one-third of all the
EQ-5D-3L health states which were observed in empirical
studies. Most previous studies estimated the EQ-5D-3L
value set based on only 42 directly measured health states
[4,7,11–16]. It can be argued that selecting health states
observed in reality is more meaningful than selecting
health states in order to evenly cover the whole space the
descriptive system defines. Some of EQ-5D-5L health
states are never or rarely observed in health surveys
because those states are either implausible (e.g. 33111) or
too severe (e.g. 33333). Valuation of those rarely observed
health states would be difficult to respondents and the
values of those states are less useful. Therefore, we
selected health states by reviewing primary EQ-5D data
collected from various populations. By directly valuing
approximately one-third of the 243 EQ-5D health states,
we substantially reduced the interpolation space for esti-
mating the value set. Theoretically, the smaller the inter-
polation space, the smaller the prediction bias (i.e.
systematic error) for the unobserved health states would be.
Nevertheless, the increase of the number of health states
was at the cost of decreasing the number of observations
for each health state, given that the sample size and the
number of health states each participant valued were fixed.
As a result, the prediction precision (i.e. random error) for
the observed health states was impaired (see below for
more discussion on prediction precision).
Third, we asked all respondents to state preferences for
the same set of health scenarios in each TTO task and
posed those scenarios to respondents in two different
orders. In contrast, TTO tasks stopped as soon as respon-
dents indicated indifference and each TTO task began with
a scenario of 5 years of full health in the MVH study. A
recent study showed that respondents tended to state
indifference within the first few TTO questions [38], and an
experiment in Norway found association between TTO
values and the starting points of the iteration procedure
[28]. Therefore, we required our respondents to consider
the full range of scenarios even though they stated indif-
ference at the beginning of the iteration process. Using this
design, we minimized premature responses and forced
respondents to think more thoroughly about their answers.
A previous study found that some respondents formed their
responses during the valuation tasks rather than knowing
their indifference points before the tasks [39]. We did not
find a significant difference in TTO values between
respondents using different starting points, suggesting that
our elicitation procedure was free from the starting point
bias.
Our modeling results suggested that the N3 model
without a constant estimated using the aggregate-level data
should be recommended for generating the EQ-5D-3L
health states in Singapore. Our modeling results were
consistent with those in previous studies in several ways.
First, aggregate-level data achieved better modeling results
than individual-level data. Two previous EQ-5D-3L valu-
ation studies also recommended a model based on aggre-
gate-level data [8,29]. Both studies directly measured
approximately 100 EQ-5D-3L states. Other studies mea-
sured only 42 or fewer states and therefore did not perform
aggregate-level modeling. Second, models with the N3
term outperformed those without the term, which was the
result in most of the previous EQ-5D-3L valuation studies
[4,8,10,11,14–16]. Third, the model without a constant
fit the data better than those with a constant. Although no
EQ-5D-3L valuation studies so far recommended such a
model specification, Brazier et al. [6] recommended a
similar model specification in a study of SG values for SF-
6D health states. The decision was based on both utility
theory and the modeling results. The existence of a positive
constant means the value for full health is lower than 1.0,
which is against the utility theory and therefore suggests
the valuation method is not optimal. Our study found that
the existence of the constant would lead to underestimated
values, especially for mild health states. It is not known
whether the same effect existed in previous studies and
therefore future studies are needed to further investigate the
N. Luo et al.
implications of the constant in the modeling of health-state
utility values.
On the other hand, the results of our study were
markedly different from those from previous studies in
two aspects. One difference was that the TTO values in
our study were lower than those observed in previous
studies. The value for 145 (59.7 %) EQ-5D-3L states was
negative in our study; there could be more such states if
we had not excluded the 47 respondents who valued all
health states as worse than death. A large number of
negative values was only observed in the UK (n=84) [4]
and Thai (n=68) [10] EQ-5D-3L value sets. As a result,
the TTO values in our study were lower than those
observed in previous studies, such as those conducted in
the US, UK, Japan, and Thailand (Fig. 3). The low TTO
values could be partially explained by data collection and
analysis procedures. First of all, as we previously dis-
cussed, all other EQ-5D-3L valuation studies perform data
transformation for negative values but we did not do this.
Also, the duration for all health states was 10 years in our
study, while in all other studies the duration for states
worse than death was less than 10 years. Evidence showed
that severe health states are less preferred if they are
longer in duration [40]. We suspect that this was the main
reason for marked censoring effects in our study. In the
meanwhile, we could not exclude the possibility that
Singaporeans do not mind dying or dying sooner in order
to avoid severe health problems than people in other
countries. Indeed, a recent study found that Singaporeans
valued EQ-5D-5L (a new EQ-5D classification system in
which each domain is described into five levels) health
states as less desirable than Chinese in China [38]. The
reason might be the concern about healthcare costs, which
are mainly borne by individuals in Singapore. A recent
study found that Singaporean cancer patients would not
choose to use better but more expensive treatment if they
had to pay for it [41].
Another major difference was that the prediction
precision of our models was low. Actually, we assessed
more models than what we reported in this study.
However, the MAE of all models was [0.10, which was
higher than the MAE in previous studies (range 0.02 [9]
to 0.08 [10]). The poorer model prediction precision was
unlikely due to the large number of health states valued
because the MAE in two previous studies, in which
approximately 100 states were valued, was less than 0.03
[8,29]. The poorer prediction precision of our models
was an indication of greater data variability, which
should be the joint effects of several design factors.
First, multiple starting points in the valuation exercise
should have resulted in greater variance in data. TTO
values were found to be associated with the starting
point [28]. Different from our study, all previous EQ-5D-
3L valuation studies followed the MVH protocol to use a
single starting point. Second, we used a relatively small
number of respondents to value a large number of health
states. As a result, each health state was only rated by
approximately 60 respondents. Third, we did not trans-
form negative values as investigators of previous studies.
The transformation performed in other studies com-
pressed data distribution and variability.
The main limitation of the study was the small number
of observations for each health state. It should have con-
tributed to the relatively poor prediction precision of our
models. However, the small number of observations per
state allowed valuation of 80 health states, which reduced
the interpolation space and the prediction bias for unob-
served health states. Another limitation of our study was
inclusion of English-speaking Singaporeans only. Due to
limited resources, we excluded Singaporeans who could
not speak English. It should be noted that it is important to
use a representative sample in such a valuation study.
However, English is the primary language in Singapore
and about 80 % of the total Singaporean population can
speak English [42]. In addition, we did not find significant
differences in valuation of EQ-5D-5L health states
Fig. 3 Time trade-off values of the 243 EQ-5D-3L health states
estimated in Singapore, Thailand, Japan, US, and UK. The health
states are ranked in decreasing order according to the values estimated
in Singapore
Valuation of EQ-5D-3L Health States in Singapore
between Singaporean Chinese who were interviewed in
English or Chinese using data from a previous study [38]
Lastly, we did not ask respondents to rank and rate the
health states before the TTO tasks, as in previous studies.
The ranking and rating exercises may familiarize partici-
pants with those health states, and result in more accurate
TTO values.
5 Conclusions
The EQ-5D-3L value set can be estimated using the TTO
method in multi-cultural, multi-ethnic Singapore. Although
the estimation precision is not optimal, the health-state
preference values generated in this study are useful to
health service researchers in the country before estimates
with smaller errors are available.
Acknowledgments This study was funded by the National Uni-
versity of Singapore.
Contribution of Authors Nan Luo designed the study; Pei Wang
analyzed the data; Nan Luo and Pei Wang drafted the manuscript; and
Julian Thumboo, Yee-Wei Lim and Hubertus Vrijhoef contributed the
interpretation of the results and commented on and/or edited the drafts
of the manuscript. Nan Luo acts as guarantor for the overall content.
Conflicts of Interest None.
References
1. Torrance GW. Measurement of health state utilities for economic
appraisal. J Health Econ. 1986;5:1–30.
2. Dolan P. Valuing health-related quality of life: issues and con-
troversies. Pharmacoeconomics. 1999;15:119–27.
3. Torrance GW, Thomas WH, Sackett DL. A utility maximization
model for evaluation of health care programs. Health Serv Res.
1972;7:118–33.
4. Dolan P. Modeling valuations for EuroQol health states. Med
Care. 1997;35:1095–108.
5. Feeny DH, Furlong WJ, Torrance GW, et al. Multiattribute and
single attribute utility functions for the health utilities index mark
3 system. Med Care. 2002;40:113–28.
6. Brazier JE, Roberts J, Deverill MD. The estimation of a prefer-
ence-based measure of health from the SF-36. J Health Econ.
2002;21:271–92.
7. Shaw JW, Johnson JA, Coons SJ. US valuation of the EQ-5D
health states: development and testing of the D1 valuation model.
Med Care. 2007;45:238–44.
8. Lee YK, Nam HS, Chuang LH, et al. South Korean time trade-off
values for EQ-5D health states: modeling with observed values
for 101 health states. Value Health. 2009;12:1187–93.
9. Tsuchiya A, Ikeda S, Ikegami N, et al. Estimating an EQ-5D popu-
lation value set: the case of Japan. Health Econ. 2002;11:341–53.
10. Tongsiri S, Cairns J. Estimating population-based values for EQ-
5D health states in Thailand. Value Health. 2011;14:1142–5.
11. Yusof FA, Goh A, Azmi S. Estimating an EQ-5D value set for
Malaysia using time trade-off and visual analogue scale methods.
Value Health. 2012;15:S85–90.
12. Golicki D, Jakubczyk M, Niewada M, et al. Valuation of EQ-5D
health states in Poland: first TTO-based social value set in central
and Eastern Europe. Value Health. 2010;13:289–97.
13. Viney R, Norman R, King MT, et al. Time trade-off derived EQ-
5D weights for Australia. Value Health. 2011;14:928–36.
14. Lamers LM, McDonnell J, Stalmeier PF, et al. The Dutch tariff:
results and arguments for an effective design for national EQ-5D
valuation studies. Health Econ. 2006;15:1121–32.
15. Zarate V, Kind P, Valenzuela P, et al. Social evaluation of EQ-5D
health states: the Chilean case. Value Health. 2011;14:1135–41.
16. Wittrup-Jensen KU, Lauridsen J, Gudex C, et al. Generation of a
Danish TTO value set for EQ-5D health states. Scand J Public
Health. 2009;37:459–66.
17. Norman R, Cronin P, Viney R, et al. International comparisons in
valuing EQ-5D health states: a review and analysis. Value Health.
2009;12:1194–200.
18. Luo N, Chew LH, Fong KY, et al. Do English and Chinese EQ-
5D versions demonstrate measurement equivalence? An explor-
atory study. Health Qual Life Outcomes. 2003;1:7.
19. Luo N, Chew LH, Fong KY, et al. Validity and reliability of the
EQ-5D self-report questionnaire in English-speaking Asian
patients with rheumatic diseases in Singapore. Qual Life Res.
2003;12:87–92.
20. Luo N, Chew LH, Fong KY, et al. Validity and reliability of the
EQ-5D self-report questionnaire in Chinese-speaking Asian
patients with rheumatic diseases in Singapore. Ann Acad Med
Singapore. 2003;32:685–90.
21. Luo N, Low S, Lau PN, et al. Is EQ-5D a valid quality of life
instrument in patients with Parkinson’s disease? A study in Sin-
gapore. Ann Acad Med Singapore. 2009;38:521–8.
22. Gao, F, Ng GY, Cheung YB, et al. The Singapore English and
Chinese version of the EQ-5D achieved measurement equiva-
lence in cancer patients. J Clin Epidemiol. 2009;62:206–13.
23. Abdin E, Subramanian M, Vaingankar JA, et al. Measuring
health-related quality of life among adults in Singapore: popu-
lation norms for the EQ-5D. Qual Life Res. 2013;22(10):
2983–91.
24. Wang YT, Lim HY, Tai D, et al. The impact of irritable bowel
syndrome on health-related quality of life: a Singapore perspec-
tive. BMC Gastroenterol. 2012;12:104.
25. Chong SA, Abdin E, Luo N, et al. Prevalence and impact of
mental and physical comorbidity in the adult Singapore popula-
tion. Ann Acad Med Singapore. 2012;41:105–14.
26. Zhang XH, Li SC, Fong KY, et al. The impact of health literacy
on health-related quality of life (HRQoL) and utility assessment
among patients with rheumatic diseases. Value Health.
2009;12:S106–9.
27. Gold MR, Patrick DL, Torrance GW, et al. Identifying and val-
uing outcomes. In: Gold MR, Siegel JE, Russell LB, editors.
Cost-effectiveness in health and medicine. New York: Oxford
University Press; 1996. p. 82–134.
28. Samuelsen CH, Augestad LA, Stavem K, et al. Anchoring effects
in the lead-time time trade-off. In: Proceedings of the 29th Eu-
roQol Plenary Meeting, 13–15 Sept 2012. The Doelen Convert
and Congress Hall, Rotterdam, The Netherlands.
29. Liu G, Wu H, Sun L, et al. Chinese valuation of EQ-5D health
states with the time trade-off method. Value Health. 2012;
15:A650.
30. Silvey SD. The lagrangian multiplier test. Ann Math Stat.
1959;30:389–407.
31. Hausman J. Specification tests in econometrics. Econometrica.
1978;46:1251–71.
N. Luo et al.
32. Breusch T, Pagan A. A simple test of heteroskedasticity and
random coefficient variation. Econometrica. 1979;47:1287–94.
33. Shapiro SS, Wilk MB. An analysis of variance test for normality
(complete samples). Biometrika. 1965;52:591–611.
34. Fayers PM, Machin D. Quality of life: assessment, analysis and
interpretation. 3rd ed. Chichester: Wiley; 2005.
35. Bland JM, Altman DG. Statistical method for assessing agree-
ment between two methods of clinical measurement. Lancet.
1986;1:307–10.
36. Department of Statistics Singapore. Census of population 2010 sta-
tistical release 1: demographic characteristics, education, language
and religion. http://www.singstat.gov.sg/publications/publications_
and_papers/cop2010/census10_stat_release1.html. Accessed 15 Jul
2013.
37. Devlin NJ, Tsuchiya A, Buckingham K, et al. A uniform time
tradeoff method for states better and worse than dead: feasibility
study of the ‘lead time’ approach. Health Econ. 2011;20:348–61.
38. Luo N, Li M, Stolk EA, et al. The effects of lead time and visual
aids in TTO valuation: a study of the EQ-VT framework. Eur J
Health Econ. 2013;14(Suppl 1):S15–24. doi:10.1007/s10198-
013-0504-1.
39. Brazier J, Ratcliffe J, Salomon J, et al. Measuring and valuing
health benefits for economic evaluation. Oxford: Oxford Uni-
versity Press; 2007.
40. Stalmeier PF, Lamers LM, Busschbach JJ, et al. On the assess-
ment of preferences for health and duration: maximal endurable
time and better than dead preferences. Med Care. 2007;
45:835–41.
41. BMJ Group Blogs. BMJ Supportive & Palliative Care. ‘‘One can
die, but cannot fall ill’’. A Survey on how costs may affect choice
of therapy in Singapore. http://blogs.bmj.com/spcare/2012/04/17/
one-can-die-but-cannot-fall-ill-a-survey-on-how-costs-may-affec
t-choice-of-therapy-in-singapore/. Accessed 15 May 2013.
42. Pakir A. Bilingual education with English as an official language:
sociocultural implications. In: Alatis JE, Tan AH, editors.
Georgetown University round table on languages and linguistics.
Washington, DC: Georgetown University Press; 1999.
Valuation of EQ-5D-3L Health States in Singapore
