Quality indicators for systemic lupus erythematosus based on the 2019 EULAR recommendations: Development and initial validation in a cohort of 220 patients

Abstract

Background
Quality of care is receiving increased attention in systemic lupus erythematosus (SLE). We developed quality indicators (QIs) for SLE based on the 2019 update of European League Against Rheumatism recommendations.

Methods
A total of 44 candidate QIs corresponding to diagnosis, monitoring and treatment, were independently rated for validity and feasibility by 12 experts and analysed by a modified Research and Development Corporation/University of California Los Angeles model. Adherence to the final set of QIs and correlation with disease outcomes (flares, hospitalisations and organ damage) was tested in a cohort of 220 SLE patients with a median monitoring of 2 years (IQR 2–4).

Results
The panel selected a total of 18 QIs as valid and feasible. On average, SLE patients received 54% (95% CI 52.3% to 56.2%) of recommended care, with adherence ranging from 44.7% (95% CI 40.8% to 48.6%) for diagnosis-related QIs to 84.3% (95% CI 80.6% to 87.5%) for treatment-related QIs. Sustained remission or low disease activity were achieved in 26.8% (95% CI 21.1% to 33.2%). Tapering of prednisone dose to less than 7.5 mg/day was achieved in 93.6% (95% CI 88.2% to 97.0%) while 73.5% (95% CI 66.6% to 79.6%) received the recommended hydroxychloroquine dose. Higher adherence to monitoring-related QIs was associated with reduced risk for a composite adverse outcome (flare, hospitalisation or damage accrual) during the last year of observation (OR 0.97 per 1% adherence rate, 95% CI 0.96 to 0.99).

Conclusion
We developed QIs for assessing and improving the care of SLE patients. Initial real-life data suggest face validity, but a variable degree of adherence and a need for further improvement.

Full text

1
Chavatza K, etal. Ann Rheum Dis 2021;0:1–8. doi:10.1136/annrheumdis-2021-220438
Systemic lupus erythematosus
CLINICAL SCIENCE
Quality indicators for systemic lupus erythematosus
based on the 2019 EULAR recommendations:
development and initial validation in a cohort of
220patients
Katerina Chavatza ,1 Myrto Kostopoulou,2 Dionysis Nikolopoulos ,1
Ourania Gioti,3 Konstantina Togia,1 Laura Andreoli,4,5 Martin Aringer ,6
John Boletis,7 Andrea Doria ,8 Frederic A Houssiau ,9 David Jayne,10
Marta Mosca,11 Elisabet Svenungsson ,12 Angela Tincani,4 George Bertsias,13
Antonis Fanouriakis ,1,3 Dimitrios T Boumpas 1,14
To cite: ChavatzaK,
KostopoulouM,
NikolopoulosD, etal.
Ann Rheum Dis Epub ahead
of print: [please include Day
Month Year]. doi:10.1136/
annrheumdis-2021-220438
Handling editor Josef S
Smolen
►Additional online
supplemental material is
published online only. To view,
please visit the journal online
(http:// dx. doi. org/ 10. 1136/
annrheumdis- 2021- 220438).
For numbered affiliations see
end of article.
Correspondence to
Dr Dimitrios T Boumpas,
Rheumatology and Clinical
Immunology, National and
Kapodistrian University of
Athens, Athens 12462, Greece;
boumpasd@ uoc. gr
KC, MK, GB, AF and DTB
contributed equally.
GB, AF and DTB are joint senior
authors.
Received 26 March 2021
Accepted 11 May 2021
© Author(s) (or their
employer(s)) 2021. No
commercial re- use. See rights
and permissions. Published
by BMJ.
ABSTRACT
Background Quality of care is receiving increased attention
in systemic lupus erythematosus (SLE). We developed
quality indicators (QIs) for SLE based on the 2019 update of
European League Against Rheumatism recommendations.
Methods A total of 44 candidate QIs corresponding to
diagnosis, monitoring and treatment, were independently
rated for validity and feasibility by 12 experts and
analysed by a modified Research and Development
Corporation/University of California Los Angeles model.
Adherence to the final set of QIs and correlation with
disease outcomes (flares, hospitalisations and organ
damage) was tested in a cohort of 220 SLE patients with
a median monitoring of 2 years (IQR 2–4).
Results The panel selected a total of 18 QIs as valid and
feasible. On average, SLE patients received 54% (95% CI
52.3% to 56.2%) of recommended care, with adherence
ranging from 44.7% (95% CI 40.8% to 48.6%) for
diagnosis- related QIs to 84.3% (95% CI 80.6% to 87.5%)
for treatment- related QIs. Sustained remission or low disease
activity were achieved in 26.8% (95% CI 21.1% to 33.2%).
Tapering of prednisone dose to less than 7.5 mg/day was
achieved in 93.6% (95% CI 88.2% to 97.0%) while 73.5%
(95% CI 66.6% to 79.6%) received the recommended
hydroxychloroquine dose. Higher adherence to monitoring-
related QIs was associated with reduced risk for a composite
adverse outcome (flare, hospitalisation or damage accrual)
during the last year of observation (OR 0.97 per 1%
adherence rate, 95% CI 0.96 to 0.99).
Conclusion We developed QIs for assessing and improving
the care of SLE patients. Initial real- life data suggest face
validity, but a variable degree of adherence and a need for
further improvement.
INTRODUCTION
Quality of healthcare is defined as ‘the degree to
which health services for individuals and popu-
lations increase the likelihood of desired health
outcomes and are consistent with current profes-
sional knowledge’ (Institute of Medicine 1999).1
The definition applies both to healthcare practi-
tioners and to all settings of care (hospitals, nursing
homes and physicians’ offices). Measurement of
quality can help to identify problems caused by
overuse, underuse or misuse of health resources.
Quality indicators (QIs) is a popular tool to
measure the degree of quality of care received by
patients. QIs are quantitative measures related to
Key messages
What is already known about this subject?
►Systemic lupus erythematosus (SLE) is a
multisystem disease with considerable
morbidity whose care is complicated by its
extreme clinical heterogeneity. The European
League Against Rheumatism (EULAR) has
developed evidence- based and expert opinion-
based recommendations for the management
of various aspects of SLE. Quality indicators
(QIs), a popular tool to measure the degree of
quality of care received by patients, have been
proposed for SLE, but for the most part they
were not based on a comprehensive systematic
literature review (SLR).
What does this study add?
►This is the first comprehensive set of QIs in
SLE based on an extensive SLR of the various
aspects of SLE, performed as part of the EULAR
recommendations for SLE. This study further
capitalises on this work by developing QIs to
detect potential gaps in SLE care and facilitate
the implementation of the guidelines. Initial
real- life data suggest a variable degree of
adherence to the recommendations and identify
areas for further improvement.
How might this impact on clinical practice or
future developments?
►These QIs can be used towards assessing and
improving patient care. QIs may facilitate the
implementation of the EULAR recommendations
by creating a checklist to be used towards
detecting gaps in lupus care and facilitating
efforts towards closing them.

2Chavatza K, etal. Ann Rheum Dis 2021;0:1–8. doi:10.1136/annrheumdis-2021-220438
Systemic lupus erythematosus
the structures, processes or outcomes of care,2 3 derived from
guidelines, systematic literature reviews (SLR) or expert panel
consensus, through the use of a systematic approach repre-
senting the current standard of care. In contrast to most guide-
lines or recommendations, QIs pertain to measurable aspects of
healthcare, describing exactly what to do, when to do it and who
is responsible for doing it, with respect to disease management
and monitoring.4 5
Systemic lupus erythematosus (SLE) is a multisystem disease
with considerable morbidity due to both the disease per se and
the complications of chronic treatment.6 Care in SLE is compli-
cated by the profound clinical heterogeneity and differences
among individual patients. During the last two decades, the
European League Against Rheumatism (EULAR) has developed
evidence- based and expert opinion- based recommendations for
the management of various aspects of SLE, including general
SLE, renal and neuropsychiatric disease, and women’s health
including fertility and pregnancy.7–9 These recommendations
were recently updated.10–12 In addition to these recommenda-
tions, other initiatives such as the treat- to- target in SLE have
also highlighted the importance of a multifaceted care targeting
remission or low disease activity.13
Towards improving patient care, detect potential gaps in
SLE care, and facilitate the implementation of the guidelines,
herein we sought to develop QIs based on the 2019 update of
the EULAR recommendations for SLE, and perform an initial
validation in one academic centre.
METHODS
Overview of the development of preliminary criteria and
selection of the final set
QIs were developed using an adaptation of the Research and
Development Corporation (RAND)/University of California Los
Angeles (UCLA) modified Delphi method, a structured system-
atic approach that combines the best available evidence from an
SLR with the collective expert opinion that has been shown to
be valid in similar applications.14–16 During a two- round process,
a panel of experts assessed the validity and feasibility of the
proposed indicators. The final set of QIs was used to evaluate the
quality of care in an SLE cohort of 220 patients and to explore
possible associations with disease outcomes (figure 1).
Identification of potential indicators and rounds of voting
An inventory of candidate QIs was developed based on the
2019 EULAR recommendations for SLE and the corresponding
SLR.7–12 17 A preliminary set of 44 QIs, which addressed seven
distinct clinical domains and was graded according to the
existing level of evidence, was evaluated by a panel of experts
(nine rheumatologists and three nephrologists from five Euro-
pean countries) and one patient representative (see online
supplemental appendix and online supplemental table 1). Every
panel member was asked to rate each QI item for validity and
feasibility, using a 9- point scale, with 9 representing the highest
possible rating (definitions for validity and feasibility and voting
instructions provided in online supplemental appendix). Panel
members were also asked to comment on the draft QIs and
suggest amendments as required. Following round 1 ratings, an
analysis of the candidate QIs was performed, as described in the
RAND/UCLA Method.14 15 For each candidate QI, the median
rating, median absolute deviation, lower and upper limit inter-
percentile range, were calculated. The Disagreement Index (DI)
was calculated using the equations provided in online supple-
mental appendix. Median validity/feasibility scores of ≤ six were
used to exclude QIs. The other measurements (deviation, agree-
ment/disagreement) were used as additional information for the
selection of the QIs for round 2 (online supplemental table 2).
The moderator (DTB) and two other panel members (GB and
AF) convened to discuss the results and revise the initial set,
based on the ratings from round 1 (online supplemental table
3). Candidate QIs were modified, merged or eliminated accord-
ingly, and the revised set was sent to the expert panel for round
2 of ratings. Experts were asked to rerate the QIs for validity and
feasibility based on the same 9- point scale. Results were analysed
and finalised based on the same principles used during round 1,
reaching the final set of QIs (table 1).
Validation
To perform an initial validation of the proposed QIs, we used
patients from the ‘Attikon’ lupus cohort, based in the largest
tertiary hospital of western Attica and considered as a referral
centre for patients with SLE.18 Patients from the cohort were
included if they (1) fulfilled the EULAR/American College of
Rheumatology (ACR) 2019 and/or 2012 Systemic Lupus Inter-
national Collaborating Clinics (SLICC) classification criteria,19 20
(2) had at least 1 year of follow- up and (3) had at least four
visits over the last year. Included patients were derived from an
Figure 1 Overview of the development of preliminary criteria and
selection of the final set of quality indicators. EULAR, European League
Against Rheumatism; QIs, quality indicators; RAND, Research and
Development Corporation; SLE, systemic lupus erythematosus; UCLA,
University of California Los Angeles.

3
Chavatza K, etal. Ann Rheum Dis 2021;0:1–8. doi:10.1136/annrheumdis-2021-220438
Systemic lupus erythematosus
inception cohort (patients followed from January 2016 (year
of establishment of the Attikon cohort) to date—60%) and a
prevalent cohort (patients with SLE diagnosis before January
2016–40%)].
Chart review and patient interviews were performed retrospec-
tively to assess patient eligibility for each QI (eg, only smokers
were eligible for the corresponding QI regarding counselling for
smoking cessation), adherence to each QI item, and to docu-
ment disease outcomes of interest, specifically flares, SLICC/ACR
damage index (SDI) and hospitalisations (due to cardiovascular
events, infections and flares). Patient interviews were performed
by physicians (KC, OG), with the completion of a patient- reported
questionnaire, for measures that were not universally available in
medical records (eg, counselling for fertility and sun protection).
Table 1 Final set of quality indicators (QIs) and rating during the second round of voting
Validity Feasibility
Median
Mean absolute
deviation Median
Mean absolute
deviation
Screening- diagnostic QIs
1If a patient has SLE
then routine laboratory tests (CBC, serum creatinine and urine analysis) and serological tests (ANA, C3/C4, dsDNA and aPL) should
be performed at diagnosis.
9 0 9 0
2If a patient has lupus nephritis or is at high risk for nephritis (young, clinical and/or serologic activity)
then follow- up tests with CBC, UPr, urine analysis, serum creatinine and lupus serology (C3/C4 and anti- dsDNA) should be performed
every 3–6 months
9 0 9 0
3If a patient with SLE has persistent proteinuria ≥500 mg and/or an unexplained decrease in glomerular filtration rate
and/or active urine sediment
then kidney biopsy is recommended
9 0 8.5 0.7
4If a patient has SLE
then stratification of CVD risk should be performed annually with assessment of both traditional* and disease- related† risk factors
and management of modifiable risk factors including smoking
9 0 8 1.5
5If a patient has SLE
then osteoporosis risk factors (age, sex, steroid use‡, smoking, low vitamin D, low BMI, family history) should be evaluated and
fracture risk§ (high, moderate, low) should be assessed and managed accordingly
9 0 8.5 0.7
Treatment QIs
6If a patient has SLE and is treated with hydroxychloroquine (HCQ)
then the HCQ dose should be ≤5 mg/kg (actual body weight) and be monitored for retinal toxicity with baseline ophthalmologic
evaluation (by visual fields and optical coherence tomography (OCT)) and annual follow- up 5 years after the initiation of HCQ-
provided that risk factors for HCQ- induced retinopathy¶ are not present
8 1.5 8.5 0.7
7If a patient with SLE, receives prednisone ≥7.5 mg for ≥3 months
then prednisone reduction should be attempted to the lowest possible dose
9 0 8 1.5
8If a patient with SLE has lupus nephritis III (±V) or IV (±V)
then immunosuppressive agents in combination with glucocorticoids are recommended
9 0 9 0
9If a patient with lupus nephritis has proteinuria ≥300–500 mg
then ACE inhibitors or ARB are recommended
9 0 9 0
10 If a woman with SLE wishes for pregnancy and has traditional risk factors for pre- eclampsia (kidney disease including
lupus nephritis, BMI ≥35, age >40, diabetes mellitus, hypertension, nulliparity)
then, low- dose aspirin is recommended during pregnancy
8 0.7 9 0
Monitoring QIs
11 If a patient has SLE
then assessment of disease activity, including SLEDAI and PGA, should be recorded in every visit
9 0 8.5 0.7
12 If a patient has SLE
then the SLICC/ACR damage index should be monitored annually
9 0 9 0
13 If a patient has SLE
then management should aim at clinical remission or- if remission cannot be achieved- at low disease activity with acceptable dose of
steroids and well tolerated immunosuppressive agents at maintenance doses
9 0 8.5 0.7
14 If a patient has SLE
then baseline tests at drug initiation and monitoring for drug toxicity should be performed
9 0 9 0
15 If a patient has SLE
then sunscreen protection is recommended
9 0 9 0
16 If a patient has SLE
then in patients with stable/inactive disease, non- live vaccines such as influenza, pneumococcal and HPV vaccines are recommended
while attenuated vaccines (such as HZV vaccine) may be considered
8.5 0.7 8.5 0.7
17 If a premenopausal woman has SLE
then reproductive health and fertility counselling should be provided including the pitfalls of oestrogen use for contraception in SLE
9 0 9 0
18 If a woman of reproductive age has SLE and wishes pregnancy
then counselling about pregnancy (eg, stable/inactive disease for at least 6–12 months) should be provided and baseline tests (eg,
anti- Ro/La, aPL), should be performed and documented
9 0 9 0
*Traditional risk factors: family history of premature CVD, primary hypercholesterolaemia, metabolic syndrome, premature menopause, dyslipidaemia and/or elevated hsCRP.
†SLE- related risk factors: persistently active or flaring disease, kidney involvement/CKD, moderate- high aPL titres, organ damage, use of GCs >5 mg/day, no use of HCQ.
‡Steroid use: with prednisone ≥2.5 mg for ≥3 months, calcium (1000–1200 mg/day) and vitamin D (600–800 IU/day) should be administered.
§Fracture Risk:High Fracture Risk(Previous Fragility Fracture, T- score ≤−2.5, FRAX score for major osteoporotic or hip fracture, beyond different thresholds according to different countries, very high GC doses)
:Antiresorptive treatment with calcium (1000–1200 mg/day) and vitamin D (600–800 IU/day) should be administered. Moderate fracture risk(FRAX score for major osteoporotic or hip fracture beyond different
thresholds according to different countries, prednisone ≥7.5 mg for ≥6 months AND: Z- score-3 OR rapid bone loss ≥10% at hip or spine over 1 year):Calcium (1000–1200 mg/day) and vitamin D (600–800 IU/day)
should be administered and antiresorptive treatment should be considered. Low fracture risk(FRAX score for major osteoporotic or hip fracture beyond different thresholds according to different countries):No need for
antiresorptive treatment. Calcium (1000–1200 mg/day) and vitamin D (600–800 IU/day) should be considered.GC doses: low dose <2.5 mg, medium 2.5–7.5 mg, high >7.5 mg, very high: prednizone ≥30 mg/day or >5
g accumulative dose in the previous year. In high GC doses, FRAX adjustment: multiplication of major osteoporotic fracture risk value (x1.15) and hip fracture value (x1.2).
¶Major risk factors for retinopathy: chronic kidney disease with GFR <60 mL/min, pre- existing retinal or macular disease, use of tamoxifen.
ACE, angiotensin- converting enzyme; ANA, antinuclear antibodies; anti- dsDNA, anti- double stranded DNA antibody; aPL, antiphospholipid antibodies; ARB, angiotensin receptor blocker; BMI, body mass index; CBC,
complete blood count; CVD, cardiovascular disease; GC, glucocorticoid; HPV, human papilloma virus; hsCRP, high- sensitivity C- reactive protein; HZV, herpes zoster virus; PGA, Physician Global Assessment; QI, quality
indicator; SLE, Systemic Lupus Erythematosus; SLEDAI, Systemic Lupus Erythematosus Disease Activity Index; SLICC/ACR, Systemic Lupus International Collaborating Clinics/American College of Rheumatology; UPr, urine
protein.

4Chavatza K, etal. Ann Rheum Dis 2021;0:1–8. doi:10.1136/annrheumdis-2021-220438
Systemic lupus erythematosus
Definitions
Performance for each QI was assessed as ‘fulfilled’, ‘not fulfilled’
or ‘missing’, in order to assess the grade of adherence. Because
most QIs consisted of more than one individual component,
‘fulfilled’ denoted that all components were met, ‘not fulfilled’
that at least one component of the corresponding QI was not
met, and ‘missing’ that data were not recorded in the chart. As
an example, QI1 would be labelled as ‘fulfilled’ if all recom-
mended laboratory and serological tests were obtained at diag-
nosis (see QI1, table 2). By contrast, QI1 would be labelled as
‘not fulfilled’, if at least one of the recommended tests was not
obtained at diagnosis.
Outcomes
Disease- related outcomes were recorded in all patients. Recorded
outcomes included (1) flares, (including major flares), defined as
a measurable increase in disease activity leading to therapeutic
intervention,21 (2) SDI increase (an increase in the SDI score
during the observation period), (3) adverse outcomes related
to glucocorticoids (GC, that is, GC- related complications), (4)
cardiovascular events or serious infections necessitating hospi-
talisation and (5) a composite adverse outcome (CAO), defined
as occurrence of at least one of the following: flares, hospital-
isations or SDI progression. The phenotype of SLE was catego-
rised as mild, moderate or severe according to the British Isles
Table 2 Adherence to 18 selected quality indicators (QIs)
Function of care
Eligible patients Fulfilled
N (%) n (%)
Screening- diagnostic QIs
1If a patient has SLE
then routine laboratory tests (CBC, serum creatinine and urine analysis) and serological tests (ANA, C3/C4, dsDNA and aPL) should be performed at
diagnosis.
132 (60) 64 (48.5)
2If a patient has lupus nephritis or is at high risk for nephritis (young, clinical and/or serologic activity)
then follow- up tests with CBC, UPr, urine analysis, serum creatinine and lupus serology (C3/C4 and dsDNA) should be performed every 3–6 months
68 (31) 33 (48.5)
3If a patient with SLE has persistent proteinuria≥500 mg and/or an unexplained decrease in glomerular filtration rate and/or active
urine sediment
then kidney biopsy is recommended
44 (20) 38 (86.4)
4If a patient has SLE
then stratification of CVD risk should be performed annually with assessment of both traditional and disease- related risk factors and management of
modifiable risk factors including smoking
220 (100) 89 (40.5)
5If a patient has SLE
then osteoporosis risk factors (age, sex, steroid use, smoking, low vitamin D, low BMI, family history) should be evaluated and fracture risk (high,
moderate, low) should be assessed and managed accordingly
220 (100) 100 (45.5)
Treatment QIs
6If a patient has SLE and is treated with hydroxychloroquine (HCQ)
then the HCQ dose should be ≤5 mg/kg (actual body weight) and be monitored for retinal toxicity with baseline ophthalmological evaluation (by
visual fields and optical coherence tomography (OCT)) and annual follow- up 5 years after the initiation of HCQ provided that risk factors for HCQ-
induced retinopathy are not present
189 (86) 139 (73.5)
7If a patient with SLE, receives prednisone ≥7.5 mg for ≥3 months
then prednisone reduction should be attempted to the lowest possible dose
141 (64) 132 (93.6)
8If a patient with SLE has lupus nephritis III (±V) or IV (±V)
then immunosuppressive agents in combination with glucocorticoids are recommended
41 (18.6) 41 (100)
9If a patient with lupus nephritis has proteinuria ≥300–500 mg
then ACE inhibitors or ARBs are recommended
25 (11.3) 22 (88)
10 If a woman with SLE wishes for pregnancy and has traditional risk factors for pre- eclampsia (Kidney disease including lupus nephritis,
BMI ≥35, age >40, diabetes mellitus, hypertension, nulliparity)
then low- dose aspirin is recommended during pregnancy
7 (3) 5 (71.4)
Monitoring QIs
11 If a patient has SLE
then assessment of disease activity, including SLEDAI and PGA, should be recorded in every visit
220 (100) 31 (14.1)
12 If a patient has SLE
then the SLICC/ACR damage index should be monitored annually
220 (100) 63 (28.6)
13 If a patient has SLE
then management should aim at clinical remission or- if remission cannot be achieved- at low disease activity with acceptable dose of steroids and
well tolerated immunosuppressive agents at maintenance doses
220 (100) 59 (26.8)
14 If a patient has SLE
then baseline tests at drug initiation and monitoring for drug toxicity should be performed
193 (87.7) 186 (96.4)
15 If a patient has SLE
then sunscreen protection is recommended
220 (100) 201 (91.4)
16 If a patient has SLE
then in patients with stable/inactive disease, non- live vaccines such as influenza, pneumococcal and HPV vaccines are recommended while
attenuated vaccines (such as HZV vaccine) may be considered
220 (100) 105 (47.7)
17 If a premenopausal woman has SLE
then reproductive health and fertility counselling should be provided including the pitfalls of oestrogen use for contraception in SLE
74 (33.6) 37 (50)
18 If a woman of reproductive age has SLE and wishes pregnancy
then counselling about pregnancy (eg, stable/inactive disease for at least 6–12 months) should be provided and baseline tests (eg, anti- Ro/La, aPL),
should be performed and documented
71 (32.2) 44 (62)
ACE, angiotensin- converting enzyme; ANA, antinuclear antibodies; anti- dsDNA, antidouble stranded DNA antibody; aPL, antiphospholipid antibodies; ARB, angiotensin receptor blocker; BMI, body
mass index; CBC, complete blood count; CVD, cardiovascular disease; HPV, human papilloma virus; HZV, herpes zoster virus; PGA, Physician Global Assessment; QI, quality indicator; SLE, Systemic
Lupus Erythematosus; SLEDAI, Systemic Lupus Erythematosus Disease Activity Index; SLICC/ACR, Systemic Lupus International Collaborating Clinics/American College of Rheumatology; UPr, urine
protein.

5
Chavatza K, etal. Ann Rheum Dis 2021;0:1–8. doi:10.1136/annrheumdis-2021-220438
Systemic lupus erythematosus
Lupus Assessment Group 2004 classification of manifestations,22
combined with expert physician judgement (DTB and AF), as
previously described.18
Statistical analysis
Descriptive statistics were used for continuous variables and
mean/SD or median/IQR values were calculated as appropriate.
Adherence to each QI was calculated as the number of patients
who received the designated care (numerator) divided by the
number of eligible patients for this particular QI (denominator).
In addition, a patient- specific mean score was calculated as the
number of QIs ‘fulfilled’ divided by the number of QIs eligible
for each patient. Accordingly, the average delivered care for each
domain was calculated as a composite score from the cases in
which recommended care was successfully delivered divided by
the number of eligibility events within each domain.
To detect potential differences in adherence between patient
subgroups, we applied three criteria: (1) disease duration (<2
years vs ≥2 years from diagnosis), (2) severity pattern (mild vs
moderate vs severe disease) and (3) origin of cohort (inception
vs prevalent). A separate analysis was performed to compare
the adherence between various QI domains (diagnostic, treat-
ment, monitoring). To compare mean values or the equality of
distribution between different categories the one- way analysis
of variance and the non- parametric Kruskal- Wallis test were
used accordingly. Logistic regression models were performed to
estimate the association between adherence to QIs and adverse
disease outcomes that occurred in two different time frames
(during the total duration of follow- up and during the last year
of follow- up). All models were adjusted for age and disease
duration. All tests were two tailed and p values less than 0.05
were considered statistically significant. Data management and
statistical analyses were performed using STATA/MP V.13.1
(StataCorp).
RESULTS
Final set of QIs
Out of 44 initial candidate QIs (online supplemental table 3),
three were removed due to a low median validity/feasibility
score. For the remaining, agreement was reached in 31 QIs and
disagreement in 10. In the former set, minor edits and discussion
based on experts’ comments resulted in 4 QIs being included
without change, 4 being retained with edits, 17 being merged to
7 separate QIs and 6 being rejected. Of the 10 QIs for which there
was disagreement, one was retained without changes, two were
retained with edits, four were excluded and three were merged
with two previously formed QIs. This resulted in a revised set
of 18 candidate QIs, which was available for Round 2 of rating
(table 1). The 18 finally selected QIs were further divided into
three categories: (1) Screening/diagnosis- related (QIs 1–5), (2)
Treatment- related (QIs 6–10) and 3) Monitoring (QIs 11–18).
More specifically, selected individual QIs pertain to diagnosis,
monitoring, therapy and its targets, fertility and pregnancy and
adjunct therapy, including prevention of cardiovascular disease
(CVD) and osteoporosis, vaccination, counselling for smoking
and sunscreen protection.
Adherence to QIs
The final set of 18 QIs was tested for adherence in all eligible
patients of our cohort (N=220)(table 2). Characteristics of the
cohort are shown in online supplemental table 4. On average,
patients received 54% (95% CI 52.30% to 56.25%) of the
indicated care. Complete laboratory work- up at diagnosis was
performed in 48.5% (95% CI 39.8% to 57.1%), with antiphos-
pholipid antibodies being the most frequently missed component
(68.9%). Disease activity evaluation in at least three out of four
visits and annual assessment of organ damage were completed in
only 14.1% (95% CI 9.4% to 18.7%) and 28.6% (95% CI 22.6%
to 34.6%), respectively. By contrast, lupus nephritis related QIs
had excellent overall adherence (88%, 95% CI 66.7% to 96.4%
for the use of ACE inhibitor/angiotensin receptor blocker, 100%
for the use of immunosuppressive treatment), except for labo-
ratory monitoring (48.5%, 95% CI 36.6% to 60.6%). Overall
adherence rate was 50% (95% CI 38.5% to 61.5%) for repro-
ductive health counselling, 62% (95% CI 49.9% to 72.7%) for
pregnancy counselling and 91.4% (95% CI 86.8% to 94.4%)
for sunscreen protection. Notably, preventive measures for
comorbidities had generally low to moderate adherence. More
specifically, overall adherence rates for cardiovascular risk modi-
fication and vaccination QIs (at least one of the available pneu-
mococcal vaccines in combination with influenza vaccine) were
40.5% (95% CI 34.1% to 47.1%) and 47.7% (95% CI 41.2%
to 54.4%), respectively. Regarding osteoporosis prevention and
treatment, the corresponding QI was fulfilled in 45.5% (95% CI
38.9% to 52.1%) of patients. A total of 73% of eligible patients
had bone mineral density measurement performed at baseline
and 58.6% at follow- up (every 2 years), while 60% of patients
Table 3 Adherence to quality indicators (QI) in subgroups of
patients
No of patients
Mean*
eligible QIs
Mean
adherence, % P value
Severity pattern 0.006
Mild 56 10.7 49.3
Moderate 60 11.1 53.9
Severe 104 12.1 57.2
Cohort 0.13
Inception 132 9.8 49.5
Prevalent 88 10.4 52.8
Disease duration 0.02
<2 years 61 10.2 54.8
≥2 years 159 9.9 49.3
*Mean number of QIs for which the patients of each group were eligible.
Table 4 Adherence to quality indicators (QIs) grouped according to function of care
Function of care No of QIs No of times eligible QIs were assessed No of times recommended care was delivered Adherence, % P value
0.03
Screening- diagnosis* 4 640 286 44.68
Treatment† 6 447 377 84.34
Monitoring‡ 8 1438 726 50.48
*QI 1–5.
†QI 6–10.
‡QI 11–18.

6Chavatza K, etal. Ann Rheum Dis 2021;0:1–8. doi:10.1136/annrheumdis-2021-220438
Systemic lupus erythematosus
belonging in the high fracture risk group received antiresorptive
treatment; almost 75% (74.3%) received calcium and vitamin D.
Of note, 63.8% of patients on GC received calcium and vitamin
D protection.
In a subgroup analysis, patients with severe disease were
more likely to receive the indicated care (57.2%) compared
with patients with moderate (53.9%) or mild (49.3%) disease
(p=0.006). Similarly, higher adherence rates were observed in
patients with short (<2 years) vs longer (≥2 years) disease dura-
tion (54.8% and 49.3% respectively, p=0.02). No significant
differences were observed between the inception and the preva-
lent cohort (table 3).
In a separate analysis according to the function of care,
treatment- related QIs were met in significantly more eligible
patients (84.3%) followed by monitoring (50.5%) and diag-
nostic (44.6%) QIs (p=0.03) (table 4).
Outcomes
Disease- related outcomes are summarised in online supple-
mental table 5. Patients were followed up for a median of 2 years
(IQR 2–4). SDI progression was observed in 22.3% of patients
incidence rate (IR)=13/100 patient- years (pys). A total of 310
flares were captured over the follow- up corresponding to 0.58
per py. The IR of hospitalisations was 15.4/100 py, attributed
mainly to major flares (7.8/100 py), serious infections (6.1/100
py) and cardiovascular events (1.5/100 py).
Overall, QI adherence did not differ among patients experi-
encing CAO and patients without CAO throughout the obser-
vation period (54.0% vs 54.7%, p=0.71). However, patients
with CAO during the last year of follow- up had lower adherence
rates in monitoring QIs when compared with patients without
a CAO (47.6% vs 53.9%, p=0.02) (online supplemental table
6). We also explored possible associations between adherence
to specific QIs and outcomes. Patients who achieved sustained
remission or Lupus Low Disease Activity State (LLDAS) (QI13),
patients who fulfilled QI16 regarding vaccination and patients
who received low- dose GC (QI7) had lower odds of experi-
encing a flare during the observation period (OR 0.15, 95% CI
0.07 to 0.31 OR 0.46, 95% CI 0.21 to 0.98 and OR 0.23, 95%
CI 0.05 to 0.94, respectively). A lower risk of CAO during the
last year of follow- up was also found in patients who met QI13
on remission/LLDAS and QI16 on vaccination (OR 0.09, 95% CI
0.04 to 0.18 and OR 0.52, 95% CI 0.28 to 0.99 respectively). As
expected, patients who achieved sustained remission or LLDAS
(QI13) had lower odds of damage accrual during the observation
period (OR 0.35, 95% CI 0.14 to 0.84). Patients assessed for
SDI accrual (QI12) and CVD risk stratification (QI4) had higher
probability to exhibit any CAO (OR 2.62, 95% CI 1.18 to 5.71
and OR 1.77, 95% CI 1.01 to 3.12, respectively) (table 5).
DISCUSSION
SLE is notorious for its clinical heterogeneity, which may in
turn increase the risk of inconsistency and variations in the care
received by patients. To ensure improved and more homoge-
neous care, EULAR has developed evidence- based and expert
opinion- based recommendations for the management of various
aspects of the disease.9–12 Nonetheless, since management recom-
mendations are often followed incompletely in real- life settings,
efforts have been made to create tools which can transform
them into easily applicable, ‘user friendly’ instructions for daily
practice. In this regard, QIs can be useful instruments for the
quantification of gaps and shortcomings in medical care. Herein,
we created a set of QIs based on the EULAR recommendations
for SLE, using a validated, systematic methodology supported
by expert opinion. In addition, we examined the adherence to
the proposed QIs in 220 patients of the ‘Attikon’ lupus cohort,
a readily available patient cohort, to take an initial ‘glimpse’ on
potential gaps of care in daily practice and assess their impact on
disease outcomes.
QIs have been previously proposed for SLE,5 23 but for the
most part they were not based on a comprehensive SLR. This
is the first set of QIs based on such a comprehensive SLR of the
various aspects of SLE (ie, diagnosis, monitoring and therapy),
which was performed in the context of the updated EULAR
recommendations. The credibility of the proposed QI set is
reinforced by the robust methodology of the procedure (ie, the
RAND/UCLA modified Delphi method), which involved assess-
ment of a large number of initial candidate QIs for validity and
feasibility, followed by two rounds of voting, all performed by a
panel of experts with expertise in SLE.
Our initial findings suggest moderate adherence (54%) with
great variability in certain types of QIs. The low rates of CVD
protection and reproductive health counselling are consistent
with data from previous studies;24 25 rates for sunscreen protec-
tion and individual components for osteoporosis and vaccination
(influenza, pneumococcal) QIs are also consistent with published
data.25 Looking for potential explanations, in the case of CVD-
related QIs, the complexity of prescribing statins by rheumatol-
ogists in some countries and, in case of osteoporosis prophylaxis
the plethora of recommendations by various scientific societies,
may account at least in part for these low adherence rates. In our
view, this reality highlights the need to actively involve nurse
specialists in the care of SLE patients, especially in the settings
of expert SLE referral centres. Such nurse practitioners could
Table 5 Risk of adverse events associated with the delivered care in an SLE cohort of 220 patients
Quality indicator (QI) Adverse event OR P value
QI12 (If a patient has SLE then the SLICC/ACR damage index should be monitored annually) CAO 2.6 0.01
QI13 (If a patient has SLE then management should aim at clinical remission or- if remission cannot be achieved—at low- disease
activity with acceptable dose of steroids and well tolerated immunosuppressive agents at maintenance doses)
SDI progression 0.4 0.02
Flares 0.2 <0.001
CAO* 0.1 <0.001
QI4 (If a patient has SLE then stratification of CVD risk should be performed annually with assessment of both traditional and
disease- related risk factors and management of modifiable risk factors including smoking
CAO* 1.8 0.04
QI16 (If a patient has SLE then in patients with stable/inactive disease, non- live vaccines such as influenza, pneumococcal and HPV
vaccines are recommended while attenuated vaccines (such as HZV vaccine) may be considered
Flares 0.5 0.04
CAO* 0.5 0.04
*CAO during the last year of follow- up.
CAO, composite adverse outcome; CVD, cardiovascular disease; HPV, human papilloma virus; HZV, herpes zoster virus; OR, odds ratio; SDI, SLICC/ACR Damage Index; SLE,
systemic lupus erythematosus.

7
Chavatza K, etal. Ann Rheum Dis 2021;0:1–8. doi:10.1136/annrheumdis-2021-220438
Systemic lupus erythematosus
monitor the assessment and fulfilment of these QIs, which may
not be a priority in a busy physician outpatient clinic.
In reference to potential causes related to better performance in
certain indicators, we found that QI adherence rates were higher
in patients with disease duration shorter than 2 years and in
patients with severe disease. These observations may reflect the fact
that physicians are more likely to adhere early after diagnosis to
ensure better disease control, and in patients who are more likely
to develop irreversible organ damage, respectively. Despite variable
rates of adherence, we did not find strong associations between
non- adherence to QIs and adverse outcomes, except that patients
who were in a low disease activity state had lower rates of flares
and damage progression. A possible explanation is that the adher-
ence to a single QI may not suffice to provide a clinically favourable
outcome, if not combined with consistent and adequate care. Thus,
in a study by Yazdany et al, patients who met ≥85% of the eligible
QIs had lower odds of damage accrual, however, the difference was
not significant for any of the individual QIs alone.26 The modest
associations between quality of care and outcomes in our study may
also reflect the relatively short follow- up, especially because many
SLE outcomes develop within years, and longer observation time
is needed to detect any association. To further address this issue,
prospective long- term follow- up studies evaluating a combination
set instead of single indicators with varied settings and outcomes
are needed.
Our study has several limitations. The duration of follow- up
was modest and data represent the experience of a single academic
centre. Consequently, our results may not be representative of other
clinical settings and in non- academic centres, gaps in patient care
may be even greater. Conversely, a tertiary care hospital that serves
as a referral centre may follow patients with a higher burden of
the disease and higher risk of progression.27 28 Risk- adjusted and
casemix models would help to account for differences in patient-
level and hospital- level risk, however, the relatively small study
sample, limited access to administrative data and the absence of
electronic health record systems prevented us from performing this
methodology. Yet, development of the current QIs was based on
an extensive systematic review and panels of experts working for
over one decade to develop recommendations for SLE. To this end,
longitudinal and nationwide population- based studies are warranted
to validate these QIs in various time and clinical settings.
In summary, we have developed a set of EULAR recommendations-
based QIs for SLE patient care, following a comprehensive SLR and
supported by expert opinion. Initial real- life data suggest a variable
degree of adherence and areas for further improvement. Neverthe-
less, these QIs may be used as a ‘checklist’ to be fulfilled in an outpa-
tient setting, in order to improve SLE patient care by facilitating the
implementation of the EULAR recommendations.
Author affiliations
1Rheumatology and Clinical Immunology, Medical School, National and
Kapodistrian University of Athens, "Attikon" University Hospital of Athens, Athens,
Greece
2Department of Nephrology, “G. Gennimatas” General Hospital, Athens, Greece
3Department of Rheumatology, "Asklepieion" General Hospital, Voula, Athens,
Greece
4Department of Clinical and Experimental Sciences, University of Brescia, Brescia,
Italy
5Unit of Rheumatology and Clinical Immunology, Spedali Civili, Brescia, Italy
6Division of Rheumatology, Department of Medicine III, University Medical Center &
Faculty of Medicine Carl Gustav Carus at the TU Dresden, Dresden, Germany
7Nephrology Department and Renal Transplantation Unit, “Laikon” Hospital, National
and Kapodistrian University of Athens, Medical School, Athens, Greece
8Rheumatology Unit, Department of Medicine, University of Padova, Padova, Italy
9Cliniques Universitaires Saint- Luc, Université catholique de Louvain, Brussels,
Belgium
10Department of Medicine, University of Cambridge, Cambridge, UK
11Rheumatology Unit, Department of Clinical and Experimental Medicine, University
of Pisa, Pisa, Italy
12Division of Rheumatology, Department of Medicine Solna, Karolinska Institutet,
Karolinska University Hospital, Stockholm, Sweden
13Rheumatology, Clinical Immunology and Allergy, University Hospital of Heraklion,
Heraklion, Crete, Greece
14Laboratory of Autoimmunity and Inflammation, Biomedical Research Foundation of
the Academy of Athens, Athens, Greece
Twitter Dimitrios T Boumpas @none
Acknowledgements We are thankful to the staff physicians and nurses of the
Rheumatology and Clinical Immunology Unit of ’Attikon’ University Hospital for
providing care to the patients with SLE and other rheumatologic diseases.
Contributors KC and OG collected data from patient medical charts and KC
drafted the manuscript. MK performed statistical analyses and edited the manuscript.
DN edited the manuscript. KT assessed patient medical charts for eligibility in the
study. LA, MA, JB, AD, FAH, DJ, MM, ES and AT evaluated the quality indicators,
provided voting and critically reviewed the manuscript. GB and AF supervised the
study and edited the manuscript. DTB conceived and supervised the study and edited
the manuscript.
Funding DTB was supported by the European Research Council (ERC) under
the European Union’s Horizon 2020 research and innovation programme (grant
agreement no. 742390). DJ was supported by the NIHR Cambridge Biomedical
Research Centre (BRC-1215 20014).
Competing interests DTB is an Editorial Board member in the Annals of the
Rheumatic Diseases. The remaining authors declare no competing interests relevant
to this work.
Patient consent for publication Not required.
Ethics approval The study was approved by the Ethics Committee of ’Attikon’
University Hospital of Athens.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the
article or uploaded as online supplemental information.
Supplemental material This content has been supplied by the author(s).
It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not
have been peer- reviewed. Any opinions or recommendations discussed are
solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all
liability and responsibility arising from any reliance placed on the content.
Where the content includes any translated material, BMJ does not warrant the
accuracy and reliability of the translations (including but not limited to local
regulations, clinical guidelines, terminology, drug names and drug dosages), and
is not responsible for any error and/or omissions arising from translation and
adaptation or otherwise.
ORCID iDs
KaterinaChavatza http:// orcid. org/ 0000- 0002- 1274- 0909
DionysisNikolopoulos http:// orcid. org/ 0000- 0002- 9894- 6966
MartinAringer http:// orcid. org/ 0000- 0003- 4471- 8375
AndreaDoria http:// orcid. org/ 0000- 0003- 0548- 4983
Frederic AHoussiau http:// orcid. org/ 0000- 0003- 1451- 083X
ElisabetSvenungsson http:// orcid. org/ 0000- 0003- 3396- 3244
AntonisFanouriakis http:// orcid. org/ 0000- 0003- 2696- 031X
Dimitrios TBoumpas http:// orcid. org/ 0000- 0002- 9812- 4671
REFERENCES
1 Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing
the quality chasm: a new health system for the 21st century. Washington (DC), 2001.
2 Donabedian A. Evaluating the quality of medical care. 1966. Milbank Q
2005;83:691–729.
3 Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260:1743–8.
4 Saag KG, Yazdany J, Alexander C, etal. Defining quality of care in rheumatology: the
American College of rheumatology white paper on quality measurement. Arthritis
Care Res 2011;63:2–9.
5 Mosca M, Tani C, Aringer M, etal. Development of quality indicators to evaluate
the monitoring of SLE patients in routine clinical practice. Autoimmun Rev
2011;10:383–8.
6 Fanouriakis A, Tziolos N, Bertsias G, etal. Update οn the diagnosis and management
of systemic lupus erythematosus. Ann Rheum Dis 2021;80:14–25.
7 Bertsias GK, Ioannidis JPA, Aringer M, etal. EULAR recommendations for the
management of systemic lupus erythematosus with neuropsychiatric manifestations:
report of a task force of the EULAR standing Committee for clinical Affairs. Ann
Rheum Dis 2010;69:2074–82.
8 Bertsias G, Ioannidis JPA, Boletis J, etal. EULAR recommendations for the
management of systemic lupus erythematosus. Report of a task force of the EULAR

8Chavatza K, etal. Ann Rheum Dis 2021;0:1–8. doi:10.1136/annrheumdis-2021-220438
Systemic lupus erythematosus
standing Committee for international clinical studies including therapeutics. Ann
Rheum Dis 2008;67:195–205.
9 Andreoli L, Bertsias GK, Agmon- Levin N, etal. EULAR recommendations for women’s
health and the management of family planning, assisted reproduction, pregnancy and
menopause in patients with systemic lupus erythematosus and/or antiphospholipid
syndrome. Ann Rheum Dis 2017;76:476–85.
10 Furer V, Rondaan C, Heijstek MW, etal. 2019 update of EULAR recommendations for
vaccination in adult patients with autoimmune inflammatory rheumatic diseases. Ann
Rheum Dis 2020;79:39–52.
11 Fanouriakis A, Kostopoulou M, Cheema K, etal. 2019 update of the joint European
League against rheumatism and European renal Association- European dialysis and
transplant association (EULAR/ERA- EDTA) recommendations for the management of
lupus nephritis. Ann Rheum Dis 2020;79:713–23.
12 Fanouriakis A, Kostopoulou M, Alunno A, etal. 2019 update of the EULAR
recommendations for the management of systemic lupus erythematosus. Ann Rheum
Dis 2019;78:736–45.
13 van Vollenhoven RF, Mosca M, Bertsias G, etal. Treat- to- target in systemic lupus
erythematosus: recommendations from an international Task force. Ann Rheum Dis
2014;73:958–67.
14 Burnand B, Aguilar MD, Steven J, etal. The RAND/UCLA Appropriateness Method
User’s Manual, 2001.
15 Brook RH. The RAND/UCLA appropriateness method: Rand Corporation, 1995.
16 Mangione- Smith R, DeCristofaro AH, Setodji CM, etal. The quality of ambulatory care
delivered to children in the United States. N Engl J Med 2007;357:1515–23.
17 Makras P, Anastasilakis AD, Antypas G, etal. The 2018 guidelines for the diagnosis
and treatment of osteoporosis in Greece. Arch Osteoporos 2019;14:39.
18 Nikolopoulos D, Kostopoulou M, Pieta A, etal. Evolving phenotype of systemic
lupus erythematosus in Caucasians: low incidence of lupus nephritis, high burden
of neuropsychiatric disease and increased rates of late- onset lupus in the ’Attikon’
cohort. Lupus 2020;29:514–22.
19 Petri M, Orbai A- M, Alarcón GS, etal. Derivation and validation of the systemic
lupus international collaborating clinics classification criteria for systemic lupus
erythematosus. Arthritis Rheum 2012;64:2677–86.
20 Aringer M, Costenbader K, Daikh D, etal. 2019 European League against
Rheumatism/American College of rheumatology classification criteria for systemic
lupus erythematosus. Arthritis Rheumatol 2019;71:1400–12.
21 Ruperto N, Hanrahan LM, Alarcón GS, etal. International consensus for a definition of
disease flare in lupus. Lupus 2011;20:453–62.
22 Nikolopoulos DS, Kostopoulou M, Pieta A, etal. Transition to severe phenotype
in systemic lupus erythematosus initially presenting with non- severe disease:
implications for the management of early disease. Lupus Sci Med 2020;7.
23 Yazdany J, Panopalis P, Gillis JZ, etal. A quality indicator set for systemic lupus
erythematosus. Arthritis Rheum 2009;61:370–7.
24 McGlynn EA, Asch SM, Adams J, etal. The quality of health care delivered to adults in
the United States. N Engl J Med Overseas Ed 2003;348:2635–45.
25 Yazdany J, Trupin L, Tonner C, etal. Quality of care in systemic lupus erythematosus:
application of quality measures to understand gaps in care. J Gen Intern Med
2012;27:1326–33.
26 Yazdany J, Trupin L, Schmajuk G, etal. Quality of care in systemic lupus
erythematosus: the association between process and outcome measures in the lupus
outcomes study. BMJ Qual Saf 2014;23:659–66.
27 Nived O, Ingvarsson RF, Jöud A, etal. Disease duration, age at diagnosis and organ
damage are important factors for cardiovascular disease in SLE. Lupus Sci Med
2020;7:e000398.
28 Laustrup H, Voss A, Green A, etal. Sle disease patterns in a Danish population- based
lupus cohort: an 8- year prospective study. Lupus 2010;19:239–46.