ACR EULAR classification criteria for systemic sclerosis 

Contexts in source publication

Context 1

... knowledge had been gained, and the rigors of classification criteria development had changed, it was felt that many classification criteria sets needed to be updated [8]. With the support of the American College of Rheumatology and European League Against Rheumatism, a joint international, multiphase effort was undertaken to develop a new classification criteria set [32 •• , 33 •• ]. The main objective was to develop a criteria set that classified a greater breadth of the spectrum of systemic sclerosis, thereby allowing more individuals the opportunity to participate in clinical trials [32 •• , 33 •• ]. In particular, the aim was to classify those with early and/or limited disease. The aim was also to try to incorporate new knowledge gained over the recent decades regard- ing systemic sclerosis, while using a state-of-the-art methodologic approach. This approach involved a balanced use of expert-based and data-driven methods, and implemen- tation of bias reduction strategies. Most notably, considerable attention was made to avoid bias introduced by circular rea- soning. This bias was an important threat to the development of other criteria sets in rheumatology [8, 28]. This is a bias that can occur when the investigators who derive the classification criteria, test the criteria in data from patients from their practice, who initially informed their concept of the disease [28]. The systemic sclerosis classification criteria steering commit- tee made efforts to ensure that systemic sclerosis experts who were asked to contribute knowledge were separate from the centers that contributed data [32 •• , 33 •• ]. The candidate criteria were nominated through two inde- pendent Delphi exercises conducted in North America through the Scleroderma Clinical Trials Consortium (SCTC) and in Europe through EUSTAR [34 • ]. The SCTC systemic sclerosis experts were asked to nominate items used in daily practice to diagnose SSc whereas the EUSTAR systemic sclerosis experts were asked to identify items suitable for diagnosis of early systemic sclerosis [34 • , 35]. The subtle differences in emphasis between the two groups allowed for a broader array of items for nomination. One hundred six experts rated 168 items. Low-scoring items were excluded, leaving 102 items. The items were rated and reduced through a consensus meeting of 16 international experts using nominal group technique. This reduced the candidate criteria to 23 items [34 • ]. Using applied Bayesian methods, the face, discriminant and construct validity of these items were tested using cases and controls sampled from the Toronto, Pittsburg, Madrid, and Berlin Connective Tissue Disease, Canadian Scleroderma Research Group, and 1,000 Faces of Lupus databases [36 • ]. The criteria were tested in 750 SSc cases and 1,071 controls (systemic lupus erythematosus, inflammatory myositis, Sjogren ’ s syndrome, Raynaud ’ s syndrome, mixed connective tissue disease, and idiopathic pulmonary arterial hypertension). Scleroderma skin changes (odds ratio (OR) 427, 95 % Credible Interval (CrI) 257, 691), telangiectasia (OR 91, 95 % CrI 58, 155), RNA polymerase III antibody (OR 75, 95 % CrI 13, 313), and puffy fingers (OR 35, 95 % CrI 24, 49) were highly discriminatory between cases and controls whereas reduced forced vital capacity (OR 0.9, 95 % CrI 0.6, 1.3) and reduced DLCO (OR 1.5, 95 % CrI 1.1, 2.0) did not discriminate well [36 • ]. Using multicriteria decision analysis (also known as conjoint analysis or forced choice methods) with 1000Minds soft- ware, the 23 candidate criteria were further reduced and weighted, and a threshold for classification identified [37 • ]. Through this process, ‘ skin thickening sparing the fingers ’ was made an exclusionary criterion. If this is present, the classification criteria should not be applied (see Fig. 1). B Skin thickening of the fingers extending proximally to the metacarpophalangeal joints ^ was a sufficient criterion. If present, the individual could be classified as systemic sclerosis. Due to poor ability to discriminate cases from controls, limited reliability in the clinical setting, or little added value in classification (low weight), digital pulp loss, dysphagia, esophageal reflux, diffusing capacity, forced vital capacity, and antinuclear antibody were removed. Acro-osteolysis, pitting scars, and digital ulcers were grouped under the heading ‘ fin- ger tip lesions. ’ Systemic sclerosis specific antibodies were also grouped under one criterion [37 • ]. This methodologic approach facilitated a system of classification that produced a measure of probability that an individual with certain features could be classified as systemic sclerosis. The draft classification system and weights were then tested against a range of cases that reflected low to high probability of having systemic sclerosis rated by experts. A total score of 9 or more was identified as the threshold with the least misclassification [32 •• , 33 •• ] (Fig. 1). The operating characteristics of the criteria set was tested in derivation and validation cohorts consisting of cases and controls from 13 North American and 10 European centers. The controls were purposefully selected to represent conditions that could be confused with SSc and included cases of eosin- ophilic fasciitis, scleromyxedema, dermatomyositis, poly- myositis, mixed connective tissue disease, undifferentiated connective tissue disease, generalized morphea, nephrogenic sclerosing fibrosis, and graft versus host disease. Using the derivation cohort, the criteria set was further refined. Finger flexion contractures, calcinosis, tendon or bursal friction rubs, renal crisis, and esophageal dilation were removed, as they did not provide added value in sensitivity or specificity. In the derivation cohort, the new classification criteria had excellent sensitivity (0.95, 95 % confidence interval (CI) 0.90, 0.98) and specificity (0.93, 95 %CI 0.86, 0.97). This remained true in the validation cohort with a sensitivity of 0.91 (95 %CI 0.87, 0.94) and specificity 0.92 (95 %CI 0.86, 0.96). The new criteria had demonstrable added value above the 1980 criteria, and the LeRoy and Medsger criteria. Other groups have independently, externally, validated the criteria, and the excellent performance characteristics hold true [38, 39] (Table 2). The improvement in classification criteria sensitivity has been partly attributed to the inclusion of Raynaud ’ s phenomenon and puffy fingers [38]. The sensitivity of the new criteria have marked improvement in limited cutaneous disease (particularly individuals without skin involvement proximal to the metacarpophalangeal joints) and those with early disease less than or equal to 3 years [38] (Table 2). Since its publication, the new classification criteria have had some criticism. One area of controversy relates to the application of the classification criteria to individuals with mixed connective tissue disease [15]. The 1980 criteria investigators made the decision to exclude mixed connective tissue disease patients from the derivation processes as they felt that it was ‘ virtually impossible ’ to distinguish from SSc [15]. Both the criteria testing [36 • ], derivation, and validation cohorts [32 •• ] of the new SSc criteria included mixed-connective-tissue- disease patients. Since many mixed connective tissue disease patients are SSc-predominant in their clinical features, it was decided that these criteria could be applied to mixed connective tissue disease patients when considering enrollment into systemic sclerosis clinical trials [32 •• ]. In a Norwegian cohort of 178 mixed connective tissue disease patients, 10 % fulfilled the new criteria [39]. It has been noted that this cohort study did not collect data on telangiectasia or nailfold capillary abnormalities, therefore 10 % may be an underestimate [9]. A second area of controversy relates to the difference between classification criteria and diagnostic criteria [40]. Most of the rheumatic diseases do not have a singular diagnostic test. Physician judgment, taking into account clinical and in- vestigational subtleties, is required to make a diagnosis. Physician expertise is gained through years of specialized ed- ucation and training [41]. This is complemented by years of clinical experience. Although it has been recognized that classification criteria inform that concept of the disease [5], the American College of Rheumatology makes a clear distinction between classification and diagnostic criteria [40]. Diagnostic criteria are intended to reflect aspects of the disease that char- acterize most cases that encompass the spectrum of the disease. Classification criteria are designed to identify a well-characterized group of patients within the spectrum of the disease. The disease prevalence, prevalence of similar diseases, and geography can affect the degree of overlap [40]. The performance of criteria depends on both the pre-test probability of the disease (prevalence of the disease and similar conditions), and the sensitivity and specificity of the criteria [40]. Racial and ethnic variation within a geographical area may also affect this. Importantly, accurate diagnosis may have impact on physician billing and patient medical coverage [40]. Diagnostic criteria that are highly specific will leave some cases undiagnosed consequently resulting in denial of treatment coverage if insurance companies and government agencies use the diagnostic criteria as a standard for reim- bursement [40]. Conversely, individuals misdiagnosed with a disease may have barriers in getting health insurance or life insurance [40]. For these reasons, the diagnosis of systemic sclerosis continues to remain in the realm of physician judgment and not classification criteria. A third area of controversy is the applicability of the new systemic sclerosis classification criteria to juvenile systemic sclerosis. Neither pediatric ...

Context 2

... for classification criteria development based on the principles of measurement science [28 – 31]. These recommendations call for increased collaboration between those with disease- specific content expertise and those with methodologic expertise (clinical epidemiologists). The recommendations call for consideration of the purpose of the criteria (clinical research, epidemiologic studies), methods of criteria generation (literature search, expert and/or patient opinion), evaluation of psychometric properties (validity, reliability, feasibili- ty), and careful selection of cases and appropriate controls [31]. A review of all the classification criteria in the rheumatic diseases, undertaken with the support of the American College of Rheumatology Classification and Response Criteria Subcommittee of the Committee on Quality Measures, found that many criteria sets in rheumatology had methodologic issues relating to the derivation sample case and control selection [8]. It was recommended that criteria sets should be independently validated in cohorts of patients and controls separate from the derivation cohort [8]. Given that disease-specific knowledge had been gained, and the rigors of classification criteria development had changed, it was felt that many classification criteria sets needed to be updated [8]. With the support of the American College of Rheumatology and European League Against Rheumatism, a joint international, multiphase effort was undertaken to develop a new classification criteria set [32 •• , 33 •• ]. The main objective was to develop a criteria set that classified a greater breadth of the spectrum of systemic sclerosis, thereby allowing more individuals the opportunity to participate in clinical trials [32 •• , 33 •• ]. In particular, the aim was to classify those with early and/or limited disease. The aim was also to try to incorporate new knowledge gained over the recent decades regard- ing systemic sclerosis, while using a state-of-the-art methodologic approach. This approach involved a balanced use of expert-based and data-driven methods, and implemen- tation of bias reduction strategies. Most notably, considerable attention was made to avoid bias introduced by circular rea- soning. This bias was an important threat to the development of other criteria sets in rheumatology [8, 28]. This is a bias that can occur when the investigators who derive the classification criteria, test the criteria in data from patients from their practice, who initially informed their concept of the disease [28]. The systemic sclerosis classification criteria steering commit- tee made efforts to ensure that systemic sclerosis experts who were asked to contribute knowledge were separate from the centers that contributed data [32 •• , 33 •• ]. The candidate criteria were nominated through two inde- pendent Delphi exercises conducted in North America through the Scleroderma Clinical Trials Consortium (SCTC) and in Europe through EUSTAR [34 • ]. The SCTC systemic sclerosis experts were asked to nominate items used in daily practice to diagnose SSc whereas the EUSTAR systemic sclerosis experts were asked to identify items suitable for diagnosis of early systemic sclerosis [34 • , 35]. The subtle differences in emphasis between the two groups allowed for a broader array of items for nomination. One hundred six experts rated 168 items. Low-scoring items were excluded, leaving 102 items. The items were rated and reduced through a consensus meeting of 16 international experts using nominal group technique. This reduced the candidate criteria to 23 items [34 • ]. Using applied Bayesian methods, the face, discriminant and construct validity of these items were tested using cases and controls sampled from the Toronto, Pittsburg, Madrid, and Berlin Connective Tissue Disease, Canadian Scleroderma Research Group, and 1,000 Faces of Lupus databases [36 • ]. The criteria were tested in 750 SSc cases and 1,071 controls (systemic lupus erythematosus, inflammatory myositis, Sjogren ’ s syndrome, Raynaud ’ s syndrome, mixed connective tissue disease, and idiopathic pulmonary arterial hypertension). Scleroderma skin changes (odds ratio (OR) 427, 95 % Credible Interval (CrI) 257, 691), telangiectasia (OR 91, 95 % CrI 58, 155), RNA polymerase III antibody (OR 75, 95 % CrI 13, 313), and puffy fingers (OR 35, 95 % CrI 24, 49) were highly discriminatory between cases and controls whereas reduced forced vital capacity (OR 0.9, 95 % CrI 0.6, 1.3) and reduced DLCO (OR 1.5, 95 % CrI 1.1, 2.0) did not discriminate well [36 • ]. Using multicriteria decision analysis (also known as conjoint analysis or forced choice methods) with 1000Minds soft- ware, the 23 candidate criteria were further reduced and weighted, and a threshold for classification identified [37 • ]. Through this process, ‘ skin thickening sparing the fingers ’ was made an exclusionary criterion. If this is present, the classification criteria should not be applied (see Fig. 1). B Skin thickening of the fingers extending proximally to the metacarpophalangeal joints ^ was a sufficient criterion. If present, the individual could be classified as systemic sclerosis. Due to poor ability to discriminate cases from controls, limited reliability in the clinical setting, or little added value in classification (low weight), digital pulp loss, dysphagia, esophageal reflux, diffusing capacity, forced vital capacity, and antinuclear antibody were removed. Acro-osteolysis, pitting scars, and digital ulcers were grouped under the heading ‘ fin- ger tip lesions. ’ Systemic sclerosis specific antibodies were also grouped under one criterion [37 • ]. This methodologic approach facilitated a system of classification that produced a measure of probability that an individual with certain features could be classified as systemic sclerosis. The draft classification system and weights were then tested against a range of cases that reflected low to high probability of having systemic sclerosis rated by experts. A total score of 9 or more was identified as the threshold with the least misclassification [32 •• , 33 •• ] (Fig. 1). The operating characteristics of the criteria set was tested in derivation and validation cohorts consisting of cases and controls from 13 North American and 10 European centers. The controls were purposefully selected to represent conditions that could be confused with SSc and included cases of eosin- ophilic fasciitis, scleromyxedema, dermatomyositis, poly- myositis, mixed connective tissue disease, undifferentiated connective tissue disease, generalized morphea, nephrogenic sclerosing fibrosis, and graft versus host disease. Using the derivation cohort, the criteria set was further refined. Finger flexion contractures, calcinosis, tendon or bursal friction rubs, renal crisis, and esophageal dilation were removed, as they did not provide added value in sensitivity or specificity. In the derivation cohort, the new classification criteria had excellent sensitivity (0.95, 95 % confidence interval (CI) 0.90, 0.98) and specificity (0.93, 95 %CI 0.86, 0.97). This remained true in the validation cohort with a sensitivity of 0.91 (95 %CI 0.87, 0.94) and specificity 0.92 (95 %CI 0.86, 0.96). The new criteria had demonstrable added value above the 1980 criteria, and the LeRoy and Medsger criteria. Other groups have independently, externally, validated the criteria, and the excellent performance characteristics hold true [38, 39] (Table 2). The improvement in classification criteria sensitivity has been partly attributed to the inclusion of Raynaud ’ s phenomenon and puffy fingers [38]. The sensitivity of the new criteria have marked improvement in limited cutaneous disease (particularly individuals without skin involvement proximal to the metacarpophalangeal joints) and those with early disease less than or equal to 3 years [38] (Table 2). Since its publication, the new classification criteria have had some criticism. One area of controversy relates to the application of the classification criteria to individuals with mixed connective tissue disease [15]. The 1980 criteria investigators made the decision to exclude mixed connective tissue disease patients from the derivation processes as they felt that it was ‘ virtually impossible ’ to distinguish from SSc [15]. Both the criteria testing [36 • ], derivation, and validation cohorts [32 •• ] of the new SSc criteria included mixed-connective-tissue- disease patients. Since many mixed connective tissue disease patients are SSc-predominant in their clinical features, it was decided that these criteria could be applied to mixed connective tissue disease patients when considering enrollment into systemic sclerosis clinical trials [32 •• ]. In a Norwegian cohort of 178 mixed connective tissue disease patients, 10 % fulfilled the new criteria [39]. It has been noted that this cohort study did not collect data on telangiectasia or nailfold capillary abnormalities, therefore 10 % may be an underestimate [9]. A second area of controversy relates to the difference between classification criteria and diagnostic criteria [40]. Most of the rheumatic diseases do not have a singular diagnostic test. Physician judgment, taking into account clinical and in- vestigational subtleties, is required to make a diagnosis. Physician expertise is gained through years of specialized ed- ucation and training [41]. This is complemented by years of clinical experience. Although it has been recognized that classification criteria inform that concept of the disease [5], the American College of Rheumatology makes a clear distinction between classification and diagnostic criteria [40]. Diagnostic criteria are intended to reflect aspects of the disease that char- acterize most cases that encompass the spectrum of the disease. Classification criteria are designed to identify a well-characterized group of patients within the spectrum of the disease. The disease ...