Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Jun;67(6):706-14.
doi: 10.1016/j.jclinepi.2013.12.009. Epub 2014 Apr 8.

Multicriteria decision analysis methods with 1000Minds for developing systemic sclerosis classification criteria

Affiliations

Multicriteria decision analysis methods with 1000Minds for developing systemic sclerosis classification criteria

Sindhu R Johnson et al. J Clin Epidemiol. 2014 Jun.

Abstract

Objectives: Classification criteria for systemic sclerosis (SSc) are being developed. The objectives were to develop an instrument for collating case data and evaluate its sensibility; use forced-choice methods to reduce and weight criteria; and explore agreement among experts on the probability that cases were classified as SSc.

Study design and setting: A standardized instrument was tested for sensibility. The instrument was applied to 20 cases covering a range of probabilities that each had SSc. Experts rank ordered cases from highest to lowest probability; reduced and weighted the criteria using forced-choice methods; and reranked the cases. Consistency in rankings was evaluated using intraclass correlation coefficients (ICCs).

Results: Experts endorsed clarity (83%), comprehensibility (100%), face and content validity (100%). Criteria were weighted (points): finger skin thickening (14-22), fingertip lesions (9-21), friction rubs (21), finger flexion contractures (16), pulmonary fibrosis (14), SSc-related antibodies (15), Raynaud phenomenon (13), calcinosis (12), pulmonary hypertension (11), renal crisis (11), telangiectasia (10), abnormal nailfold capillaries (10), esophageal dilation (7), and puffy fingers (5). The ICC across experts was 0.73 [95% confidence interval (CI): 0.58, 0.86] and improved to 0.80 (95% CI: 0.68, 0.90).

Conclusions: Using a sensible instrument and forced-choice methods, the number of criteria were reduced by 39% (range, 23-14) and weighted. Our methods reflect the rigors of measurement science and serve as a template for developing classification criteria.

Keywords: Classification criteria; Conjoint analysis; Decision analysis; Forced-choice; Scleroderma; Sensibility; Systemic sclerosis.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Conceptual framework for SSc classification criteria development
Figure 2
Figure 2
Example of how questions were posed to participants in the discrete choice experiment.
Figure 3
Figure 3
Experts’ rankings of the relative probability that each of the 20 cases has systemic sclerosis in first ranking exercise. Note: Cases are presented in order of the median rank (X-axis). The ranks (Y-axis) for each case are those assigned by each expert (A – G). Some points shown represent multiple coinciding data points.
Figure 4
Figure 4
Experts’ rankings of the relative probability that each of the 20 cases has systemic sclerosis in second ranking exercise. Note: Cases are presented in order of the median rank (X-axis). The ranks (Y-axis) for each case are those assigned by each expert (A – K). Some points shown represent multiple coinciding data points.

Similar articles

Cited by

  • Core items to be included in a definition of moderate psoriatic arthritis: literature review and expert opinion.
    Urruticoechea-Arana A, Álvarez-Vega JL, García-Vivar ML, Pinto-Tasende JA, García de Yébenes MJ, Carmona L, Queiro R. Urruticoechea-Arana A, et al. Rheumatol Int. 2024 Nov;44(11):2327-2336. doi: 10.1007/s00296-024-05644-y. Epub 2024 Jul 6. Rheumatol Int. 2024. PMID: 38969942 Review.
  • Defining the key clinician skills and attributes for competency in managing patients with osteoporosis and fragility fractures.
    Jackson LE, Saag KG, Johnson SR, Danila MI. Jackson LE, et al. J Bone Miner Res. 2024 May 2;39(4):425-432. doi: 10.1093/jbmr/zjae019. J Bone Miner Res. 2024. PMID: 38477794
  • The 2023 ACR/EULAR classification criteria for calcium pyrophosphate deposition disease.
    Abhishek A, Tedeschi SK, Pascart T, Latourte A, Dalbeth N, Neogi T, Fuller A, Rosenthal A, Becce F, Bardin T, Ea HK, Filippou G, Fitzgerald J, Iagnocco A, Lioté F, McCarthy GM, Ramonda R, Richette P, Sivera F, Andrés M, Cipolletta E, Doherty M, Pascual E, Perez-Ruiz F, So A, Jansen TL, Kohler MJ, Stamp LK, Yinh J, Adinolfi A, Arad U, Aung T, Benillouche E, Bortoluzzi A, Dau J, Maningding E, Fang MA, Figus FA, Filippucci E, Haslett J, Janssen M, Kaldas M, Kimoto M, Leamy K, Navarro GM, Sarzi-Puttini P, Scirè C, Silvagni E, Sirotti S, Stack JR, Truong L, Xie C, Yokose C, Hendry AM, Terkeltaub R, Taylor WJ, Choi HK. Abhishek A, et al. Ann Rheum Dis. 2023 Oct;82(10):1248-1257. doi: 10.1136/ard-2023-224575. Epub 2023 Jul 26. Ann Rheum Dis. 2023. PMID: 37495237 Free PMC article.
  • The 2023 ACR/EULAR Classification Criteria for Calcium Pyrophosphate Deposition Disease.
    Abhishek A, Tedeschi SK, Pascart T, Latourte A, Dalbeth N, Neogi T, Fuller A, Rosenthal A, Becce F, Bardin T, Ea HK, Filippou G, FitzGerald J, Iagnocco A, Lioté F, McCarthy GM, Ramonda R, Richette P, Sivera F, Andres M, Cipolletta E, Doherty M, Pascual E, Perez-Ruiz F, So A, Jansen TL, Kohler MJ, Stamp LK, Yinh J, Adinolfi A, Arad U, Aung T, Benillouche E, Bortoluzzi A, Dau J, Maningding E, Fang MA, Figus FA, Filippucci E, Haslett J, Janssen M, Kaldas M, Kimoto M, Leamy K, Navarro GM, Sarzi-Puttini P, Scirè C, Silvagni E, Sirotti S, Stack JR, Truong L, Xie C, Yokose C, Hendry AM, Terkeltaub R, Taylor WJ, Choi HK. Abhishek A, et al. Arthritis Rheumatol. 2023 Oct;75(10):1703-1713. doi: 10.1002/art.42619. Epub 2023 Jul 26. Arthritis Rheumatol. 2023. PMID: 37494275 Free PMC article.
  • Advanced Autoantibody Testing in Systemic Sclerosis.
    Almaabdi K, Ahmad Z, Johnson SR. Almaabdi K, et al. Diagnostics (Basel). 2023 Feb 23;13(5):851. doi: 10.3390/diagnostics13050851. Diagnostics (Basel). 2023. PMID: 36899995 Free PMC article. Review.

References

    1. Felson DT, Anderson JJ. Methodological and statistical approaches to criteria development in rheumatic diseases. Baillieres Clin Rheumatol. 1995 May;9(2):253–66. - PubMed
    1. Singh JA, Solomon DH, Dougados M, Felson D, Hawker G, Katz P, et al. Development of classification and response criteria for rheumatic diseases. Arthritis Rheum. 2006 Jun 15;55(3):348–52. - PubMed
    1. Johnson SR, Goek ON, Singh-Grewal D, Vlad SC, Feldman BM, Felson DT, et al. Classification criteria in rheumatic diseases: a review of methodologic properties. Arthritis Rheum. 2007 Oct 15;57(7):1119–33. - PubMed
    1. Hudson M, Taillefer S, Steele R, Dunne J, Johnson SR, Jones N, et al. Improving the sensitivity of the American College of Rheumatology classification criteria for systemic sclerosis. Clin Exp Rheumatol. 2007 Sep-Oct;25(5):754–7. - PubMed
    1. Preliminary criteria for the classification of systemic sclerosis (scleroderma) Bull Rheum Dis. 1981;31(1):1–6. - PubMed

Publication types