Figure 1 - available via license: Creative Commons Attribution 3.0 Unported
Content may be subject to copyright.
Graphical description of symmetric and asymmetric ROC curves. The dotted lines show, for reference, TPP = 1 − FPP (the negative diagonal) and the lines FPP = a (vertical) and TPP = 1 − a (horizontal). The FPP coordinate of point A = a, and the FPP coordinate of point C = a*, such that a < a*. The solid line is a symmetric ROC curve passing through the points A (a, b) and B (a 1 , b 1 ) (such that a 1 = 1 − b, b 1 = 1 − a). Point C (a*, 1 − a*) also lies on the symmetric ROC curve. Asymmetries are defined by reference to the symmetric curve passing through point A, as follows. The dashed line is a TPPasymmetric ROC curve passing through the points A (a, b) and D (a 2 , b 2 ) (such that a 2 > 1 − b, b 2 = 1 − a). The dot-dashed line is a TNP-asymmetric ROC curve passing through the points A (a, b) and E (a 3 , b 3 ) (such that a 3 < 1 − b, b 3 = 1 − a).
Source publication
Receiver operating characteristic (ROC) curves have application in analysis of the performance of diagnostic indicators used in the assessment of disease risk in clinical and veterinary medicine and in crop protection. For a binary indicator, an ROC curve summarizes the two distributions of risk scores obtained by retrospectively categorizing subje...
Contexts in source publication
Context 1
... refer to this kind of skew as TPP-asymmetry, and to the kind of skew where the curve clings to the top edge of the ROC space longer than it does to the left as TNP-asymmetry [17]. Figure 1 provides graphical definitions of these symmetry and asymmetry properties. Graphical description of symmetric and asymmetric ROC curves. ...
Context 2
... For a numerical example, consider Killeen and Taylor's Figure 1 (top) in [14]. In this example, the distribution of risk scores for cases f 1 (x) is Normal with mean μ 1 = 3.4 and standard deviation σ 1 = 1 and the distribution of risk scores for controls f 2 (x) is Normal with mean μ 2 = 2 and standard deviation σ 2 = 1. ...
Context 3
... This is illustrated in Figure 3, using values of μ 1 and μ 2 from Killeen and Taylor (Figure 1 in [14]). We note also from Figure 3 that the point where the two curves intersect characterizes the symmetric ROC curve with I(f 1 ,f 2 ) = I(f 2 ,f 1 ) = 0.980 nits. . ...
Context 4
... graphical plot of 1−F 1 (x) against 1−F 2 (x) then provides the ROC curve. Such ROC curves are TPP-asymmetric (as described in Figure 1) (see, e.g., Figure 1 in [25]). ...
Context 5
... graphical plot of 1−F 1 (x) against 1−F 2 (x) then provides the ROC curve. Such ROC curves are TPP-asymmetric (as described in Figure 1) (see, e.g., Figure 1 in [25]). ...
Similar publications
The second-law aspect of an electrically conducting thermal boundary-layer flow inside a permeable pipe moving along its axis under Joule heating effect is considered here. The governing equations are normalised by suitable similarity transformations and solved analytically. An expression for average entropy number is obtained by integrating entrop...
Citations
... Hughes and Bhattacharya [12] characterize symmetry properties of bi-gamma and binormal ROC curves in the terms of Kullback-Leibler divergences (KLD) between the diseased and non-diseased groups. For further contributions in this direction, we refer to Ferreira et al. [7]. ...
The receiver operating characteristics (ROC) analysis is commonly used in clinical settings to check the performance of a single threshold for distinguishing population-wise bimodal-distributed test results. However, for population-wise three-modal distributed test results, a single threshold ROC (stROC) analysis showed poor discriminative performance. The purpose of this study is to use a double-threshold ROC analysis for the three-modal distributed test results to provide better discriminative performance than the stROC analysis. A double-threshold receiver operating characteristic plot (dtROC) is constructed by replacing the single threshold with a double threshold. The sensitivity and specificity coordinates are chosen to maximize sensitivity for a given specificity value. Besides a simulation study assuming a mixture of lognormal, Poisson, and Weibull distributions, a clinical application is examined by a secondary data analysis of palpation test results of the C7 spinous process using the modified thorax-rib static technique. For the assumed mixture models , the discrimination performance of dtROC analysis outperforms the stROC analysis (area under ROC (AUROC) increased from 0.436 to 0.983 for lognormal distributed test results, 0.676 to 0.752 for the Poisson distribution, and 0.674 to 0.804 for Weibull distribution).
... Visually, the resulting ROC curves appear to be either symmetric about the negative diagonal of the graphical plot ( Figure 1A), or skewed towards the upper axis (referred to as TNP-asymmetry, Figure 1B) or the left-hand axis (referred to as TPP-asymmetry, Figure 1C) of the plot. More formally, the shape (symmetry) properties of ROC curves have been characterized in terms of the Kullback-Leibler divergences D(f c f nc ) and D(f nc f c ) between the case and non-case distributions [9][10][11] (with D(f c f nc ) and D(f nc f c ) ≥ 0, and equality only if the case and non-case distributions are identical). In particular, binormal ROC curves may be symmetric, TNP-asymmetric or TPP-asymmetric [10]; for symmetric binormal ROC curves, D(f c f nc ) = D(f nc f c ); while for TNP-asymmetric binormal ROC curves, D(f c f nc ) < D(f nc f c ), and for TPP-asymmetric binormal ROC curves, D(f c f nc ) > D(f nc f c ). ...
... More formally, the shape (symmetry) properties of ROC curves have been characterized in terms of the Kullback-Leibler divergences D(f c f nc ) and D(f nc f c ) between the case and non-case distributions [9][10][11] (with D(f c f nc ) and D(f nc f c ) ≥ 0, and equality only if the case and non-case distributions are identical). In particular, binormal ROC curves may be symmetric, TNP-asymmetric or TPP-asymmetric [10]; for symmetric binormal ROC curves, D(f c f nc ) = D(f nc f c ); while for TNP-asymmetric binormal ROC curves, D(f c f nc ) < D(f nc f c ), and for TPP-asymmetric binormal ROC curves, D(f c f nc ) > D(f nc f c ). For the binormal ROC curve in particular, these conditions reduce to symmetry when σ c = σ nc , TNP-asymmetry when σ c < σ nc , and TPP-asymmetry when σ c > σ nc [10] (Table 2, Figure 1). ...
... In particular, binormal ROC curves may be symmetric, TNP-asymmetric or TPP-asymmetric [10]; for symmetric binormal ROC curves, D(f c f nc ) = D(f nc f c ); while for TNP-asymmetric binormal ROC curves, D(f c f nc ) < D(f nc f c ), and for TPP-asymmetric binormal ROC curves, D(f c f nc ) > D(f nc f c ). For the binormal ROC curve in particular, these conditions reduce to symmetry when σ c = σ nc , TNP-asymmetry when σ c < σ nc , and TPP-asymmetry when σ c > σ nc [10] (Table 2, Figure 1). [1] Figure 6 1.134 1.704 0 1 TPP-asymmetric Improper From above i Mean; ii Standard deviation; iii Terminology of [10]; iv Terminology of [12]; v As FPP increases; vi Prevalence = 0.7; vii Prevalence = 0.3. ...
The predictive receiver operating characteristic (PROC) curve is a diagrammatic format with application in the statistical evaluation of probabilistic disease forecasts. The PROC curve differs from the more well-known receiver operating characteristic (ROC) curve in that it provides a basis for evaluation using metrics defined conditionally on the outcome of the forecast rather than metrics defined conditionally on the actual disease status. Starting from the binormal ROC curve formulation, an overview of some previously published binormal PROC curves is presented in order to place the PROC curve in the context of other methods used in statistical evaluation of probabilistic disease forecasts based on the analysis of predictive values; in particular, the index of separation (PSEP) and the leaf plot. An information theoretic perspective on evaluation is also outlined. Five straightforward recommendations are made with a view to aiding understanding and interpretation of the sometimes-complex patterns generated by PROC curve analysis. The PROC curve and related analyses augment the perspective provided by traditional ROC curve analysis. Here, the binormal ROC model provides the exemplar for investigation of the PROC curve, but potential application extends to analysis based on other distributional models as well as to empirical analysis.
... To satisfy the properties for a ROC curve to be proper in discrete ordinal category data, Metz and Pan 15 proposed a proper binormal model and a new algorithm using the monotonic transformation of the likelihood ratio. Alternative models were explored for the proper ROC curve, such as the bi-gamma model 14,16 and the bi-beta model. 17 A stochastic ordering technique was introduced in Gelfand and Kuo 18 and developed in Gelfand and Kottas. ...
There has been a recent increase in the diagnosis of diseases through radiographic images such as x-rays, magnetic resonance imaging, and computed tomography. The outcome of a radiological diagnostic test is often in the form of discrete ordinal data, and we usually summarize the performance of the diagnostic test using the receiver operating characteristic (ROC) curve and the area under the curve (AUC). The ROC curve will be concave and called proper when the outcomes of the diagnostic test in the actually positive subjects are higher than in the actually negative subjects. The diagnostic test for disease detection is clinically useful when a ROC curve is proper. In this study, we develop a hierarchical Bayesian model to estimate the proper ROC curve and AUC using stochastic ordering in several domains when the outcome of the diagnostic test is discrete ordinal data and compare it with the model without stochastic ordering. The model without stochastic ordering can estimate the improper ROC curve with a nonconcave shape or a hook when the true ROC curve of the population is a proper ROC curve. Therefore, the model with stochastic ordering is preferable over the model without stochastic ordering to estimate the proper ROC curve with clinical usefulness for ordinal data.
... To estimate the proper ROC curve for discrete ordinal data, Metz and Pan (1999) proposed a proper binormal model and a new algorithm using the monotonic transformation of the likelihood ratio. In the frequentist method, the bi-gamma (Dorfman et al., 1996;Hughes and Bhattacharya, 2013) and the bi-beta model (Mossman and Peng, 2016) were proposed. Recently, Nandram and Peiris (2018) developed a robust Bayesian model using stochastic ordering to obtain proper ROC curves. ...
Diagnostic tests in medical fields detect or diagnose a disease with results measured by continuous or discrete ordinal data. The performance of a diagnostic test is summarized using the receiver operating characteristic (ROC) curve and the area under the curve (AUC). The diagnostic test is considered clinically useful if the outcomes in actually-positive cases are higher than actually-negative cases and the ROC curve is concave. In this study, we apply the stochastic ordering method in a Bayesian hierarchical model to estimate the proper ROC curve and AUC when the diagnostic test results are measured in discrete ordinal data. We compare the conventional binormal model and binormal model under stochastic ordering. The simulation results and real data analysis for breast cancer indicate that the binormal model under stochastic ordering can be used to estimate the proper ROC curve with a small bias even though the sample sizes were small or the sample size of actually-negative cases varied from actually-positive cases. Therefore, it is appropriate to consider the binormal model under stochastic ordering in the presence of large differences for a sample size between actually-negative and actually-positive groups. © 2019 The Korean Statistical Society, and Korean International Statistical Society.
... Thus, ROC-based analysis has become a useful tool to evaluate the performance of classifiers in various fields, including medical diagnosis [11]. Different approaches to estimating the ROC curve have been investigated, whether being non-parametric [12], [13], semiparametric [14], [15], or parametric [16], [17]. A summarized ROC (SROC) curve was also proposed for meta-analysis, which has the same properties as the ROC. ...
The receiver operating characteristic (ROC) curve
is a useful tool to evaluate the performance of classifiers, and
is widely used in signal detection, pattern recognition and
machine learning. For complex object classification, multiple
single classifiers are often used and they are concatenated
into a multistage classification system. Thus, it is necessary to
obtain the overal ROC curve, because the ROC curves of the
individual classifiers are not useful for the overall system since it
has multi-level decision thresholds. In this paper, a systematic
approach was introduced for measuring the performance of
multistage systems via estimating the overall ROC curve. Two
new ROC models sharing the same properties of classical ROC
curves were proposed, inspired by the Gaussian and logistic
distributions. The models were then experimented on a recently
introduced multistage system for epileptic spike classification
from electroencephalogram data. Experimental results indicated
that the proposed ROC models can be used for multistage
classification systems.
... According to Signal Detection Theory(SDT), we assume that there are two probability distributions of the random variables X and Y, one associated with the signal event s and other with the non signal event n [11]; these probability (or density) distributions of a given observation x and y are conditional upon the occurrence of s and n [2]. ...
... The marginal distribution function is the double integral of equation (11). Therefore, the posterior probability density function of σ and β given the data (t 1, t 2, …, t n ) is obtained by dividing the joint posterior density function over the marginal distribution function as ...
A Receiver Operating Characteristic (ROC) curve provides quick access to the quality of classification in many medical diagnoses. The Weibull distribution has been observed as one of the most useful distributions, for modeling and analyzing lifetime data in Engineering, Biology, Survival and other fields. Studies have been done vigorously in the literature to determine the best method in estimating its parameters. In this paper, we examine the performance of Bayesian Estimator using Jeffreys‘ Prior Information and Extension of Jeffreys‘ Prior Information with three Loss functions, namely, the Linear Exponential Loss,General Entropy Loss, and Square Error Loss for estimating the AUC values for Constant Shape Bi-Weibull failure time distribution. Theoretical results are validated by simulation studies. Simulations indicated that estimate of AUC values were good even for relatively small sample sizes (n=25). When AUC≤0.6, which indicated a marked overlap between the outcomes in diseased and non-diseased populations. An illustrative example is also provided to explain the concepts.
... 4. If f(x) and g(y) denote the continuous PDF for non-diseased and diseased groups respectively. Let ( , ) KL f g denote the Kullback -Leibler (KL) divergence between the distributions of non-diseased and diseased group with f(x) as the comparison distribution and g(y) as the reference distribution (Hughes and Bhattacharya, 2013). Then ...
Receiver Operating Characteristic (ROC) Curve is used for assessing the ability of a biomarker/screening test to discriminate between non-diseased and diseased subject. In this paper, the parametric ROC curve is studied by assuming two-parameter exponential distribution to the biomarker values. The ROC model developed under this assumption is called bi-exponential ROC (EROC) model. Here, the research interest is to know how far the biomarker will make a distinction between diseased and non-diseased subjects when the gold standard is available using parametric EROC curve and its Area Under the EROC Curve (AUC). Here, the standard error is used as an estimate of the precision of the accuracy measure AUC. The properties of EROC curve that explains the behavior of the EROC curve are also discussed. The AUC along with its asymptotic variance and confidence interval are derived.
... This KLD measure has become popular in classification theory to explain the extent of accuracy in terms of closeness between two densities and the asymptotic properties of ROC Curve. Recently, the divergence measure KLD is used in Bi-normal ROC Curve, Bi-exponential ROC model and Bi-gamma ROC model to explain the symmetric and asymmetric properties (Hughes and Bhattacharya, 2013). Further, (Balaswamy, et al., 2014) considered the concept of KLD in classification to study and interpret the Single Truncated ROC Curve as well as used to identify the closeness between both distributions of normal and abnormal populations. ...
Classifying objects/individuals is common problem of interest. Receiver Operating Characteristic (ROC) curve is one such tool which helps in classifying the objects/individuals into one of the two known groups or populations. The present work focuses on proposing a Hybrid version of the ROC model. Usually the test scores of the two populations namely normal and abnormal tend to follow some particular distribution, here in this study it is considered that the test scores of normal follow Half Normal and abnormal follow Rayleigh distributions respectively. The characteristics of the proposed ROC model along with measures such as AUC and KLD are derived and demonstrated using a real data set and simulation data sets.
... Volkau et al. [38], Cabella et al. [39] promoted KLD as an application tool for analyzing the magnetic resonance images and also to compare the performance of two separate tests respectively. Hughes and Bhattacharya [40] characterized the symmetry properties of Bi-Normal and Bi-Gamma ROC curves in terms of KLD between two probability distributions which are of cases and controls. ...
In assessing the performance of a diagnostic test, the widely used classification technique is the Receiver Operating Characteristic (ROC) Curve. The Binormal model is commonly used when the test scores in the diseased and healthy populations follow Normal Distribution. It is possible that in real applications the two distributions are different but having a continuous density function. In this paper we considered a model in which healthy and diseased populations follow half normal and exponential distributions respectively, hence named it as the Hybrid ROC (HROC) Curve. The properties and Area under the curve (AUC) expressions were derived. Further, to measure the distance between the defined distributions, a popular divergence measure namely Kullback Leibler Divergence (KLD) has been used. Simulation studies were conducted to study the functional behavior of Hybrid ROC curve and to show the importance of KLD in classification.
... The ROC curves provide an efficient way to display the relationship between sensitivity and specificity and the cut-off point for positive and negative tests [22,23] . The ROC curves describe the performance of a test used to discriminate between normal and abnormal cases based on a variable measured on a continuous scale. ...
Researchers in the field of protein secondary structure prediction use typical three states of secondary structures, namely: alpha helices (H) beta strands (E), and coils (C). The series of amino acids polymers linked together into adjacent chains are known as proteins. Protein secondary structure prediction is a fundamental step in determining the final structure and functions of a protein. In this work we developed a prediction machine for protein secondary structure. By investigating the amino acids benchmark data sets, it was observed that the data is grouped into two distinct states or groups almost 50% each. In this scheme, researchers classify any state which is not classified as helix or strands as coils. Hence, in this work a new way of looking to the data set is adopted. For this type of data, the Receiver Operating Characteristic (ROC) analysis is considered for analysing and interpreting the results of assessing the protein secondary structure classifier. The results revealed that ROC analysis showed similar results to that obtained using other non ROC classification methods. The ROC curves were able to discriminate the coil states from non-coil states by 72% prediction accuracy with very small standard error.
