Content uploaded by Marios S Pattichis
Author content
All content in this area was uploaded by Marios S Pattichis on Jan 08, 2014
Content may be subject to copyright.
Detection of Hard Exudates and Red Lesions in the Macula Using a Multiscale
Approach
Carla Agurto*,†, Honggang Yu†, Victor Murray*, Marios S. Pattichis*, Simon Barriga†, Peter Soliz†
*Electrical and Computer Engineering Department
University of New Mexico, Albuquerque, New Mexico 87131
Emails: capaagri@unm.edu, vmurray@ieee.org, pattichis@ece.unm.edu
†VisionQuest Biomedical LLC, Albuquerque, New Mexico 87106
Emails: hyu@visionquest-bio.com, sbarriga@visionquest-bio.com, psoliz@visionquest-bio.com
Abstract—Diabetic retinopathy (DR) is a complication of
diabetes that causes blindness to 1.8 million people in the
world. The risk of vision loss from DR increases when
pathologies present on the macula. In this paper, we present a n
automatic system to detect pathologies on the macula such as
hard exudates microaneurysms, and hemorrhages. Our
approach is a bottom-up implementation, which tries to
capture each abnormal structure in the macula in order to
detect DR lesions. This technique starts by eliminating the non-
uniform illumination thereby enhancing the contrast of red
lesions in the images. Possible DR lesion (hard exudates and
red lesions) candidates on the macula are extracted by using
amplitude-modulation frequency-modulation (AM-FM)
features. AM-FM features extract texture information from
different frequency scales, providing for an effective method
for the detection of hard exudates and red lesions. For each
lesion candidate, we also extract shape, color and other texture
features that are then combined with AM-FM features.
Pathologies in the macula are detected from the candidate
lesions using supervised classification with Partial Least
Squares.
Diabetic Retinopathy; Amplitude-modulation Frequency-
modulation (AM-FM); Partial Least Squares
I. INTRODUCTION
Located at the center of the macula, the fovea contains
the highest density of photoreceptors in the retina and is
responsible for the central vision. Many pathologies
occurring on or near the fovea, such as clinically significant
macular edema (CSME), represent a high risk for vision loss.
For example, there is an association between hard exudates
near the fovea and CSME [1]. However, not all the types of
lesions in the macula represent a comparable risk to patients.
For example, drusen, which look similar in shape and color
to exudates are not immediately sight threatening; thus, must
be differentiated from hard exudates, which are high risk for
sight threatening disease and demand an alternate clinical
pathway of patient management.
Many approaches have been proposed as a means for
automatic DR screening. Most of them utilize “bottom-up”
techniques in which segmentation of the lesions is required
in order to detect DR. Other approaches [2, 3] are "top
down" where segmentation and grading of specific lesions is
not necessary to classify the image as normal or abnormal.
Much of the reported work has focused either on the
detection of red lesions such as microaneurysms and
hemorrhages on the fundus images [4, 5, 6] or on the
detection of bright lesions such as exudates and cotton wool
spots [7, 8, 9]. The extraction of features on the retinal
images is commonly the basis for most automatic
classification systems. Morphological methods [10], Gabor
filters [7], and Wavelet transforms are the most popular
methods for feature extraction [5]. A number of different
classifiers have been used to process the extracted features.
Sopharak et al. [8] used an unsupervised method called
Fuzzy C-means. The authors in [7] used Neural Networks in
which each pixel is associated with a soft label indicating the
probability of a pixel being bright. In a different approach to
detect red lesions, Niemeijer et al. [6], used k-nearest
neighbors classifier with Neural Networks.
Our approach uses an optimization approach to select the
most promising Amplitude-Modulation Frequency-
Modulation (AM-FM) features. Contrary to other methods,
the same system is applied to detect red lesions and hard
exudates.
II. DATA DESCRIPTION
The images were acquired at the University of Texas
Health Science Center, San Antonio (UTHSCSA). 153
macula-centered digital fundus photographs were used to
train/test our algorithm. The images were acquired using
2392x2048 pixels and 60 degrees field of view. Pixel
footprint is about 9 Pm. A region of 1 disc diameter (DD)
centered in the fovea (1DD = 400pixels) was extracted for
each image. Lesions such as hard exudates and red lesions
(microaneurysms and hemorrhages) were marked by a
certified ophthalmic medical technologist. N=35 images
were graded as normal, and the remaining images presented
two types of lesions. N=79 images presented hard exudates
in the macula, and N=81 images presented red lesions. N=
42 present with both types of lesions. The normal cases also
contained images with non-pathological features such as
retinal sheen, foveal reflex, and low contrast. Other images
presented drusen which are pathologies related to age-related
macular degeneration (AMD).
III. METHODOLOGY
Fig. 1 shows the methodology used to detect the bright and
dark lesions on the macula. The pre-processing block is
applied only to detect red lesions. In the following
13978-1-4673-1830-3/12/$31.00 ©2012 IEEE SSIAI 2012
subsections, we explain the optimization procedure used to
obtain the best performance of our algorithm.
A. Pre-processing for red lesion detection
For the detection of red lesions, the images are pre-
processed following a three-step approach. First, we apply
illumination correction using a shade correction technique
[11]. Second, non-overlapping windows of 30x30 pixels are
selected from the image, the bright pixels are detected and
then they are replaced by the mean average value of the
remaining pixels in that window. To find the optimal
threshold for selecting the brightest intensity pixels, the
second derivative of the histogram of the intensity pixel
values is calculated. After doing so, the image is smoothed
with a 9x9 average filter. Finally, the contrast is enhanced
using contrast limited adaptive histogram equalization
(CLAHE), as shown in Fig. 2.
B. Amplitude-Modulation Frequency-Modulation
Amplitude Modulation-Frequency Modulation (AM-
FM) [12] represents an image in terms of its instantaneous
amplitude and instantaneous frequency components as:
¦
|
M
1n
nn
y)(x,y)cos(x,a),(
M
yxI
(1)
where M is the number of AM-FM components,
),( yxa
n
denote instantaneous amplitude functions (IA) and
),( yx
n
M
denote the instantaneous phase functions. For each AM-FM
component, the instantaneous frequency (IF) is defined in
terms of the gradient of the phase
n
M
:
¸
¸
¹
·
¨
¨
©
§
w
w
w
w
y
yx
x
yx
yx
nn
n
),(
,
),(
),(
MM
M
(2)
In terms of textural features, for each component, three
estimates of the AM-FM outputs are used: instantaneous
amplitude, instantaneous frequency magnitude (IFm) and
angle (IFangle). These AM-FM estimates were calculated at
5 different frequency scales which correspond to the
following bands of frequencies: High (H), Medium (M),
Low (L), Very Low (VL) and Ultra Low (U). Then we
merged them with the low pass filter (LPF) in 13 different
combinations: U-H, LPF, VL, L, M, LPF-H, U-VL, VL-L,
L-M, M+H, H, U.
C. Parameter Optimization
Estimates of the IA, IF magnitude, and IF angle are
calculated for the 13 different combinations of scales. Thus,
a total of 39 different AM-FM feature images are obtained
for each image. From them, binary maps are created by
thresholding the generated AM-FM feature images. In order
to find the optimal threshold value to create the binary
maps, an optimization technique was used based on a subset
of 53 images. Thirty different percentiles were used to test
for the optimal threshold value. Here, note that lesions are
characterized by low or high values of the AM-FM features
images.
By using the reader-based ground-truth for hard
exudates and red lesions on the macula, sensitivity and
specificity were obtained and the distance to the ideal point
(100%/100%) is calculated for each of the 1170 points (39
AM-FM feature images times 30 percentile values) for each
image and the two types of lesions. The binary AM-FM
feature images with a threshold that has the minimum
distance to the ideal point (set to be less than 0.3
heuristically on training data) will be selected as an input for
our system. Fig. 3 shows some examples of distances for
different thresholds and different types of lesions. It can be
noticed that in Fig. 3a for percentiles near 67th, the distance
to the ideal point is lower than the acceptable minimum
distance = 0.3 (dark blue) for most of the AM-FM feature
images so many of them are useful to detect hard exudates.
On the other hand, Fig 3b shows fewer cases with distances
lower than 0.3 meaning that only few of them are going to
be used to detect red lesions.
Figure 1. Block diagram of our approach to detect hard exudates and red
lesions in the macula.
Figure 2. Pre-processing for detecting red lesions in the macula. (a)
Macula from the original green channel, (b) Macula of the retinal image
after apply the pre-processing block explained in section III.A.
14
Figure 3. Map of distance values for the different thresholds. The y-axis
represents the AM-FM representation for the 13 different combinations in
the order specified in section A for IA(1-13), IFm (14-26), and IFangle (27-
39). (a) Results of distances after applying the threshold to find lower
thresholds to detect hard exudates. (b) Results of distances after applying
the threshold to find upper thresholds to detect red lesions.
Figure 4. Extraction of lesion candidates to detect red lesions. (a) Original
image, (b) Binary map obtained with the relevant scales and optimal
parameters of AM-FM representation, (c) Constrained binary maps to dark
pixels, (d) Candidates of red lesions after applying morphological
operations
D. Color constraint
Color constraints are applied to the AM-FM binary
output using a sliding window of 100x100 pixels. The bright
pixels that are higher than the 95th percentile of the content
of this window are maintained for the bright lesion
detection. For red lesions detection, the threshold was set to
the 7th percentile. The intensity pixels below this threshold
are used to mask the AM-FM binary maps. Since there is a
great amount of dark pixels in the fovea and lesions are the
darkest among them, a special mask of approximately the
size of the fovea is used to reduce the false positives rate
detection. The pixels with intensity smaller than the 3rd
percentile of the content within this window are kept to
generate the dark lesions mask. Similarly, pixels that are
higher than 99th percentile in the fovea are kept for the
bright pixels mask. This modification was implemented
after noticing that the foveal reflex, an imaging artifact, is
darker than exudates when they are close to each other.
After these masks are generated, they are multiplied by
the AM-FM binary maps, as it is shown in Fig 4(c).
Afterwards, morphological operations to remove small
objects that are considered to be noise or vessel lines are
applied, as shown in Fig. 4(d).
E. Extraction of features
Sixty-four features are used to characterize each of the
possible candidate objects in order to determine its type
(non-lesion, hard exudate, red lesion). By using the pixel
information of each candidate, we extracted 3 types of
features: 1) Color information within the candidate and a
neighborhood of pixels outside the candidate, 2) Shape
(area, position, eccentricity, major and minor axis length,
solidity, perimeter), and 3) Texture information using gray
level concurrence matrix (contrast, energy, homogeneity,
correlation). The features are normalized to have zero mean
and standard deviation 1.
F. Classification
The features obtained in Section E are the inputs of a
linear regression classifier based on partial least squares
(PLS) [3]. The classifier is trained for each type of lesion
(red lesions and hard exudates) by using the 53 images
selected for training purposes.
IV. RESULTS
The optimal parameters obtained with the optimization
process for the detection of hard exudates and red lesions are
applied to our testing set consisting of 100 images. After the
candidates for each type of lesion are extracted, the model
created with the training images is applied to the images. We
present results for all the candidates that we extracted for
each type of lesion in Fig. 5. However, since we are
interested in the detection of abnormalities in the macula per
retinal image, a point of the ROC curve that has very high
specificity is used to reduce the amount of false positives in
the classification per image.
A. Hard Exudates
Fig. 5 shows the ROC curve for the detection of lesion
candidates for red lesions and hard exudates. An area under
the ROC curve (AUC) of 0.95 is obtained in the candidate
classification. By setting the threshold to obtain a sens/spec =
66%/98%, the classifier trained to detect maculas with
exudates in a testing set of 100 images (51 with exudates, 49
without exudates) achieved 100% sensitivity, with specificity
of 58%.
B. Red lesions
An AUC of 0.90 is obtained in the candidate
classification. By setting the threshold to obtain a
sens/spec = 74%/90%, the detection of maculas with red
15
lesions in the testing set (60 with red lesions, 49 with
non-red lesions) is 92%/55%.
V. DISCUSSIO N
Structures captured with AM-FM features offer rich
information of the analyzed lesions: hard exudates and red
lesions. In addition to this, the optimization process helps us
obtain high sensitivity in the detection of these lesions even
in the candidate selection process prior to the classification.
Fine drusen, also a bright lesion, is a problem for the
algorithm, so a post-processing to focus on the shape of these
small lesions may improve its performance. As it is known,
the shape of the drusen is more similar to a circle than
exudates. Image enhancement may also help identify drusen
first and help avoid false exudates detection.
The problem of detecting red lesions is very challenging
since they present very irregular shapes and have variable
texture characteristics. However, the results obtained with
this optimized method are encouraging.
Vessels with branches or portions of vessels are another
cause of misclassification. A more detailed analysis is going
to be performed in order to eliminate those from the
candidates.
VI. CONCLUSIONS
A computer-aided detection algorithm based on
generalized optimization scheme of image decompositions is
presented. Given the optimization process and the flexibility
of the implementation, this methodology could be extended
to the detection of different types of lesions. In addition, the
system only requires image enhancement for red lesions
since the exudates are well captured with AM-FM features.
The system achieves 100% sensitivity in detecting maculas
with hard exudates and 92% sensitivity in detecting maculas
with red lesions. In future implementations, post-processing
is going to be added in order to increase the specificity of the
system.
ACKNOWLEDGMENT
This work was supported by NEI grants: EY020015,
RC3EY020749. S. Nemeth for marking the lesions used in
this study. UTHSCSA for the data.
Figure 5. ROC curve of the detection of lesions of the extracted
candidates. (a) Detection of exudates, (b) Detection of red lesions.
Figure 6. Results of the lesion detection in the macula algorithm for
maculas with exudates (1st and 3rd columns), and for red lesions (2nd
column).
REFERENCES
[1] S. Resnikoff et al., "Global data on visual impairment in the year
2002", Bull. World Health Organ., vol. 82, pp. 844-851, 2004.
[2] E. Chaum, T.P. Karnowski, V.P. Govindasamy, M. Abdelrahman,
K.W. Tobin, “Automated Diagnosis of Retinopathy By Content-
Based Image Retrieval,” Retina:November/December 2008. 2008;
28(10):1463-1477.
[3] C. Agurto, V. Murray, E. Barriga E. S., et al., “Multiscale AM-FM
Methods for Diabetic Retinopathy Lesion Detection,” IEEE
Transactions on Medical Imaging. 2010; 29(2): 502-512.
[4] M. Larsen, J. Godt, N. Larsen, et al. “Automated detection of fundus
photographic red lesions in diabetic retinopathy”. Invest Ophthalmol
Vis Sci. 2003; 44(2):761-6.
[5] G. Quellec, M. Lamard, P.-M. Josselin, G. Cazuguel, B. Cochener, C.
Roux, “Optimal wavelet transform for the detection of
microaneurysms in retina photographs,” IEEE Transactions on
Medical Imaging. 2010. 2008; 27(9):1230-1241.
[6] M. Niemeijer, B. vanGinneken, J. Staal, M.S.A. Suttorp-Schulten, M.
D. Abràmoff, “Automatic Detection of Red Lesions in Digital Color
Fundus Photographs,” IEEE Transactions on Medical Imaging. 2005;
24(5):584-592.
[7] M. Niemeijer, S.R. Russell, M. A. Suttorp, B. vanGinneken, M. D.
Abràmoff, “Automated Detection and Differentiation of Drusen,
Exudates, and Cotton-wool Spots in Digital Color Fundus
Photographs for Early Diagnosis of Diabetic Retinopathy,” Invest
Ophthalmol Vis Sci. 2007; 48(5):2260-2267.
[8] A. Sopharak, B. Uyyanonvara, “Automatic Exudates Detection from
Diabetic Retinopathy retinal image using Fuzzy C-means and
Morphological methods,” Proceedings of the Third IASTED
International Conference Advances in Computer Science and
Technology, Phuket, Thailand.2007; pp. 359-364.
[9] A. Osareh, M. Mirmehdi, B. Thomas and R. Markham, “Automated
Identification of Diabetic Retinal Exudates in Digital Colour Images,”
British Journal of Ophthalmology. 2003: 87(10): 1220-1223.
[10] T. Walter, J. C. Klein, P. Massin, A. Erginay, “A Contribution of
Image Processing to the Diagnosis of Diabetic Retinopathy-Detection
of Exudates in Colour Fundus Images of the Human Retina,” IE EE
Transactions on Medical Imaging. 2002; 21: 1236-1243.
[11] Archie Sharma, "Automated Depth analysis of optic nerve head from
stereo fundus image", Master thesis, University of Texas Tech.
[12] V. Murray, P. Rodriguez, M.S. Pattichis, “Multi-scale AM-FM
Demodulation and Reconstruction Methods with Improved
Accuracy,” IEEE Transactions on Image Processing. 2010; 19(5):
1138-1152.
16
