Figure 1 - uploaded by Radim Kolar
Content may be subject to copyright.
A - Control grid before registration. B - Control grid after the affine registration. C - Control grid after flexible registration showing restitution of a shear distortion of a frame.
Source publication
The paper describes restitution of geometrical distortions and improvement of signal-to-noise ratio of auto-fluorescence retinal images, finally aimed at segmentation and area estimation of the lipofuscin spots as one of the features to be included in glaucoma diagnosis. The main problems - geometrical and illumination incompatibility of frames in...
Context in source publication
Context 1
... in the 2D case, x x 1 x 2 is a separable n th order B-spline convolution kernel and c ( x i ) are their respective weighting coefficients. This B-spline interpolation scheme allows us to compute image derivatives required for efficient computation of criterion derivatives in optimization. We have used a multilevel (multi-resolution) approach to optimization, with different types of optimizers on individual levels - according to dimensionality of the optimization problem on the level, and also with respect to the risk of getting stuck in a local minimum. First, images are sub-sampled to the resolution 128 x 128 pixels and then rough translation parameters are found. As the next step, the affine transform parameters are searched for, using the results of the previous step as initiation. In these two steps, the controlled random search (CRS) optimizing algorithm is used, being a contraction process where an initial set of N points in Φ space is iteratively contracted by replacing the worst point with a better one; for details see [3]. In both steps, the nearest-neighbour (0-order B-spline) interpolation of transformed image values has been found (surprisingly) as best for the criterion computation. After the two steps, the found parameter vector may be assumed as close enough to the global optimum so that the similarity metrics can be considered a quadratic form and consequently, the Powell’s method may be used to quickly improve precision of the globally optimal Φ of affine transformation. This method finds the N -dimensional minimum of C by repeatedly minimizing C in a single dimension, gradually along N different directions; it only requires evaluations of C itself and not of its gradient (see [7] for details). In this step, 2 nd -order B-spline brightness interpolation is used in the criterion computation. No regularization ( w =w = 0) was used in the affine registration step. As the next step, flexible registration is applied using images down-sampled to 256 x 256 pixels. First, a coarse registration with spatial transform controlled by 11 x 11 control-nodes is done. Here we make use of the limited- memory Broyden, Fletcher, Goldfarb and Shannon method (L-BFGS); a quasi-Newton unconstrained nonlinear optimi- zer, utilising both function values and gradients to build up a picture of the surface to be optimized (see [8] for details). 2 nd order B-splines for image value interpolation and regularised optimum criterion with w s =0.0001, w v =0 were used. Finally, the found transform has been up-sampled to FFD controlled by 31 x 31 points, initiating the last step of elastic registration, which is done with this B-spline resolution. It requires 1922 parameters of the FFD to be optimized with respect to the criterion C using the regularization weights set to w s =0.01, w v =0.0001. The typical results of registration process are shown on the Fig. 1. IV. A VERAGING OF HRA S EQUENCES The proposed registration algorithm has been used for aligning image sequences each containing 9 images acquired from 15 patients. Of this data, 7 sequences were of 1024 x 1024 pixel resolution, others were of 512 x 512 resolution. In all cases the registration using the proposed algorithm has been successful. The quality of the registration was evaluated both subjectively via movies of each sequence after registration, detecting any misalignment as a movement, and objectively. The quantitative evaluation of the registration quality has been done on the registered images combined into an image by one of the three techniques: averaging, the principle component analysis (PCA) with taking the first component [9], and the minimum noise fraction transform (MNF) [9] also with the first component. The average SNR of a single image in the sequence was computed and compared to the SNR of the image combined without registration, after rigid registration and after elastic registration. An assumingly constant background area Ω was selected using image thresholding and morphological operations, and SNR was computed as the ratio between signal range (maximum minus minimum image value) and standard deviation over Ω ...
Citations
... To address these problems, several algorithms with varied levels of success have been proposed through the years 21,[43][44][45][46][47][48][49][50][51] . In our method, to accurately register relatively lowquality clinical FA images, we utilize a two-step non-rigid registration approach: a robust global vessel based registration method based on the RANdom SAmpling & Consequence (RANSAC) algorithm 52 , followed by a more accurate non-rigid intensity multi-resolution registration of FA images. ...
Purpose: To create and validate software to automatically segment leakage area in real-world clinical fluorescein angiography (FA) images of subjects with diabetic macular edema (DME). Methods: FA images obtained from 24 eyes of 24 subjects with DME were retrospectively analyzed. Both video and still frame images were obtained using a Heidelberg Spectralis 6-mode HRA/OCT unit. We aligned early and late FA frames in the video by a two-step non-rigid registration method. To remove background artifacts, we subtracted early and late FA frames. Finally, after post-processing steps, including detection and inpainting of the vessels, a robust active contour method was utilized to obtain leakage area in a 1500 µm radius circular region centered at the fovea. Images were captured at different fields-of-view (FOV) and were often contaminated with outliers, as is the case in real-world clinical imaging. Our algorithm was applied to these images with no manual input. Separately, all images were manually segmented by two retina specialists. The sensitivity, specificity, and accuracy of manual interobserver, manual intraobserver, and automatic methods were calculated. Results: The mean accuracy was 0.86±0.08 for automatic versus manual, 0.83±0.16 for manual interobserver, and 0.90±0.08 for manual intraobserver segmentation methods. Conclusions: Our fully-automated algorithm can reproducibly and accurately quantify the area of leakage of clinical grade FA video and is congruent with expert manual segmentation. The performance was reliable for different DME sub-types. This approach has the potential to reduce time and labor costs and may yield objective and reproducible quantitative measurements of DME imaging biomarkers.
Copyright © 2015 by Association for Research in Vision and Ophthalmology.
... Intensity-based algorithms [14], [15], [17]- [20] are preferable if there is no change of content between images, as every pixel can be used to drive the registration and no feature extraction is required. Change of content and severe occlusions make similarity metrics based on pixel intensity unusable. ...
... Dreo et al. [18] use an intensity-based approach to rigidly register fundus FA sequences. Kubeka et al. [20] use mutual information to register FA small field of view (20 • ) images combining a global affine transformation and a local free form deformation based on B-splines. Tsai et al. [25] adapt their previous work on dual-bootstrap, 12-parameter quadratic transformation registration based on Lowe keypoint generation and matching [21] to align small FOV fundus FA sequences. ...
... This kind of preprocessing requires total coherence among repetitions, which cannot be guaranteed in retinal imaging due to eye movements between individual images and, in case of scanning imaging as with scanning laser tomography (SLT), even during individual scans, so that the scan lines become mutually shifted. To prevent these inconsistencies, the images must be flexibly registered taking into account the characteristic types of distortions ahead of averaging; this particular problem has been treated in [41] and [42]. As the method belongs to the more generic area of retinal image registration, it will be treated in section 2.2. ...
... The registration of retinal images, as a preparatory step for fusion required for diagnostic purposes [6,14,54], iteB6, has certain particularities specific to individual applications that must be taken into account when designing the registration methods for particular purposes, as in [33,37,41,42]. All the registration methods [52,72] have a common approach -searching for a geometric transform T (α) that best describes the spatial relationship between the details in the reference image and the corresponding details of the registered image. ...
... Already mentioned preregistration of AF retinal images before averaging [34,42] used in principle the same approach based on eq. (7); however with some differences crucial for successful registration of the SLT sequences [41]. ...
The paper presents an overview of image analysis activities of the Brno DAR group in the medical application area of retinal imaging. Particularly, illumination correction and SNR enhancement by registered averaging as preprocessing steps are briefly described; further mono-and multimodal registration methods developed for specific types of oph-thalmological images, and methods for segmentation of optical disc, retinal vessel tree and autofluorescence areas are presented. Finally, the designed methods for neural fibre layer detection and evaluation on retinal images, utilising different combined texture analysis ap-proaches and several types of classifiers, are shown. The results in all the areas are shortly commented on at the respective sections. In order to emphasise methodological aspects, the methods and results are ordered according to consequential phases of processing rather then divided according to individual medical applications.
... This kind of preprocessing requires total coherence among repetitions, which cannot be guaranteed in retinal imaging due to eye movements between individual images and, in case of scanning imaging as with scanning laser tomography (SLT), even during individual scans, so that the scan lines become mutually shifted. To prevent these inconsistencies, the images must be flexibly registered taking into account the characteristic types of distortions ahead of averaging; this particular problem has been treated in [41] and [42]. As the method belongs to the more generic area of retinal image registration, it will be treated in section 2.2. ...
... The registration of retinal images, as a preparatory step for fusion required for diagnostic purposes [6,14,54], iteB6, has certain particularities specific to individual applications that must be taken into account when designing the registration methods for particular purposes, as in [33,37,41,42]. All the registration methods [52,72] have a common approach -searching for a geometric transform T (α) that best describes the spatial relationship between the details in the reference image and the corresponding details of the registered image. ...
... Already mentioned preregistration of AF retinal images before averaging [34,42] used in principle the same approach based on eq. (7); however with some differences crucial for successful registration of the SLT sequences [41]. ...
-The contribution summarises the results of a long-term project concerning processing and analysis of multimodal retinal image data, run in cooperation between Brno University of Technology -Dept. of Biomedical Engineering and Erlangen University -Clinic of Ophthalmology. From the medical application point of view, the main stimulus is the improvement of diagnostics (primarily of glaucoma but other diseases as well) by making the image segmentation and following analysis reproducible and possibly independent on the evaluator. Concerning the methodology, different image processing approaches had to be combined and modified in order to achieve reliable clinically applicable procedures.
... The simple registration algorithm, using only spatial shift, is implemented in original HRA software. There was also attempt in ongoing research to increase the registration accuracy with the help of non-rigid registration [84]. This possibility won't be discussed here and therefore, the averaged AF image from HRA software will be used in next research. ...
... Obviously, there can be more complicated discrepancies between images, caused by movement of the eye during the scanning process. These kind of distortions would need flexible transformations [84]. To make this method practicable (i.e. to speed up the computations), ...
"Obor Biomedicínské inženýrství" Zkrácená verze habilitační práce--Vysoké učení technické v Brně, Fakulta elektrotechniky a komunikačních technologií, Ústav biomedicínského inženýrství, 2009
Based registration of retinal images proved to be very successful especially for minimally overlapping images. The most commonly used transformation method uses a quadratic model to represent the geometry of the retinal surface. Although this model has been used for more than one decade, there is no literature that studies the model errors for abnormal eye geometries. In this work, we present a study of the registration errors of the quadratic model in case of diseased eyes. The study includes two basic models of the retinal surface for eyes suffering from: myopia; and retinal diseases (e.g. age related macular degeneration). In addition, real datasets of age related macular degeneration (AMD) patients have been used to quantify the registration error. The simulation results show that the average error can be as high as 13 pixels at extreme conditions of myopia and retinal diseases. For real datasets with typical disease conditions, the error was found to be 2.6 pixels.
The lecture will summarise results of the long-term project concerning processing and analysis of multimodal retinal image data. The project is run at the Dept. of BME, FEEC, Brno University of Technology in frame of the DAR research centre coordinated by the Inst. of Information Theory and Automation, Cz.Ac.Sci. Prague, in cooperation with the Clinic of Ophthalmology, University Erlangen (D) and also with the Ophthalmologic Centre, Zlin (CZ). From the medical application point of view, the main idea is the improvement of retina based diagnostics (primarily of glaucoma) utilising automatic reproducible image segmentation and analysis, independent on the evaluator. The used methodology encompasses many approaches that are to be combined and partially modified in order to achieve - in the end - reliable, clinically applicable procedures. Besides describing the long term development of the research, particular interest will be devoted to some of the latest results, concerning blood vessel segmentation using 2D matched filtering and retinal neural layer detection via texture analysis.
We present RERBEE (robust efficient registration via bifurcations and elongated elements), a novel feature-based registration algorithm able to correct local deformations in high-resolution ultra-wide field-of-view (UWFV) fluorescein angiogram (FA) sequences of the retina. The algorithm is able to cope with peripheral blurring, severe occlusions, presence of retinal pathologies and the change of image content due to the perfusion of the fluorescein dye in time. We have used the computational power of a graphics processor to increase the performance of the most computationally expensive parts of the algorithm by a factor of over × 1300, enabling the algorithm to register a pair of 3900 × 3072 UWFV FA images in 5-10 min instead of the 5-7 h required using only the CPU. We demonstrate accurate results on real data with 267 image pairs from a total of 277 (96.4%) graded as correctly registered by a clinician and 10 (3.6%) graded as correctly registered with minor errors but usable for clinical purposes. Quantitative comparison with state-of-the-art intensity-based and feature-based registration methods using synthetic data is also reported. We also show some potential usage of a correctly aligned sequence for vein/artery discrimination and automatic lesion detection.
This article deals with registration and fusion of multimodal opththalmologic images obtained by means of a laser scanning device (Heidelberg retina angiograph). The registration framework has been designed and tested for combination of autofluorescent and infrared images. This process is a necessary step for consecutive pixel level fusion and analysis utilizing information from both modalities. Two fusion methods are presented and compared.
A semiautomatic approach to the detection and evaluation of the autofluorescent zones in retinal images, recognized as having a diagnostic value, has been designed based on fusing information from two Heidelberg Retina Angiograph imaging modalities - autofluorescent and infrared modes. The procedure, initiated by automatic preprocessing and region-of-interest determination continues by manually initiated segmentation via constrained region growing and ends with evaluating the size and geometrical coordinates of the AF regions with respect to the centre of the optic disc. Results are compared with those obtained by experienced ophthalmologists.
