Figure - available from: Neuroinformatics
This content is subject to copyright. Terms and conditions apply.
Fluorescence microscopy image of a neuron with manually indicated junctions (red circles) and terminations (yellow circles). The radius of each annotated critical-point region reflects the size of the underlying image structure
Source publication
Digital reconstruction of neuronal cell morphology is an important step toward understanding the functionality of neuronal networks. Neurons are tree-like structures whose description depends critically on the junctions and terminations, collectively called critical points, making the correct localization and identification of these points a crucia...
Citations
... Methods for detecting vessel bifurcation points directly from images can be divided into model-based methods and learning-based methods. Model-based methods include vector field-based [15] , fuzzylogic-based [16] , local Gaussian models [17] , Hessian-based [18] , AdaBoost-based [19] , scale-space behavior-based [20] , and so on. These methods adopt specific models for specific types of vessel bifurcation points. ...
Background and objective
: Accurate detection of vessel bifurcation points from mesoscopic whole-brain images plays an important role in reconstructing cerebrovascular networks and understanding the pathogenesis of brain diseases. Existing detection methods are either less accurate or inefficient. In this paper, we propose VBNet, an end-to-end, one-stage neural network to detect vessel bifurcation points in 3D images.
Methods
: Firstly, we designed a 3D convolutional neural network (CNN), which input a 3D image and output the coordinates of bifurcation points in this image. The network contains a two-scale architecture to detect large bifurcation points and small bifurcation points, respectively, which takes into account the accuracy and efficiency of detection. Then, to solve the problem of low accuracy caused by the imbalance between the numbers of large bifurcations and small bifurcations, we designed a weighted loss function based on the radius distribution of blood vessels. Finally, we extended the method to detect bifurcation points in large-scale volumes.
Results
: The proposed method was tested on two mouse cerebral vascular datasets and a synthetic dataset. In the synthetic dataset, the F1-score of the proposed method reached 96.37%. In two real datasets, the F1-score was 92.35% and 86.18%, respectively. The detection effect of the proposed method reached the state-of-the-art level.
Conclusions
: We proposed a novel method for detecting vessel bifurcation points in 3D images. It can be used to precisely locate vessel bifurcations from various cerebrovascular images. This method can be further used to reconstruct and analyze vascular networks, and also for researchers to design detection methods for other targets in 3D biomedical images.
... In the past few decades, many traditional methods have been proposed to detect junctions in biomedical images. These methods can be categorized into skeleton-based method [23]- [26] and model-based method [27] [28]. A skeleton-based method RLRSM [26] was proposed to efficiently detect junctions and terminations for curvilinear structures in segmented images. ...
... Some prototypes are carefully selected to construct the trainable COSFIRE [27] filters which are effective to detect junctions similar to prototypes. Another model-based approach, Neuron Pinpointer (NP) [28], was presented to detect the junctions in 2D neuron images. This method is based on the directional filtering and angular profile analysis, in combination with a two-stage fuzzy-logic based reasoning system. ...
... Because we focus on 2D images, and the 3D neuron image usually has a lower resolution in the Z direction [20], the Maximum Intensity Projection (MIP) method is used along the Z direction to obtain the training and testing neuron images. Nevertheless, the junctions in 2D neuron images are also critical points that can provide valuable information about the neuron structures [28]. There are many neuron reconstruction works that analyze the 3D images based on the detected 2D critical points [20] [52]. ...
Junction plays an important role in biomedical research such as retinal biometric identification, retinal image registration, eye-related disease diagnosis and neuron reconstruction. However, junction detection in original biomedical images is extremely challenging. For example, retinal images contain many tiny blood vessels with complicated structures and low contrast, which makes it challenging to detect junctions. In this paper, we propose an O-shape Network architecture with Attention modules (Attention O-Net), which includes Junction Detection Branch (JDB) and Local Enhancement Branch (LEB) to detect junctions in biomedical images without segmentation. In JDB, the heatmap indicating the probabilities of junctions is estimated and followed by choosing the positions with the local highest value as the junctions, whereas it is challenging to detect junctions when the images contain weak filament signals. Therefore, LEB is constructed to enhance the thin branch foreground and make the network pay more attention to the regions with low contrast, which is helpful to alleviate the imbalance of the foreground between thin and thick branches and to detect the junctions of the thin branch. Furthermore, attention modules are utilized to introduce the feature maps from LEB to JDB, which can establish a complementary relationship and further integrate local features and contextual information between the two branches. The proposed method achieves the highest average F1-scores of 0.82, 0.73 and 0.94 in two retinal datasets and one neuron dataset, respectively. The experimental results confirm that Attention O-Net outperforms other state-of-the-art detection methods, and is helpful for retinal biometric identification.
... In recent years, some studies on neuronal structure critical point detection and dendritic spines detection have been developed, such as combination of shifted filter response (COSFIRE) [21] and Neuron Pinpointer (NP) [22]. Moreover, there are some methods for detecting 2-D crossings in other kinds of images, such as chromosome images [23]. ...
Morphology reconstruction of neurons from 3D microscopic images is essential to neuroscience research. However, many reconstructions may contain errors and ambiguities because of the cross-over neuronal fibers. In this paper, an automatic algorithm is proposed for the detection and separation of cross-over structures and is applied to neuron tracing for improving the neuron reconstruction results. First, an SPE-Net is employed to detect the 3D neuron cross-over points and locate the cross-over structures in neuron volumetric images. Second, a multiscale upgraded ray-shooting model (MSURS) is proposed to obtain robust results at different scales with high confidence and is employed to extract the cross-over neuronal structure features. Then, a cross-over structure separation method (CSS) is developed to eliminate the false connections of cross-over structures and generate deformed separated neuronal fibers based on the extracted features to replace the original neurites signals. Experiments demonstrate that the SPE-Net for cross-over point detection achieves average precision and recall rates of 73.89% and 79.66% respectively and demonstrate the proposed CSS method can improve 20.46% the performance of the reconstructions on average. The results confirm that the proposed method can effectively improve the neuron tracing results in volumetric images.
... It is an extremely challenging task for the regular interest points detection methods [21] to directly detect the 3-D junction points in image stacks. A directional filtering and feature extraction algorithm in combination with a two-stage fuzzy-logic-based reasoning system was proposed in [22] to detect the 2-D termination points and junction points in neuron images. Nevertheless, it has not been extended to detect the 3-D junction points. ...
... However, the existing ray-shooting model takes too much irrelevant pixel information into account, which is inefficient in extracting branch features. Furthermore, the existing junction points detection methods [22]- [24] mainly focus on the tracing applications and have not been extended to the proof-editing applications. ...
The digital reconstruction of neurons is essential to various neuroscientific studies. Due to the existence of gaps and ambiguities in neuron images, the neuron tracing results generated by most automatic reconstruction algorithms may be incomplete, resulting in false negatives (FNs), which need to be repaired in proof editing. However, the automatic proof-editing methods for repairing FN branches have rarely been explored. In this study, we propose a proof-editing algorithm for automatically detecting and repairing the FN branches of the initial reconstruction, which is based on a multiscale upgraded ray (MUR)-shooting model and an MOST-based repairer. The MUR detects the FN branch and the corresponding branch direction vector by analyzing the multiscale intensity distribution features around a topological feature point. The topological feature points contain the junction points detected from the neuron image and the tip nodes extracted from the initial reconstruction. The MOST-based repairer is proposed to prevent the redundant reconstructions by assigning the detected branch direction vector as the initial tracing direction, which rejects the nodes returning to the traced area. The experimental results demonstrate clearly that the proposed method can reduce 20% of the false-negative rate at most. The experimental results confirm that the proposed method is extremely helpful for generating faithful reconstructions.
... Junction points can be obtained either by manual annotation, for example using Vaa3D-Neuron [11], or by automatic detection using image filtering and pattern recognition methods [24], [27], [37]- [40]. However, manual annotation is very timeconsuming given the typical size and complexity of the data sets, and thus automation is the only viable option. ...
... The COSFIRE method uses a set of trainable keypoint detectors that we call Combination Of Shifted Filter Responses (COSFIRE filters) to detect vascular bifurcations in segmented retinal images [27]. Since each type of junction points may vary considerably in terms of geometry and image intensity, designing a different number of filters (the parameters for each filter need to be adjusted) for each possible case is time consuming [24]. ...
... We further evaluated the proposed method on 2D neuron images, by comparing its performance with the COSFIRE method and Neuron Pinpointer (NP) [24], a state-of-the-art neuron junction detection method. For the COSFIRE method, we trained a set of COSFIRE filters for two neuron datasets separately as in [27]. ...
Detection and localization of terminations and junctions is a key step in the morphological reconstruction of tree-like structures in images. Previously, a ray-shooting model was proposed to detect termination points automatically. In this paper, we propose an automatic method for 3D junction points detection in biomedical images, relying on a circular sampling model and a 2D-to-3D reverse mapping approach. First, the existing ray-shooting model is improved to a circular sampling model to extract the pixel intensity distribution feature across the potential branches around the point of interest. The computation cost can be reduced dramatically compared to the existing ray-shooting model. Then, the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm is employed to detect 2D junction points in maximum intensity projections (MIPs) of sub-volume images in a given 3D image, by determining the number of branches in the candidate junction region. Further, a 2D-to-3D reverse mapping approach is used to map these detected 2D junction points in MIPs to the 3D junction points in the original 3D images. The proposed 3D junction point detection method is implemented as a build-in tool in the Vaa3D platform. Experiments on multiple 2D images and 3D images show average precision and recall rates of 87.11% and 88.33% respectively. In addition, the proposed algorithm is dozens of times faster than the existing deep-learning based model. The proposed method has excellent performance in both detection precision and computation efficiency for junction detection even in large-scale biomedical images.
... For instance, the human brain has 10 11 neurons. Sophisticated pattern recognition and image filtering techniques can automatically detect critical points [17]- [22]. However, the existing automatic methods only focus on the detection of one or two types of critical points. ...
... The method proposed in [38] mainly detects terminations and BPs without discriminating between three-branch points (TBs) and four-branch points (FBs). In addition, most of the existing methods, such as the Neuron Pinpointer (NP) [17], focus on only one particular type of data set. Since the regions around different types of critical points have different pixel intensity distribution features, it is challenging for existing methods to simultaneously detect all types of critical points. ...
... The terminations and TBs can be easily extracted from the reconstruction outputs using APP2, by identifying the nodes with more than one parent or children from the reconstruction results. This kind of evaluation was also used in [17]. We can see that the proposed method also has very reliable performance, especially in the precision rate. ...
The correct location and identification of critical points, including terminations, three-branch points and fourbranch points, for curvilinear structures in images is a crucial task in many curvilinear structure reconstruction processes. Few methods have been proposed to detect all three types of critical points simultaneously. In this paper, we upgraded the existing rayshooting model to a multifunctional ring-like ray-shooting model that can efficiently detect all types of critical points for curvilinear structures at the same time. First, we measured the minimum and maximum radii of the region of interest based on a sphere-growing method and the modified ray-burst model, respectively. Second, the estimated minimum and maximum radii are used to determine the inner and outer radii of the ring-like ray-shooting model, allowing the proposed model to adaptively extract the pixel intensity distribution feature along the potential branches. Finally, the density-based spatial clustering of applications with noise (DBSCAN) algorithm is used to detect critical points of curvilinear structures by analyzing the number of clusters of pixel intensity distribution extracted from the ring-like ray-shooting model. The experimental results on multiple datasets confirmed that the proposed method can achieve very accurate detection results efficiently and provide the reconstruction of curvilinear structures with beneficial information.
... Crossover points are defined as the points of visual intersection of two neuronal structures, falsely suggesting a junction of four or more branches. 1 Terminations and branch points are helpful to determine the topology and faithfulness of the reconstruction results [8]. While cross-over points help to identify the potential wrong connections in the automated reconstruction results, which is useful to refine the automated neuron reconstruction results or avoid wrong connections during the reconstruction procedure. ...
... Although there are existing methods for termination detection, junction detection and dendritic spines detection of treelike structures [22]- [24], a method that can simultaneously detect all types of critical points has barely been explored for neuron reconstruction. Neuron Pinpointer [8] was proposed to detect 2D neuron terminations and junctions in fluorescence microscopy images. This algorithm is based on feature extraction and analysis of an angular profile, in combination with a two-stage fuzzy-logic system. ...
... The neuron critical points are helpful to determine the topology and faithfulness of the neuron structure reconstructions [8]. Many reconstruction algorithms and tools can be used to reconstruct the neuron morphology based on the critical points, e.g., the Probabilistic Neuron Reconstructor (PNR) [16]. ...
Digital reconstruction of neuronal structures is very important to neuroscience research. Many existing reconstruction algorithms require a set of good seed points. 3D neuron critical points, including terminations, branch points and cross-over points, are good candidates for such seed points. However, a method that can simultaneously detect all types of critical points has barely been explored. In this work, we present a method to simultaneously detect all 3 types of 3D critical points in neuron microscopy images, based on a spherical-patches extraction (SPE) method and a 2D multi-stream convolutional neural network (CNN). SPE uses a set of concentric spherical surfaces centered at a given critical point candidate to extract intensity distribution features around the point. Then, a group of 2D spherical patches is generated by projecting the surfaces into 2D rectangular image patches according to the orders of the azimuth and the polar angles. Finally, a 2D multi-stream CNN, in which each stream receives one spherical patch as input, is designed to learn the intensity distribution features from those spherical patches and classify the given critical point candidate into one of four classes: termination, branch point, cross-over point or non-critical point. Experimental results confirm that the proposed method outperforms other state-of-the-art critical points detection methods. The critical points based neuron reconstruction results demonstrate the potential of the detected neuron critical points to be good seed points for neuron reconstruction. Additionally, we have established a public dataset dedicated for neuron critical points detection, which has been released along with this paper.
... Junction points can be obtained either by manual annotation, for example using Vaa3D-Neuron [11], or by automatic detection using image filtering and pattern recognition methods [24], [27], [37]- [40]. However, manual annotation is very timeconsuming given the typical size and complexity of the data sets, and thus automation is the only viable option. ...
... The COSFIRE method uses a set of trainable keypoint detectors that we call Combination Of Shifted Filter Responses (COSFIRE filters) to detect vascular bifurcations in segmented retinal images [27]. Since each type of junction points may vary considerably in terms of geometry and image intensity, designing a different number of filters (the parameters for each filter need to be adjusted) for each possible case is time consuming [24]. ...
... We further evaluated the proposed method on 2D neuron images, by comparing its performance with the COSFIRE method and Neuron Pinpointer (NP) [24], a state-of-the-art neuron junction detection method. For the COSFIRE method, we trained a set of COSFIRE filters for two neuron datasets separately as in [27]. ...
... For neuronal shape reconstruction, the tracing methods usually extract the skeletons of neurites, which form the neuronal skeleton, and then reconstruct the neurite shape based on its extracted skeleton. Most of the tracing methods focus on how to extract neurite skeleton and are constructed from diverse mathematical models such as the graph-based model (Peng et al. 2011;Turetken et al. 2011;Basu et al. 2013;De et al. 2016), open snake model (Xu and Prince 1998;Wang et al. 2011;Luo et al. 2015), principle curve model (Bas and Erdogmus 2011;Quan et al. 2016;Li et al. 2017a, b), iterative back tracking (Liu et al. 2016, minimal path (Yang et al. 2018), probabilistic model (Radojevic and Meijering 2017;Skibbe et al. 2018), fuzzy logic model (Radojević et al. 2016), machine learning model (Chen et al. 2015;Megjhani et al. 2015;Gu et al. 2017) and others. Series of advances in this field have provided accurate identification of a single neuron from a local neuronal population De et al. 2016;Quan et al. 2016), single neuron reconstruction in ultra-volume dataset (Peng et al. 2017) and even population reconstruction at brain-wide scale (Zhou et al. 2018a, b;Winnubst et al. 2019). ...
Neuronal shape reconstruction is a helpful technique for establishing neuron identity, inferring neuronal connections, mapping neuronal circuits, and so on. Advances in optical imaging techniques have enabled data collection that includes the shape of a neuron across the whole brain, considerably extending the scope of neuronal anatomy. However, such datasets often include many fuzzy neurites and many crossover regions that neurites are closely attached, which make neuronal shape reconstruction more challenging. In this study, we proposed a convex image segmentation model for neuronal shape reconstruction that segments a neurite into cross sections along its traced skeleton. Both the sparse nature of gradient images and the rule that fuzzy neurites usually have a small radius are utilized to improve neuronal shape reconstruction in regions with fuzzy neurites. Because the model is closely related to the traced skeleton point, we can use this relationship for identifying neurite with crossover regions. We demonstrated the performance of our model on various datasets, including those with fuzzy neurites and neurites with crossover regions, and we verified that our model could robustly reconstruct the neuron shape on a brain-wide scale.
... Depending on the subproblems being solved, modules can operate independently, or work together for example to combine local and global processing, possibly requiring multiple iterations. Several subproblems that can be identified in the literature include image prefiltering and segmentation (Zhou et al. 2015;Türetken et al. 2011;Sironi et al. 2016;Mukherjee and Acton 2013), soma (cell body) detection and segmentation (Quan et al. 2013), landmark points extraction (Al-Kofahi et al. 2008;Wang et al. 2011;Choromanska et al. 2012;Baboiu and Hamarneh 2012;Su et al. 2012;Radojević et al. 2016), neuron arbor tracing (Zhao et al. 2011;Liu et al. 2016;Leandro et al. 2009;Radojević and Meijering 2017a;Xiao and Peng 2013), and assembling the final tree-like graph structure Türetken et al. 2011;Yuan et al. 2009). In the remainder of this section we briefly review techniques for solving each of these subproblems. ...
... An alternative is to use shape fitting approaches (Quan et al. 2013). Next, to initialize and/or guide the segmentation of the arbor, landmark points are often extracted using image filters that specifically enhance tubular structures Türetken et al. 2011;Choromanska et al. 2012;Su et al. 2012;Radojević et al. 2016), a popular one being the so-called "vesselness filter" (Frangi et al. 1998). In our proposed method we have adopted classical approaches for soma and seed point detection as detailed in the next section. ...
... To this end, the radius r s of the structuring element needs to be larger than the largest expected branch radius in a given data set, and smaller than the expected soma radius. The resulting image is then smoothed using a Gaussian filter with standard deviation equal to r s and segmented using max-entropy thresholding (Radojević et al. 2016) to obtain a blob corresponding to the a b c d e f soma. For computational efficiency both the erosion and the Gaussian smoothing operation are carried out by separable filtering. ...
Microscopic images of neuronal cells provide essential structural information about the key constituents of the brain and form the basis of many neuroscientific studies. Computational analyses of the morphological properties of the captured neurons require first converting the structural information into digital tree-like reconstructions. Many dedicated computational methods and corresponding software tools have been and are continuously being developed with the aim to automate this step while achieving human-comparable reconstruction accuracy. This pursuit is hampered by the immense diversity and intricacy of neuronal morphologies as well as the often low quality and ambiguity of the images. Here we present a novel method we developed in an effort to improve the robustness of digital reconstruction against these complicating factors. The method is based on probabilistic filtering by sequential Monte Carlo estimation and uses prediction and update models designed specifically for tracing neuronal branches in microscopic image stacks. Moreover, it uses multiple probabilistic traces to arrive at a more robust, ensemble reconstruction. The proposed method was evaluated on fluorescence microscopy image stacks of single neurons and dense neuronal networks with expert manual annotations serving as the gold standard, as well as on synthetic images with known ground truth. The results indicate that our method performs well under varying experimental conditions and compares favorably to state-of-the-art alternative methods.
