Figure - available from: Frontiers in Neuroscience
This content is subject to copyright.
Mouse cortex neural tissue acquired by ATUM-SEM. (A) An example of an aligned image stack that covers approximately 20 × 20 × 10 μm through the ATUM-SEM method. (B,C) Examples of mitochondria and other ultrastructures. The green arrows indicate mitochondria; the red arrows indicate vesicles; the yellow arrow indicates the Golgi body, and the purple arrow indicates the endoplasmic reticulum. (D–F) Examples of mitochondria segmentation in our training dataset.
Source publication
Together, mitochondria and the endoplasmic reticulum (ER) occupy more than 20% of a cell's volume, and morphological abnormality may lead to cellular function disorders. With the rapid development of large-scale electron microscopy (EM), manual contouring and three-dimensional (3D) reconstruction of these organelles has previously been accomplished...
Citations
... For large-scale analysis of mitochondria and cristae in 3D to be realistic, an automated approach is highly desirable. Recent advances in machine learning have made large-scale automatic segmentation of mitochondria feasible 22,23 but until now, such automated tools have not allowed segmentation and analysis of the structures most intimately connected to mitochondrial function: the cristae. There may be several reasons for this. ...
... In our study, 9% of the evaluated mitochondria were of this type. An additional challenge is that mitochondria gross shape parameters are affected by cellular location 13,23,43 . Separating mitochondria into subpopulations depending on location for example in the cell soma or in processes requires larger 3D volumes with concomitant increased imaging times and data amounts. ...
Mitochondria are the main suppliers of energy for cells and their bioenergetic function is regulated by mitochondrial dynamics: the constant changes in mitochondria size, shape, and cristae morphology to secure cell homeostasis. Although changes in mitochondrial function are implicated in a wide range of diseases, our understanding is challenged by a lack of reliable ways to extract spatial features from the cristae, the detailed visualization of which requires electron microscopy (EM). Here, we present a semi-automatic method for the segmentation, 3D reconstruction, and shape analysis of mitochondria, cristae, and intracristal spaces based on 2D EM images of the murine hippocampus. We show that our method provides a more accurate characterization of mitochondrial ultrastructure in 3D than common 2D approaches and propose an operational index of mitochondria’s internal organization. With an improved consistency of 3D shape analysis and a decrease in the workload needed for large-scale analysis, we speculate that this tool will help increase our understanding of mitochondrial dynamics in health and disease.
... The only top-down approach from the reviewed works in this paper is the one proposed by Liu et al. (2020a). They introduced a pipeline that complements Mask-RCNN. ...
Electron microscopy (EM) enables high-resolution imaging of tissues and cells based on 2D and 3D imaging techniques. Due to the laborious and time-consuming nature of manual segmentation of large-scale EM datasets, automated segmentation approaches are crucial. This review focuses on the progress of deep learning-based segmentation techniques in large-scale cellular EM throughout the last six years, during which significant progress has been made in both semantic and instance segmentation. A detailed account is given for the key datasets that contributed to the proliferation of deep learning in 2D and 3D EM segmentation. The review covers supervised, unsupervised, and self-supervised learning methods and examines how these algorithms were adapted to the task of segmenting cellular and sub-cellular structures in EM images. The special challenges posed by such images, like heterogeneity and spatial complexity, and the network architectures that overcame some of them are described. Moreover, an overview of the evaluation measures used to benchmark EM datasets in various segmentation tasks is provided. Finally, an outlook of current trends and future prospects of EM segmentation is given, especially with large-scale models and unlabeled images to learn generic features across EM datasets.
... These mechanisms greatly enhance the learning ability of ERnet in classifying ER structures in the spatio-temporal domain. While machine learning methods have previously been implemented for denoising images of ER structures 35 , reconstructing ER structures based on electron microscopy images 36 and identification ER stress marker whorls 37 , ERnet is capable of video-rate image segmentation and analysis of live cells, further extending the deep learning toolbox for biomedical research. ...
The ability to quantify structural changes of the endoplasmic reticulum (ER) is crucial for understanding the structure and function of this organelle. However, the rapid movement and complex topology of ER networks make this challenging. Here, we construct a state-of-the-art semantic segmentation method that we call ERnet for the automatic classification of sheet and tubular ER domains inside individual cells. Data are skeletonized and represented by connectivity graphs, enabling precise and efficient quantification of network connectivity. ERnet generates metrics on topology and integrity of ER structures and quantifies structural change in response to genetic or metabolic manipulation. We validate ERnet using data obtained by various ER-imaging methods from different cell types as well as ground truth images of synthetic ER structures. ERnet can be deployed in an automatic high-throughput and unbiased fashion and identifies subtle changes in ER phenotypes that may inform on disease progression and response to therapy.
... Another top-down method [22] aims to increase the amplification direction of its visibility so that the mitochondrial contour can be detected with high accuracy. In addition to the R-CNN architecture, a recursive cutting subnet is also used. ...
Mitochondria are the organelles that generate energy for the cells. Many studies have suggested that mitochondrial dysfunction or impairment may be related to cancer and other neurodegenerative disorders such as Alzheimer’s and Parkinson’s diseases. Therefore, morphologically detailed alterations in mitochondria and 3D reconstruction of mitochondria are highly demanded research problems in the performance of clinical diagnosis. Nevertheless, manual mitochondria segmentation over 3D electron microscopy volumes is not a trivial task. This study proposes a two-stage cascaded CNN architecture to achieve automated 3D mitochondria segmentation, combining the merits of top-down and bottom-up approaches. For top-down approaches, the segmentation is conducted on objects’ localization so that the delineations of objects’ contours can be more precise. However, the combinations of 2D segmentation from the top-down approaches are inadequate to perform proper 3D segmentation without the information on connectivity among frames. On the other hand, the bottom-up approach finds coherent groups of pixels and takes the information of 3D connectivity into account in segmentation to avoid the drawbacks of the 2D top-down approach. However, many small areas that share similar pixel properties with mitochondria become false positives due to insufficient information on objects’ localization. In the proposed method, the detection of mitochondria is carried out with multi-slice fusion in the first stage, forming the segmentation cues. Subsequently, the second stage is to perform 3D CNN segmentation that learns the pixel properties and the information of 3D connectivity under the supervision of cues from the detection stage. Experimental results show that the proposed structure alleviates the problems in both the top-down and bottom-up approaches, which significantly accomplishes better performance in segmentation and expedites clinical analysis.
... Recent volume EM advances have enabled 3D imaging 1,2 of increasingly large specimens, notably in connectomics [3][4][5] where deep learning (DL) algorithms are actively employed to generate wiring diagrams and enable quantitative analyses. [6][7][8][9] Similar methods have also been used to investigate organellar structures in other systems, [10][11][12][13][14] generating biological insights at unprecedented scales. The typical DL workflow for these applications is to densely annotate features in a 3D region of interest (ROI), train a model on these annotations, run inference on the full dataset, and then proofread the model output. ...
Mitochondria are extremely pleomorphic organelles. Automatically annotating each one accurately and precisely in any 2D or volume electron microscopy (EM) image is an unsolved computational challenge. Current deep learning-based approaches train models on images that provide limited cellular contexts, precluding generality. To address this, we amassed a highly heterogeneous ∼1.5 × 106 image 2D unlabeled cellular EM dataset and segmented ∼135,000 mitochondrial instances therein. MitoNet, a model trained on these resources, performs well on challenging benchmarks and on previously unseen volume EM datasets containing tens of thousands of mitochondria. We release a Python package and napari plugin, empanada, to rapidly run inference, visualize, and proofread instance segmentations. A record of this paper's transparent peer review process is included in the supplemental information.
... We applied ASEM to three-dimensional FIB-SEM images of cells prepared by either CF or HPFS. We validated our approach by segmenting mitochondria, ER, and Golgi apparatus, as these organelles had been studied previously in similar efforts (Žerovnik Mekuč et al., 2020;Žerovnik Mekuč et al., 2022;Heinrich et al., 2021;Liu et al., 2020), and then used ASEM to recognize much smaller structures, nuclear pores, and clathrin-coated pits and vesicles. For nuclear pores in interphase, we can segment nearly all the pores in the nuclear membrane. ...
Volume electron microscopy is an important imaging modality in contemporary cell biology. Identification of intracellular structures is a laborious process limiting the effective use of this potentially powerful tool. We resolved this bottleneck with automated segmentation of intracellular substructures in electron microscopy (ASEM), a new pipeline to train a convolutional neural network to detect structures of a wide range in size and complexity. We obtained dedicated models for each structure based on a small number of sparsely annotated ground truth images from only one or two cells. Model generalization was improved with a rapid, computationally effective strategy to refine a trained model by including a few additional annotations. We identified mitochondria, Golgi apparatus, endoplasmic reticulum, nuclear pore complexes, caveolae, clathrin-coated pits, and vesicles imaged by focused ion beam scanning electron microscopy. We uncovered a wide range of membrane-nuclear pore diameters within a single cell and derived morphological metrics from clathrin-coated pits and vesicles, consistent with the classical constant-growth assembly model.
... For large scale analysis of mitochondria and cristae in 3D to be realistic, an automated approach is highly desirable. Recent advances in machine learning have made large-scale automatic segmentation of mitochondria feasible [23,24] but until now, such automated tools have not allowed segmentation and analysis of the structures most intimately connected to mitochondrial function: the cristae. There may be several reasons for this. ...
... In our study 9% of the evaluated mitochodnria were of this type. An additional challenge is that mitochondria gross shape parameters are affected by cellular location [24,14,45]. Separating mitochondria into subpopulations depending on location for example in the cell soma or in processes require larger 3D volumes with concomitant increased imaging times and data amounts. ...
Mitochondria are the main suppliers of energy for cells and their bioenergetic function is regulated by mitochondrial dynamics: the constant changes in mitochondria size, shape, and cristae morphology to secure cell homeostasis. Although mitochondrial dysfunction is implicated in a wide range of diseases, our understanding of mitochondrial function remains limited by the complexity of inferring these spatial features from 2D electron microscopical (EM) images of intact tissue. Here, we present a semi-automatic method for segmentation and 3D reconstruction of mitochondria, cristae, and intracristal spaces based on 2D EM images of the murine hippocampus. We show that our method provides a more accurate characterization of mitochondrial ultrastructure in 3D than common 2D approaches and propose an operational index of mitochondria's internal organization. We speculate that this tool may help increase our understanding of mitochondrial dynamics in health and disease.
... In mammalian cells, it was reported that 3D reconstructed individual mitochondria communicated with each other by nanotunnels allowing the formation of dynamic mitochondrial networks 41 . Moreover, the 3D organisation of mitochondria in neuronal cells showed the predominance of tubular structures in axons and dendrites suggesting their crucial role in the maintenance of cellular energy homeostasis 42 . ...
In the ovarian follicle, a bilateral cell-to-cell communication exists between the female germ cell and the cumulus cells which surround the oocyte. This communication allows the transit of small size molecules known to impact oocyte developmental competence. Pyruvate derivatives produced by mitochondria, are one of these transferred molecules. Interestingly, mitochondria may adopt a variety of morphologies to regulate their functions. In this study, we described mitochondrial morphologies in porcine cumulus cells. Active mitochondria were stained with TMRM (Tetramethylrhodamine, Methyl Ester, Perchlorate) and observed with 2D confocal microscopy showing mitochondria of different morphologies such as short, intermediate, long, and very long. The number of mitochondria of each phenotype was quantified in cells and the results showed that most cells contained elongated mitochondria. Scanning electron microscopy (SEM) analysis confirmed at nanoscale resolution the different mitochondrial morphologies including round, short, intermediate, and long. Interestingly, 3D visualisation by focused ion-beam scanning electron microscopy (FIB-SEM) revealed different complex mitochondrial morphologies including connected clusters of different sizes, branched mitochondria, as well as individual mitochondria. Since mitochondrial dynamics is a key regulator of function, the description of the mitochondrial network organisation will allow to further study mitochondrial dynamics in cumulus cells in response to various conditions such as in vitro maturation.
... Deep learning methods have been developed to recognize organelles in EM images, mostly one at a time (Haberl et al., 2018;Xiao et al., 2018;Zhang et al., 2019). Advancement of EM imaging techniques and computing resources allow deep learning-based 3D analysis of MCSs based on organelle segmentation in high-resolution volumetric data, which demands days of machine time for EM imaging and high-performance computing resources for the analysis of a single cell (Heinrich et al., 2021;Liu et al., 2020). However, cross-sample comparisons of biological samples can easily be complicated by the heterogeneity of the cells (Yang et al., 2018), requiring a large sample size for deep learning-based statistical analysis, which hardly adapts to the time and resource demands of 3D analysis at the current stage. ...
Membrane contact site (MCS)-mediated organelle interactions play essential roles in the cell. Quantitative analysis of MCSs reveals vital clues for cellular responses under various physiological and pathological conditions. However, an efficient tool is lacking. Here, we developed DeepContact, a deep-learning protocol for optimizing organelle segmentation and contact analysis based on label-free EM. DeepContact presents high efficiency and flexibility in interactive visualizations, accommodating new morphologies of organelles and recognizing contacts in versatile width ranges, which enables statistical analysis of various types of MCSs in multiple systems. DeepContact profiled previously unidentified coordinative rearrangements of MCS types in cultured cells with combined nutritional conditions. DeepContact also unveiled a subtle wave of ER–mitochondrial entanglement in Sertoli cells during the seminiferous epithelial cycle, indicating its potential in bridging MCS dynamics to physiological and pathological processes.
... A clear motive for this work is evidence that links alterations of organelle structure to neurodegenerative diseases and cancer. 4,5,6,133 The high axial resolution of SBF-SEM and especially FIB-SEM data allows for accurate segmentation of cell organelles. Due to the diversity of organelles and cell types as well as a lack of publicly available training data, automated organelle segmentation has not experienced the same surge as in connectomics, which has benefited from years of substantial manual annotation effort. ...
Detailed knowledge of biological structure has been key in understanding biology at several levels of organization, from organs to cells and proteins. Volume electron microscopy (volume EM) provides high resolution 3D structural information about tissues on the nanometer scale. However, the throughput rate of conventional electron microscopes has limited the volume size and number of samples that can be imaged. Recent improvements in methodology are currently driving a revolution in volume EM, making possible the structural imaging of whole organs and small organisms. In turn, these recent developments in image acquisition have created or stressed bottlenecks in other parts of the pipeline, like sample preparation, image analysis and data management. While the progress in image analysis is stunning due to the advent of automatic segmentation and server-based annotation tools, several challenges remain. Here we discuss recent trends in volume EM, emerging methods for increasing throughput and implications for sample preparation, image analysis and data management. This article is protected by copyright. All rights reserved.
