FIGURE 8 - uploaded by Xiaokang Liu
Content may be subject to copyright.
Visualization of the feature maps of different layers from Inception-ResNet-v2. From convd2_3 to mixed _7b (Supplementary Fig. S1), layers become deeper. First column of each layer is the averaged feature map (A), and the remaining column feature maps are nine examples of this layer (B-J). The dimensions of convd2_3, convd2_5, mixed_5b, mixed_6a, mixed_7a, and conv_7b are 147 × 147 × 64, 71 × 71 × 192, 35 × 35 × 320, 17 × 17 × 1088, 8 × 8 × 2080, and 8 × 8 × 1536, respectively. A schematic of the Inception-ResNet-v2 architecture is provided in Supplementary Fig. S1. Yellow (blue) pixels correspond to higher (lower) activations.
Source publication
The rapid and accurate taxonomic identification of fossils is of great significance in paleontology, biostratigraphy, and other fields. However, taxonomic identification is often labor-intensive and tedious, and the requisition of extensive prior knowledge about a taxonomic group also requires long-term training. Moreover, identification results ar...
Contexts in source publication
Context 1
... regions correspond to a high (low) score for predicting the label in Grad-CAM considering the average activation of the 1536 feature maps from Inception-ResNetv2. Normally, the attention area of each feature map can be focused on the limited or unique characteristics ( Selvaraju et al. 2016). A similar pattern is observed in the feature maps (Fig. 8), and some of the feature maps highlight the umbilicus, ribs, and inner whorl of the ammonoid. In particular, Inception-ResNet-v2 identifies different structures for each feature map in deep layers. (e.g., layers of mixed_7a and mixed _7b). Some feature maps are highly activated in a region limited to several pixels, indicating feature ...
Context 2
... DCNNs, which compress the size (length and width) and increase the number (channels) of feature images through repeated convolutions. The feature maps from different convolutional layers demonstrate that shallow layers are sensitive to low-level features, such as brightness, edges, curves, and other conjunctions (layers conv2d_3 and conv2d_5 in Fig. 8) ( Zeiler et al. 2011;Zeiler and Fergus 2014). Conversely, deeper layers The activation patterns and feature maps show that the DCNN architecture can capture the fine-grained details of different fossils. For instance, the discriminative features extracted from ammonoids are mainly concentrated in the circular features merging from the ...
Citations
... The results of this process can vary greatly depending on the quality of the data and the number of datasets used to train the model and the parameters chosen for optimization [41], [42]. However, with a large enough dataset and proper parameter adjustments, it is possible to achieve a high level of accuracy [43], [44], [45] in the classification of herbal plants. ...
Herbal plants are a source of natural materials used in alternative medicine and traditional therapies to maintain health. The purpose of this research is to develop an intelligent system application that is able to assist people in independently detecting herbal plants around them, provide education, and most importantly, find the optimal value based on certain parameters. This research uses several values for the parameters studied, namely the epoch value which varies between 10, 50, 100, 250, 750, and 1000; the batch size value which varies between 16, 32, 64, 128, 256, and 512; and the learning rate value which varies between 0.00001, 0.0001, 0.001, 0.01, 0.1, and 1. A total of 10,000 training data samples (1,000 samples in 10 classes) were used in Teachable Machine. The method used is to utilize the TensorFlow framework in the Teachable Machine service to train image data. This framework provides Convolutional Neural Networks (CNN) algorithms that can perform image classification with a high degree of accuracy. The test results for more than three months showed that the highest optimal value was achieved at the 50th epoch value, with a learning rate of 0.00001, and a batch size of 32, which resulted in an accuracy rate between 98% and 100%. Based on these results, a mobile web-based intelligent system application service was developed using the TensorFlow framework in Teachable Machine. This application is expected to be widely implemented for the benefit of the community. However, the challenges and limitations in training this test data are the large number of data classes that will be very good so that machine learning can learn to recognize objects but will take hours to train, then the training image object data has a clean background from other objects so that when tested it is not detected and influenced as another object or can result in a decrease in the percentage value.
... With pre-trained models on ImageNet, their classification results showed high accuracies of >90%, and the more balanced dataset prevented overfitting in biotic structures. Following this, Liu et al. (2023) presented the Fossil Image Dataset (FID) with 415,339 images from 50 fossil clades (including various invertebrates, vertebrates, plants, microfossils, and trace fossils), and an online model (www.ai-fossil.com) is also available for fossil image identification. They showed that certain clades were more difficult to identify than others and proportions of complete to fragmentary fossil hugely influence the rate of correct identification. ...
... Most datasets surveyed here were manually collected and annotated, but there are also exceptions that use web crawling and published literature (Liu et al., 2023) for collection and crowdsourcing for annotation (Wong, 2011), which are both common practice in preparing large-scale AI training datasets. However, paleontology is a specialist field that needs expert knowledge and long-term practice in order to generate training sets, making automated and crowdsourced dataset creation a more difficult prospect. ...
... Automated species classification has always been a focus in paleontological AI studies, and such an idea was repeatedly proposed by Gaston andO'Neill (2004), MacLeod et al. (2010) and many others. Now there is prototype system developed based on large datasets and deep learning have been developed techniques (Liu et al., 2023). From KBS to CNN, most (if not all) paleontological AI studies surveyed here are within the range of supervised learning, meaning that there needs to be a solid labeled dataset for model training, with clear taxonomy for fossil classification. ...
The accumulation of large datasets and increasing data availability have led to the emergence of data-driven paleontological studies, which reveal an unprecedented picture of evolutionary history. However, the fast-growing quantity and complication of data modalities make data processing laborious and inconsistent, while also lacking clear benchmarks to evaluate data collection and generation, and the performances of different methods on similar tasks. Recently, artificial intelligence (AI) has become widely practiced across scientific disciplines, but not so much to date in paleontology where traditionally manual workflows have been more usual. In this study, we review >70 paleontological AI studies since the 1980s, covering major tasks including micro-and macrofossil classification, image segmentation, and prediction. These studies feature a wide range of techniques such as Knowledge-Based Systems (KBS), neural networks, transfer learning, and many other machine learning methods to automate a variety of paleontological research workflows. Here, we discuss their methods, datasets, and performance and compare them with more conventional AI studies. We attribute the recent increase in paleontological AI studies most to the lowering of the entry bar in training and deployment of AI models rather than innovations in fossil data compilation and methods. We also present recently developed AI implementations such as diffusion model content generation and Large Language Models (LLMs) that may interface with paleontological research in the future. Even though AI has not yet been a significant part of the paleontologist's toolkit, successful implementation of AI is growing and shows promise for paradigm-transformative effects on paleontological research in the years to come.
... It uses statistical models implementing original and fixed data to make predictions, as opposed to classic statistical models, that undertakes analyses to draw conclusions and identify associations between variables (Mohri et al., 2018). Several ML algorithms have been applied to different fossil images, SEM, and CT scans of different fossils (De Baets, 2021;Ferreira-Chacua and Koeshidayatullah, 2023;Liu et al., 2023) to automatically recognize paleontological images; these results yielded an overall accuracy of 85% (percentage of properly identified taxa). In most cases, the results were comparable to those of human experts (Reid et al., 2019). ...
In the last 5 decades, paleontological research has exploded where fossils have enabled robust dating of rocks, improved understanding of origination/extinction rates or mass extinction events, biogeography, adaptive strategies, and many more. New molecular technologies have enabled intensive analyses of vertebrates and invertebrates, plant fossils, fossilized microbes, trace fossils, and fossil molecules, alike. Paleontological research has become interdisciplinary with inputs from geology, chemistry, biology, astronomy, and archaeology. Herein, we review the principles of promising molecular technologies and explore their applications and limitations vis-à-vis paleontological research. This review will attempt to provide a roadmap that can be used for future research directions. Advanced chemical imaging provides the ability to identify and quantify chemical characteristics to evaluate taphonomic damage, original biological structures, or fossils microbes. Molecular methods (e.g., molecular clock, DNA barcode, racemization dating, and biomarkers) offer a unique source of information and provide robust clues into the co-evolution of life in modern and past environments. Two main limitations are noted and include an exceptional preservation of the organic material, which is not always the case, and the complexity and cost of the instruments involved in the analyses. These difficulties are limiting the factual applications in paleontological analysis. Although very little research has been carried out on the aforementioned methods, they however, provide improved answers to highly debated and unsolved biological and climatic issues and a window to better understanding the origin of life. Biomarker proxies will be further developed and refined to answer emerging questions in the Quaternary Period.
... It uses statistical models implementing original and fixed data to make predictions, as opposed to classic statistical models, that undertakes analyses to draw conclusions and identify associations between variables (Mohri et al., 2018). Several ML algorithms have been applied to different fossil images, SEM, and CT scans of different fossils (De Baets, 2021;Ferreira-Chacua and Koeshidayatullah, 2023;Liu et al., 2023) to automatically recognize paleontological images; these results yielded an overall accuracy of 85% (percentage of properly identified taxa). In most cases, the results were comparable to those of human experts (Reid et al., 2019). ...
A systematic horizon scanning was undertaken to identify the up-to-date perspectives on paleontological research. A summarized evaluation (applicability and acceptability) was also provided to identify the challenges and opportunities of paleontological techniques. Present-day advances in molecular analyses and scanning techniques generate valuable new data to test old and recent systematic problems and provide a revolution in systematic paleontology. Integrating non-destructive high-resolution virtual solutions such as X-ray computed tomography and 3D-laser scanning with machine learning can be widely used for the analysis of internal features of fossils and more efficiently for automated taxonomy. The slow pace of the revolution in paleontological techniques can be attributed to the limited advanced statistical training and the cost of the instruments, software, and hardware needed for digitization and imaging. In addition, molecular techniques offer a unique source of information (e.g., biomarkers), however, costs and difficulties are limiting their applications. Sclerochronology using carbonate shells of well-preserved fossils (e.g., mollusks, corals, and fish) has the potential to reconstruct the paleoclimate at very high resolution (daily, seasonal, and annual). These approaches are revolutionary and will grow continuously to substitute traditional methods and will reduce time and human efforts.
... Accurate genus and species-level identification are reported as a hard task, and taxonomic agreement is achieved around 70 %, even if for the experienced domain experts [34]. Planktonic foraminifera has highly complex and variable morphologies for each genus and species to define microfossil specimens via one-by-one examination under the microscope [22,30,34,65]. Furthermore, in some cases, genetic analysis is required to provide genus or species distinction for the existence of morphological end-members. ...
... However, this technique has only been applied in a limited way. In other cases, morphological variation is noted as an important verification as well as the genetic differentiation for each microfossil genus and species [9,10,21,[65][66][67]. ...
... On this point, computer vision and machine learning models can provide an effective quick way to not only automate a task that relatively few trained paleontologists are able to do (i.e., identification of all genera and species in terms of accurate decision) but also to undertake a level of consistency due to subjectivity and/or bias [9,10,21,[65][66][67]. ...
The applicability of digital imaging techniques and machine learning models to paleontological datasets is
exploring the possibility of predicting microfossils extracted from the rock samples instead of the traditional
identifying methodologies under the microscope in a one-by-one way via a domain expert. However, these
processes, including labeling, are carried out manually and take a high time-consuming, especially for many
quantities and diversity of complex morphological microfossil specimens. In this work, we propose a transfer
learning framework based on a custom model CNN (Convolutional Neural Network) and diverse pre-trained deep
models (ResNet50, Xception, InceptionV3, VGG6, MobileNet) trained with the millions of images for Globotruncanita genus and Globotruncana genus in genus-level and species-level prediction. The second primary
advantage of our framework is able to provide better and more robust decisions for a limited number of
microfossil images captured by the low-cost light microscope imaging technology. The comparison of the diverse
methods was evaluated with different performance metrics, and the observation of the framework was made to
perform high prediction scores reaching up to the outcomes (>99 % accuracy and > 0.99 AUC score for genuslevel/>81 % accuracy and > 0.89 AUC score for species-level). As far as we know, this research study is the first
attempt to investigate a transfer learning framework to predict the Globotruncanita genus and Globotruncana
genus families at the genus-level and species-level microfossils. Overall, it may extend the existing literature on
paleontological science and automated/quick classification manner.
... Pires et al. 29 introduced a transfer learning technique for the classification of a dataset comprising eight distinct fusulinid genera. Liu et al. 30 evaluated the results of three typical deep CNN (convolutional neural network) architectures on a large-scale fossil dataset of more than 50 clades including invertebrates, vertebrates, plants, microfossils, and fossil traces from five hyperclades. Hou et al. 31 proposed a multi-perspective framework that uses original, gray, and skeleton images of each fossil for training. ...
Graptolites, fossils significant for evolutionary studies and shale gas exploration, are traditionally identified visually by taxonomists due to their intricate morphologies and preservation challenges. Artificial intelligence (AI) holds great promise for transforming such meticulous tasks. In this paper, we demonstrate that graptolites can be identified with taxonomist accuracy using a deep learning model. We construct the most sophisticated and largest professional single organisms image dataset to date, which is composed of >34,000 images of 113 graptolite species annotated at pixel-level resolution to train the model, develop, and evaluate deep learning networks to classify graptolites. The model’s performance surpassed taxonomists in accuracy, time, and generalization, achieving 86% and 81% accuracy in identifying graptolite genus and species, respectively. This AI-based method, capable of recognizing minute morphological details better than taxonomists, can be integrated into web and mobile apps, extending graptolite identification beyond research institutes and enhancing shale gas exploration efficiency.
... Nonetheless, recent advances in the use of deep learning models for taxonomic identification have shown promising prospects for the application on fossil taxa, including foraminifera (Hsiang et al., 2019;Marchant et al., 2020;Mitra et al., 2019;Pires de Lima et al., 2020), graptolites (Niu & Xu, 2022), fossil leaves (Wilf et al., 2021), pollen (Punyasena et al., 2022) and multiple-body-fossil mixture (Liu et al., 2022). The identification of modern foraminifera could be well compared to that of fossil foraminifera due to their close morphology and modes of preservation, and they are also among the first to be tested for species identification using deep learning. ...
Identification of fossil species is crucial to evolutionary studies. Recent advances from deep learning have shown promising prospects in fossil image identification. However, the quantity and quality of labelled fossil images are often limited due to fossil preservation, conditioned sampling and expensive and inconsistent label annotation by domain experts, which pose great challenges to training deep learning‐based image classification models.
To address these challenges, we follow the idea of the wisdom of crowds and propose a multiview ensemble framework, which collects Original (O), Grey (G) and Skeleton (S) views of each fossil image reflecting its different characteristics to train multiple base models, and then makes the final decision via soft voting.
Experiments on the largest fusulinid dataset with 2400 images show that the proposed OGS consistently outperforms baselines (using a single model for each view), and obtains superior or comparable performance compared to OOO (using three base models for three the same Original views). Besides, as the training data decreases, the proposed framework achieves more gains. While considering the identification consistency estimation with respect to human experts, OGS receives the highest agreement with the original labels of dataset and with the re‐identifications of two human experts. The validation performance provides a quantitative estimation of consistency across different experts and genera.
We conclude that the proposed framework can present state‐of‐the‐art performance in the fusulinid fossil identification case study. This framework is designed for general fossil identification and it is expected to see applications to other fossil datasets in future work. Notably, the result, which shows more performance gains as train set size decreases or over a smaller imbalance fossil dataset, suggests the potential application to identify rare fossil images. The proposed framework also demonstrates its potential for assessing and resolving inconsistencies in fossil identification.
... Automated species-level identification methods (Hsiang et al., 2019;Karaderi et al., 2022;Marchant et al., 2020) were developed based on this dataset, and their performances are comparable to human experts. Several image datasets for fossil organisms have been made available, including foraminifera (Hsiang et al., 2019;Mitra et al., 2019), graptolites (Niu & Xu, 2022), fossil leaves (Wilf et al., 2021) and multiple-body-fossil mixture (Liu et al., 2022). However, only a few efforts have been seen in fusulinid studies, for example, Pires de Lima et al. (2020) utilized roughly 300 photos to automatically identify eight genera. ...
Fusulinid foraminifera are among the most common microfossils of the Late Palaeozoic and act as key fossils for stratigraphic correlation, paleogeographic and paleoenvironmental indication, and evolutionary studies of marine life. Accurate and efficient identification forms the basis of such research involving fusulinids but is limited by the lack of digitized image datasets. This article presents the first large image dataset of fusulinids containing 2,400 images of individual samples subjected to 16 genera of all six fusulinid families and labelled to species level. These images were collected from the literature and our unpublished samples through an automatic segmentation procedure implementing BlendMask, a deep learning model. The dataset shows promise for the efficient accumulation of fossil images through automated procedures and will facilitate taxonomists in future morphologic and systematic studies.
... A limitation with understanding species from fossils alone is that their identifications and classifications are uncertain. Recently, artificial intelligence (AI) has been used to classify fossil specimens and develop taxonomies 292,293 . Also, labeling anatomical data is time consuming and subject to observer error. ...
Fossil endocasts record features of brains from the past: size, shape, vasculature, and gyrification. These data, alongside experimental and comparative evidence, are needed to resolve questions about brain energetics, cognitive specializations, and developmental plasticity. Through the application of interdisciplinary techniques to the fossil record, paleoneurology has been leading major innovations. Neuroimaging is shedding light on fossil brain organization and behaviors. Inferences about the development and physiology of the brains of extinct species can be experimentally investigated through brain organoids and transgenic models based on ancient DNA. Phylogenetic comparative methods integrate data across species and associate genotypes to phenotypes, and brains to behaviors. Meanwhile, fossil and archeological discoveries continuously contribute new knowledge. Through cooperation, the scientific community can accelerate knowledge acquisition. Sharing digitized museum collections improves the availability of rare fossils and artifacts. Comparative neuroanatomical data are available through online databases, along with tools for their measurement and analysis. In the context of these advances, the paleoneurological record provides ample opportunity for future research. Biomedical and ecological sciences can benefit from paleoneurology’s approach to understanding the mind as well as its novel research pipelines that establish connections between neuroanatomy, genes and behavior.
... Nonetheless, recent advances in the use of deep learning models for taxonomic identification have shown promising prospects for the application on fossil taxa, including foraminifera (Hsiang et al., 2019;Marchant et al., 2020;Mitra et al., 2019;Pires de Lima et al., 2020), graptolites (Niu & Xu, 2022), fossil leaves (Wilf et al., 2021), pollen (Punyasena et al., 2022) and multiple-body-fossil mixture (Liu et al., 2022). The identification of modern foraminifera could be well compared to that of fossil foraminifera due to their close morphology and modes of preservation, and they are also among the first to be tested for species identification using deep learning. ...
Identification of fossil species is crucial to evolutionary studies. Recent advances from deep learning have shown promising prospects in fossil image identification. However, the quantity and quality of labeled fossil images are often limited due to fossil preservation, conditioned sampling, and expensive and inconsistent label annotation by domain experts, which pose great challenges to the training of deep learning based image classification models. To address these challenges, we follow the idea of the wisdom of crowds and propose a novel multiview ensemble framework, which collects multiple views of each fossil specimen image reflecting its different characteristics to train multiple base deep learning models and then makes final decisions via soft voting. We further develop OGS method that integrates original, gray, and skeleton views under this framework to demonstrate the effectiveness. Experimental results on the fusulinid fossil dataset over five deep learning based milestone models show that OGS using three base models consistently outperforms the baseline using a single base model, and the ablation study verifies the usefulness of each selected view. Besides, OGS obtains the superior or comparable performance compared to the method under well-known bagging framework. Moreover, as the available training data decreases, the proposed framework achieves more performance gains compared to the baseline. Furthermore, a consistency test with two human experts shows that OGS obtains the highest agreement with both the labels of dataset and the two experts. Notably, this methodology is designed for general fossil identification and it is expected to see applications on other fossil datasets. The results suggest the potential application when the quantity and quality of labeled data are particularly restricted, e.g., to identify rare fossil images.
