Fig 2 - uploaded by Xiangwen Wang
Content may be subject to copyright.
Three generations of artificial neural networks (ANNs). MLP, multilayer perceptron; MP, McCulloch-Pitts; SNNs, spiking neural networks.
Source publication
As a new brain-inspired computational model of the artificial neural network, a spiking neural network encodes and processes neural information through precisely timed spike trains. Spiking neural networks are composed of biologically plausible spiking neurons, which have become suitable tools for processing complex temporal or spatiotemporal infor...
Contexts in source publication
Context 1
... and play important roles in information processing and pattern recognition. An ANN is a computational model consisting of neurons as basic computational units. Information exchange between neurons is accomplished through synapses. According to their computational units, ANN models can be divided into three different generations [10], as shown in Fig. ...
Context 2
... for recurrent SNNs possess this property, such as the algorithm presented by Brea et al. [197] and FOLLOW [202]. However, there are still some supervised learning algorithms for recurrent SNNs that do not possess this property, such as SPTT [193]. A taxonomy for supervised learning algorithms from the locality property dimension is shown in Fig. 20. Meanwhile, some supervised learning algorithms have been proved to not possess stability of the optimal solution; for example, the algorithm presented by Legenstein et al. [79]. However, there are still many supervised learning algorithms for which it cannot be determined whether they possess stability of the optimal solution; for ...
Context 3
... Spiking neuron models. Fig. 22 shows a taxonomy for supervised learning algorithms from the spiking neuron model dimension. It can be seen that some supervised learning algorithms, especially some gradientdescent-based algorithms [152,154,133,130,193], can use only the neurons in which the internal state can be described by an analytic ex- pression, such as the SRM ...
Similar publications
Mixed-signal analog/digital circuits emulate spiking neurons and synapses with extremely high energy efficiency, an approach known as “neuromorphic engineering”. However, analog circuits are sensitive to process-induced variation among transistors in a chip (“device mismatch”). For neuromorphic implementation of Spiking Neural Networks (SNNs), mism...
Neural spikes can encode a rich set of information, ranging from the perceived intensity of light sources to the likelihood associated with decisions made in the cortex. Among these capabilities, previous studies demonstrated that spikes can also encode in their activity multiple frequencies at the same time, such as those generated by skin vibrati...
Over the past years Spiking Neural Networks (SNNs) models became attractive as a possible bridge to enable low-power event-driven neuromorphic hardware. SNNs have a high computational power due to the implicit employment of the biologically inspired input times. SNNs employ various parameters such as neuron threshold, synaptic delays, and weights i...
Memristors provide a tempting solution for weighted synapse connections in neuromorphic computing due to their size and non-volatile nature. However, memristors are unreliable in the commonly used voltage-pulse-based programming approaches and require precisely shaped pulses to avoid programming failure. In this paper, we demonstrate a current-limi...
Cyber-attacks on specialized industrial control systems are increasing in frequency and sophistication, which means stronger countermeasures need to be implemented, requiring the designers of the equipment in question to re-evaluate and redefine their methods for actively protecting against advanced mass cyber-attacks. The attacks in question have...
Citations
... The weights are adjusted to adapt to the underlying data patterns. The way weights are altered to achieve the desired outcome is known as the "learning algorithm", and it is a pivotal element influencing the neural network's performance [27]. The backpropagation error is a widely utilized learning algorithm in the realm of neural networks. ...
Lithium, a critical natural resource integral to modern technology, has influenced diverse industries since its discovery in the 1950s. Of particular interest is lithium-7, the most prevalent lithium isotope on Earth, playing a vital role in applications such as batteries, metal alloys, medicine, and nuclear research. However, its extraction presents significant environmental and logistical challenges. This article explores the potential for lithium exploration on the Moon, driven by its value as a resource and the prospect of cost reduction due to the Moon’s lower gravity, which holds promise for future space exploration endeavors. Additionally, the presence of lithium in the solar wind and its implications for material transport across celestial bodies are subjects of intrigue. Drawing from a limited dataset collected during the Apollo missions (Apollo 12, 15, 16, and 17) and leveraging artificial intelligence techniques and sample expansion through bootstrapping, this study develops predictive models for lithium-7 concentration based on spectral patterns. The study areas encompass the Aitken crater, Hadley Rima, and the Taurus–Littrow Valley, where higher lithium concentrations are observed in basaltic lunar regions. This research bridges lunar geology and the formation of the solar system, providing valuable insights into celestial resources and enhancing our understanding of space. The data used in this study were obtained from the imaging sensors (infrared, visible, and ultraviolet) of the Clementine satellite, which significantly contributed to the success of our research. Furthermore, the study addresses various aspects related to statistical analysis, sample quality validation, resampling, and bootstrapping. Supervised machine learning model training and validation, as well as data import and export, were explored. The analysis of data generated by the Clementine probe in the near-infrared (NIR) and ultraviolet-visible (UVVIS) spectra revealed evidence of the presence of lithium-7 (Li-7) on the lunar surface. The distribution of Li-7 on the lunar surface is non-uniform, with varying concentrations in different regions of the Moon identified, supporting the initial hypothesis associating surface Li-7 concentration with exposure to solar wind. While a direct numerical relationship between lunar topography and Li-7 concentration has not been established due to morphological diversity and methodological limitations, preliminary results suggest significant economic and technological potential in lunar lithium exploration and extraction.
... Intelligent algorithms are used in multiple disciplines with their powerful learning and computational capabilities (Rahman & Szabó, 2021;Wang et al., 2022a). For example, the neural network algorithm, which is a deep learning model, has its origin in the human brain's nervous system and is able to simulate the brain's figurative thinking process (Wang et al., 2020). Its most important feature is that it can learn autonomously from the input training samples, so as to deal with events with non-linear relationships (Su & Fu, 2021). ...
Based on the land use data in 1986, 1996, 2000, 2004, 2009, 2015 and 2020 in the Pingshuo opencast coal mine area, the land use information entropy and dynamic degree were calculated, and the land use transfer matrix and complex network were constructed. The evolution characteristics of land usage were analyzed from several perspectives. Under the future economic-ecological dual scenario, an improved genetic algorithm was proposed to optimize the land use structure of the mining region, while the Markov model was used for comparative validation. The research results showed that cultivated land, forest land and grassland decreased between 1986 and 2020, while residential land and mining land rose dramatically, with cultivated land serving as the primary source. The entropy value of land use increased and the dominance decreased. Moreover, the integrated changes in the land use became more intense in 1986–2020, with the most drastic change in residential land. The land use system’s overall stability was low, but showed a slowly increasing trend. In addition, after the optimization of multi-objective genetic algorithm, the land use structure of the mining area can provide greater comprehensive benefits, especially the ecological benefit. This aided in the improvement of the mining area’s delicate ecosystem. The research results reveal the evolutionary process of land use over the past 30 years in the Pingshuo opencast coal mine area and provide theoretical support for regional land use planning and sustainable development.
... Our experiments spanned supervised, semisupervised, and unsupervised learning tasks to demonstrate EventZoom's efficacy under different label availability conditions. We used spiking neural networks (SNNs) [38,39,40,41] for validation of each group of experiments. Additionally, to align with current training paradigms and ensure fair comparison with existing methodologies, we adopted the common practice of converting event sequences into frames to facilitate neural network training. ...
Event data captured by Dynamic Vision Sensors (DVS) offers a unique approach to visual processing that differs from traditional video capture, showcasing its efficiency in dynamic and real-time scenarios. Despite advantages such as high temporal resolution and low energy consumption, the application of event data faces challenges due to limited dataset size and diversity. To address this, we developed EventZoom -- a data augmentation strategy specifically designed for event data. EventZoom employs a progressive temporal strategy that intelligently blends time and space to enhance the diversity and complexity of the data while maintaining its authenticity. This method aims to improve the quality of data for model training and enhance the adaptability and robustness of algorithms in handling complex dynamic scenes. We have experimentally validated EventZoom across various supervised learning frameworks, including supervised, semi-supervised, and unsupervised learning. Our results demonstrate that EventZoom consistently outperforms other data augmentation methods, confirming its effectiveness and applicability as a powerful event-based data augmentation tool in diverse learning settings.
... This even holds when the noise cannot be measured since ANP still functions when we compare two noisy passes rather than a clean and a noisy pass (Dalm et al., 2023). Additionally, the gradient-free nature of the approach becomes valuable in settings where the computational graph contains non-differentiable components as in spiking recurrent neural networks; a type of network where effective training methods are still under exploration (Tavanaei et al., 2019;Wang et al., 2020;Neftci et al., 2019). ...
Recurrent neural networks (RNNs) hold immense potential for computations due to their Tur-ing completeness and sequential processing capabilities, yet existing methods for their training encounter efficiency challenges. Backpropagation through time (BPTT), the prevailing method, extends the backpropagation (BP) algorithm by unrolling the RNN over time. However, this approach suffers from significant drawbacks, including the need to interleave forward and backward phases and store exact gradient information. Furthermore, BPTT has been shown to struggle with propagating gradient information for long sequences, leading to vanishing gradients. An alternative strategy to using gradient-based methods like BPTT involves stochastically approximating gradients through perturbation-based methods. This learning approach is exceptionally simple, necessitating only forward passes in the network and a global reinforcement signal as feedback. Despite its simplicity, the random nature of its updates typically leads to inefficient optimization, limiting its effectiveness in training neural networks. In this study, we present a new approach to perturbation-based learning in RNNs whose performance is competitive with BPTT, while maintaining the inherent advantages over gradient-based learning. To this end, we extend the recently introduced activity-based node perturbation (ANP) method to operate in the time domain, leading to more efficient learning and generalization. Subsequently, we conduct a range of experiments to validate our approach. Our results show similar performance, convergence time and scala-bility when compared to BPTT, strongly outperforming standard node perturbation and weight perturbation methods. These findings suggest that perturbation-based learning methods offer a versatile alternative to gradient-based methods for training RNNs which can be ideally suited for neuromorphic computing applications.
... Typically, neural networks are divided into three generations, each of which mimics the multilayered structure of the human brain while displaying unique behaviors [166]. The first generation has binary (0,1) neuron output, which is derived from simple weighted synaptic input thresholding. ...
Simultaneous Localization and Mapping (SLAM) is a crucial function for most autonomous systems, allowing them to both navigate through and create maps of unfamiliar surroundings. Traditional Visual SLAM, also commonly known as VSLAM, relies on frame-based cameras and structured processing pipelines, which face challenges in dynamic or low-light environments. However, recent advancements in event camera technology and neuromorphic processing offer promising opportunities to overcome these limitations. Event cameras inspired by biological vision systems capture the scenes asynchronously consuming minimal power but with higher temporal resolution. Neuromorphic processors, which are designed to mimic the parallel processing capabilities of the human brain, offer efficient computation for real-time data processing of event-based data streams. This paper provides a comprehensive overview of recent research efforts in integrating event cameras and neuromorphic processors into VSLAM systems. It discusses the principles behind event cameras and neuromorphic processors, highlighting their advantages over traditional sensing and processing methods. Furthermore, an in-depth survey was conducted on state-of-the-art approaches in event-based SLAM, including feature extraction, motion estimation, and map reconstruction techniques. Additionally, the integration of event cameras with neuromorphic processors, focusing on their synergistic benefits in terms of energy efficiency, robustness, and real-time performance was explored. The paper also discusses the challenges and open research questions in this emerging field, such as sensor calibration, data fusion, and algorithmic development. Finally, the potential applications and future directions for event-based SLAM systems are outlined, ranging from robotics and autonomous vehicles to augmented reality.
... This even holds when the noise cannot be measured since ANP still functions when we compare two noisy passes rather than a clean and a noisy pass (Dalm et al., 2023). Additionally, the gradient-free nature of the approach becomes valuable in settings where the computational graph contains non-differentiable components as in spiking recurrent neural networks; a type of network where effective training methods are still under exploration (Tavanaei et al., 2019;Wang et al., 2020;Neftci et al., 2019). ...
Recurrent neural networks (RNNs) hold immense potential for computations due to their Turing completeness and sequential processing capabilities, yet existing methods for their training encounter efficiency challenges. Backpropagation through time (BPTT), the prevailing method, extends the backpropagation (BP) algorithm by unrolling the RNN over time. However, this approach suffers from significant drawbacks, including the need to interleave forward and backward phases and store exact gradient information. Furthermore, BPTT has been shown to struggle with propagating gradient information for long sequences, leading to vanishing gradients. An alternative strategy to using gradient-based methods like BPTT involves stochastically approximating gradients through perturbation-based methods. This learning approach is exceptionally simple, necessitating only forward passes in the network and a global reinforcement signal as feedback. Despite its simplicity, the random nature of its updates typically leads to inefficient optimization, limiting its effectiveness in training neural networks. In this study, we present a new approach to perturbation-based learning in RNNs whose performance is competitive with BPTT, while maintaining the inherent advantages over gradient-based learning. To this end, we extend the recently introduced activity-based node perturbation (ANP) method to operate in the time domain, leading to more efficient learning and generalization. Subsequently, we conduct a range of experiments to validate our approach. Our results show similar performance, convergence time and scalability when compared to BPTT, strongly outperforming standard node perturbation and weight perturbation methods. These findings suggest that perturbation-based learning methods offer a versatile alternative to gradient-based methods for training RNNs.
... It should be noted that it's essential to carefully design and train the network in NIR analysis, considering the specific requirements of the analysis task. BPANN specifically refers to an ANN trained using the backpropagation algorithm, and is a supervised learning algorithm where the network learns from labeled data by adjusting its weights to minimize the error between predicted and actual outputs (Wang, Lin, & Dang, 2020). BPANN may excel in capturing intricate patterns in high-dimensional data but might require more computational resources for training. ...
NIR sensors, in conjunction with advanced chemometric algorithms, have proven to be a powerful and efficient tool for intelligent quality evaluation of sweetpotato roots throughout the entire supply chain. By leveraging NIR data in different wavelength ranges, the physicochemical, nutritional and antioxidant compositions, as well as variety classification of sweetpotato roots during the different stages were adequately evaluated, and all findings involving quantitative and qualitative investigations from the beginning to the present were summarized and analyzed comprehensively. All chemometric algorithms including both linear and nonlinear employed in NIR analysis of sweetpotato roots were introduced in detail and their calibration performances in terms of regression and classification were assessed and discussed. The challenges and limitations of current NIR application in quality evaluation of sweetpotato roots are emphasized. The prospects and trends covering the ongoing advancements in software and hardware are suggested to support the sustainable and efficient sweetpotato processing and utilization.
... The light source is assumed to be in the form of a point. According to the physics law, the intensity of the pointed light has reverse relation with the square of the distance between two fireflies [19]. Equation (9) computes the intensity of the pointed light as: ...
... Firefly's attractiveness is proportional to the light intensity seen by adjacent fireflies. The attractiveness of a firefly can be expressed by equation (10) as [19]: ...
... where β refers to the received attractiveness, β 0 is the maximum attractiveness of the source (attractiveness at r = 0) and γ is the light absorption coefficient. Equation (11) presents a new position of each firefly [19]: ...
The aim of this study is to explore the characteristics of an active Free-Piston Stirling Engine (AFPSE) through the use of machine learning methods. Due to the time-intensive nature of extracting simulation results from complex thermal equations, an Artificial Neural Network (ANN) is utilized to expedite the process. To construct a nonlinear model, 5000 samples are extracted from simulation results. Input parameters included in the model are the hot and cold source temperatures, the voltage given to the DC motor, spring stiffness, and the mass of the power piston, while output parameters are the amplitude and frequency of power piston displacement. The proposed ANN model structure comprises two hidden layer with 10 and 20 neurons, respectively, indicating the applicability of the ANN model in estimating significant parameters of AFPSE in a shorter amount of time. The firefly optimization algorithm is utilized to determine the unknown input parameters of ANN and maximize the output power. Results indicate that a maximum output power of 23.07 W can be attained by applying 8.5 V voltage on the DC motor. This study highlights the potential of machine learning techniques to explore the primary features of AFPSE.
... To reduce the labelling costs of data, unsupervised learning has emerged as a powerful technique for the learning of video data. In the traditional image domain, unsupervised methods are expected to outperform their supervised counterparts [13,14,16,60,61] due to their potential to train data without labels. ...
Tracking fish movements and sizes of fish is crucial to understanding their ecology and behaviour. Knowing where fish migrate, how they interact with their environment, and how their size affects their behaviour can help ecologists develop more effective conservation and management strategies to protect fish populations and their habitats. Deep learning is a promising tool to analyse fish ecology from underwater videos. However, training deep neural networks (DNNs) for fish tracking and segmentation requires high-quality labels, which are expensive to obtain. We propose an alternative unsupervised approach that relies on spatial and temporal variations in video data to generate noisy pseudo-ground-truth labels. We train a multi-task DNN using these pseudo-labels. Our framework consists of three stages: (1) an optical flow model generates the pseudo-labels using spatial and temporal consistency between frames, (2) a self-supervised model refines the pseudo-labels incrementally, and (3) a segmentation network uses the refined labels for training. Consequently, we perform extensive experiments to validate our method on three public underwater video datasets and demonstrate its effectiveness for video annotation and segmentation. We also evaluate its robustness to different imaging conditions and discuss its limitations.
... When the weight of the linear transformation (weighting of the input) changes, the output of the neural network also changes, i.e., weight learning is possible. Therefore, neural networks are used as models for machine learning and for various classes of tasks, such as classification, regression, and generation, with or without supervision [17][18][19][20]. Applications include pattern recognition, a type of natural information processing that selects and extracts objects that have certain rules and meanings from data that include miscellaneous information such as images and sounds, data mining, and technology that extracts knowledge by comprehensively applying data evaluation techniques such as statistics, pattern recognition, and AI to large amounts of data (e.g., image recognition, recommendation) [21]. ...
Computational intelligence (CI) uses applied computational methods for problem-solving inspired by the behavior of humans and animals. Biological systems are used to construct software to solve complex problems, and one type of such system is an artificial immune system (AIS), which imitates the immune system of a living body. AISs have been used to solve problems that require identification and learning, such as computer virus identification and removal, image identification, and function optimization problems. In the body’s immune system, a wide variety of cells work together to distinguish between the self and non-self and to eliminate the non-self. AISs enable learning and discrimination by imitating part or all of the mechanisms of a living body’s immune system. Certainly, some deep neural networks have exceptional performance that far surpasses that of humans in certain tasks, but to build such a network, a huge amount of data is first required. These networks are used in a wide range of applications, such as extracting knowledge from a large amount of data, learning from past actions, and creating the optimal solution (the optimization problem). A new technique for pre-training natural language processing (NLP) software ver.9.1by using transformers called Bidirectional Encoder Representations (BERT) builds on recent research in pre-training contextual representations, including Semi-Supervised Sequence Learning, Generative Pre-Training, ELMo (Embeddings from Language Models), which is a method for obtaining distributed representations that consider context, and ULMFit (Universal Language Model Fine-Tuning). BERT is a method that can address the issue of the need for large amounts of data, which is inherent in large-scale models, by using pre-learning with unlabeled data. An optimization problem involves “finding a solution that maximizes or minimizes an objective function under given constraints”. In recent years, machine learning approaches that consider pattern recognition as an optimization problem have become popular. This pattern recognition is an operation that associates patterns observed as spatial and temporal changes in signals with classes to which they belong. It involves identifying and retrieving predetermined features and rules from data; however, the features and rules here are not logical information, but are found in images, sounds, etc. Therefore, pattern recognition is generally conducted by supervised learning. Based on a new theory that deals with the process by which the immune system learns from past infection experiences, the clonal selection of immune cells can be viewed as a learning rule of reinforcement learning.
