Figure 1 - uploaded by Hamed Khorasgani
Content may be subject to copyright.
Functional block diagram of the LADEE spacecraft (image credit: NASA/ARC) https://directory.eoportal.org/web/eoportal/satellite-missions/l/ladee
Source publication
his paper discusses a mixed method that combines unsupervised learning methods and human expert input for analyzing telemetry data from long-duration robotic space missions. Our goal is to develop more automated methods for detecting anomalies in time series data. Once anomalies are identified using unsupervised learning methods we use feature sele...
Contexts in source publication
Context 1
... a case study, we an- alyze telemetry data that was generated by NASA's Lunar At- mosphere and Dust Environment Explorer (LADEE) space- craft 1 , a robotic mission that orbited the moon to gather de- tailed information about the structure and composition of the thin lunar atmosphere, and determine whether dust is lofted into the lunar sky Hine et al. (2010). The LADEE system block diagram, shown in Figure 1, shows the four primary subsystems of the spacecraft: (1) the Integrated Avionics sys- tem, (2) the Propulsion system, (3) the Attitude Control sys- tem (ACS)), and (4) the Electrical Power Subsystem (EPS). Using the lessons learned from this case study, our overall goal is to develop a general data-driven monitoring approach for telemetry (i.e., streaming time series) data for purposes of health monitoring, which includes fault and anomaly detec- tion, prognosis, and performance analysis of the monitored system. ...
Context 2
... experts confirmed after studying the mission operator logs that the reaction wheels only went off once during the mis- sion, and the second zero in the figure was a case of bad data. Figure 10 shows that different currents in the SATORI board 3 #2 were the most significant features for this cluster. Figure 11 shows that the SATORI #2 current variable for both data objects in group 2 repeatedly exceeded the 95% confidence bounds of the current during nominal operations. ...
Context 3
... 10 shows that different currents in the SATORI board 3 #2 were the most significant features for this cluster. Figure 11 shows that the SATORI #2 current variable for both data objects in group 2 repeatedly exceeded the 95% confidence bounds of the current during nominal operations. ...
Context 4
... time interval was about six minutes long. Two different currents in the Power-switching and Pyro Inte- gration boards (PAPI) board 4 #2 were the significant features that characterized this group (see Figure 10). Figure 13 shows turned off that the PAPI #2 high pressure current number 7 during these three time intervals. ...
Context 5
... different currents in the Power-switching and Pyro Inte- gration boards (PAPI) board 4 #2 were the significant features that characterized this group (see Figure 10). Figure 13 shows turned off that the PAPI #2 high pressure current number 7 during these three time intervals. The high amplitude in the PAPI # 2 propulsion subsystem current was the second significant fea- ture for this cluster. ...
Context 6
... the help of our experts, we found out that the valve driver unit, which controls the propulsion subsystem and the pressurant tank heaters, (part of the propulsion subsystem (see Figure 1)) were ON for the three time intervals. This corresponded to a unique behavior, however, our experts con- firmed that the behavior was not anomalous. ...
Context 7
... cluster included two time windows, each about 20 min- utes in duration. Figure 15 shows the SATORI #1 current, which was the most significant feature for this cluster. The current values exceeded the 95% upper bound for nominal operations. ...
Context 8
... high battery current caused the battery to discharge below acceptable levels, and, the bat- tery voltage dropped below set thresholds. Our experts char- acterized this as an anomaly in operations because the laser communications test led to unintended consequences of the q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Figure 11. SATORI #2 high pressure current for cluster 2 objects battery voltage dropping below specified thresholds. ...
Context 9
... should also be noted that the sampling rate was also signif- icantly lower, because this was the end of the mission. The most significant feature for this group was the battery volt- age, which fell below the 95% bounds of normal operation (see Figure 16). The drop in battery voltage led to drops in the SATORI #1 and SATORI #2 voltages. ...
Context 10
... drop in battery voltage led to drops in the SATORI #1 and SATORI #2 voltages. Figure 17 shows SATORI #1 and SATORI #2 voltages were the next set of significant features. ...
Context 11
... battery volt- age was again the most significant feature that distinguished this group from nominal operations. Figure 18 shows battery voltage during this time interval. The remaining significant features for this cluster and their importance factors are pre- sented in Figure 19. ...
Context 12
... 18 shows battery voltage during this time interval. The remaining significant features for this cluster and their importance factors are pre- sented in Figure 19. ...
Context 13
... preserve the health of the battery, several loads were switched off to reduce energy consumption and give the bat- tery a chance to recharge. Figure 18 shows that the battery voltage came back to an acceptable level during this mode. Our experts explained that the data points in this group repre- sented a unique behavior in spacecraft operations. ...
Context 14
... order to demonstrate the power of diagnosis driven prog- nostic framework, we conducted a design for prognostics study to improve prognostics capabilities in the LADEE power system model. The prognostics recommendation re- port for the LADEE system, shown in Figure 21 (table on the right-side), proposes adding 5 degradation detection type tests that will look across multiple existing sensors to detect voltage degradation functions. This motivated the develop- ment of the Battery Voltage Anomaly Detector test point as shown in TEAMS model in Figure 20. ...
Citations
... The question arises: how can symbolic representations in case of WFM be derived from time series data, and how can inconsistencies be detected? Biswas et al. [5] presents an approach that centers on anomaly detection utilizing regions of unsupervised learned nominal behavior, which collectively encompass the system's operational behaviors. Within our framework, the stringent delineation of operational conditions forces us to refer to these as observational states. ...
Consistency-based diagnosis is an established approach to diagnose technical applications, but suffers from significant modeling efforts-especially for dynamic multi-modal time series. Machine learning seems to be an obvious solution, which becomes less obvious when looking at details: Which notion of consistency can be used? If logical calculi are still to be used, how can dynamic time series be transferred into the discrete world? This paper presents the methodology Discret2Di for automated learning of logical expressions for consistency-based diagnosis. While these logical calculi have advantages by providing a clear notion of consistency, they have the key problem of relying on a discretization of the dynamic system. The solution presented combines machine learning from both the time series and the symbolic domain to automate the learning of logical rules for consistency-based diagnosis.
... The authors proved the effectiveness of applying these clustering algorithms in their case study where some telemetry channels tended to deliver anomalous values. In [23], the authors proposed a mixed method for mode and anomaly detection of spacecraft systems. This method combines human expert input with unsupervised learning methods to analyze spacecraft telemetry data. ...
Satellite telemetry data plays an ever-important role in both the safety and the reliability of a satellite. These two factors are extremely significant in the field of space systems and space missions. Since it is challenging to repair space systems in orbit, health monitoring and early anomaly detection approaches are crucial for the success of space missions. A large number of efficient and accurate methods for health monitoring and anomaly detection have been proposed in aerospace systems but without showing enough concern for the patterns that can be mined from normal operational telemetry data. Concerning this, the present paper proposes DCLOP, an intelligent Deep Clustering-based Local Outlier Probabilities approach that aims at detecting anomalies alongside extracting realistic and reasonable patterns from the normal operational telemetry data. The proposed approach combines (i) a new deep clustering method that uses a dynamically weighted loss function with (ii) the adapted version of Local Outlier Probabilities based on the results of deep clustering. The DCLOP approach effectively monitors the health status of a spacecraft and detects the early warnings of its on-orbit failures. Therefore, this approach enhances the validity and accuracy of anomaly detection systems. The performance of the suggested approach is assessed using actual cube satellite telemetry data. The experimental findings prove that the suggested approach is competitive to the currently used techniques in terms of effectiveness, viability, and validity.
... An active research field between causality reasearch and fault diagnosis are bond-graph approaches [24,25]. Gao et al. [26] provide a good overview over bond-graphs and other model-based diagnosis methods. ...
In this article we describe a novel diagnosis methodology for physical systems such as industrial production systems. The article consists of two parts: Part one analyzes the differences between using sensor values and using residual values for fault diagnosis. Residual values denote the health of a component by comparing sensor values to a predefined model of normal behaviour. We further analyse how faults propagate through components of a physical system and argue for the use of residual values for diagnosing physical systems. In part two we extend the theory of established consistency-based diagnosis algorithms to use residual values. We also illustrate how users of the presented diagnosis methodology are free to substitute the residual generating equations and the diagnosis algorithm to suit their specific needs. For diagnosis, we present the algorithm HySD, based on Satisfiability Modulo Linear Arithmetic. We present an implementation of HySD using threshold values and a symbolic diagnosis approach. However, the approach is also suitable to integrate modern machine learning methods for anomaly detection and combine them with a multitude of diagnosis approaches. Through experiments on the process-industry benchmark Tennessee Eastman Process and another benchmark consisting of multiple tank systems we show the feasibility of our approach. Overall we show how our novel diagnosis approach offers a practical methodology that allows industry to advance from current state of the art anomaly detection to automated fault diagnosis.
Keywords: Diagnosis; Fault detection and isolation; Qualitative physics; Satisfiability
... Many diagnosis approaches have been tried out in industrial use cases. For example in rotor systems (Babu Rao & Mallikarjuna Reddy, 2021;Leitão et al., 2020), smartgrids (Jiang, Zhang, Gao, & Wu, 2014), wind turbines (Svärd & Nyberg, 2011), automotive (Struss, 2002;Stein, 2009), power electronics (Poon et al., 2017), embedded systems (Zoeteweij, Pietersma, Abreu, Feldman, & Van Gemund, 2008), process industry (Kallesoe, Cocquempot, & Izadi-Zamanabadi, 2006;Struss & Ertl, 2009), and spacecraft (Bajwa, Sweet, & Korsmeyer, 2003;Balaban, Narasimhan, Cannon, & Brownston, 2007;Biswas et al., 2016bBiswas et al., , 2016a). ...
This article presents a novel approach to diagnose faults in production machinery. A novel data-driven approach is presented to learn an approximation of dependencies between variables using Spearman correlation. It is further shown, how the approximation of the dependencies are used to create propositional logic rules for fault diagnosis. The article presents two novel algorithms: 1) to estimate dependencies from process data and 2) to create propositional logic diagnosis rules from those connections and perform consistency-based fault diagnosis. The presented approach was validated using three experiments. The first two show that the presented approach works well for injection molding machines and a simulation of a four-tank system. The limits of the presented method are shown with the third experiment containing sets of highly correlated signals.
... Many diagnosis approaches have been tried out in industrial use cases. For example in rotor systems (Babu Rao & Mallikarjuna Reddy, 2021;Leitão et al., 2020), smartgrids (Jiang, Zhang, Gao, & Wu, 2014), wind turbines (Svärd & Nyberg, 2011), automotive (Struss, 2002;Stein, 2009), power electronics (Poon et al., 2017), embedded systems (Zoeteweij, Pietersma, Abreu, Feldman, & Van Gemund, 2008), process industry (Kallesoe, Cocquempot, & Izadi-Zamanabadi, 2006;Struss & Ertl, 2009), and spacecraft (Bajwa, Sweet, & Korsmeyer, 2003;Balaban, Narasimhan, Cannon, & Brownston, 2007;Biswas et al., 2016bBiswas et al., , 2016a). ...
This article presents a novel approach to diagnose faults in injection molding machines. A novel data-driven approach is presented to learn an approximation of dependencies between variables using Spearman correlation. It is further shown, how the approximation of the dependencies are used to create propositional logic rules for fault diagnosis. The article presents two novel algorithms: 1) to estimate dependencies from process data and 2) to create propositional logic diagnosis rules from those connections and perform consistency-based fault diagnosis. The presented approach was validated using three experiments. The first two show that the presented approach works well for injection molding machines and a simulation of a four-tank system. The limits of the presented method are shown with the third experiment containing sets of highly correlated signals.
... The main reason for this is the computation of the dynamic weights involves domain information (like the track segment), whereas the fixed weight is an empirical value calculated based on trail runs. Including contextual information has been found effective in different machine learning applications of data-driven anomaly detection [55], face recognition [56], speech recognition, and query classification [57]. In our setup, the use of contextual information in the reward function helps us compute a superior dynamic weights compared to fixed weights. ...
Cyber Physical Systems (CPS) have increasingly started using Learning Enabled Components (LECs) for performing perception-based control tasks. The simple design approach, and their capability to continuously learn has led to their widespread use in different autonomous applications. Despite their simplicity and impressive capabilities, these models are difficult to assure, which makes their use challenging. The problem of assuring CPS with untrusted controllers has been achieved using the Simplex Architecture. This architecture integrates the system to be assured with a safe controller and provides a decision logic to switch between the decisions of these controllers. However, the key challenges in using the Simplex Architecture are: (1) designing an effective decision logic, and (2) sudden transitions between controller decisions lead to inconsistent system performance. To address these research challenges, we make three key contributions: (1) dynamic-weighted simplex strategy-we introduce "weighted simplex strategy" as the weighted ensemble extension of the classical Simplex Architecture. We then provide a reinforcement learning based mechanism to find dynamic ensemble weights, (2) middleware framework-we design a framework that allows the use of the dynamic-weighted simplex strategy, and provides a resource manager to monitor the computational resources, and (3) hardware testbed-we design a remote-controlled car testbed called DeepNNCar to test and demonstrate the aforementioned key concepts. Using the hardware, we show that the dynamic-weighted simplex strategy has 60% fewer out-of-track occurrences (soft constraint violations), while demonstrating higher optimized speed (performance) of 0.4 m/s during indoor driving than the original LEC driven system.
... -System based on LSTM Neural Networks was deployed by NASA to control the Curiosity rover operated by NASA [6]. -Application of a fully automatized data processing workflow [1]. ...
The fast growing number of low earth orbit exploitation and deep space missions results in enormous volumes of telemetry data. In order to operate efficiently satellites constellations as well as spacecrafts, DMSS offers a self-learning visual platform for anomaly detection in telemetry data coming from embedded sensors. As use-case, the data of two space missions operated by the European Space Agency were analyzed: Mars Express and GAIA.KeywordsSpacecraft telemetryAnomaly detectionVisualization
... Additionally, it does not consider the different operational modes and contexts of the working environment in performing the arbitration, which could be crucial for the systems performance. Biswas, Gautam, et al. [13] have shown that mode detection is a crucial problem and datadriven anomaly detection methods should be context sensitive. They also do not provide any confidence metric that can be used to evaluate the decisions of the LEC if safety violations occur. ...
... Applying the weighted simplex strategy to our system could (1) improves the safety of the system, while optimizing for speed (see Figure 3), and (2) allows us to integrate contextsensitive weights to compute the systems output. Biswas et al [13] have addressed the importance of mode and contextsensitive information in data-driven anomaly detection methods. Also, context-aware machine learning approaches have been found effective in different applications of face recognition [15], speech recognition, and query classification [16]. ...
Learning enabled components (LECs) trained using data-driven algorithms are increasingly being used in autonomous robots commonly found in factories, hospitals, and educational laboratories. However, these LECs do not provide any safety guarantees, and testing them is challenging. In this paper, we introduce a framework that performs weighted simplex strategy based supervised safety control, resource management and confidence estimation of autonomous robots. Specifically, we describe two weighted simplex strategies: (a) simple weighted simplex strategy (SW-Simplex) that computes a weighted controller output by comparing the decisions between a safety supervisor and an LEC, and (b) a context-sensitive weighted simplex strategy (CSW-Simplex) that computes a context-aware weighted controller output. We use reinforcement learning to learn the contextual weights. We also introduce a system monitor that uses the current state information and a Bayesian network model learned from past data to estimate the probability of the robotic system staying in the safe working region. To aid resource constrained robots in performing complex computations of these weighted simplex strategies, we describe a resource manager that offloads tasks to an available fog nodes. The paper also describes a hardware testbed called DeepNNCar, which is a low cost resource-constrained RC car, built to perform autonomous driving. Using the hardware, we show that both SW-Simplex and CSW-Simplex have 40% and 60% fewer safety violations, while demonstrating higher optimized speed during indoor driving (∼ 0.40 m/s) than the original system (using only LECs).
... Wavelet-based preprocessing is another possibility to extract frequency features of different timescales [17]. In this approach, the entire mission data is then clustered regarding these features in an unsupervised fashion using Euclidean distances. ...
The possibilities to observe and interact with any given spacecraft are naturally limited compared to groundbased systems due to a number of factors. These include but are not limited to the availability and bandwidth of their connection to ground, the availability of staff, communication latencies and power budgets. While a minimum level of autonomy is required for every spacecraft, past experiments and missions have shown that introducing more sophisticated autonomy mechanisms can drastically increase the efficiency for many missions in terms of reliability, science output and required operational effort. Artificial intelligence poses an increasingly popular approach for implementing on-board autonomy. The number of techniques and variants of artificial intelligence available in the literature is, however, just as diversified as their potential field of application. To provide an overview of the current state of the art of artificial intelligence and its application for space systems, this paper provides an extensive survey on existing techniques and algorithms as well as existing and potential applications on board spacecraft and on ground. The survey focuses on autonomous planning and scheduling of operations, self-awareness, anomaly detection and Fault Detection Isolation and Recovery (FDIR), on-board data analysis as well as onboard operations and processing of earth-observation data.
... However, the problem with these approaches is that they use fixed arbitration weights irrespective of the different operational modes and contexts of the working environment. Biswas et al. [16] have shown that mode detection is a crucial problem and data-driven anomaly detection methods should be mode and context sensitive. We hypothesize the same is true for Simplex Architectures. ...
This paper deals with resource constrained autonomous robots commonly found in factories, hospitals, and education laboratories, which popularly use learning enabled components (LEC) to make control actions. However, these LECs do not provide any safety guarantees, and testing them is challenging. To overcome these challenges, we introduce a framework that performs confidence estimation, resource management, and supervised safety control of autonomous systems with LECs. Using this framework, we make the following contributions: (1) allow for seamless integration of safety controllers and different simplex strategies to aid the LEC, (2) introduce RL-Simplex and illustrate the use of Q-learning to learn the optimal weights for the arbitration logic of the Simplex Architecture, (3) design a system level monitor that uses the current state information and a discrete Bayesian network model learned from past data to estimate a metric, which indicates if the car will remain in the safe region, and (4) a Resource Manager which performs dynamic task offloading depending on the resource temperature and CPU utilization while continually adjusting vehicle speed to compensate for the latency overhead. We compare the speed, steering and safety performance of the different controllers and simplex strategies, and we find RL-Simplex to have 60\% fewer safety violations and higher optimized speed during indoor driving ($\sim\,0.40\,m/s$) than the original system (using only LEC).
