Lothar Thiele's research while affiliated with ETH Zurich and other places

Multitask real-time embedded systems are often restricted by tight energy budgets, whilst they usually have environmental interactions through software-controlled energy-hungry peripheral modules like LTE, WiFi, and GSM. The way that the driver calls are used within the embedded software to do such a control introduces program energy-hotspots (EHs) from the peripheral module perspective, namely the code pieces wasting the system energy. By the energy waste, we mean that the energy consumption is reducible via some program code modifications without threatening the system schedulability and logical correctness. This paper examines the program EHs of fixed-priority real-time tasks where two types of energy inefficiency can occur: Intra-task type, causing energy waste even if a task runs individually, and inter-task type, happening due to the interaction between different system tasks, namely preemption scenarios even if there is no intra-task EH. The main cause of such EHs is the unnecessary time intervals between the driver calls, causing extra energy consumption by peripheral modules. We propose some static analysis methods to automatically detect and eliminate both types of intra-and inter-task EHs regarding their mutual relevance, according to the extreme (worst-case and best-case) execution times of certain task code parts. Our manipulations on the tasks to eliminate the EHs include some program code modifications with the awareness of system schedulability and logical correctness, and changing some scheduling decisions, namely limiting the preemption points. After applying our proposed method to the test tasks, our simulation results show an energy reduction of up to 19 percent.

Future intelligent robots are expected to process multiple inputs simultaneously (such as image and audio data) and generate multiple outputs accordingly (such as gender and emotion), similar to humans. Recent research has shown that multi-input single-output (MISO) deep neural networks (DNN) outperform traditional single-input single-output (SISO) models, representing a significant step towards this goal. In this paper, we propose MIMONet, a novel on-device multi-input multi-output (MIMO) DNN framework that achieves high accuracy and on-device efficiency in terms of critical performance metrics such as latency, energy, and memory usage. Leveraging existing SISO model compression techniques, MIMONet develops a new deep-compression method that is specifically tailored to MIMO models. This new method explores unique yet non-trivial properties of the MIMO model, resulting in boosted accuracy and on-device efficiency. Extensive experiments on three embedded platforms commonly used in robotic systems, as well as a case study using the TurtleBot3 robot, demonstrate that MIMONet achieves higher accuracy and superior on-device efficiency compared to state-of-the-art SISO and MISO models, as well as a baseline MIMO model we constructed. Our evaluation highlights the real-world applicability of MIMONet and its potential to significantly enhance the performance of intelligent robotic systems.

Spatial-temporal graph models are prevailing for abstracting and modelling spatial and temporal dependencies. In this work, we ask the following question: \textit{whether and to what extent can we localise spatial-temporal graph models?} We limit our scope to adaptive spatial-temporal graph neural networks (ASTGNNs), the state-of-the-art model architecture. Our approach to localisation involves sparsifying the spatial graph adjacency matrices. To this end, we propose Adaptive Graph Sparsification (AGS), a graph sparsification algorithm which successfully enables the localisation of ASTGNNs to an extreme extent (fully localisation). We apply AGS to two distinct ASTGNN architectures and nine spatial-temporal datasets. Intriguingly, we observe that spatial graphs in ASTGNNs can be sparsified by over 99.5\% without any decline in test accuracy. Furthermore, even when ASTGNNs are fully localised, becoming graph-less and purely temporal, we record no drop in accuracy for the majority of tested datasets, with only minor accuracy deterioration observed in the remaining datasets. However, when the partially or fully localised ASTGNNs are reinitialised and retrained on the same data, there is a considerable and consistent drop in accuracy. Based on these observations, we reckon that \textit{(i)} in the tested data, the information provided by the spatial dependencies is primarily included in the information provided by the temporal dependencies and, thus, can be essentially ignored for inference; and \textit{(ii)} although the spatial dependencies provide redundant information, it is vital for the effective training of ASTGNNs and thus cannot be ignored during training. Furthermore, the localisation of ASTGNNs holds the potential to reduce the heavy computation overhead required on large-scale spatial-temporal data and further enable the distributed deployment of ASTGNNs.

Deep models lack robustness to simple input transformations such as rotation, scaling, and translation, unless they feature a particular invariant architecture or undergo specific training, e.g., learning the desired robustness from data augmentations. Alternatively, input transformations can be treated as a domain shift problem, and solved by post-deployment model adaptation. Although a large number of methods deal with transformed inputs, the fundamental relation between input transformations and optimal model weights is unknown. In this paper, we put forward the configuration subspace hypothesis that model weights optimal for parameterized continuous transformations can reside in low-dimensional linear subspaces. We introduce subspace-configurable networks to learn these subspaces and observe their structure and surprisingly low dimensionality on all tested transformations, datasets and architectures from computer vision and audio signal processing domains. Our findings enable efficient model reconfiguration, especially when limited storage and computing resources are at stake.

Distributed embedded systems are pervasive components jointly operating in a wide range of applications. Moving towards energy harvesting powered systems enables their long-term, sustainable, scalable, and maintenance-free operation. When these systems are used as components of an automatic control system to sense a control plant, energy availability limits when and how often sensed data is obtainable, and therefore when and how often control updates can be performed. The time-varying and non-deterministic availability of harvested energy and the necessity to plan the energy usage of the energy harvesting sensor nodes ahead of time, on the one hand, have to be balanced with the dynamically changing and complex demand for control updates from the automatic control plant and thus energy usage, on the other hand. We propose a hierarchical approach with which the resources of the energy harvesting sensor nodes are managed on a long time horizon and on a faster time scale, self-triggered model predictive control controls the plant. The controller of the harvesting-based nodes’ resources schedules the future energy usage ahead of time and the self-triggered model predictive control incorporates these time-varying energy constraints. For this novel combination of energy harvesting and automatic control systems, we derive provable properties in terms of correctness, feasibility, and performance. We evaluate the approach on a double integrator and demonstrate its usability and performance in a room temperature and air quality control case study.

In many real-world wireless IoT networks, the application dictates the location of the nodes and therefore the link characteristics are inhomogeneous. Furthermore, nodes may in many scenarios only communicate with the Internet-attached gateway via multiple hops. If an energy-efficient short-range modulation scheme is used, nodes that are reachable only via high-path-loss links cannot communicate. Using a more energy-demanding long-range modulation allows connecting more nodes but would be inefficient for nodes that are easily reachable via low-path-loss links. Combining multiple modulations is challenging as low-power radios usually only support the use of a single modulation at a time. In this paper, we present the Long-Short-Range (LSR) protocol which supports low-power multi-hop communication using multiple modulations and is suited for networks with inhomogeneous link characteristics. To reduce the inherent redundancy of long-range modulations, we present a method to determine the connectivity graph of the network during regular data communication without adding significant overhead. In simulations, we show that LSR allows for reducing power consumption significantly for many scenarios when compared to a state-of-the-art multi-hop communication protocol using a single long-range modulation. We demonstrate the applicability of LSR with an implementation on real hardware and a testbed with long-range links.

Vision Transformers (ViTs) have recently achieved promising results in various computer vision tasks. However, ViTs have high computation costs and a large number of parameters due to the stacked multi-head self-attention (MHSA) and expanded feed-forward network (FFN) modules. Since the complexity of Transformer-based models is quadratic with the length of the input tokens, most current efforts focus on reducing the number of tokens in ViTs to improve the model efficiency. Unlike previous studies, we argue that diverse redundant features help ViTs understand the data comprehensively. In this paper, we propose GhostViT, which achieves both computation and storage efficiency. The key concept of GhostViT is to generate more diverse features using cheap operations in the MHSA and FFN modules. We experimentally demonstrate that our GhostViT can significantly reduce both the parameters and FLOPs of ViTs while achieving the similar or better accuracy. For example, about 14% of parameters and 17% of FLOPs of the DeiT-tiny model are reduced without any accuracy loss on the ImageNet-1 K dataset. The codes and trained models can be found at https://github.com/HuCaoFighting/GhostViT .