Figure 1
The top image shows the depth map from the scene at the bottom. The colors are applied depending on the distance in meters as shown in the scale on the right part of the image. At the light gray wall with thin horizontal stripes, the algorithm of the depth sensor wrongly estimates an object close to the camera.
Contexts in source publication
Context 1
... two-dimensional images, data in three dimensions is reconstructed. Therefore, it seems only natural that in certain cases the generated depth images might contain wrong data as shown in Figure 1. Specifically, when used in larger outdoor environments at different weather conditions like drones would exhibit, the set of parameters and requirements might be very different from other common use cases for depth sensors as found in indoor scenarios like finger tracking or gesture recognition. ...
Context 2
... mentioned in Section III, we discovered some cases in which depth values in the depth map were not accurate and disturb the obstacle avoidance algorithms. Figure 1 shows one example. We provide another case in Figure 3. ...
Context 3
... the intermediate images in X and Y dimension, we apply the absolute function and convert them into an 8-bit format (line 10, 11). We add both images together and apply a binary threshold on the mask (lines 13, 14). Using the case from Figure 3, we visualize these processing steps in Figure 5. ...
Similar publications
Drones equipped with cameras are being used for surveillance purposes. These surveillance systems need vision-based object detection of ground objects which look very small because of the altitude of drones. We propose an improved YOLOv4 model targeted for vision-based small object detection. We investigated the performance of state of the art YOLO...
O objetivo deste artigo é mapear usos, críticas e controvérsias em torno do que abordamos como dispositivos de (contra)vigilância em favelas no Rio de Janeiro. O termo (contra)vigilância sintetiza dinâmicas sociotécnicas articuladas a partir de dois eixos: dispositivos de vigilância institucionais, como câmeras e drones policiais, que implicam em r...
Citations
... As a result, the temperature values on the 3D coordinates of all objects located at any distance from the system can be determined with improved accuracy. With the inherent 3D measurement and our proposed compensation model, it opens more opportunities in applying for drone flight in distant measurement and spectral (400-2500 nm) analysis [16]. In addition, 3D plant phenotyping and breeding [17], investigating the unusual disturbances and damages on high-voltage electrical transformers [18] under high ambient air temperature, and 3D spectral analyzing [19,20] can be accomplished. ...
A highly accurate and low-cost mobile platform for multiple object temperature measurement in 3D coordinate positions is introduced. The key idea relies on not only a combination of a 3D optical depth sensor and the 2D infrared thermal imager but also a determined distant-dependent compensation model. As a result, a cost-effective yet wider ( ${\gt}{100.0}^\circ {\rm C}$ > 100.0 ∘ C ) temperature measurement range under the fluctuation of working distant shift is realized with a low standard deviation of 0.16°C at the working distance of 1.0 m. Accurate temperature measurement at different positions of objects along the suitable working distance is also demonstrated.
... Several studies have demonstrated the value of stereopsis in controlling drone flights, e.g. [120][121][122][123][124], but all have a stage where they explicitly compute disparity. Skipping this step may enable more efficient stereoscopically guided control of behaviour. ...
Stereopsis has traditionally been considered a complex visual ability, restricted to large-brained animals. The discovery in the 1980s that insects, too, have stereopsis, therefore, challenged theories of stereopsis. How can such simple brains see in three dimensions? A likely answer is that insect stereopsis has evolved to produce simple behaviour, such as orienting towards the closer of two objects or triggering a strike when prey comes within range. Scientific thinking about stereopsis has been unduly anthropomorphic, for example assuming that stereopsis must require binocular fusion or a solution of the stereo correspondence problem. In fact, useful behaviour can be produced with very basic stereoscopic algorithms which make no attempt to achieve fusion or correspondence, or to produce even a coarse map of depth across the visual field. This may explain why some aspects of insect stereopsis seem poorly designed from an engineering point of view: for example, paying no attention to whether interocular contrast or velocities match. Such algorithms demonstrably work well enough in practice for their species, and may prove useful in particular autonomous applications.
This article is part of a discussion meeting issue ‘New approaches to 3D vision’.
... Thus we disable the IR projector for this reason, as well as the fact that it is not needed for outdoor operation and consumes power. For the rest of the camera settings we refer to (Pohl et al., 2019), which provides RealSense settings for drone collision avoidance applications. ...
... A great deal of time was spent at the flight testing sites diagnosing and fixing hardware and software issues unrelated to the actual obstacle-avoidance methodology. Initially, the Intel RealSense D435 depth camera output too many false-positive points within the point cloud, which was largely resolved by using the settings presented in (Pohl et al., 2019). Another issue arose due to USB3 connection to the Intel Realsense, which interfered with the GPS signal. ...
Agile, fixed-wing, aircraft have been proposed for diverse applications, due to their enhanced flight efficiency, compared to rotorcraft, and their superior maneuverability, relative to conventional, fixed-wing, aircraft. We present a novel, reactive, obstacle-avoidance algorithm that enables autonomous flight through unknown, cluttered environments using only on-board sensing and computation. The method selects a reference trajectory in real-time from a pre-computed library, based on goal location, instantaneous point cloud data, and the aircraft states. At each time-step, a cost is assigned to candidate trajectories that are collision-free and lead to the edge of the obstacle sensor’s field-of-view, with cost based on both distance to obstacles, and the goal. The lowest cost reference trajectory is then tracked. If all potential trajectories result in a collision, the aircraft has enough space to come to a stop, which theoretically guarantees collision-free flight. Our work demonstrates autonomous flight in unknown and unstructured environments using only on-board sensing (stereo camera, IMU, and GPS) and computation with an agile, fixed-wing, aircraft in both simulation and outdoor flight tests. During flight testing, the aircraft cumulatively flew 4.4km autonomously in outdoor environments with trees as obstacles with an average speed of 8.1ms−1 and a top speed of 14.4ms−1. To the best of our knowledge, ours is the first obstacle-avoidance algorithm suitable for agile, fixed-wing, aircraft that can theoretically guarantee collision-free flight and has been validated experimentally using only on-board sensing and computation in an unknown environment.
... For instance, if the inspection camera is monocular, photogrammetry techniques can be employed to retrieve measurements from a sequence of images taken from different angles [31,32]. Also, recent studies have demonstrated that the use of binocular cameras (generally, a regular high-resolution camera coupled with a depth sensor camera) can be effectively employed to obtain distance information from a single photo [33,34]. ...
The advent of parallel computing capabilities, further boosted through the exploitation of graphics processing units, has resulted in the surge of new, previously infeasible, algorithmic schemes for structural health monitoring (SHM) tasks, such as the use of convolutional neural networks (CNNs) for vision-based SHM. This work proposes a novel approach for crack recognition in digital images based on coupling of CNNs and suited image processing techniques. The proposed method is applied on a dataset comprising images of the welding joints of a long-span steel bridge, collected via high-resolution consumer-grade digital cameras. The studied dataset includes photos taken in sub-optimal light and exposure conditions, with several noise contamination sources such as handwriting scripts, varying material textures, and, in some cases, under presence of external objects. The reference pixels representing the cracks, together with the crack width and length, are available and used for training and validating the proposed model. Although the proposed framework requires some knowledge of the "damaged areas", it alleviates the need for precise labeling of the cracks in the training dataset. Validation of the model by means of application on an unlabeled image set reveals promising results in terms of accuracy and robustness to noise sources.
... Using a passive method, two cameras are calibrated; the difference in the distance to the same point between the left and right cameras can be used to obtain a short distance when the distance between the two cameras is sufficiently large. Another method is based on the binocular difference, which is a short distance [6], [7]. When using a single camera, the structure-from-motion method, which estimates the difference in three-dimensional distance while tracking multiple identical points that change over time, is also widely applied [8][9][10]. ...
Estimating a road surface or planes for applying AR(Augmented Reality) or an autonomous vehicle using a camera requires significant computation. Vision sensors have lower accuracy in distance measurement than other types of sensor, and have the difficulty that additional algorithms for estimating data must be included. However, using a camera has the advantage of being able to extract various information such as weather conditions, sign information, and road markings that are difficult to measure with other sensors. Various methods differing in sensor type and configuration have been applied. Many of the existing studies had generally researched by performing the depth estimation after the feature extraction. However, recent studies have suggested using deep learning to skip multiple processes and use a single DNN(Deep Neural Network). Also, a method using a limited single camera instead of a method using a plurality of sensors has been proposed. This paper presents a single-camera method that performs quickly and efficiently by employing a DNN to extract distance information using a single camera, and proposes a modified method for using a depth map to obtain real-time surface characteristics. First, a DNN is used to estimate the depth map, and then for quick operation, normal vector that can connect similar planes to depth is calculated, and a clustering method that can be connected is provided. An experiment is used to show the validity of our method, and to evaluate the calculation time.
Nowadays, many industries use robots and cameras in tandem to detect specific objects and perform specific tasks. However, misdetection can occur due to inconsistencies in lighting, background, and environment. In order to address the aforementioned issues, this study proposes using a dual arm six-degree-of-freedom (6-DoF) collaborative robot, ABB YuMi, and red, green, blue-depth (RGB-D) camera with YOLOv5 in a pick-and-place application. In order to prepare the dataset, the images are collected and labeled. The dataset has been trained with the YOLOv5 machine learning algorithm. It has taken on the role of weight for real-time detection. When RGB images from a camera are sent to YOLOv5, data pertaining to the bottle’s position x-y and color are extracted from the depth and color images. The position of the robot is used to control its movement. There are three parts to the experiment. To begin, YOLOv5 is tested with and without trained images. Second, YOLOv5 is tested with real-time camera images. Finally, we assume that YOLOv5 has perfect detection and grasping ability. The results were 95, 90, and 90%.
Cette thèse s’inscrit dans le cadre de la navigation perceptuelle. Notre objectif est d’étudier et de concevoir un système d’interprétation d’une scène d’environnement intérieur, observée par un système multi-capteurs réunissant un capteur ultrason et une caméra RVB. Le système proposé peut être employé pour équiper un dispositif intelligent d’assistance aux non voyants, ou encore un robot opérant dans des espaces meublés. Dans un système d’interprétation de scène, les acquisitions faites par les capteurs, présentent des restrictions du monde réel et se trouvent affectées d’imperfections, qu’il convient de prendre en compte au lieu de les ignorer. Leur prise en compte dans notre système d’interprétation a été effectuée par l’emploi de la théorie des possibilités lors de la modélisation des données acquises. Les modèles adoptés sont des distributions de possibilités. L’analyse et l’interprétation de la scène acquise s’est en suite basée sur ces connaissances possibilistes. Le système d’aide à la navigation proposé dans ce travail, présente une description de la scène environnante selon un modèle simpliste, partageant le champ intercepté par les capteurs en trois zones majeures, à savoir : face, gauche et droite. Il fournit à l’utilisateur des informations concernant la distance qui le sépare des objets détectés, la rigidité matérielle de ces objets, ainsi que leur positionnement dans la scène (objet à gauche, objet en face, objet à droite). Les performances du système d’interprétation proposé sont évaluées en utilisant le prototype "NA_System", développé par l’équipe "Cybernics team" du laboratoire "CEM_Lab" de l’École nationale d’ingénieurs de Sfax (ENIS). Les résultats obtenus sont encourageants et montrent l’efficacité de la théorie des possibilités comme cadre de représentation de données acquises de différents capteurs. La stratégie d’interprétation de scène proposée s’est montrée efficace pour intégrer les informations issues de multiples sources de connaissances. Dans la chaine de traitement de données adoptées pour l’analyse et l’interprétation de la scène, de nouvelles approches ont été proposées, notamment pour la sélection d’attributs, la détection d’objets saillants, la classification, la fusion et le recalage de données issues de deux sources.
Recently, there have been many advances in the algorithms required for autonomous navigation in unknown environments, such as mapping, collision avoidance, trajectory planning, and motion control. These components have been integrated into drones with high‐end computers and graphics processors. However, further development is required to enable compute‐constrained platforms with such autonomous navigation capabilities. To address this issue, in this paper, we present an autonomous navigation framework for reaching a goal in unknown three‐dimensional cluttered environments. The framework consists of three main components. The first component is a computationally efficient method for mapping the environment from the disparity measurements obtained from a depth sensor. The second component is a stochastic approach to generate a path to a given goal, taking into account the field of view constraints on the space that is assumed to be safe for navigation. The third method is a fast algorithm for the online generation of motion plans, taking into account the robot's dynamic constraints, model and environmental uncertainty, and disturbances. We provide a qualitative and quantitative comparison with existing reaching a goal and exploration methods, showing the superior performance of our approach. Additionally, we present indoors and outdoors experiments using a robotic platform based on the Intel Ready to Fly drone kit, which represents the implementation, in the most computational constrained platform, of autonomous navigation in unknown cluttered environments demonstrated to date. Open source code is available at: https://github.com/IntelLabs/autonomousmavs. The video of the experimental results can be found in https://youtu.be/79IFfQfvXLE.
