No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
40. Detection of irrigation malfunctions based on thermal imaging
Abstract
Frequent monitoring of the irrigation system is an important task for efficient farm management. However, since it is costly and labor-intensive it is not implemented on a regular basis. Irrigation malfunctions are usually associated with improper maintenance or operation of the irrigation system and appear as leaks and clogs in irrigation lines. These malfunctions cause within-field yield variability by direct (e.g., leaking to groundwater) and indirect effects (e.g., decrease in yield and quality). This study presents algorithms for monitoring and mapping irrigation system failures based on airborne thermal imaging data.
Data of olive groves in north Israel, near Kibbutz Gshur were collected in 2012 using an airborne thermal camera (Long Infra-Red range, 8-14μm). About 50 hectares of olive groves were imaged at a spatial resolution of 0.35m. Ground truth was determined by scouting. Each visually observed malfunction was geo-tagged, resulting in 100 malfunctions.
The detection model's predictor variables were derived using standard image processing tools. To remove the soil pixels, three segmentation algorithms (Otsu's Method, Full-Width Half-Max, and Continuous Max-Flow-Min-Cut) and merges between them were implemented and evaluated. Further to the segmentation process, to avoid mixed pixels, a Subpixel Edge Detection method was applied. Fourteen features were derived from the processed thermal images and were used as predictor variables. Classification models were developed and evaluated including RUSBoost, Linear SVM, and SUBspace KNN.
The best segmentation method was a merge of Continuous Max-Flow-Min-Cut and Otsu using the Minimum operation. Consequent classification achieved using the Bootstrap aggregation (Bagging) with Random Forest probably since it can deal with the imbalanced data. The accuracy achieved in leaks and clogs detection was 90.8% and 87.5%, and the reliability values were 78.5% and 100% respectively. This is an improvement of 10% to the accuracy of leaks' detection reported with the same data in a previous study of the same group (ECPA 2015).
Ongoing work is aimed at using higher resolution images and at improving algorithms along with extending analyses for additional crop types.
Keywords: thermal imaging, irrigation malfunctions
... A preliminary study within our group that was presented in a conference (Kalo et al., 2021) showed the feasibility of identifying irrigation malfunctions in drip systems by thermal images using machine learning for a case study of olive plants. Data from 100 ha of olive groves were collected using an airborne thermal camera. ...
... Data from 100 ha of olive groves were collected using an airborne thermal camera. Results were 89.5% and 87.5% accurate in detecting leaks and clogs, respectively, using random forest (RF) algorithms (Kalo et al., 2021). ...
Leaks and clogs in drip-irrigated orchards lead to variable yields, reduced efficiency and profitability. Frequent monitoring of irrigation systems by farmers is important but costly, labor-intensive, and not easily implementable on a regular basis. Moreover, in subsur-face drip-irrigation systems, it is difficult to visually detect malfunctions. The objective of this study was to develop processing methodologies based on thermal remote sensing, to produce classification models for detecting irrigation malfunctions in orchards, and distinguish between different types of malfunctions. A thermal camera mounted on an unmanned aerial vehicle platform was used to acquire thermal images in three commercial almond and jojoba plantations with subsurface drip irrigation. An image-processing pipeline was developed to extract plant-specific features, and classification models were used to detect malfunctions in individual plants. Plants were segmented using four algorithms: Otsu, continuous max-flow and min-cut, full-width-half-max, and watershed. Thirty-two features were extracted from the canopy temperature of each plant and normalized with meteorological data. The most significant features were selected using a recursive feature elimination method. Three classification models (multiclass, binary, hierarchical) were constructed using five classification algorithms. Performance was evaluated with k-fold cross-validation and an independent test set. In the almond plants orchard, the hierarchical classification approach with support vector machine (SVM) algorithms yielded 68% accuracy and 33% false-positive rate (FPR) for clog detection and 2.8% FPR for leak detection. In the jojoba plantation, the multiclass classification approach with SVM algorithms gave 82% accuracy for clog and leak detection with 0% FPR.
... Nonetheless, despite their effectiveness in detecting larger leaks caused by hydrants, these methods primarily focused on substantial puddles and lacked a complete automation. On the other hand, an automated algorithm was developed to monitor irrigation system malfunctions in olive orchards using airborne TIR data [37]. This method included image segmentation through the merging of Continuous Max-Flow-Min-Cut with the Otsu method [38] and subpixel edge detection to address mixed pixels. ...
Water is essential for maintaining plant health and optimal growth in agriculture. While some crops depend on irrigation, others can rely on rainfed water, depending on regional climatic conditions. This is exemplified by grapevines, which have specific water level requirements, and irrigation systems are needed. However, these systems can be susceptible to damage or leaks, which are not always easy to detect, requiring meticulous and time-consuming inspection. This study presents a methodology for identifying potential damage or leaks in vineyard irrigation systems using RGB and thermal infrared (TIR) imagery acquired by unmanned aerial vehicles (UAVs). The RGB imagery was used to distinguish between grapevine and non-grapevine pixels, enabling the division of TIR data into three raster products: temperature from grapevines, from non-grapevine areas, and from the entire evaluated vineyard plot. By analyzing the mean temperature values from equally spaced row sections, different threshold values were calculated to estimate and map potential leaks. These thresholds included the lower quintile value, the mean temperature minus the standard deviation (Tmean−σ), and the mean temperature minus two times the standard deviation (Tmean−2σ). The lower quintile threshold showed the best performance in identifying known leak areas and highlighting the closest rows that need inspection in the field. This approach presents a promising solution for inspecting vineyard irrigation systems. By using UAVs, larger areas can be covered on-demand, improving the efficiency and scope of the inspection process. This not only reduces water wastage in viticulture and eases grapevine water stress but also optimizes viticulture practices.
... (3) Vegetation segmentation (VS) based on [29] written in the Matlab R2020a (Mathworks Inc., Matick, MA, USA): ...
Accurate canopy extraction and temperature calculations are crucial to minimizing inaccuracies in thermal image-based estimation of orchard water status. Currently, no quantitative comparison of canopy extraction methods exists in the context of precision irrigation. The accuracies of four canopy extraction methods were compared and the effect on water status estimation was explored: 2-pixel erosion (2PE) where non-canopy pixels were removed by thresholding and morphological erosion; edge detection (ED) where edges were identified and morphologically dilated; vegetation segmentation (VS) using temperature histogram analysis and spatial watershed segmentation; and RGB-binary masking (RGB-BM) where a binary canopy layer was statistically extracted from an RGB image for thermal image masking. Field experiments occurred in a four ha commercial peach orchard during the primary fruit growth stage (III). The relationship between stem water potential (SWP) and crop water stress index (CWSI) was established in 2018. During 2019, a large dataset of ten thermal infrared and two RGB images was acquired. The canopy extraction methods had different accuracies: on 12 Aug, overall accuracy was 83% (2PE), 77% (ED), 84% (VS), and 90% (RGB-BM). Despite the high accuracy of the RGB-BM method, canopy edges and between-row weeds were misidentified as canopy. Canopy temperature and CWSI were calculated using the average of 100% of canopy pixels (CWSI_T100%) and the average of the coolest 33% of canopy pixels (CWSI_T33%). The CWSI_T33% dataset produced similar SWP-CWSI models irrespective of canopy extraction method while CWSI_T100% yielded different and inferior models. The results highlighted that: 1) The contribution of the RGB images is not significant for canopy extraction. Canopy pixels can be extracted with high accuracy and reliability solely with thermal images; and 2) The T33% approach to canopy temperature calculation is more robust and superior to the simple mean of all canopy pixels. These noteworthy findings are a step forward in implementing thermal imagery in precision irrigation management.
Numerous agronomical applications of remote sensing have been proposed in recent years, including water stress assessment at field by thermal imagery. The miniaturization of thermal cameras allows carrying them onboard the unmanned aerial vehicles (UAVs), but these systems have no temperature control and, consequently, drifts during data acquisition have to be carefully corrected. This manuscript presents a comprehensive methodology for radiometric correction of UAV remotely-sensed thermal images to obtain (combined with visible and near-infrared data) multispectral ortho-mosaics, as a previous step for further image-based assessment of tree response to water stress. On summer 2013, UAV flights were performed over an apple tree orchard located in Southern France, and 4 dates and 5 h of the day were tested. The 6400 m2 field plot comprised 520 apple trees, half well-irrigated and half submitted to progressive summer water stress. Temperatures of four different on-ground stable reference targets were continuously measured by thermo-radiometers for radiometric calibration purposes. By using self-developed software, frames were automatically extracted from the thermal video files, and then radiometrically calibrated using the thermal targets data. Once ortho-mosaics were obtained, root mean squared error (RMSE) was calculated. The accuracy obtained allowed multi-temporal mosaic comparison. Results showed a good relationship between calibrated images and on-ground data. Significantly higher canopy temperatures were found in water-stressed trees compared to well-irrigated ones. As high resolution field ortho-mosaics were obtained, comparison between trees opens the possibility of using multispectral data as phenotypic variables for the characterization of individual plant response to drought.
In computer aided medical system, many practical classification applications are confronted to the massive multiplication of collection and storage of data, this is especially the case in areas such as the prediction of medical test efficiency, the classification of tumors and the detection of cancers. Data with known class labels (labeled data) can be limited but unlabeled data (with unknown class labels) are more readily available. Semi-supervised learning deals with methods for exploiting the unlabeled data in addition to the labeled data to improve performance on the classification task. In this paper, we consider the problem of using a large amount of unlabeled data to improve the efficiency of feature selection in large dimensional datasets, when only a small set of labeled examples is available. We propose a new semi-supervised feature evaluation method called Optimized co-Forest for Feature Selection (OFFS) that combines ideas from co-forest and the embedded principle of selecting in Random Forest based by the permutation of out-of-bag set. We provide empirical results on several medical and biological benchmark datasets, indicating an overall significant improvement of OFFS compared to four other feature selection approaches using filter, wrapper and embedded manner in semi-supervised learning. Our method proves its ability and effectiveness to select and measure importance to improve the performance of the hypothesis learned with a small amount of labeled samples by exploiting unlabeled samples.
In the current study, the use of thermal remote sensing to detect irrigation-system malfunctions in olive orchards and table grape vineyards was evaluated. In the first part of the study, irrigation malfunctions were simulated. In the olive orchard, where deficit irrigation is routinely applied, both simulated leaks and clogs were detected. In grapevines, where full irrigation is applied, only simulated long-term dripper clogs were detectable. In the second part of the study, the accuracy of the automatic detection system was evaluated under commercial conditions. The accuracy of leak detection by thermal remote sensing was 75-90% and the reliability values of leak and clog detection were 90 and 70%, respectively. Thermal remote sensing seems to be a useful tool for detecting drip-irrigation malfunctions, thereby enhancing water-use efficiency and saving on labor.
Thermal imaging has been used in the past for remote detection of regions of canopy showing symptoms of stress, including water deficit stress. Stress indices derived from thermal images have been used as an indicator of canopy water status, but these depend on the choice of reference surfaces and environmental conditions and can be confounded by variations in complex canopy structure. Therefore, in this work, instead of using stress indices, information from thermal and visible light imagery was combined along with machine learning techniques to identify regions of canopy showing a response to soil water deficit. Thermal and visible light images of a spinach canopy with different levels of soil moisture were captured. Statistical measurements from these images were extracted and used to classify between canopies growing in well-watered soil or under soil moisture deficit using Support Vector Machines (SVM) and Gaussian Processes Classifier (GPC) and a combination of both the classifiers. The classification results show a high correlation with soil moisture. We demonstrate that regions of a spinach crop responding to soil water deficit can be identified by using machine learning techniques with a high accuracy of 97%. This method could, in principle, be applied to any crop at a range of scales.
The estimation of edge features, such as sub-pixel position, orientation, curvature and change in intensity at both sides of the edge, from the computation of the gradient vector in each pixel is usually inexact, even in ideal images. In this paper, we present a new edge detector based on an edge and acquisition model derived from the partial area effect, which does not assume continuity in the image values. The main goal of this method consists in achieving a highly accurate extraction of the position, orientation, curvature and contrast ofthe edges, even in difficult conditions, such as noisy images, blurred edges, low contrast areas or very close contours. For this purpose, we first analyze the influence of perfectly straight or circular edges in the surrounding region, in such a way that, when these conditions are fulfilled, the features can exactly be determined. Afterwards, we extend it to more realistic situations considering how adverse conditions can be tackled and presenting an iterative scheme for improving the results. We have tested this method in real as well as in sets of synthetic images with extremely difficult edges, and in both cases a highly accurate characterization has been achieved.
Thermal imaging has been progressively introduced as a promising technique for irrigation scheduling and the assessment of the crop-water status, especially when deficit irrigation (DI) strategies are being implemented. However, up to day, one of the most important limitations of this technique is related to the data interpretation derived from these sensors; which is in many cases, a severe limitation to its practical usage by farmers and technicians. This work evaluates the potential and usefulness of the non-water stress baselines (NWSB) extracted from thermal information, in order to define a practical protocol for taking decisions related to irrigation scheduling in almond plantations. The present study was developed during the kernel-filling period of three cultivars of mature almond trees (cvs. Guara, Lauranne, and Marta), subjected to different irrigation treatments. Thermal information was obtained by using a Thermal Camera (Flir SC660) with a high resolution, and subsequently, confronted with other related plant physiological parameters [leaf water potential (Ψleaf) and stomatal conductance (gs )]. Moreover, once defined the NWSB, there were obtained similar functions taking as reference the information from those irrigation treatments under different water stress levels. Finally, and taking into account the yield values, there were defined the most advisable functions in order to maximize the water savings, minimizing the yield losses. Here we demonstrate that the findings allow concluding that thermal information is enough robust to know the crop-water status and define a proper irrigation scheduling for almond plantations, being necessary to consider different thermal functions depending on the cultivar.
In order to meet the demand for increased global food production under limited water resources, implementation of suitable irrigation scheduling technique is crucial, particularly in irrigated basins experiencing water stress. Optimizing water use in agriculture requires innovations in detection of plant water stress, at various stages of the growing season to minimize crop physiological damage, and yield loss. Remotely sensed plant stress indicators, based on the visible and near-infrared spectral regions, have the advantage of high spatial and spectral resolutions, low cost, and quick turnaround time. This paper outlines recent developments in monitoring crop water stress, for scheduling irrigation, some of the constraints experienced, and future research needs.
Precision agriculture (PA) utilizes tools and technologies to identify in-field soil and crop variability for improving farming practices and optimizing agronomic inputs. Traditionally, optical remote sensing (RS) that utilizes visible light and infrared regions of the electromagnetic spectrum has been used as an integral part of PA for crop and soil monitoring. Optical RS, however, is slow in differentiating stress levels in crops until visual symptoms become noticeable. Surface temperature is considered to be a rapid response variable that can indicate crop stresses prior to their visual symptoms. By measuring estimates of surface temperature, thermal RS has been found to be a promising tool for PA. Compared to optical RS, applications of thermal RS for PA have been limited. Until recently (i.e., before the advancement of low cost RS platforms such as unmanned aerial systems (UAVs)), the availability of high resolution thermal images was limited due to high acquisition costs. Given recent developments in UAVs, thermal images with high spatial and temporal resolutions have become available at a low cost, which has increased opportunities to understand in-field variability of crop and soil conditions useful for various agronomic decision-making. Before thermal RS is adopted as a routine tool for crop and environmental monitoring, there is a need to understand its current and potential applications as well as issues and concerns. This review focuses on current and potential applications of thermal RS in PA as well as some concerns relating to its application. The application areas of thermal RS in agriculture discussed here include irrigation scheduling, drought monitoring, crop disease detection, and mapping of soil properties, residues and tillage, field tiles, and crop maturity and yield. Some of the issues related to its application include spatial and temporal resolution, atmospheric conditions, and crop growth stages.
Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, ***, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.
The problem of imbalanced data between classes prevails in various applications such as bioinformatics. The correctness of prediction in case of imbalanced data is usually biased towards the majority class. However, in several applications, the accuracy of prediction in minority class is also significant as much as in majority class. Previously, there were many techniques proposed to increase the accuracy in minority class. These techniques are based on the concept of re-sampling, which can be over-sampling and under-sampling, during the training process. Those re-sampling techniques did not considered how the data are scattered in the space. In this paper, we proposed a new technique based on the fact that the location of separating function in between any two sub-clusters in different classes is defined only by the boundary data of each sub-cluster. In addition, the accuracy is measured only by the testing set. Our technique adapted the concept of bootstrapping to estimate new region of each sub-cluster and synthesize the new boundary data. The new region is for coping with the unseen testing data. All new synthesized data were classified by using the concept of AdaBoost algorithm. Our results outperformed the other techniques under several performance evaluating functions.
Leaf temperature is directly determined by leaf energy and water balance. If diminished water availability decreases latent heat flux at the leaf surface, a complementary increase of sensible heat will occur and create a larger temperature difference between foliage and air. Radiation, air temperature, humidity and wind speed modify leaf temperature and may mask indications of water stress. The position, inclination and orientation of leaves within the canopy also produce considerable variation of leaf temperature. These factors were incorporated in a linear transpiration model, using physical and physiological characteristics of cotton. Water stress was simulated by imposing a limit value for stomatal conductance. The energy balance equation was solved as a function of angle between leaf and solar beam, to determine leaf temperature frequency distribution. The results show that stress induced temperature rise occurs over a small percentage of the total leaf area. Detection of moderate stress requires a normalizing procedure which takes into account meteorological conditions. The leaf temperature distribution is a better indicator of stress than the average value.
The infrared thermometer integrates thermal radiation emitted by the foliage included in its field of view. The model simulated the temperature composition of the field of view as a function of sighting angle and orientation relative to the incident solar beam. The spatial averaging of the signal in the field of view, attenuated the sensitivity of the measurement to water stress. The results also indicated that pointing the infrared thermometer towards the canopy in the same direction as the sun, and at an angle of incidence as close as possible to the solar zenith angle, improved the ability to detect water stress.
With the continuous expansion of data availability in many large-scale, complex, and networked systems, such as surveillance, security, Internet, and finance, it becomes critical to advance the fundamental understanding of knowledge discovery and analysis from raw data to support decision-making processes. Although existing knowledge discovery and data engineering techniques have shown great success in many real-world applications, the problem of learning from imbalanced data (the imbalanced learning problem) is a relatively new challenge that has attracted growing attention from both academia and industry. The imbalanced learning problem is concerned with the performance of learning algorithms in the presence of underrepresented data and severe class distribution skews. Due to the inherent complex characteristics of imbalanced data sets, learning from such data requires new understandings, principles, algorithms, and tools to transform vast amounts of raw data efficiently into information and knowledge representation. In this paper, we provide a comprehensive review of the development of research in learning from imbalanced data. Our focus is to provide a critical review of the nature of the problem, the state-of-the-art technologies, and the current assessment metrics used to evaluate learning performance under the imbalanced learning scenario. Furthermore, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potential important research directions for learning from imbalanced data.
We propose and study novel max-flow models in the continuous setting, which directly map the discrete graph-based max-flow problem to its continuous optimization formulation. We show such a continuous max-flow model leads to an equivalent min-cut problem in a natural way, as the corresponding dual model. In this regard, we revisit basic conceptions used in discrete max-flow / min-cut models and give their new explanations from a variational perspective. We also propose corresponding continuous max-flow and min-cut models constrained by priori supervised information and apply them to interactive image segmentation/labeling problems. We prove that the proposed continuous max-flow and min-cut models, with or without supervised constraints, give rise to a series of global binary solutions λ*(x) ϵ {0,1}, which globally solves the original nonconvex image partitioning problems. In addition, we propose novel and reliable multiplier-based max-flow algorithms. Their convergence is guaranteed by classical optimization theories. Experiments on image segmentation, unsupervised and supervised, validate the effectiveness of the discussed continuous max-flow and min-cut models and suggested max-flow based algorithms.