No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Improving Automatic Target Recognition in Low Data Regime using Semi-Supervised Learning and Generative Data Augmentation
ResearchGate has not been able to resolve any citations for this publication.
Restart techniques are common in gradient-free optimization to deal with multimodal functions. Partial restarts are also gaining popularity in gradient-based optimization to improve the rate of convergence in accelerated gradient schemes to deal with ill-conditioned functions. In this paper, we propose a simple restart technique for stochastic gradient descent to improve its anytime performance when training deep neural networks. We empirically study its performance on CIFAR-10 and CIFAR-100 datasets where we demonstrate new state-of-the-art results below 4\% and 19\%, respectively. Our source code is available at https://github.com/loshchil/SGDR.
We study the potential benefits to classification pre- diction that arise from having access to unlabeled samples. We compare learning in the semi-supervised model to the standard, supervised PAC (distribu- tion free) model, considering both the realizable and the unrealizable (agnostic) settings. Roughly speaking, our conclusion is that access to unlabeled samples cannot provide sample size guarantees that are better than those obtainable with- out access to unlabeled data, unless one postulates very strong assumptions about the distribution of the labels. In particular, we prove that for basic hypothesis classes over the real line, if the distribution of un- labeled data is 'smooth', knowledge of that dis- tribution cannot improve the labeled sample com- plexity by more than a constant factor (e.g., 2). We conjecture that a similar phenomena holds for any hypothesis class and any unlabeled data distribu- tion. We also discuss the utility of semi-supervised learning under the common cluster assumptioncon- cerning the distribution of labels, and show that even in the most accommodating cases, where data is generated by two uni-modal label-homogeneous distributions, common SSL paradigms may be mis- leading and may result in poor prediction perfor- mance.
Noise in field seismic data is usually complicated. Most signal-to-noise (S/N) ratio enhancement methods are designed for some specific noise type and they will become less effective when dealing with complex noise. In this article, we propose to use a trilateral weighted sparse coding (TWSC) scheme within the block matching framework for complex noise attenuation. Block matching-based approaches take the repetitive nature of effective signals into account which facilitates the S/N ratio enhancement. Block matching requires the identification of similar signal patches through a data set. This becomes more challenging for low-quality data. Thus, the signal matching criterion is of great importance. The Euclidean distance is the most widely used criterion for similarity measurement in block matching. It becomes a bottleneck for matching low-quality data or for data with rotations. To overcome this obstacle, we modify the Euclidean distance criterion by computing distances using Zernike moments which are rotation invariant and add noise robustness. This in turn benefits the subsequent filtering stage by TWSC, which uses two weight matrices to characterize the complex noise properties, and a third matrix to characterize the sparsity priors of signal. The improved sparse coding model is solved by the alternating direction method of multipliers. Tests on synthetic and several field data sets show that the proposed strategy achieves better performance in dealing with complex noise.
Tea storage period is difficult to be assessed by traditional sensory and chemical analysis methods. In this research, hyperspectral imaging (HSI) covering the spectral range of 400–1000 nm was combined with chemometric methods to predict the storage period of green tea samples. Tea samples were incubated under two storage temperatures of 4 °C (Sample set 1) and 25 °C (Sample set 2) for 360 days. The reflectance spectra were acquired every 90 days. Based on the preprocessed full spectra, partial least squares-discriminant analysis (PLS-DA)- and support vector machine (SVM)-based models had acceptable outputs for the two sample sets with an accuracy of over 90%. In addition, regression coefficients (RCs) of PLS-based models, second derivative (2-Der), and successive projections algorithm (SPA) were applied to select the optimal wavelengths. Satisfactory results were achieved by SVM-based models on the basis of optimal wavelengths. The SPA-SVM- and 2-Der-SVM-based models had the best performance to discriminate the storage period of the two sample sets with the prediction accuracies of 98% and 98%, respectively. The results demonstrated that HSI could be used to evaluate the storage period of green tea, providing an alternative for a real-time monitoring of tea quality.
Large deep neural networks are powerful, but exhibit undesirable behaviors such as memorization and sensitivity to adversarial examples. In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their labels. By doing so, mixup regularizes the neural network to favor simple linear behavior in-between training examples. Our experiments on the ImageNet-2012, CIFAR-10, CIFAR-100, Google commands and UCI datasets show that mixup improves the generalization of state-of-the-art neural network architectures. We also find that mixup reduces the memorization of corrupt labels, increases the robustness to adversarial examples, and stabilizes the training of generative adversarial networks.
Confidence calibration -- the problem of predicting probability estimates representative of the true correctness likelihood -- is important for classification models in many applications. We discover that modern neural networks, unlike those from a decade ago, are poorly calibrated. Through extensive experiments, we observe that depth, width, weight decay, and Batch Normalization are important factors influencing calibration. We evaluate the performance of various post-processing calibration methods on state-of-the-art architectures with image and document classification datasets. Our analysis and experiments not only offer insights into neural network learning, but also provide a simple and straightforward recipe for practical settings: on most datasets, temperature scaling -- a single-parameter variant of Platt Scaling -- is surprisingly effective at calibrating predictions.
We present YOLO, a unified pipeline for object detection. Prior work on
object detection repurposes classifiers to perform detection. Instead, we frame
object detection as a regression problem to spatially separated bounding boxes
and associated class probabilities. A single neural network predicts bounding
boxes and class probabilities directly from full images in one evaluation.
Since the whole detection pipeline is a single network, it can be optimized
end-to-end directly on detection performance. Our unified architecture is also
extremely fast; YOLO processes images in real-time at 45 frames per second,
hundreds to thousands of times faster than existing detection systems. Our
system uses global image context to detect and localize objects, making it less
prone to background errors than top detection systems like R-CNN. By itself,
YOLO detects objects at unprecedented speeds with moderate accuracy. When
combined with state-of-the-art detectors, YOLO boosts performance by 2-3%
points mAP.
A boosting algorithm seeks to minimize em- pirically a loss function in a greedy fash- ion. The resulted estimator takes an addi- tive function form and is built iteratively by applying a base estimator (or learner) to up- dated samples depending on the previous it- erations. This paper studies convergence of boosting when it is carried out over the lin- ear span of a family of basis functions. For general loss functions, we prove the conver- gence of boosting's greedy optimization to the infinimum of the loss function over the linear span. As a side product, these re- sults reveal the importance of restricting the greedy search step sizes, as known in practice through the works of Friedman and others.
A comprehensive review of an area of machine learning that deals with the use of unlabeled data in classification problems: state-of-the-art algorithms, a taxonomy of the field, applications, benchmark experiments, and directions for future research.
In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). Interest in SSL has increased in recent years, particularly because of application domains in which unlabeled data are plentiful, such as images, text, and bioinformatics. This first comprehensive overview of SSL presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research.Semi-Supervised Learning first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods, organized according to algorithmic strategies. After an examination of generative models, the book describes algorithms that implement the low-density separation assumption, graph-based methods, and algorithms that perform two-step learning. The book then discusses SSL applications and offers guidelines for SSL practitioners by analyzing the results of extensive benchmark experiments. Finally, the book looks at interesting directions for SSL research. The book closes with a discussion of the relationship between semi-supervised learning and transduction.
Mixmatch: A holistic approach to semi-supervised learning
- David Berthelot
- Nicholas Carlini
- Ian Goodfellow
- Nicolas Papernot
- Avital Oliver
- Colin Raffel
Deep learning and generative adversarial networks for automatic target recognition
- Chen Gan
- Wei Wang
- Weijie Liu
Transfer learning-based automatic target recognition in sar images
- Xiaohui Hu
- Yufeng Huang
- Xiaodong Cheng
- Xiaoping Liu
- Huan Zhao
Understanding softmax confidence and uncertainty
- Tim Pearce
- Alexandra Brintrup
- Jun Zhu
Augmented small target detection in infrared images based on a novel weight-shared siamese feature extraction network
- Xuyang Li
- Xing Zhang
- Mengjie Han
- Weihua Liu
Deep learning for automatic target recognition in infrared imagery: a survey
- luo
A comparative study of target detection and classification techniques in infrared imagery
- Saptarshi Mukherjee
- Arka Mallick
- Rishabh Agrawal
Automatic target recognition from infrared imagery: challenges and solutions
- John Smith
- Mary Jones
- Sarah Brown
Feature extraction for automatic target recognition in infrared imagery: a survey
- Raja Syamsul Azmir Raja Ghazali
- Syamsiah Mashohor
- Rozi Mahmud
A survey of automatic target recognition in forward-looking infrared images
- Zhuofu Li
- Xinyu Zuo
- Xiangyu Sun
- Hong Zhao
Infrared target recognition based on deep learning
- Haofeng Wang
- Siwei Zou
- Hong Liu
- Jun Yang
Visualizing data using t-sne
- Laurens Van Der Maaten
- Geoffrey Hinton
Target tracking in infrared images with template matching and adaptive correlation filters
- Wei Wang
- Xiaopeng Xu
- Xiaoshuai Wu
- Jian Yang
- Dajiang Wei
- Hongbin Zhang
Multi-target tracking in infrared videos using multi-level feature representation and re-detection
- Qichao Yang
- Jianfeng Li
- Di Huang
- Qi Lu
- Jun Sun
Learning deep representations of fine-grained visual information for remote sensing scene classification
- Dan Xu
- Elisa Ricci
- Wanli Ouyang
- Xiaogang Wang
- Nicu Sebe
Deep learning for automatic target recognition
- yang
A fast and robust target detection and recognition algorithm for infrared search and track system
- Yapeng Zhang
- Honggang Qi
- Bin Xiao
- Jianping Liu
- Jianping Shi
Augmented few-shot learning for remote sensing image classification
- Yingying Zhang
- Zihan Wei
- Hefei Wu
- Hai Huang
- Jie Chen
A survey on deep learning-based automatic target recognition: From shallow to deep
- Qingsong Zhao
- Jialin Liu
- Wei Zhang
- Zhen Chen
- Jiqing Jiang
Semi-supervised learning based small infrared target detection using multi-level feature fusion and an attention mechanism
- Yongping Zhai
- Shuiping Zheng
- Qiong Liu
- Tingting Zhang
- Xiang Ren
- Jing Zhang
Data augmentation for deep learning-based small target detection in infrared images
- Jingjing Zheng
- Zhihui Lai
- Xiaobo He
- Xiaoyu Liu
- Qin Zheng
Understanding softmax confidence and uncertainty
- pearce
A comparative study of target detection and classification techniques in infrared imagery
- mukherjee
Automatic target recognition from infrared imagery: challenges and solutions
- smith
Visualizing data using t-sne
- van der maaten
Target tracking in infrared images with template matching and adaptive correlation filters
- wang
Semi-supervised learning based small infrared target detection using multi-level feature fusion and an attention mechanism
- zhai
A survey on deep learning-based automatic target recognition: From shallow to deep
- zhao
Augmented small target detection in infrared images based on a novel weight-shared siamese feature extraction network
- li
Feature extraction for automatic target recognition in infrared imagery: a survey
- azmir raja ghazali
A fast and robust target detection and recognition algorithm for infrared search and track system
- zhang
Data augmentation for deep learning-based small target detection in infrared images
- zheng
Mixmatch: A holistic approach to semi-supervised learning
- berthelot
Learning deep representations of fine-grained visual information for remote sensing scene classification
- xu
Multi-target tracking in infrared videos using multi-level feature representation and re-detection
- yang
Deep learning and generative adversarial networks for automatic target recognition
- gan