Figure - available from: International Journal of Computer Vision
This content is subject to copyright. Terms and conditions apply.
![The proposed Hadamard Matrix Guided Online Hashing framework. Each time when a set of streaming data arrives (a), the data points from the same class (denoted by one common shape and color) are assigned with a column (denoted by the same color) from a pre-generated Hadamard Matrix as the target code (r∗=4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r^{*} = 4$$\end{document} bits in this case) to be learned in the Hamming space (b). And each row is regarded as a set of binary labels (-1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-1$$\end{document} or +1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+1$$\end{document}). The goal of our framework is to learn r∗\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r^{*}$$\end{document} separate binary classifiers to predict each bit (c)](publication/341200621/figure/fig1/AS:962206107713537@1606419221474/The-proposed-Hadamard-Matrix-Guided-Online-Hashing-framework-Each-time-when-a-set-of.png)
The proposed Hadamard Matrix Guided Online Hashing framework. Each time when a set of streaming data arrives (a), the data points from the same class (denoted by one common shape and color) are assigned with a column (denoted by the same color) from a pre-generated Hadamard Matrix as the target code (r∗=4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r^{*} = 4$$\end{document} bits in this case) to be learned in the Hamming space (b). And each row is regarded as a set of binary labels (-1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-1$$\end{document} or +1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+1$$\end{document}). The goal of our framework is to learn r∗\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r^{*}$$\end{document} separate binary classifiers to predict each bit (c)
Source publication
Online image hashing has attracted increasing research attention recently, which receives large-scale data in a streaming manner to update the hash functions on-the-fly. Its key challenge lies in the difficulty of balancing the learning timeliness and model accuracy. To this end, most works follow a supervised setting, i.e., using class labels to b...
Citations
... To overcome such limitations, a series of methods termed online hashing [9] have been proposed and achieved satisfactory performance. More specifically, existing online hashing methods can be classified into three categories, i.e., uni-modal [10,11,12,13,14,15,16], cross-modal [17,18], and multi-modal [19,20,21]. Uni-modal methods are specialized in retrieving data within the same modality, facilitating intra-modal searches. ...
... Different from batch-based hashing, online hashing is born for online retrieval tasks and is capable of efficiently learning from streaming data as they could be updated only based on the newly coming data while preserving knowledge learned from old seen data. More specifically, existing online hashing methods can be classified into three categories, i.e., uni-modal [10,39,12,13,14,15,16], cross-modal [17,40,18], and multi-modal [19,20]. Online uni-modal hashing adopts queries from one modality to search for similar instances within the same modality [41,42,43,44,45]. ...
In the real world, multi-modal data often appears in a streaming fashion, and there is a growing demand for similarity retrieval from such non-stationary data, especially at a large scale. In response to this need, online multi-modal hashing has gained significant attention. However, existing online multi-modal hashing methods face challenges related to the inconsistency of hash codes during long-term learning and inefficient fusion of different modalities. In this paper, we present a novel approach to supervised online multi-modal hashing, called High-level Codes, Fine-grained Weights (HCFW). To address these problems, HCFW is designed by its non-trivial contributions from two primary dimensions: 1) Online Hashing Perspective. To ensure the long-term consistency of hash codes, especially in incremental learning scenarios, HCFW learns high-level codes derived from category-level semantics. Besides, these codes are adept at handling the category-incremental challenge. 2) Multi-modal Hashing Aspect. HCFW introduces the concept of fine-grained weights designed to facilitate the seamless fusion of complementary multi-modal data, thereby generating multi-modal weights at the instance level and enhancing the overall hashing performance. A comprehensive battery of experiments conducted on two benchmark datasets convincingly underscores the effectiveness and efficiency of HCFW.
... Online hashing can be generally divided into unsupervised hashing (Leng et al. 2015;Chen et al. 2017) and supervised hashing (Lin et al. 2020;Fang, Zhang, and Liu 2021). Unsupervised online hashing is roughly based on the idea of "sketching" (Clarkson and Woodruff 2009). ...
... BSODH sets two balancing factors to solve the "imbalance problem" caused by the asymmetric graphs and optimises them by means of discretization. Hadamard Matrix Guided Online Hashing (HMOH) (Lin et al. 2020) considers the Hadamard matrix as a more discriminative codebook. By assigning each column of the Hadamard matrix a unique label as target, the hash functions are updated. ...
... In order to reveal the superiority of FOH, we compare with several state-of-the-art online hashing baselines, including Online Kernel Hashing (OKH) (Huang, Yang, and Zheng 2013), Adaptive hashing (AdaptHash) (Cakir and Sclaroff 2015a), Online Supervised Hashing (OSH) (Cakir and Sclaroff 2015b), OH with Mutual Information (MIHash) , Towards Optimal Discrete Online Hashing with Balanced Similarity (BSODH) (Lin et al. 2019b) and Hadamard Matrix Guided Online Hashing (HMOH) (Lin et al. 2020). We use a pre-trained VGG16 (Simonyan and Zisserman 2015) for all the baselines to extract the original real-value image features. ...
Hashing has been widely researched to solve the large-scale approximate nearest neighbor search problem owing to its time and storage superiority. In recent years, a number of online hashing methods have emerged, which can update the hash functions to adapt to the new stream data and realize dynamic retrieval. However, existing online hashing methods are required to update the whole database with the latest hash functions when a query arrives, which leads to low retrieval efficiency with the continuous increase of the stream data. On the other hand, these methods ignore the supervision relationship among the examples, especially in the multi-label case. In this paper, we propose a novel Fast Online Hashing (FOH) method which only updates the binary codes of a small part of the database. To be specific, we first build a query pool in which the nearest neighbors of each central point are recorded. When a new query arrives, only the binary codes of the corresponding potential neighbors are updated. In addition, we create a similarity matrix which takes the multi-label supervision information into account and bring in the multi-label projection loss to further preserve the similarity among the multi-label data. The experimental results on two common benchmarks show that the proposed FOH can achieve dramatic superiority on query time up to 6.28 seconds less than state-of-the-art baselines with competitive retrieval accuracy.
... Recently, hashing-based approximate nearest neighbor (ANN) search methods [1], [7], [8], [9], [10], [11], [12], [13], [14], [15] have been developed to encode high-dimensional data into binary codes and perform fast non-exhaustive search on binary codes. Specifically, by learning similaritypreserving mappings [16], [17], [18], hashing methods aim to generate compact binary codes to preserve Euclidean distance or semantic similarity of original data, and use these codes as direct indices (addresses) into hash table(s). ...
... . With h 1 = q and i starting from 2, h i is obtained by flipping the j ′th bit of the p j ′ (i) th element in the sequence in each iteration according to Eq. (9). Therefore, the sequence extension for each iteration i in our proposed CSE is formulated as ...
Representing visual data using compact binary codes is attracting increasing attention as binary codes are used as direct indices into hash table(s) for fast non-exhaustive search. Recent methods show that ranking binary codes using weighted Hamming distance (WHD) rather than Hamming distance (HD) by generating query-adaptive weights for each bit can better retrieve query-related items. However, search using WHD is slower than that using HD. One main challenge is that the complexity of extending a monotone increasing sequence using WHD to probe buckets in hash table(s) for existing methods is at least proportional to the square of the sequence length, while that using HD is proportional to the sequence length. To overcome this challenge, we propose a novel fast non-exhaustive search method using WHD. The key idea is to design a constant sequence extension algorithm to perform each sequence extension in constant computational complexity and the total complexity is proportional to the sequence length, which is justified by theoretical analysis. Experimental results show that our method is faster than other WHD-based search methods. Also, compared with the HD-based non-exhaustive search method, our method has comparable efficiency but retrieves more query-related items for the dataset of up to one billion items.
... Several classic online hashing methods have emerged in recent years, and they are all data-dependent. Representative works include Online Hashing (OKH) [24], Adaptive Hashing (AdaptHash) [32], Online Supervised Hashing (OSH) [33], Online Hashing with Mutual Information (MIHash) [34], Balanced Similarity for Online Discrete Hashing (BSODH) [35], Supervised Online Hashing via Hadamard Codebook Learning (HCOH) [36], Hadamard Matrix Guided Online Hashing (HMOH) [37], etc. ...
... In this section, online hashing algorithms, such as OKH [24], AdaptHash [32], OSH [33], MIHash [34], BSODH [35], HCOH [36], HMOH [37], SketchHash [38] and FROSH [39] are introduced. Among them, all are supervised hashing methods except SketchHash and FROSH. ...
... Then, LSH is used to convert the codebook into a binary code adapted to the number of hash bits. In an improved version of HMOH [37], hash linear regression is processed as a binary-classification issue, and the case of multi-label is considered as well. ...
Online hashing is a valid storage and online retrieval scheme, which is meeting the rapid increase in data in the optical-sensor network and the real-time processing needs of users in the era of big data. Existing online-hashing algorithms rely on data tags excessively to construct the hash function, and ignore the mining of the structural features of the data itself, resulting in a serious loss of the image-streaming features and the reduction in retrieval accuracy. In this paper, an online hashing model that fuses global and local dual semantics is proposed. First, to preserve the local features of the streaming data, an anchor hash model, which is based on the idea of manifold learning, is constructed. Second, a global similarity matrix, which is used to constrain hash codes is built by the balanced similarity between the newly arrived data and previous data, which makes hash codes retain global data features as much as possible. Then, under a unified framework, an online hash model that integrates global and local dual semantics is learned, and an effective discrete binary-optimization solution is proposed. A large number of experiments on three datasets, including CIFAR10, MNIST and Places205, show that our proposed algorithm improves the efficiency of image retrieval effectively, compared with several existing advanced online-hashing algorithms.
... Although earlier hashing methods [13,14,20,24,27,40,43,58] are easy to apply in practice, their performance is typically inferior to more recent deep learning counterparts. Deep supervised hashing methods [6,12,31,35,36,55,57,63,66] usually achieve better performance over unsupervised ones by using additionally the semantic class labels. However, they are limited in scalability as class label annotation is costly and even impossible in extreme cases (e.g., rare objects). ...
Existing unsupervised hashing methods typically adopt a feature similarity preservation paradigm. As a result, they overlook the intrinsic similarity capacity discrepancy between the continuous feature and discrete hash code spaces. Specifically, since the feature similarity distribution is intrinsically biased (e.g., moderately positive similarity scores on negative pairs), the hash code similarities of positive and negative pairs often become inseparable (i.e., the similarity collapse problem). To solve this problem, in this paper a novel Similarity Distribution Calibration (SDC) method is introduced. Instead of matching individual pairwise similarity scores, SDC aligns the hash code similarity distribution towards a calibration distribution (e.g., beta distribution) with sufficient spread across the entire similarity capacity/range, to alleviate the similarity collapse problem. Extensive experiments show that our SDC outperforms the state-of-the-art alternatives on both coarse category-level and instance-level image retrieval tasks, often by a large margin. Code is available at https://github.com/kamwoh/sdc.
... However, most existing works fail to solve it and only a few solutions have been proposed. Some works rise to the challenge by means of offline coding strategies, such as Error Correcting Output Codes and Hadamard matrix (Lin et al. 2018(Lin et al. , 2020. Some methods try to increase the number of hash bits for better representation (Mandal, Annadani, and Biswas 2018). ...
With the vigorous development of multimedia equipments and applications, efficient retrieval of large-scale multi-modal data has become a trendy research topic. Thereinto, hashing has become a prevalent choice due to its retrieval efficiency and low storage cost. Although multi-modal hashing has drawn lots of attention in recent years, there still remain some problems. The first point is that existing methods are mainly designed in batch mode and not able to efficiently handle streaming multi-modal data. The second point is that all existing online multi-modal hashing methods fail to effectively handle unseen new classes which come continuously with streaming data chunks. In this paper, we propose a new model, termed Online enhAnced SemantIc haShing (OASIS). We design novel semantic-enhanced representation for data, which could help handle the new coming classes, and thereby construct the enhanced semantic objective function. An efficient and effective discrete online optimization algorithm is further proposed for OASIS. Extensive experiments show that our method can exceed the state-of-the-art models. For good reproducibility and benefiting the community, our code and data are already publicly available.
... However, most existing works fail to solve it and only a few solutions have been proposed. Some works rise to the challenge by means of offline coding strategies, such as Error Correcting Output Codes and Hadamard matrix (Lin et al. 2018(Lin et al. , 2020. Some methods try to increase the number of hash bits for better representation (Mandal, Annadani, and Biswas 2018). ...
With the vigorous development of multimedia equipment and applications, efficient retrieval of large-scale multi-modal data has become a trendy research topic. Thereinto, hashing has become a prevalent choice due to its retrieval efficiency and low storage cost. Although multi-modal hashing has drawn lots of attention in recent years, there still remain some problems. The first point is that existing methods are mainly designed in batch mode and not able to efficiently handle streaming multi-modal data. The second point is that all existing online multi-modal hashing methods fail to effectively handle unseen new classes which come continuously with streaming data chunks. In this paper, we propose a new model, termed Online enhAnced SemantIc haShing (OASIS). We design novel semantic-enhanced representation for data, which could help handle the new coming classes, and thereby construct the enhanced semantic objective function. An efficient and effective discrete online optimization algorithm is further proposed for OASIS. Extensive experiments show that our method can exceed the state-of-the-art models. For good reproducibility and benefiting the community, our code and data are already available in supplementary material and will be made publicly available.
... Yuan et al. [31] analyzed the limitations of the pairwise/triplet based hashing methods, and proposed the Central Similarity Quantization (CSQ) for image retrieval that not only captures the data similarity globally but also improves the learning efficiency. The existing central similarity methods utilize the row vectors or column vectors of Hadamard matrix to generate hash centers [17,3,15,31]. As shown in Figure 1 (a), a single center is used as the target to learn the hash function for single-label image retrieval. ...
Deep hashing has been widely applied to large-scale image retrieval by encoding high-dimensional data points into binary codes for efficient retrieval. Compared with pairwise/triplet similarity based hash learning, central similarity based hashing can more efficiently capture the global data distribution. For multi-label image retrieval, however, previous methods only use multiple hash centers with equal weights to generate one centroid as the learning target, which ignores the relationship between the weights of hash centers and the proportion of instance regions in the image. To address the above issue, we propose a two-step alternative optimization approach, Instance-weighted Central Similarity (ICS), to automatically learn the center weight corresponding to a hash code. Firstly, we apply the maximum entropy regularizer to prevent one hash center from dominating the loss function, and compute the center weights via projection gradient descent. Secondly, we update neural network parameters by standard back-propagation with fixed center weights. More importantly, the learned center weights can well reflect the proportion of foreground instances in the image. Our method achieves the state-of-the-art performance on the image retrieval benchmarks, and especially improves the mAP by 1.6%-6.4% on the MS COCO dataset.
... Balanced Similarity for Online Discrete Hashing (BSODH) [Lin et al. 2019] uses an asymmetric graph regularization to preserve the similarity between the streaming data and the existing dataset and update the hash function according to the graph. Online Supervised Hashing (OSupH) and Hadamard Matrix Guided Online Hashing (HMOH) [Lin et al. 2020a] explore how to generate the target codes according to the label information and update the hash functions according to the target codes. Although the current online hashing methods can achieve good search performance and are efficient in learning the hash functions online, they have to recompute the binary codes for the database when the hash functions are updated. ...
... If two data points are similar, the inner product of their corresponding vectors after linear transformation should be larger than that of dissimilar ones. To learn the similarity function using the point-wise label information, inspired by HMOH [Lin et al. 2020a], we can generate l-bit binary target code g ∈ {−1, 1} l according to the label information so that each column u ∈ R D in U is a projection vector and corresponds to one bit of the binary target code. So is each column v ∈ R b in V. ...
... Generating the l-bit binary target codes that can preserve the label information has been explored in some online hashing methods Weng and Zhu 2020c;Lin et al. 2020aLin et al. , 2018b. Here, we adopt HMOH [Lin et al. 2020a] to introduce the Hadamard matrix and regard each column of Hadamard matrix as the binary target code g ∈ {−1, 1} l for each class label, which by nature satisfies several desired properties of binary hash codes. ...
Online hashing methods usually learn the hash functions online, aiming to efficiently adapt to the data variations in the streaming environment. However, when the hash functions are updated, the binary codes for the whole database have to be updated to be consistent with the hash functions, resulting in the inefficiency in the online image retrieval process. In this paper, we propose a novel online hashing framework without updating binary codes. In the proposed framework, the hash functions are fixed and a parametric similarity function for the binary codes is learnt online to adapt to the streaming data. Specifically, a parametric similarity function that has a bilinear form is adopted and a metric learning algorithm is proposed to learn the similarity function online based on the characteristics of the hashing methods. The experiments on two multi-label image datasets show that our method is competitive or outperforms the state-of-the-art online hashing methods in terms of both accuracy and efficiency for multi-label image retrieval.
... However, such a setting cannot handle dynamic application scenarios where data are fed into the system in a streaming fashion. Therefore, online hashing has attracted much research attention recently [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37]. It aims to online update the hash functions from the sequentially arriving data instances, which merits in its superior adaptivity and scalability for large-scale online retrieval applications. ...
Online image hashing has received increasing research attention recently, which processes large-scale data in a streaming fashion to update the hash functions on-the-fly. To this end, most existing works exploit this problem under a supervised setting, i.e., using class labels to boost the hashing performance, which suffers from the defects in both adaptivity and efficiency: First, large amounts of training batches are required to learn up-to-date hash functions, which leads to poor online adaptivity. Second, the training is time-consuming, which contradicts with the core need of online learning. In this paper, a novel supervised online hashing scheme, termed Fast Class-wise Updating for Online Hashing (FCOH), is proposed to address the above two challenges by introducing a novel and efficient inner product operation. To achieve fast online adaptivity, a class-wise updating method is developed to decompose the binary code learning and alternatively renew the hash functions in a class-wise fashion, which well addresses the burden on large amounts of training batches. Quantitatively, such a decomposition further leads to at least 75% storage saving. To further achieve online efficiency, we propose a semi-relaxation optimization, which accelerates the online training by treating different binary constraints independently. Without additional constraints and variables, the time complexity is significantly reduced. Such a scheme is also quantitatively shown to well preserve past information during updating hashing functions. We have quantitatively demonstrated that the collective effort of class-wise updating and semi-relaxation optimization provides a superior performance comparing to various state-of-the-art methods, which is verified through extensive experiments on three widely-used datasets.
