Content Based Image Retrieval - Science topic

Pierre De Handschutter FLAN or BF matchers with SIFT, SURF or ORB like key point descriptors would work reasonably only when the same or part of the query image is appeared (in different scale, orientation, illumination and with other variations) in the matching images. But that is not the case in Corel dataset hence those method would not work as expected. If you really need to use them, try using Bag-of-Feature models of SIFT, SURF and ORB with SVM like classifier. You may refer the following code.

Color histograms (including all the color spaces such as RGB, HSV and Lab etc) have their limits and its obvious they do not hold the necessary information about the shapes of the objects appeared in the image. Therefore no wonder about the poor results in color histograms based CBIR.

LBP is better at discriminating objects in a particular domain (like faces, vehicles etc) and it is vulnerable in general image retrieval applications where the images having highly cluttered background.

GLCM along has the same problem where it only characterizes the texture details which may not be the discriminating feature for a pair of images. Hu in the other hand should not use for natural image classification where it performs reasonably with single object having a well defined shape.

The best non deep learning based approach I found so far is the Fused Color Gabor Pyramidal Histogram of Oriented Gradients (FC-GPHOG) descriptor classified with nearest neighbor on Enhanced Fisher Model (EFM). You can find my implementation of FC-GPHOG from:

The code related to the work mentioned in the paper :

EFM is nothing but the applying LDA on truncated PCA projection of the FC-GPHOG descriptor.

Hello Ali,

Thank you for your answer. I've already read this paper actually and it was indeed an interesting comparison between several techniques, including CNN, but I was also looking for a comparison between DL and traditional ML such as SVM.

However, you're right that it is a good starting point. I also found this one : A Comparison of Active Classification Methods for Content-Based Image Retrieval

from Gosselin which compares several ML techniques (though no DL) and it seems interesting but I haven't read it in details yet.

Regards

This is an important question in computer vision.

Cutting edge CBIR research is being done in the following areas:

Colour images:

Chawki, Youness(MRC-UMLIST-C); El Asnaoui, Khalid(MRC-UMLIST-C); Ouanan, Mohammed(MRC-UMLIST-C); Aksasse, Brahim(MRC-UMLIST-C)

Content frequency and shape features based on CBIR: application to colour images. Int. J. Dyn. Syst. Differ. Equ. 8 (2018), no. 1-2, 123–135

According to the authors, the precision average can be reached as 74.49%, which is surprisingly low.

Distance:

Herwindiati, Dyah E.; Isa, Sani M.; Sagara, Rahmat

The new notion distance of content based image retrieval (CBIR). (English, Indonesian summary) J. Indones. Math. Soc. 16 (2010), no. 1, 51–67.

This paper introduces a notion of distance in the CBIR process that is derived from the measure of multivariate dispersion called vector variance called the minimum vector variance (MVV) estimator.

thank for all your replay please suggest real time problem and which image should i take

Hello, Vijaylakshmi Sajwan

Congratulations to you have successfully formulated a distance model in CBIR. Please you can share with us for what kind of imagery, and how to formulate the distance. Thank you

Here are their policy https://www.instagram.com/developer/ (I don't see any info that told you need a commercial account to access API):

Before you start using the API Platform, we have a few guidelines that we'd like to tell you about. Please make sure to read the full Platform Policy. Here's what you'll read about:

Check an alternative description at the appendix of the following paper:

Krishnamachari S., Yamada A., Abdel-Mottaleb M., Kasutani E. (2000) Multimedia Content Filtering, Browsing, and Matching Using MPEG-7 Compact Color Descriptors. In: Laurini R. (eds) Advances in Visual Information Systems. VISUAL 2000. Lecture Notes in Computer Science, vol 1929. Springer, Berlin, Heidelberg

(similar version available at)

The chisquare attack is a simble and famous method to test the Robustness of the security system against the attacks. It depend on probability analysis of stego image after put the information inside it using LSB method then compare it with the probability analysis of original image to check the difference between them. If the difference near zero it means no information inside the image otherwise if near one it means the image held the information.

Best regard

Zeyad Safaa Younus Saffawi

Hello.

Kindly check the attachment. Hope that will be of some help to you.

Regards

Rebika Rai

Dear Manisha mam,

I got some of the video data sets mentioned above. But I am not happy with those data sets. Have you found any other data sets

principle component analysis or PCA and independent component analysis or ICA are among the most common methods for dimension reduction

I suppose that you multiplied values in MATLAB. I think that your signal samples are the numbers of 'uint8' type. In order to multiply signal samples properly you have to convert them on 'double' type. I checked your example in MATLAB Command Window. The listing is below:

>> a=uint8(17)

a=

17

>> 64*a

ans =

255

>> 64*double(a)

ans=

1088

I hope that this is solution of your problem.

best regards

Robert Wielgat

The inverted index should map each unique word in the vocabulary tree to a list of image ids in the database, sorted in an increasing order.

Mustafa,

As said by Khan Muhammad, the current trend in classification and detection on images is motivated by Deep Learning, precisely, Convolutional Deep Neural Networks. In case you have an abundance of data to train, then go ahead with models like Alexnet but in case, you do not have sufficient data, then learn about already trained caffenet and retrain it with your data so that it adjusts itself to your problem domain. 

You can train models on Caffe/Theano frameworks and use their wrappers in your C# application for inference(testing) phase.

Hope this helps.

kind regards,

Ankit Tripathi

 https://sites.google.com/site/dctresearch/Home/content-based-image-retrieval

Hello Dear

Have a good day. I hope you are good. please use this code and change images

For example the contour of a circle.

It can be made of dots. An edge

detector would recognize each dot as

a very short edge, but this is not

what we want. To mitigate the problem

one could apply blur and a threshold

then; now the edge detector would

yield something useful.

A different approach without edge

detector would be a snake, some kind

of springy lasso that tries to minimize

mechanical energy by adapting to the

contour with the help of its shape

memory.

What also would work is a Hough

transformation that has an

accumulator space with 3 dimensions

for x,y, radius. That method is

called generalized Hough transformation,

but for every type of contour a

specialized accumulator and way

of filling it must be defined.

Regards,

Joachim

The h-index can be calculated automatically in Web of Science and Scopus or manually in other databases that provide citation information (e.g. SciFinder, PsychINFO, Google Scholar). The index is based on a list of publications ranked in descending order by the number of citations these publications received. The value of h is equal to the number of papers (N) in the list that have N or more citations.

Before you can calculate your h-index, you will need a complete publication list.

Calculating your citations and H-index at Scholar.google.com kindly go through this link: https://www.wageningenur.nl/upload_mm/3/1/c/890c6a4a-4ce2-479f-98f0-fabb7ac92a54_Calculating%20your%20citation%20index%20with%20Google%20scholar.pdf

You can refer some tools for that. or 

You can request your colleagues and friends to draw the outline or border of the regions. Then you can integrate all the outlines and form the ground truth. I do not remember the tool for drawing outline manually. you can google it.

The following content maybe helpful. There are several algorithms to compute the quality score of an image.

It depends what you are looking for. The specific goal of the CBIR for the lung (x-ray or CT) has to come from the clinical side. For instance, the challenge for finding similar cases when we are dealing with pulmonary nodules is mainly the low discrimination between the suspicious mass and surrounding regions, specially in x-ray. In such cases some sort of pre-processing is required first to amplify the nodule boundary against the surroundings. The actual search can be executed afterwards.    

 Thank you a lot Mr France.

I whould like to execute vl_sift but i  don't know how to do that?

Matlab showed the error message "Attempt to execute SCRIPT vl_sift as a function:"

From view of  textural analysis the class of medical images could be considered as a grand class covers a wide spectrum of textural images, some of images can be effectively considered in terms of edge density, others in terms of shading characteristics, others in terms of color spectral behaviors, others in terms of the geometrical (uniform or non-uniform) behaviors, and so on.

So, I think it is better to identify the types of medical sub-classes you want to work on, then decide to choose the proper type(s) of features you have to depend on for making the retrieval decisions.

Regards... 

The ANMRR method by MPEG-7 standard. You'll need the ground-truth of the images.

Introduction to MPEG-7: Multimedia Content Description Interface, B. S. Manjunath et al, ISBN: 978-0-471-48678-7. Pag 183-185

Best

In order to design features for image retrieval, I think it's very important to first understand what makes two of these images similar.  I have no knowledge of these types of images so I can't help you with this, but there could be many possibilities, such as histograms of the image, shapes of the branches, number of branches, etc.  It's always better to be able to understand the basic idea of what these images similar and then design features to encode this information.

A simple distance method for CBIR is a predefined metric used to find the best similarity between a query image and a number of images in a database. e.g. Euclidean, Manhattan, Chi square, Mahalanobis, etc.

On the other hand. the learned distance methods attempt to find the best metric to achieve the best query result, the distance is learned with specialized algorithms such as Large Margin Nearest Neighbor or Neighbourhood components analysis.

I found a new metric which is invariant to data scale, noise and outliers, it is perfect for CBIR systems, a student of mine is making excellent results using this distance, you may find the paper at:

Mohamed,

Yeah that is d simple code to apply threshold ....

did it  helps you?

Dear Nisreen

You need to define each image by a feature vector, those features should be invariant to scale, rotation and translation, and you may perform the following steps:

1- Extract some features from the query image, and save it an array called QArray.

2- For each image in the database, extract the same features in step 1 in the same order and save the features in array called ImageArray_i.

3- For each ImageArray_i , calculate the similarity with QArray, using a similarity (distance) function

4- Define a threshold t,

5- For each i in ImageArray_i  if the distance from QArray to ImageArray_i < t, then retrieve image i, else do nothing,

====

What are the features?

There are hundreds of them, you may use:

1- color histogram (probabilities of the RGB colors in the image

2- you may use the intensity histogram, (probabilities of the (R+G+B)/3 in the image

3- Statistic features, such as mean, mode, media stdev, moments, uniformity, entropy, etc.

4- Hundreds more, read about them.

====

What are the similarity measures?

1- Euclidian distance

2- Manhattan distance

3- Hassanat Distance, this one is invented by me, and it is better than the previous ones because it is invariant to the outliers. you may find the link of the paper at: https://www.researchgate.net/publication/264995324_Dimensionality_Invariant_Similarity_Measure

4- tens of other distances..

Contributions in such research is highly dependent on the features used.

For your first move, you may start with the color features.

Please see some papers we published listed on my page. our research results indicate that you need to use a large number of clusters for images with complex colour and texture variations. This finding coincides with the so-called "bag of words" approach for CBIR.

This is a good question with many possible answers.

A good place to start looking for an answer is in

J.-M. Chang, Classification of the Grassmannians: Theory and Applications, Ph.D. thesis, Colorado State University, 2008:

See Section 3.1, starting on page 19: Matrix representation of points on the Grassmann manifold. 

I have to break off at this point but will return to this interesting question a bit later.

(Unfortunately I could not attach the doc file containing the following text. Therefore I just pasted the text here.)

 First order histogram: (MATLAB)

      im = imread('tire.tif');

  nf = 20; featureVec = imhist(im, nf); % with any arbitrary value of nf

  % OR

  featureVec = imhist(im); % with default nf=256

 Second order histogram, here GLCM (MATLAB)

  im = imread('tire.tif');

  glcm = graycomatrix(im);

  temp = graycoprops(glcm);

  featureVec(1) = temp.Contrast;

  featureVec(2) = temp.Correlation;

  featureVec(3) = temp.Energy;

  featureVec(4) = temp.Homogeneity;

 First order histogram: (MATLAB)

  im = imread('tire.tif');

  [Nx, Ny] = size(im); % suppose the image is a single-band image

  w = 3; % neighborhood window --> (2w+1)-by-(2w+1)

  nf = 50;

  extIm = padarray(im, [w, w], 'symmetric');

  featureVec = zeros(Nx, Ny, nf); % memory allocation

  for x = 1+w:Nx+w

      for y = 1+w:Ny+w

         WIN = extIm(x-w:x+w, y-w:y+w);

         featureVec(x-w, y-w, :) = imhist(WIN, nf);

     end

  end

 Second order histogram, here GLCM (MATLAB)

   im = imread('tire.tif');

   [Nx, Ny] = size(im); % suppose the image is a single-band image

   w = 3; % neighborhood window --> (2w+1)-by-(2w+1)

   extIm = padarray(im, [w, w], 'symmetric');

   nf = 4; % graycoprops gives 4 features

   featureVec = zeros(Nx, Ny, nf);

   for x = 1+w:Nx+w

      for y = 1+w:Ny+w

         WIN = extIm(x-w:x+w, y-w:y+w);

         glcm = graycomatrix(WIN);

         temp = graycoprops(glcm);

         featureVec(x-w, y-w, 1) = temp.Contrast;

         featureVec(x-w, y-w, 2) = temp.Correlation;

         featureVec(x-w, y-w, 3) = temp.Energy;

         featureVec(x-w, y-w, 4) = temp.Homogeneity;

      end

   end

I was using IBSR database for segmentation purposes: http://www.nitrc.org/projects/ibsr

I've found some more links:

Hi  Abdul Hameed, 

Thank you for your reponse.

I would like to  know how does the Relevance  Feedback work in the attached code .

In my research field, Average Precision (for each query) and Mean Average Precision (for all queries) are the most widely used. They reflect both precision and recall information.

If we can regard the "retrieval" method as a solution of "recognition", we can also use ROC curve and/or AUC as the measure. They also reflect both precision and recall information.

Other measures include discounted cumulative gain, mean reciprocal rank, etc., but I never saw any people really used them in their studies (at least in my research field).

I am not sure if understand you correctly, but I am afraid you are confusing some things.

A similaritry measure is not the outcome of the classification. The result of your classifier is a class assignment for each instance / sample. A similarity measure can be used for this class assignment. If I understand you correct you got a typical 2-class classification scenario, you got a set of samples and you want to decide if the samples either belong to class A or class B.

In a nutshell:

- calculate the features for each of your samples

- this results in a n-dimensional feature vector as a representation of each sample in feature space

- find the features which are most discriminative and weight them higher

- find a rule (e.g. a similarity measure) to distinguish between class A and B

There are lots approaches for similarity measures. A very simple example would be: if you have two feature vectors you can compare them by calculating the euclidean distance (or any other distance measure) between them.

There are lots approaches for weighting features, a very simple example would be to use weighting factors. For example, if you have two features 'a' and 'b' and 'b' is more discriminative than 'a', you could use a weight of 0.5 for feature 'a' and 2.0 for feature 'b' when making the decission. This would cause feature 'b' to have 4-times the impact on the classification decission compared to feature 'a'.

A small and very simple example:

feature vector 1: {3, 2}

feature vector 2: {1, 5}

euclidean distance between 1 and 2: (3-1)^2 + (2-5)^2 = 4+9 = 13

distance = 13

now we weight feature 'a' with 0.5 and 'b' with 2.0: 0.5((3-1)^2) + 2.0((2-5)^2) = 2+18 = 20

distance = 20

I hope that helps a bit ...

Hi,

I'm not sure about relevance feedback, but the "most discriminative" feature should be a feature that can be used to help tell the deference between the target classes. The "most dominant" feature would be the one that occurs the most. The most dominant may not be the most discriminative, for instance in document classification the most dominant feature, in a bag-of-words representation, maybe stop words like "the", "and", "it", but these are not discriminative as most documents will have them, whereas the feature "tennis" may be highly discriminative in telling whether a document is about the game of tennis, but may occur less often than the stop words.

As far as I understand you have images

of the skin, say from arms. Not from the scalp.

Three methods come in mind for removing

the hair: outlier detection (statistics), or

edge detection (Canny algorithm), or

clustering in RGB histograms: hairs should

cause small sharp peaks.

After recognition you can replace the hairs

by an interpolated hue from the near environment.

A more sophisticated approach would try

to identify the hair as an object and could

follow the trajectories of the hairs for

doing the removal.

Regards,

Joachim

I think the previous two references provided by @Christos P Loizou and @Lucia Ballerini are new and excellent to start with.

The question is so general which makes it trivial to provide a specific answer. For example, Neural Network has been widely used for years, while it is still being enhanced (for example: Convolutional NN). Among most recent image classification approaches, sparse representation has recently opened a new demanding research area. The following paper is the latest one:

G. Shenghua, I. W. Tsang, M. Yi, "Learning Category-Specific Dictionary and Shared Dictionary for Fine-Grained Image Categorization,", IEEE Transaction on Image Processing, vol.23, no.2, pp.623,634, Feb. 2014

I think classification based on sparse representation is an easier choice for proposing innovations and enhancements, compared to NN and KSVM which are somehow saturated.

There are a number of ways to extract key frames from a video. You can do so for example in MATLAB by:

1. Calculating the histogram difference between two consecutive frames. If the difference is greater than a threshold (selected by user) then consider the frame as a key. You do this for all frames in the video.

2. Another way is using the image entropy. Calculate the Entropy of each frame. Whenever the difference between two consecutive frames is more than a Threshold then mark this frame as a shot. Take for example the frames having maximum Entropy.

Thank you somuch. I will try to do it.....

Circular harmonics are a solution to Laplace's equation in polar coordiniates. This is a planar solution to the spatial solution provided by spherical harmonics (http://en.wikipedia.org/wiki/Spherical_harmonics ). A nice introduction to Circular solutions is given by Christopher Baird of U. Mass-Lowell in his EM lecture notes. ( http://faculty.uml.edu/cbaird/95.657(2011)/Lecture4.pdf ).

Dear Latch Shun, I am not sure that how the following links are helpful for you. With Good Wishes. Good Luck!

Using LIBSVM, you will get only prediction and accuracy of the class. You need to train each class examples.

Have a first start by looking at the work of the team of Prof. Stiefelhagen at KIT: http://cvhci.anthropomatik.kit.edu/research/multimedia-analysis

MPEG7 is about the standardization of Image Content description: http://www.micc.unifi.it/delbimbo/wp-content/uploads/2011/03/reference_papers/ieee_multimedia_mpeg-7_part1.pdf