Weight Sharing in Convolution Neural Networks ?

Dear @Nitesh Jindal,

It could be understood that in CNN, different filters would be applied to extract features map from the images. You could assume the pixels of filters (filters aka kernels are represented in the middle of the attached photo) as weights and they are fixed values according to filter types. During the convolution process, the filter passes through the whole image by holding the same weights. Therefore, it could be concluded that in CNN there would be weight sharing.

With bests,

Mansurbek Abdullaev

Dear Nitesh,

The answer and photo shared by Mansurbek Urol ugli Abdullaev is quiet explanatory. A CNN, is comprised of multiple filters. The filters are actually feature detectors. Each filter highlights a specific feature present in the image such as edges, vertical lines, horizontal lines. These filters are usually 2*2 or 3*3 matrices of weights that move across the image (layer input) in a sliding window pattern.

The weight sharing is referred to using fixed weights for each filter across the entire input.

Hello @Nitesh Jindal,

Each filter of 3x3 has 9 grid-boxes that each one of these gird box run through all possible options inside them, for Example each of 3x3 try the following :

000 001 111

010 & 010 & 101 & ...

000 100 111

By running all these filters the machine capture the edges, curves, ... and other features and running all these possible options are still in layer 1 filter. Then next layer let's say another 3x3 filter can learn more features let's say circles and so on. Deeper layers, means more layers, will give better accuracy, but longer computation time. So you amy need to find the optimum number of filter that honor your objective.

A CNN has multiple layers. Weight sharing happens across the receptive field of the neurons(filters) in a particular layer.Weights are the numbers within each filter. So essentially we are trying to learn a filter. These filters act on a certain receptive field/ small section of the image. When the filter moves through the image, the filter does not change. The idea being, if an edge is important to learn in a particular part of an image, it is important in other parts of the image too.

Fereshteh Hassani Mansurbek Urol ugli Abdullaev Thank you everyone, my understanding from the explanation is the weight sharing basically comes into picture when kernel is slided over the input tensor and it results in a dot product with the receptive input area to output a scalar value as in the diagram (it is 155 in diagram)