Figure 1 - available via license: CC BY
Content may be subject to copyright.
Unlike people with normal vision system, people with dichromatic vision system are able to easily recognize the word ''NO.''
Source publication
Color vision deficiencies affect visual perception of colors and, more generally, color images. Several sciences such as genetics, biology, medicine, and computer vision are involved in studying and analyzing vision deficiencies. As we know from visual saliency findings, human visual system tends to fix some specific points and regions of the image...
Contexts in source publication
Context 1
... this section, we describe the steps of our work, starting from the eye-tracking session, we aim at determining the most meaningful differences between normal and color visiondeficient systems with respect to a fixed number of images, then we tackle the segmentation and the recoloring of the regions with different saliency levels; at last, a further eye-tracking session assesses the enhancement of the images. We point out that we are interested in detecting differences in HVS behavior among people with normal vision system and people affected by color vision deficiencies (Figure 1 shows that only dichromatic people will be able to easily recognize the word NO standing out from the background). ...
Context 2
... the purpose of our experiments, we created an ad hoc dataset by merging almost 90 images from different public datasets: MIT1003 (Judd et al., 2009a), CAT2000 All the images have in common a prevalence of red-green chromatic contrasts (of interest for protanopia and deuteranopia deficiencies); we did not take into account images with yellow-blue chromatic contrast because we did not have available people affected by tritanopia vision deficiency. All the fixation point maps we collected during two experimental sessions have been gathered into a public available ground-truth under the name of EToCVD (Bruno et al., 2018). ...
Context 3
... some imperfections in the recoloration of the segmented region (Figure 8(a)), we noticed that the overall distribution of the fixation points (Figure 8(b), (c), and (d)) over the images is closer to the corresponding ground truth map than the distribution of the fixation points obtained during the first eye-tracking session. As shown in Figure 11(a), (b), (c), and (d), the eye movements of the observers affected by protanopia and deuteranopia can be really different than those of people with normal color vision system. The interesting thing is that, analyzing the improvements obtained with our enhancement method by observing the fixation points map of subjects affected by protanopia (Figure 12(c)) and deuteranopia (Figure 12(d)), the improvement is noticeable because the fixation points are quite closer to our ground truth (Figure 11(a)). ...
Context 4
... shown in Figure 11(a), (b), (c), and (d), the eye movements of the observers affected by protanopia and deuteranopia can be really different than those of people with normal color vision system. The interesting thing is that, analyzing the improvements obtained with our enhancement method by observing the fixation points map of subjects affected by protanopia (Figure 12(c)) and deuteranopia (Figure 12(d)), the improvement is noticeable because the fixation points are quite closer to our ground truth (Figure 11(a)). ...
Context 5
... shown in Figure 11(a), (b), (c), and (d), the eye movements of the observers affected by protanopia and deuteranopia can be really different than those of people with normal color vision system. The interesting thing is that, analyzing the improvements obtained with our enhancement method by observing the fixation points map of subjects affected by protanopia (Figure 12(c)) and deuteranopia (Figure 12(d)), the improvement is noticeable because the fixation points are quite closer to our ground truth (Figure 11(a)). ...
Context 6
... shown in Figure 11(a), (b), (c), and (d), the eye movements of the observers affected by protanopia and deuteranopia can be really different than those of people with normal color vision system. The interesting thing is that, analyzing the improvements obtained with our enhancement method by observing the fixation points map of subjects affected by protanopia (Figure 12(c)) and deuteranopia (Figure 12(d)), the improvement is noticeable because the fixation points are quite closer to our ground truth (Figure 11(a)). ...
Context 7
... results reached an average score increase of approximately 0.08 AUC and 0.5 NSS (both excluding the first 200 milliseconds). Figure 13 shows some examples with different AUC and NSS values related to quite meaningful images with the corresponding eye-tracking fixation point map of the first eye-tracking session and the fixation point map related to the second eye-tracking session, giving us a visual and qualitative demonstration of the improvement we achieved. In Figure 10, we plotted the histogram graph of AUC and NSS average score increase with respect to the deuteranopia case study and we showed meaningful images and the corresponding fixation point maps. ...
Context 8
... 13 shows some examples with different AUC and NSS values related to quite meaningful images with the corresponding eye-tracking fixation point map of the first eye-tracking session and the fixation point map related to the second eye-tracking session, giving us a visual and qualitative demonstration of the improvement we achieved. In Figure 10, we plotted the histogram graph of AUC and NSS average score increase with respect to the deuteranopia case study and we showed meaningful images and the corresponding fixation point maps. It is noticeable that in the case of observers with protanopia, we reached an average score increase of approximately 0.05 AUC and 0.3 NSS (both excluding the first 200 milliseconds). ...
Context 9
... applied a color mapping function who takes into account both deuteranopia and protanopia effect, and we looked for a trade-off mapping function allowing to achieve the best improving results for both kind of color vision deficiencies. So far, results showed a better Figure 12. The enhancement assessment on (a and b) the images is supported by the fixation point maps for observers with (c) protanopia and (d) deuteranopia. ...
Context 10
... enhancement assessment on (a and b) the images is supported by the fixation point maps for observers with (c) protanopia and (d) deuteranopia. improvement for people affected by deuteranopia (in Figure 14 you can see some experimental results with respect to deuteranopia case study); this may be explained by referring to the effectiveness of the negative color mapping as in Equation 9 that is more appropriated with respect to deuteranopia than protanopia. We will be focusing on two different mapping functions to be tuned on the two color deficiencies differently. ...
Context 11
... reason behind is that the saliency method we adopted as saliency map extraction is mainly based on the number of scale invariant feature transform (SIFT) keypoint (Lowe, 1999) detected in the image which in turn depends on the size of the image and on the textured regions in the image (the finer the texture in the image, the greater the number of scale-invariant feature transform keypoints we have in the image). The experiments have been conducted by using a Tobii EyeX eye-tracker recording the eye movements with a sampling rate of about 55 Hz; the data have been processed in MathWorks Figure 13. For a given image (first column), we collected the fixation points from normal observer (second column), from observers with protanopia (third column), from observers with protanopia looking at the enhanced image (fourth column). ...
Context 12
... used a workstation with a quad-core 2.4 GHz processor and 16 GB of RAM for our experiments. Figure 14. For a given image (first column), we collected the fixation points from normal observers (second column), from observers with deuteranopia (third column), from observers with deuteranopia looking at the enhanced image (fourth column). ...
Citations
... The case study described in this paper focuses on employing a webcam-based eye-tracking platform for detecting how eye movements differ between people with colour vision deficiency (CVD) and those with normal vision (as seen in Figure 1). A previous study conducted in 2019 based on an infrared eye-tracking campaign revealed different visual attention patterns between participants who had dichromatic vision deficiency and those with normal vision [Bruno et al. 2019]. We are interested in replicating these results with webcam eye tracking, i.e., based on ordinary optical cameras without infrared support, motivated by the scalability of such a solution. ...
Webcam-based eye-tracking platforms have recently re-emerged due to improvements in machine learning-supported calibration processes and offer a scalable option for conducting eye movement studies. Although not yet comparable to the infrared-based ones regarding accuracy and frequency, some compelling performances have been observed, especially in those scenarios with medium-sized AOI (Areas of Interest) in images. In this study, we test the reliability of webcam-based eye-tracking on a specific task: Eye movement distribution analysis for CVD (Colour Vision Deficiency) detection. We introduce a new publicly available eye movement dataset based on a pilot study (n=12) on images with dominant red colour (previously shown to be difficult with dichromatic AOI to investigate CVD by comparing attention patterns obtained in webcam eye-tracking sessions). We hypothesized that webcam eye tracking without infrared support could detect differing attention patterns between CVD and non-CVD participants and observed statistically significant differences, allowing the retention of our hypothesis.
... The output of saliency models is used in several topics such as image quality assessment [1,2,[6][7][8], image and video compression [29], image captioning and description [9], image search and retrieval [36], image enhancement for people with CVD (Colour Vision Deficiency) [5], saliency led painting restoration [37] and so forth [12]. ...
Cultural heritage understanding and preservation is an important issue for society as it represents a fundamental aspect of its identity. Paintings represent a significant part of cultural heritage, and are the subject of study continuously. However, the way viewers perceive paintings is strictly related to the so-called HVS (Human Vision System) behaviour. This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. In further details, we introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans, including the fundamental understanding of a scene, and then extend it to painting images. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention. We use an FCNN (Fully Convolutional Neural Network), in which we exploit a differentiable channel-wise selection and Soft-Argmax modules. We also incorporate learnable Gaussian distributions onto the network bottleneck to simulate visual attention process bias in natural scene images. Furthermore, to reduce the effect of shifts between different domains (i.e. natural images, painting), we urge the model to learn unsupervised general features from other domains using a gradient reversal classifier. The results obtained by our model outperform existing state-of-the-art ones in terms of accuracy and efficiency.
... 1 Code and dataset will be availble here : https://github.com/ kmamine/SSLArtScanpath 2 Funded by the TIC-ART project, Regional fund (Region Centre-Val de Loire) modelling and prediction of saliency and scanpaths became a cornerstone task that improves the efficiency of many other computer vision applications like indoor localization [19], image quality [1], image watermarking [25], image compression [39],image search and retrieval [49] or image enhancement for people with CVD (Colour Vision Deficiency) [9]. ...
In our paper, we propose a novel strategy to learn distortion invariant latent representation from painting pictures for visual attention modelling downstream task. In further detail, we design an unsupervised framework that jointly maximises the mutual information over different painting styles. To show the effectiveness of our approach, we firstly propose a lightweight scanpath baseline model and compare its performance to some state-of-the-art methods. Secondly, we train the encoder of our baseline model on large-scale painting images to study the efficiency of the proposed self-supervised strategy. The lightweight decoder proves effective in learning from the self-supervised pre-trained encoder with better performances than the end-to-end fine-tuned supervised baseline on two painting datasets, including a proposed new visual attention modelling dataset.
... Due to its broad relevance, predicting human eye movement patterns and visual saliency has an impressive range of applications in computer vision and related fields such as image compression [35], image captioning [22], image retrieval [31], image re-targeting [61], quality assessment of multimedia content (i.e. image [3], [17], [18], stereo [62], 3D meshes [?], [1], etc.), remote sensing [30], watermarking [34], map viewing [51] [5], indoor localization [29], perception [14], image enhancement [12], [19], healthcare [38] among many others. ...
Human visual Attention modelling is a persistent interdisciplinary research challenge, gaining new interestin recent years mainly due to the latest developments in deep learning. That is particularly evident insaliency benchmarks. Novel deep learning-based visual saliency models show promising results in capturinghigh-level (top-down) human visual attention processes. Therefore, they strongly differ from the earlierapproaches, mainly characterised by low-level (bottom-up) visual features. These developments account forinnate human selectivity mechanisms that are reliant on both high- and low-level factors. Moreover, the twofactors interact with each other. Motivated by the importance of these interactions, in this project, we tacklevisual saliency modelling holistically, examining if we could consider both high- and low-level featuresthat govern human attention. Specifically, we propose a novel method SAtSal (Self-Attention Saliency).SAtSal leverages both high and low-level features using a multilevel merging of skip connections duringthe decoding stage. Consequently, we incorporate convolutional self-attention modules on skip connectionfrom the encoder to the decoder network to properly integrate the valuable signals from multilevel spatialfeatures. Thus, the self-attention modules learn to filter out the latent representation of the salient regionsfrom the other irrelevant information in an embedded and joint manner with the main encoder-decodermodel backbone. Finally, we evaluate SAtSal against various existing solutions to validate our approach,using the well-known standard saliency benchmark MIT300. To further examine SAtSal’s robustness onother image types, we also evaluate it on the Le-Meur saliency painting benchmark.
... The focus of recent research is on Daltonization based on image content. Content-dependent categories include histogram-based [5], neighbourhood-based [20,21], and clustering-based [22][23][24] methods; optimization [13,[25][26][27][28][29][30], [31,32], and feature vector-based [33] methods can more effectively analyse image colour information. In content-dependent methods, the final pixel colour value depends on the initial colour area of the image, the histogram, or the spatial location of the pixels. ...
To help CVD (colour vision deficiency) observers distinguish among different colours in digital images, this study proposes a content‐dependent Daltonization algorithm based on lightness and chroma information. This improves the trade‐offs in contrast, naturalness, and colour consistency deficiencies found in existing methods. Chroma remapping and brightness adjustments on image content are utilized to increase contrast and maximize the preservation of the original hue. In a quantitative study, the proposed method proved more useful for dichromats than did existing methods. The evaluation by research subjects showed that the method was superior to the current methods in balancing contrast, naturalness, and colour consistency.
... Deficiencies [22] The contributions of this work is to detect the main differences between the aforementioned human visual systems related to color vision deficiencies by analyzing real fixation maps among people with and without color vision deficiencies. Another contribution is to provide a method to enhance color regions of the image by using a detailed color mapping of the segmented salient regions of the given image. ...
Color blindness is a condition that affects the cones in the eyes, it can be congenital or acquired and is considered a medium disability that affects about 8.5% of the world population and it occurs in children, who have special difficulties to the newly enter an educational environment with materials developed for people with normal vision, this work focuses on the development one technology , to allow people with a visual disability known as color blindness, to improve their daily activities which in turn leads to a better inclusion in the social and educational environment. To help the inclusion of these people an application was made that allows an identification of the type of condition of the user through the Ishihara test, which worked with two versions, the traditional and a variation to work with children, and subsequently the result It is taken into account to work with another Augmented Reality application which first uses an identification of the colors of an image through a color classification system, for this different algorithms were implemented one with automatic programming and another with a particle swarm optimization (PSO), once the algorithm identifies the colors it modifies them in real time to a spectrum of colors that if distinguishable by the student but at the same time identifies the centroids of the objects and labels them in real time with the real color, two forms were used for labeling, the word with the color and a color code ColorADD proposed or by Neiva in 2008.
... As many searches show the results that "Image Thresholding" can be used to detected or inspected an abnormality, which appear on an object, by converting grayscale image to binary image (black & white image). Hence, this technique is the most effective in image with high level of contrast [1] - [4]. Figure 1 is shown the result of input image after applied an image thresholding. ...
This paper presented an approach of Digital Image Processing (DIP) and measurement properties of image region for inspecting the part in Head Gimbal Assembly (HGA) process. Current, the inspection process is performed by the skilled employee, which depends on employee experiences. The inspection method proposed by using digital image processing to do an image enhancement in order to detect the abnormalities and defects on part. The measurement properties of image region is proposed to identify the abnormality occurred on HGA part after image enhancement, in production process. This research aimed to develop the algorithm to inspected HGA part by considered the number of pads, convex area value, and eccentricity value. Then used measurement properties of image region to enhance the algorithm for classifying part in production process. This approach has been developed and proved with the real part in HGA production process. It’s proved that the algorithm and method proposed can be classified the part in HGA production process.
Despite the existence of numerous methods for recoloring images with diverse effects, challenges such as unnatural and inharmonious colors of the converted objects still persist. To address these issues, we have developed a novel approach to image recoloration. Our method ensures that the resulting images possess three crucial properties: naturalness, harmonization, and distinguishability, making them accessible to individuals with color vision deficiencies. Our approach comprises two main components: recommended palette generation and image recoloring. The former allows us to learn the color distribution of various natural objects, while the latter enables us to recolor the image with the recommended palette. Our results demonstrate that our method outperforms existing approaches to some extent and warrants further exploration.
The Region of interest (ROI) analysis is widely used in image analytics, video coding, computer graphics, computer vision, medical imaging, nuclear medicine, computer tomography and many other areas in medical applications. This ROI determination process using subjective method (e.g. using human vision) often differ from the objective ones (e.g. using mathematical modelling). However, there is no existing method in the literature that could provide a single decision when both methods’ ROI data is available. To address this limitation, a robust algorithm is developed by combining the human eye tracking (subjective) and the graph-based visual saliency modelling (objective) information to determine a more realistic ROI for a scene. To carry out this process, in one hand, several different independent human visual saliency factors such as pupil size, pupil dilation, central tendency, fixation pattern, and gaze plot for a group of twenty-two participants are collected by applying on a set of publicly available eighteen video sequences. On the other hand, the features of Graph based visual saliency (GBVS) highlights conspicuity in the scene. Gleaned from these two pieces of information, the proposed algorithm determines the final ROI based on some heuristics. Experimental results show that for a wide range of video sequences and compared to the existing deep learning based (MxSalNet) and depth pixel (DP) based ROI, the proposed ROI is more consistent to the benchmark ROI, which was previously decided by a group of video coding experts. As the subjective and objective options frequently create an ambiguity to reach a single decision on ROI, the proposed algorithm could determine an ultimate decision, which is eventually validated by experts’ opinion.
