Figure 14 - uploaded by Graca Bressan
Content may be subject to copyright.
Performance comparison based on MOS values between DASH algorithms with and without considering SDF.
Source publication
Dynamic adaptive streaming over HTTP (DASH) has become a promising solution for video
delivery services over the Internet in the last few years. Currently, several video content
providers use the DASH solution to improve the users’ quality of experience (QoE) by
automatically switching video quality levels (VQLs) according to the network status.
Ho...
Similar publications
Popularity of streaming services for gaming videos has increased tremendously over the last years, e.g. Twitch and Youtube Gaming. Compared to classical video streaming applications, gaming videos have additional requirements. For example, it is important that videos are streamed live with only a small delay. In addition, users expect low stalling,...
HTTP Adaptive Streaming (HAS) is the de facto standard for video streaming services over the Internet. In HAS, each video is temporally segmented and stored in different qualities. The client selects the quality level for every video segment based on network conditions, allowing a smooth playback with the best possible Quality of Experience (QoE)....
![Figure 2-2 Tree structure [20]](profile/Abiy-Biru-Chebudie/publication/317570242/figure/fig2/AS:504873226641408@1497382565222/2-Tree-structure-20_Q320.jpg)
![Figure 2-1 Datasets in machine learning [13]](profile/Abiy-Biru-Chebudie/publication/317570242/figure/fig1/AS:504873227042816@1497382565178/1-Datasets-in-machine-learning-13_Q320.jpg)
The video streaming applications contribute to a major share of the Internet traffic. Consequently, monitoring and management of video streaming quality has gained a significant importance in the recent years. The disturbances in the video, such as, amount of buffering and bitrate adaptations affect user Quality of Experience (QoE). Network operato...
Managing quality of experience (QoE) is now widely accepted as a critical objective for multimedia applications and the supporting communication systems. In general, QoE management encompasses: (1) monitoring of the key influence factors and QoE indicators, and (2) deciding on the appropriate control actions as specified by the management goal. Man...
Uploading Virtual Reality (VR) video over cellular networks is expected to boom in near future, as general consumers could generate high-quality VR videos with portable 360-degree cameras and are willing to share with others. Consequently, concerns of uplink bandwidth and delay arose for current popular technology of tile-based VR video streaming,...
Citations
... • Stalling events are generally assumed to have the worst impact on QoE [6], [7] • Multiple quality switches have a negative effect on QoE [1], [8] • High amplitude quality switches decrease the QoE [1], [3] Nevertheless, no research has compared multiple quality switches, high amplitude quality switches, and stalling events, which have been proven to impact QoE negatively. This paper has conducted evaluations and analyses to provide further insights into these areas. ...
HTTP Adaptive Streaming (HAS) is nowadays a popular solution for multimedia delivery. The novelty of HAS lies in the possibility of continuously adapting the streaming session to current network conditions, facilitated by Adaptive Bitrate (ABR) algorithms. Various popular streaming and Video on Demand services such as Netflix, Amazon Prime Video, and Twitch use this method. Given this broad consumer base, ABR algorithms continuously improve to increase user satisfaction. The insights for these improvements are, among others, gathered within the research area of Quality of Experience (QoE). Within this field, various researchers have dedicated their works to identifying potential impairments and testing their impact on viewers' QoE. Two frequently discussed visual impairments influencing QoE are stalling events and quality switches. So far, it is commonly assumed that those stalling events have the worst impact on QoE. This paper challenged this belief and reviewed this assumption by comparing stalling events with multiple quality and high amplitude quality switches. Two subjective studies were conducted. During the first subjective study, participants received a monetary incentive, while the second subjective study was carried out with volunteers. The statistical analysis demonstrated that stalling events do not result in the worst degradation of QoE. These findings suggest that a reevaluation of the effect of stalling events in QoE research is needed. Therefore, these findings may be used for further research and to improve current adaptation strategies in ABR algorithms.
... A HAS oriented QoE metric proposed in [35] considers the main issues related to the impact of VQL switching on QoE: -Frequency of video quality level (VQL) switching; -Types of VQL changes (in the DASH scenario, there are more than two versions of the same video on a video server, so it is possible that there are more types of VQL change events); -Temporal location of a VQL switching event. ...
... Rodríguez et al. [31] found that switching frequency, switching type (i.e., spatial and temporal resolutions), and switching temporal location are three critical factors of media quality switches that impact the QoE. Tran et al. [17] conducted subjective tests to formulate a multi-factor QoE model. ...
With the recent growth of multimedia traffic over the Internet and emerging multimedia streaming service providers, improving Quality of Experience (QoE) for HTTP Adaptive Streaming (HAS) becomes more important. Alongside other factors, such as the media quality, HAS relies on the performance of the media player’s Adaptive Bitrate (ABR) algorithm to optimize QoE in multimedia streaming sessions. QoE in HAS suffers from weak or unstable internet connections and suboptimal ABR decisions. As a result of imperfect adaptiveness to the characteristics and conditions of the internet connection, stall events and quality level switches could occur and with different durations that negatively affect the QoE. In this paper, we address various identified open issues related to the QoE for HAS, notably (i) the minimum noticeable duration for stall events in HAS; (ii) the correlation between the media quality and the impact of stall events on QoE; (iii) the end-user preference regarding multiple shorter stall events versus a single longer stall event; and (iv) the end-user preference of media quality switches over stall events. Therefore, we have studied these open issues from both objective and subjective evaluation perspectives and presented the correlation between the two types of evaluations. The findings documented in this paper can be used as a baseline for improving ABR algorithms and policies in HAS.
... This experiment is just one of the series of tests that they will conduct. In [15], Rodríguez et al. determined that changes in video quality level (VQL) had an effect on the user QoE. They proposed a DASH algorithm, including a decision parameter named the switching degradation factor (SDF) that captured a correlation between the QoE and VQL switching types, the frequency of VQL switching events and their temporal locations. ...
Given a large number of online video viewers, video streaming, over various networks, is important communication technology. The multitude of viewers makes it challenging for service providers to provide a good viewing experience for subscribers. Video streaming capabilities are designed based on concepts including quality, viewing flexibility, changing network conditions, and specifications for different customer devices. Adjusting the quality levels, and controlling various relevant parameters to stream the video content with good quality and without interruption is vital. This paper proposes an adaptive framework to balance the average video bitrates with respect to appropriate quality switches, making the transition to higher switches more seamless. The quality adaptation scheme increases the bitrates to the maximum value at their current quality switch before shifting to a higher level. This reduced switching times between levels and guarantees the stability of viewing and avoids interruptions. The use of a dynamic system ensures optimal performance, by controlling system parameters and making the algorithm more tunable. We built the system using an open-source DASH library (Libdash) with QuickTime player, studied the video load changes on two performance parameters, Central Processing Unit and Memory usages that have a high impact on multimedia quality. Consequently, the values of parameters that affected the performance of video streaming could be decreased, permitting users to regulate the parameters according to their preferences. Further, reducing the switching levels will reduce the overloads that occur while transferring from one level to another.
... Our work is most relevant to three previous studies. Bampis et al. [21] and our previous studies [22], [23] collect human opinion scores of realistic streaming videos generated by a set of ABR algorithms. Aside from the fact that the ABR algorithms under evaluation are severely dated (proposed before 2015), the scope of these studies are limited to three dimensions including encoding, network throughput and the choice of ABR algorithms. ...
The diversity of video delivery pipeline poses a grand challenge to the evaluation of adaptive bitrate (ABR) streaming algorithms and objective quality-of-experience (QoE) models. Here we introduce so-far the largest subject-rated database of its kind, namely WaterlooSQoE-IV, consisting of 1350 adaptive streaming videos created from diverse source contents, video encoders, network traces, ABR algorithms, and viewing devices. We collect human opinions for each video with a series of carefully designed subjective experiments. Subsequent data analysis and testing/comparison of ABR algorithms and QoE models using the database lead to a series of novel observations and interesting findings, in terms of the effectiveness of subjective experiment methodologies, the interactions between user experience and source content, viewing device and encoder type, the heterogeneities in the bias and preference of user experiences, the behaviors of ABR algorithms, and the performance of objective QoE models. Most importantly, our results suggest that a better objective QoE model, or a better understanding of human perceptual experience and behaviour, is the most dominating factor in improving the performance of ABR algorithms, as opposed to advanced optimization frameworks, machine learning strategies or bandwidth predictors, where a majority of ABR research has been focused on in the past decade. On the other hand, our performance evaluation of 11 QoE models shows only a moderate correlation between state-of-the-art QoE models and subjective ratings, implying rooms for improvement in both QoE modeling and ABR algorithms. The database is made publicly available at: \url{https://ece.uwaterloo.ca/~zduanmu/waterloosqoe4/}.
... As stated before, the analysis of signal quality plays an important role in many applications to obtain a better performance [22]- [24], and different methods are implemented to improve the signal quality in different areas, such as communication systems [25]- [32]. Thus, the improvement of the signal quality has a direct impact on the user quality of experience [33]- [36]. ...
The iris biometric recognition process is one of the most consistent biometric technology among others that are currently available. However, its efficiency and accuracy can be affected by low-quality iris images used as input to a recognition system, thus, the global performance is decreased. In this context, this paper proposes an evaluation study to determine the impact of the iris image quality on the iris biometric system performance, using the key metrics presented in the ISO/IEC 29794-6:2015 standard. The experimental tests are performed using an iris image database and the OSIRIS biometric recognition software, both of them widely accepted and referenced in the latest research. Experimental results show the interval values of each quality metric and the number of images that accomplish the minimum required values. The performance of the biometric system is assessed by True-Match (TM) and False Non-Match (FNM) parameters; thus, it was possible to identify that the higher the level of image quality, the smaller the FNM value, therefore, the system performance is improved.
... As stated before, the analysis of signal quality plays an important role in many applications to obtain a better performance [22]- [24], and different methods are implemented to improve the signal quality in different areas, such as communication systems [25]- [32]. Thus, the improvement of the signal quality has a direct impact on the user quality of experience [33]- [36]. ...
The iris biometric recognition process is one of the most consistent biometric technology among others that are currently available. However, its efficiency and accuracy can be affected by low-quality iris images used as input to a recognition system, thus, the global performance is decreased. In this context, this paper proposes an evaluation study to determine the impact of the iris image quality on the iris biometric system performance, using the key metricspresented in the ISO/IEC 29794-6:2015 standard. The experimental tests are performed using an iris image database and the OSIRIS biometric recognition software, both of them widely accepted and referenced in the latest research. Experimental results show the interval values of each quality metric and the number of images that accomplish the minimum required values. The performance of the biometric system is assessed by True-Match (TM) and False Non-Match (FNM) parameters; thus, it was possible to identify that the higher the level of image quality, the smaller the FNM value; therefore, the system performance is improved.
... Previous studies have investigated, both qualitatively and quantitatively, different factors affecting the QoE of HAS sessions [2][3][4]. In general, there are three key factors, namely initial delay, varying perceptual quality, and interruptions as shown in Fig. 1. ...
... In the literature, the impact of varying perceptual quality was modeled using some statistics such as the number of switches [6,7], the average [4,6,8,9], the median [10], the minimum [10], and the standard deviation of segment quality values [6]. As for interruptions, their impact was modeled using some statistics such as the number of interruptions [11,12], the average [11], the maximum [11], and the sum [12,13] of interruption durations. ...
... Aforementioned, the factor of varying perceptual quality could be divided into two sub-factors of quality levels and quality variations. The contributions of quality levels were investigated in many existing studies [4,6,8,9]. It was found that, given a quality level, its contribution depends on its total presence time during a session [12,20]. ...
HTTP Adaptive Streaming (HAS) has become a cost-effective means for multimedia delivery nowadays. However, how the quality of experience (QoE) is jointly affected by 1) varying perceptual quality and 2) interruptions is not well-understood. In this paper, we present the first attempt to quantitatively quantify the relative impacts of these factors on the QoE of streaming sessions. To achieve this purpose, we first model the impacts of the factors using histograms, which represent the frequency distributions of the individual factors in a session. By using a large dataset, various insights into the relative impacts of these factors are then provided, serving as suggestions to improve the QoE of streaming sessions.
... Reference [9] focuses on the impact of interruption in the video playback process on QoE of the streaming media services and establishes an exponential model between the MOS and the duration of interruption. Reference [10] also studied the effect of switching of the quality of a video QoE, and concluded that the switching frequency between different Video Quality Levels (VQL) interferes with the user's attention. It also affects a user's QoE, and by combining the exponential and logarithmic model the effect of video quality switching on the user's QoE can be quantified. ...
DASH (Dynamic Adaptive Streaming over HTTP (HyperText Transfer Protocol)) as a universal unified multimedia streaming standard selects the appropriate video bitrate to improve the user’s Quality of Experience (QoE) according to network conditions, client status, etc. Considering that the quantitative expression of the user’s QoE is also a difficult point in itself, this paper researched the distortion caused due to video compression, network transmission and other aspects, and then proposes a video QoE metric for dynamic adaptive streaming services. Three-Dimensional Convolutional Neural Networks (3D CNN) and Long Short-Term Memory (LSTM) are used together to extract the deep spatial-temporal features to represent the content characteristics of the video. While accounting for the fluctuation in the quality of a video caused by bitrate switching on the QoE, other factors such as video content characteristics, video quality and video fluency, are combined to form the input feature vector. The ridge regression method is adopted to establish a QoE metric that enables to dynamically describe the relationship between the input feature vector and the value of the Mean Opinion Score (MOS). The experimental results on different datasets demonstrate that the prediction accuracy of the proposed method can achieve superior performance over the state-of-the-art methods, which proves the proposed QoE model can effectively guide the client’s bitrate selection in dynamic adaptive streaming media services.
... This experiment is just one of the series of tests that they will conduct. In [15], Rodríguez et al. determined that changes in video quality level (VQL) had an effect on the user QoE. They proposed a DASH algorithm, including a decision parameter named the switching degradation factor (SDF) that captured a correlation between the QoE and VQL switching types, the frequency of VQL switching events and their temporal locations. ...
