Figure 4 - available via license: CC BY
Content may be subject to copyright.
Source publication
360-degree video streaming for high-quality virtual reality (VR) is challenging for current wireless systems because of the huge bandwidth it requires. However, millimeter wave (mmWave) communications in the 60 GHz band has gained considerable interest from the industry and academia because it promises gigabit wireless connectivity in the huge unli...
Contexts in source publication
Context 1
... mmWave devices are still under development and there are few that are commercially available. To examine how real-life data transfer rates can be achieved, we used a mmWave VR dongle [15] that acts as a USB 3.0 network adapter with parameters defined in Table 1 to communicate with each other as shown in Figure 4a. In particular, current antennas present 10 dBi of gain, a transmitted equivalent isotropic radiated power (EIRP) of 13 dBm and IEEE 802.11ad single carrier MCS (Modulation and Coding Scheme) of 7. ...
Context 2
... the obstacle test, the achievable data rates fall up to 23% when disturbed by a human head. Also, the achievable data rate reduces according to the misaligned degree α i or β i between the transmitter and receiver, as shown in Figure 4b. These experiments indicate that the mmWave channel is quite sensitive to obstacles and misalignments that can suddenly lead to a drop in the bandwidth. ...
Context 3
... TileColumnWidthArray and TileRowHeightArray options are used to adjust the resolutions of each tile. We executed this experiment using the conventional and proposed methods, as shown in Figure 14. For real-time decoding, open source OpenHEVC decoder was used that provides an Android development branch [28]. ...
Similar publications
Ultra High Definition (UHD) video streaming to portable devices has become topical. Two standardized codecs are current, H.264/Advanced Video Coding (AVC) and the more recent High Efficiency Video Coding (HEVC). This paper compares the two codecs’ robustness to packet loss, after making allowances for relative coding gain. A significant finding fro...
Citations
... With the rapid advances in computing, graphics, and networking technologies enabling video streaming services that require 4K, 8K bandwidth, the multimedia service environment has evolved from 2D video streaming environment to a virtual environment capable of streaming 3D or 360-VR video. Recently, many researchers have been studying 360 degree panorama video streaming [1][2][3][4][5][6][7][8]. However, since the streaming bandwidth of the HMD (Head Mounted Display) device that provides multimedia service in a virtual environment is limited, there is a limit to streaming high quality video images having a bit rate of 8K or higher. ...
This paper proposes a technique for 360-degree panoramic video generating with multiple actual and virtual cameras in real-time. Our method acquires multiple images consisting of real images from physical cameras and virtual images from virtual cameras by GPU-based texture and encoding. Then, our method generates 360-degree panoramic images by ORB feature detection and matching to virtual and real images, automatic homography estimation based on the random absent consensus algorithm, multi-band blending and produces high-resolution video stream by Nvidia encoder based Hardware accelerated video encoding. From experimental results, we verified that our method takes at least 45fps at 2K resolution and 33fps at 4K resolution when transmitting the head mounted display stereo, enabling real-time 360-degree panoramic video generation as well as transmission.
... Nguyen et al. [62] also address pre-fetching but using the new package prioritization feature from HTTP/2 to delivery high-quality tiles. Then, in a new direction, Nguyen et al. [63] investigated multi-path streaming but for decoding offloading. They consider a mobile VR device and a PC with access to the stream server and connected using a millimetre-wave (mmWave) network in the 60 GHz band. ...
Omnidirectional videos (ODVs) have gone beyond the passive paradigm of traditional video, offering higher degrees of immersion and interaction. The revolutionary novelty of this technology is the possibility for users to interact with the surrounding environment, and to feel a sense of engagement and presence in a virtual space. Users are clearly the main driving force of immersive applications and consequentially the services need to be properly tailored to them. In this context, this chapter highlights the importance of the new role of users in ODV streaming applications, and thus the need for understanding their behaviour while navigating within ODVs. A comprehensive overview of the research efforts aimed at advancing ODV streaming systems is also presented. In particular, the state-of-the-art solutions under examination in this chapter are distinguished in terms of system-centric and user-centric streaming approaches: the former approach comes from a quite straightforward extension of well-established solutions for the 2D video pipeline while the latter one takes the benefit of understanding users’ behaviour and enable more personalised ODV streaming.
... This popular project method encodes a distorted version of the spherical surface and assigns more bits to a particular view of the cubic. Other measurement studies discovered that the impacts of delay on 360°VoD streaming mainly stem from the impacts of delay on viewport-adaptive streaming algorithms [24,25,28]. Despite all the efforts to understand 360°VoD systems, none considers the 360°camera and the management of a live streaming session, which are essential components in 360°video camera sensing. ...
360° video camera sensing is an increasingly popular technology. Compared with traditional 2D video systems, it is challenging to ensure the viewing experience in 360° video camera sensing because the massive omnidirectional data introduce adverse effects on start-up delay, event-to-eye delay, and frame rate. Therefore, understanding the time consumption of computing tasks in 360° video camera sensing becomes the prerequisite to improving the system's delay performance and viewing experience. Despite the prior measurement studies on 360° video systems, none of them delves into the system pipeline and dissects the latency at the task level. In this paper, we perform the first in-depth measurement study of task-level time consumption for 360° video camera sensing. We start with identifying the subtle relationship between the three delay metrics and the time consumption breakdown across the system computing task. Next, we develop an open research prototype Zeus to characterize this relationship in various realistic usage scenarios. Our measurement of task-level time consumption demonstrates the importance of the camera CPU-GPU transfer and the server initialization, as well as the negligible effect of 360° video stitching on the delay metrics. Finally, we compare Zeus with a commercial system to validate that our results are representative and can be used to improve today's 360° video camera sensing systems.
... T HE emergence of high-quality video including ultrahigh-definition (UHD) video, 360-degree immersive video, initiates new and exciting applications in virtual reality (VR), augmented reality (AR) and mixed reality (MR) in education, training, entertainment, and other markets [1] [2]. The bandwidth-intensive nature of these new generation video contents poses challenges in handling transmission and storage burdens while ensuring low latency delivery [3]. Furthermore, based on different user necessities including heterogeneous network capacities, display, power, and computing capabilities, the need for other video formats (e.g. ...
Video coding using dynamic background frame achieves better compression compared to the traditional techniques by encoding background and foreground separately. This process reduces coding bits for the overall frame significantly; however, encoding background still requires many bits that can be compressed further for achieving better coding efficiency. The cuboid coding framework has been proven to be one of the most effective methods of image compression which exploits homogeneous pixel correlation within a frame and has better alignment with object boundary compared to traditional block-based coding. In a video sequence, the cuboid-based frame partitioning varies with the changes of the foreground. However, since the background remains static for a group of pictures, the cuboid coding exploits better spatial pixel homogeneity. In this work, the impact of cuboid coding on the background frame for high-resolution videos (Ultra-High-Definition (UHD) and 360-degree videos) is investigated using the multilayer framework of SHVC. After the cuboid partitioning, the method of coarse frame generation has been improved with a novel idea by keeping human-visual sensitive information. Unlike the traditional SHVC scheme, in the proposed method, cuboid coded background and the foreground are encoded in separate layers in an implicit manner. Simulation results show that the proposed video coding method achieves an average BD-Rate reduction of 26.69% and BD-PSNR gain of 1.51 dB against SHVC with significant encoding time reduction for both UHD and 360 videos. It also achieves an average of 13.88% BD-Rate reduction and 0.78 dB BD-PSNR gain compared to the existing relevant method proposed by X. HoangVan [18].
... In HAS, the original video content is converted into several bitrates; each bitrate is further converted into several segmenter segments, which may be called chunks. Each video segment or chunk has a fixed duration video in seconds [8,9]. The chunks' path or address is stored in a high-level language file called a media presentation description or a manifest file. ...
The demand for multimedia content over the Internet protocol network is growing exponentially with Internet users’ growth. Despite high reliability and well-defined infrastructure for
Internet protocol communication, Quality of Experience (QoE) is the primary focus of multimedia
users while getting multimedia contents with flawless or smooth video streaming in less time with
high availability. Failure to provide satisfactory QoE results in the churning of the viewers. QoE depends on various factors, such as those related to the network infrastructure that significantly affects
perceived quality. Furthermore, the video delivery’s impact also plays an essential role in the overall
QoE that can be made efficient by delivering content through specialized content delivery architectures called Content Delivery Networks (CDNs). This article proposes a design that enables effective
and efficient streaming, distribution, and caching multimedia content. Moreover, experiments are
carried out for the factors impacting QoE, and their behavior is evaluated. The statistical data is taken
from real architecture and analysis. Likewise, we have compared the response time and throughput
with the varying segment size in adaptive bitrate video streaming. Moreover, resource usage is also
analyzed by incorporating the effect of CPU consumption and energy consumption over segment
size, which will be counted as effective efforts for sustainable development of multimedia systems.
The proposed architecture is validated and indulged as a core component for video streaming based
on the use case of a Mobile IPTV solution for 4G/LTE Users.
... 4,6,7 Furthermore, ordinary horizontal FOV is defined as 120°, which demands a 360 VR viewport, and the FOV corresponding to the VR viewport requires 4K resolution. Therefore, a minimum of 12K resolution and 100 FPS is necessary to stream high-quality 360 VR videos and reduce VR sickness, 8 which is another challenge. ...
... 6 That is, it causes a prediction mismatch, and the solution, which creates the generated reference picture, still has overhead problems. 6,[8][9][10] Many studies have been conducted with the objective of solving the problem outlined above. 4 However, it requires H/W decoding to provide highquality 360 VR streaming with 8K and 16K resolution video. ...
... The coordinates of the vertexes of HMD output area can be calculated using the seeing direction (f, u, c) and the values of U, V after applying equations (10) to (8) ...
To support 360 virtual reality video streaming services, high resolutions of over 8K and network streaming technology that guarantees consistent quality of service are required. To this end, we propose 360 virtual reality video player technology and a streaming protocol based on MPEG Dynamic Adaptive Streaming over HTTP Spatial Representation Description to support the player. The player renders the downsized video as the base layer, which has a quarter of the resolution of the original video, and high-quality video tiles consisting of tiles obtained from the tiled-encoded high-quality video (over 16K resolution) as the enhanced layer. Furthermore, we implemented the system and conducted experiments to measure the network bandwidth for 16K video streaming and switching latency arising from changes in the viewport. From the results, we confirmed that the player has a switching latency of less than 1000 ms and a maximum network download bandwidth requirement of 100 Mbps.
... This means that the amount of data handled by the HMD increases rapidly, which is very challenging. For this reason, asymmetric core processing [1], [2], data offloading over mmWave [3], [4] for a mobile devices, and view location based asymmetric down-sampling method [5] have proposed. Consequently, the motion-constrained tile set (MCTS) [6] has been reported to process the viewport of the user. ...
Immersive video streaming has become very popular. To increase the quality of experience (QoE) with immersive media, user movement adaptive video streaming, three degrees of freedom plus (3DoF+), has emerged and is expected to meet this growing demand. Satisfying the limit of the bandwidth, providing high-quality immersive experience is challenging because 3DoF+ system requires high resolution, multi-view video transmission. This paper proposes a stride based 3DoF+ 360 video streaming system and introduces two main ideas: (i) a multi-view video redundancy removal method using view synthesis, (ii) a multi-view video residual packing method. The proposed multi-view video compression method removes redundancy between videos and packs them into one video, and it exhibits a BD-rate saving of 36.0%in maximum compared to the results of the high-efficiency video coding reference model. In addition, the proposed system requires fewer number of decoders for the clients, and it decreases the burden for immersive video streaming.
... The images are then segmented into tiles and each tile is encoded in multiple versions of quality. To achieve a reasonable performance, viewport-dependent streaming was actively studied [11], [12], [13], [19]. However, these tile-based streaming methods require repetitive decoding operations in a streaming environment as they cannot reference information of other tiles in video encoding. ...
In 360-degree video streaming, Most solutions are based on tile-based streaming that divides videos into tiles and streams the high-quality tiles corresponding to the user's viewport areas. However, these methods cannot transmit different combinations of tile coding efficiently. In this paper, we experimented with streaming 360-degree videos using a motion-constrained tile set (MCTS) technique that allows encoding with constraining motion vectors such that each tile can be decoded and transmitted independently. Moreover, we have used a tile-based approach using a saliency map that integrates the information of human visual attention with the contents to deliver high-quality tiles to the region of interest (ROI). We encoded the 360-degree videos at various quality representations with MCTS techniques and assigned a tile quality representation using a saliency map predicted by the existing convolutional neural network (CNN) model. We proposed a novel heuristic algorithm to assign appropriate quality to the tiles on the centerline. Consequently, mixed quality videos based on the saliency map enable efficient streaming in 360-degree videos. Using the Salient360! dataset, the proposed method shows an improvement in terms of bandwidth with little loss of viewport image quality.
The moving picture experts group (MPEG) immersive video (MIV) coding standard enables the implementation of a practical six degrees of freedom (6DoF) virtual reality (VR) video streaming system by two approaches: 1) removing the redundancy between multi-view videos or 2) selecting and transmitting representative views among multi-view videos using traditional 2-D video compression standards. This paper presents a strategy for providing immersive video considering two aspects: 1) minimizing the fatigue in the synthesized and rendered viewport, and 2) providing rate-distortion optimization (RDO) to yield high-quality videos at a low bitrate. The results of the two approaches in the MIV coding standard using high-efficiency video coding (HEVC) are discussed, and the advantages and drawbacks of each method for 2-D display and head-mounted display are detailed.
