Next Article in Journal
HRV Features as Viable Physiological Markers for Stress Detection Using Wearable Devices
Next Article in Special Issue
Smart Sensor Technologies for IoT
Previous Article in Journal
A Novel Runtime Algorithm for the Real-Time Analysis and Detection of Unexpected Changes in a Real-Size SHM Network with Quasi-Distributed FBG Sensors
Previous Article in Special Issue
Wi-Fi-Based Location-Independent Human Activity Recognition via Meta Learning
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 21
Issue 8
10.3390/s21082872
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Impact of Scene Content on High Resolution Video Quality

by
Miroslav Uhrina
*,
Anna Holesova
,
Juraj Bienik
and
Lukas Sevcik
Department of Multimedia and Information-Communication Technology, University of Zilina, Univerzitna 1, 010 26 Zilina, Slovakia
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(8), 2872; https://doi.org/10.3390/s21082872
Submission received: 1 March 2021 / Revised: 11 April 2021 / Accepted: 13 April 2021 / Published: 19 April 2021
(This article belongs to the Special Issue Smart Sensor Technologies for IoT)
Browse Figures
Review Reports Versions Notes

Abstract

This paper deals with the impact of content on the perceived video quality evaluated using the subjective Absolute Category Rating (ACR) method. The assessment was conducted on eight types of video sequences with diverse content obtained from the SJTU dataset. The sequences were encoded at 5 different constant bitrates in two widely video compression standards H.264/AVC and H.265/HEVC at Full HD and Ultra HD resolutions, which means 160 annotated video sequences were created. The length of Group of Pictures (GOP) was set to half the framerate value, as is typical for video intended for transmission over a noisy communication channel. The evaluation was performed in two laboratories: one situated at the University of Zilina, and the second at the VSB—Technical University in Ostrava. The results acquired in both laboratories reached/showed a high correlation. Notwithstanding the fact that the sequences with low Spatial Information (SI) and Temporal Information (TI) values reached better Mean Opinion Score (MOS) score than the sequences with higher SI and TI values, these two parameters are not sufficient for scene description, and this domain should be the subject of further research. The evaluation results led us to the conclusion that it is unnecessary to use the H.265/HEVC codec for compression of Full HD sequences and the compression efficiency of the H.265 codec by the Ultra HD resolution reaches the compression efficiency of both codecs by the Full HD resolution. This paper also includes the recommendations for minimum bitrate thresholds at which the video sequences at both resolutions retain good and fair subjectively perceived quality.

1. Introduction

In recent years, the number of various types of surveillance and data collection cameras located both indoors and outdoors have been constantly increasing. Popularity of home security cameras is also growing as even high-quality models become more affordable. Typical surveillance cameras applications include public safety, protection of facilities against theft or vandalism, remote video monitoring, traffic surveillance, weather monitoring, or more special cases, such as animal monitoring or data collection, for statistical or marketing purposes. Today, due to the pandemic situation, face recognition with and without a protective mask is also becoming a point of interest for researchers in cooperation with technology companies [1,2,3]. It is important to realize that each such employed sensor produces a tremendous amount of data to be subsequently transmitted over the network or further processed, which calls for effective video compression. Furthermore, whether the image or video is presented to a live person or a machine learning algorithm (most often for its classification or segmentation), the best results can be achieved when the image is of the highest achievable quality. This implies one common goal for the distributors, communication service providers, or even broadcasting companies, to optimally set the compression parameters so that perceived video quality is maximal, while the bandwidth requirements are minimal. This challenge leads to increased interest in the analysis of video content followed by the individual setting of the compression parameters of video sequences with different types of scene content. Even though many studies deal with the video quality assessment using subjective methods, the demand exceeds the supply; there is still a lack of video quality datasets, as well as recorded subjective tests, conducted on these datasets. Very popular and extensively used datasets, such as References [4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27], come from the University of Texas and were developed by the Laboratory for Image and Video Engineering. Another very popular option is the VQEG-HDTV database [28], which is a result of international project of VQEG (Video Quality Experts Group) consortium. Other well-known datasets are BVI-HD [29], BVI textures [30] and BVI-HFR [31] developed at the University of Bristol, AVT-VQDB-UHD-1 database [32] made by the Ilmenau University of Technology, Ultra Video Group (UVG) dataset [33] composed at the Tampere University, SJTU 4K video quality database [34] from the Shanghai Jiao Tong University, Image and Video Processing Subjective Quality Video Database [35] developed at the Chinese University of Hong Kong, collection of IRCCyN/IVP databases from the Institut de Recherche en Communications et Cybernétique de Nantes [36], Konstanz Natural Video Database (KoNViD-1k) made by the Universitaet Konstanz [37,38], MCL-V [39], and [40] databases from the MSC University of Southern California, Scalable Video Database [41,42] composed at the EPFL, ReTRiEVED Video Quality Database [43] made by the Universita Degli Studi or TUM databases [44,45] developed by the Technical University of Munich. Taking into account the demand and importance of measuring the performance of video quality assessment techniques, a number of studies on perceptual evaluation was presented. Most of them merely compare the quality of video sequences with various characteristics evaluated by different subjective methods and do not examine the content aspect. Rerabek et al. [46] examined a rate-distortion performance analysis and mutual comparison of one of the latest video coding standards H.265/HEVC with VP9 codec. Ramzan et al. [47] presented a performance evaluation of three coding standards—Advanced Video Coding (H.264/MPEG-AVC), High-Efficiency Video Coding (H.265/MPEG-HEVC), and VP9, based on subjective and objective quality evaluations. Two different sequences at both resolutions (Full HD and Ultra HD) were tested using the DSIS method. Bienik et al. [48] measured the impact of the compression formats, namely H.264, H.265, and VP9 on perceived video quality. The evaluation was performed on four Full HD sequences using the Absolute Category Rating (ACR) and DSCQS methods. Xu et al. [49] presented a subjective video quality assessment on 4K Ultra-High Definition (UHD) videos using the DSCQS method. Six different test sequences were used for the evaluation.Herrou et al. [50] focused on a performance comparison between HEVC and VP9 in the HDR context through both objective and subjective evaluations. Dumic et al. [51] offered findings on subjective assessment of H.265 versus H.264 Video Coding for High-Definition Video Systems. For the evaluation, a database consisting of 120 degraded HD video sequences with 4 contents encoded at various compression rates to H.265/HEVC and H.264/AVC formats was compiled. Milovanovic et al. [52] subjectively compared the coding efficiency of three video coding standards (MPEG-H HEVC, H.264/MPEG-4 AVC, and H.262/MPEG-2). Sotelo et al. [53] presented a subjective quality assessment of HEVC/H.265 compressed 4K Ultra-High-Definition (UHD) videos in a laboratory viewing environment. Kufa et al. [54] explored coding efficiency performance of High Efficiency Video Coding (HEVC) and VP9 compression formats on video content in Full HD and UHD resolutions. Deep et al. [55] focused on the comparison of HEVC and VP9 based on both subjective and objective evaluation on various (720p, 1080p, and 2160p) test videos. Akyazi et al. [56] examined the compression efficiency of HEVC/H.265, VP9, and AV1 codecs based on subjective quality assessment. Our survey of research papers shows that there is still a lack of databases of video sequences annotated according to subjective evaluation. Therefore, this paper brings new subjective results and also explores the impact of the video content on the subjective assessment. We decided to compare today’s most used compression standards—H.264/AVC and H.265/HEVC—on video sequences at Full HD and Ultra HD resolutions. Our publication follows Reference [57], where a new 4K video dataset was compiled with full subjective scores (Mean Opinion Score (MOS)) of videos at different bitrates compressed by HEVC/H.265 codec evaluated by the Double Stimulus Impairment Scale (DSIS) method, variant II. For our measurements, we decided to use the Absolute Category Rating (ACR) method.

2. Dataset Description and Preparation

2.1. Dataset Description

For our measurements, we used the dataset from the Media Lab of the Shanghai Jiao Tang University [34]. We selected eight sequences with various scene content, illustrated in Figure 1, classified according to the Temporal Information (TI) and Spatial Information (SI) from this database. SI defines the amount of spatial detail in an image and is higher for more spatially complex scenes, while TI represents the number of temporal changes in a video sequence and is higher for high motion sequences [58]. The spatial perceptual information is based on the Sobel filter and is represented by the formula:
S I = m a x t i m e [ s t d s p a c e [ S o b e l ( F n ) ] ] ,
where F n stands for video frame, and s t d s p a c e for the standard deviation over the pixels in each Sobel-filtered frame. The temporal information is computed as:
T I = m a x t i m e [ s t d s p a c e [ M n ( i , j ) ] ] ,
where M n ( i , j ) is the difference between pixels at the same position in the frame belonging to two consecutive frames, i.e.,
M n ( i , j ) = F n ( i , j ) − F n − 1 ( i , j ) ,
where F n ( i , j ) is the pixel at the i-th row, and j-th column of n-th frame in time [58]. Both of these parameters were calculated for each sequence using the Mitsu tool [59] and plotted in Figure 2. The general specification of the dataset is given in Table 1, and the content of individual sequences is briefly described in Table 2.

2.2. Dataset Preparation

In our research, we decided to explore the quality of 8-bit video sequences at two commonly used resolutions, i.e., Full HD (FHD) and Ultra HD (UHD) with a typical chroma subsampled YUV 4:2:0 format. Because original sequences were uncompressed and YUV 4:4:4 color format at Ultra HD resolution with 10-bit depth was used, we had to convert them to the appropriate formats. Therefore, all test sequences were first chroma subsampled from YUV 4:4:4 to YUV 4:2:0 format and also the bit depth was changed from 10 to 8 bits per channel. Subsequently, all these conversion steps were repeated for Full HD resolution utilizing the FFmpeg tool [61]. As we wanted to assess also Full HD in addition to Ultra HD, the resolution also had to be altered. For all these conversion steps we used once again the FFmpeg tool [61]. Correspondingly, two uncompressed test sequences were generated (Figure 3) for each type of content, which adds up to 16 videos. We call them the source video sequences (SRCs) for the rest of this paper.

2.3. Coding Process

All these test video sequences (SRCs) were afterwards encoded to both compression standards to be evaluated, i.e., H.264/AVC and H.265/HEVC. As a quality restriction parameter, we decided to use the constant bitrate. We selected 5 various target bitrates: 1, 3, 5, 10 and 15 Mbps based on our previous research [62] which have shown the efficiency of codecs growing nonlinearly with increasing bitrate. We have limited the number of bitrates to 5 as a compromise between the complexity and time requirements of subjective testing and precision of the measurements. For the purposes of our research, we decided to use the Group of Pictures (GOP) length typical for video intended for transfer over a noisy communication channel. The GOP length is based on the framerate of used video sequences and is commonly set to half of the framerate value. Accordingly, given that test video sequences had a framerate of 30 fps (frames per second), we chose the GOP length of 15 frames, i.e., M = 3, N = 15. The first number, labeled with M letter, expresses the distance between two anchor frames (I or P) and the second number, denoted with N letter, stands for the distance between two key frames (I). For this coding process, we used once again the FFmpeg tool, which contains libraries x264 and x265 for H.264/AVC and H.265/HEVC codec, respectively [61], creating the total of 160 video sequences for the subjective quality assessment. We refer to them as PVSs (Processed Video Sequences) for the rest of this paper. The FFmpeg command example for encoding the Wood test sequence to the H.264 format at 1 Mbps bitrate is: ffmpeg -i Wood_1920x1080_30fps_420_8bit_YUV.yuv -vcodec libx264 -command example-params keyint=15:min-keyint=15:bframes=3:b-adapt=1:bitrate=1000:vbv-maxrate=1000:vbv-bufsize=1000 Wood_1920x1080_30fps_420_8bit_H264_01M.mp4.

3. Subjective Quality Assessment

During the subjective testing, all created PVSs were shown to people of different ages and genders to evaluate their quality. We decided to use the Absolute Category Rating (ACR) method [58,63] which belongs to the category of Single Stimulus (SS) subjective video quality assessment techniques. The principle of this method is that the degraded sequences are presented to the observers one at a time, and they are asked to rate its quality on a five-level grading scale, where 1 indicates the bad quality, and 5 stands for the excellent quality. The measurement was conducted in two laboratories separately: one situated at the University of Zilina (UNIZA), and the second at the VŠB – Technical University in Ostrava. The video sequences were presented on three types of displays (Table 3) depending on the resolution of the test sequences in the laboratories under normal indoor illumination conditions.
Thirty participants, mostly students, were involved in the testing in each laboratory. All of them were naive observers which means they had no expertise in the image artefacts that may be introduced by the system under test. Naturally, they were thoroughly acquainted with the method of assessment, types of impairment, grading scale, sequence, and timing as required by Reference [58]. The statistical distribution of the number of men and women who took part in the tests, as well as the average age of all observers, is shown in Table 4. The course of the entire subjective assessment process is represented by Figure 4.

4. Statistical Analysis and Presentation of the Results

After performing the subjective tests, we processed all collected results statistically; for each test sequence, codec, and resolution, the Mean Opinion Score (MOS) and 95 percent Confidence Interval (CI) in accordance with Reference [64] were calculated and plotted in graphs, a shown below. The presentation of the results could be divided into five parts. In the first part, the cross-comparison of the results obtained from different laboratories, i.e., from UNIZA and VŠB, is performed using the Pearson correlation coefficient (PCC) and the Root Mean Square Error (RMSE). In the second part, the bitrate impact on the perceived video quality depending on the scene content is plotted. The third part deals with the Analysis of Variance (ANOVA) which was applied on the acquired data. In the fourth part, the impact of the bitrate on the perceived video quality in terms of the used codec and resolution is presented. Finally, in the fifth part, the minimum bitrate thresholds at which the video sequence should be encoded to reach certain quality are determined.

4.1. Correlation between the Results from Individual Laboratories

To compare the MOS values obtained from both laboratories, i.e., from UNIZA and VŠB, and, to find out the correlation, the Pearson correlation coefficient (PCC), as well the Root Mean Square Error (RMSE) were calculated. All computations were done for both codecs and resolutions, as well as for all test sequences. The results are plotted in Figure 5 and Figure 6 and are shown in Table 5.
As we can see from Figure 5 and Figure 6, as well as from Table 5, there is a high correlation between the results from both laboratories. The lowest correlation was reached by the combination of Full HD resolution and H.264 codec. This is most likely due to the different displays used in the assessments; at the UNIZA laboratory, the Full HD display was used, while, at the VŠB laboratory, the Ultra HD display was used. Vice versa, the highest correlation rate was achieved by video sequences encoded to H.264 at UHD resolution.

4.2. Impact of Bitrate on Video Quality Depending on Scene Content

Figure 7 shows the impact of the bitrate on the perceived video quality (defined by the MOS with associated CI). In this figure, eight graphs are inserted considering used codec, resolution, and laboratory where the evaluation was conducted. Sequences with different scene contents are color-coded in the graphs; each curve represents MOS values for a given test sequence. Figure 8 shows the average MOS values obtained from UNIZA and VSB laboratories.
It is apparent from the graphs that the sequences with the lowest SI and TI values, such as the “Bund Nightscape” and the “Construction Field”, reached the best MOS value. Vice versa, the observers rated the sequences situated in the middle of the SI-TI diagram, such as the “Marathon” or “Runners”, as of worst quality. Interesting cases are the “Campfire Party” and the “Fountains” sequences. The “Campfire Party” contains a lot of movement (high TI values) but not many details (low SI values) and reached low MOS value, while the “Fountains” sequence lies near to the “Bund Nightscape” and the “Construction Field” sequences, meaning it has low both TI and SI values and also scored low on the MOS scale. A special case is the “Wood” sequence which is situated at the upper right corner of the SI-TI diagram. Nevertheless, its quality was perceived as similar to the sequences “Fountains” and “Runners”. All these differences are more pronounced:
  • at low bitrates—with increasing bitrate, the perceived quality rises, too, and approaches the perceived quality of sequences with low SI-TI values,
  • at Ultra HD resolution rather than at Full HD resolution, and
  • at H.265 codec rather than at H.264 codec.
Based on these results, we can state that the compression efficiency and related video quality depends on the content of the sequences. However, the sequence representation and description only by the spatial and temporal information is not sufficient and should be the subject of further research. We suggest other parameters should be used to describe the scene, such as, for instance, the luminance and contrast or the colors occurring in the scene. In addition, the psychological factors should be considered. Based on the results, we can also state that the temporal information has greater impact on the perceived quality than the number of the objects defined by the spatial information.

4.3. Analysis of Variance

To verify what stemmed from the graphical representation of the subjective evaluation results, the ANOVA was applied on the data [65]. The three-way ANOVA was used to compare the significance and influence of individual sequence parameters on the resulting perceived video quality. The interaction between three independent variables, bitrate (X1), content (scene type) (X2), and resolution (X3) in Table 6 or compression standard (X3) in Table 7 was examined, with video quality being considered a dependent variable. Table 6 and Table 7 depict the three-way ANOVA matrices. The F-value, also called the F-ratio is calculated as the variance of the group means divided by the mean of the within group variances (Mean Squared Error). Greater F-value indicates more significant variation. In ANOVA, the p-value, i.e., the probability of getting the observed result at random, is also determined. For the source of variation to be regarded as insignificant, the p-value must be higher than a given alpha level, commonly set to 0.05. When performing ANOVA, the p-value is also determined to investigate the probability of rejecting the hypothesis.
Based on the analysis of the tables, the following conclusions can be drawn. Table 6 indicates that for H.265 encoded sequences, the effect of resolution can be ignored, since this variable was deemed statistically insignificant. In contrast, in the case of the H.264 codec, this negative phenomenon does not occur and resolution is the second most important parameter that determines the subjectively perceived quality. For both codecs, an alteration in bitrate results in a maximum change in the subjective MOS. According to Table 7, the impact of compression format on the perceived quality is considered statistically insignificant for Full HD video sequences. However, that is not the case for Ultra HD resolution, where deployed codec is the second most influential variable. Equivalently to Table 6, the bitrate has the greatest effect on the subjective video quality assessment results. All remaining ANOVA test results in both tables can be regarded statistically significant based on their p-values.

4.4. Impact of Bitrate on Video Quality Depending on Codec and Resolution

Figure 9 shows the impact of the bitrate on the perceived video quality (defined by the MOS with associated CI) plotted separately for each type of video sequence. In this figure, eight graphs are inset, considering examined test sequence, which show the impact of used codec and resolution on the perceived quality of a given sequence; curve represents averaged MOS values from both laboratories for a given codec and resolution.
In Figure 10, the averaged MOS value from both laboratories from all used test sequences for each codec and resolution is plotted.
We can draw several conclusions from Figure 9 and Figure 10. Firstly, it is apparent that the H.265 compression standard yields better quality than the H.264 codec. This is a generally known fact and we expected it. But what is interesting and important is that the efficiency difference between these two codecs is negligible for the Full HD video sequences. Therefore, it is inessential to use H.265 compression standard at this resolution, as the observers will not see any notable differences. The use of H.265 codec is relevant only for the videos at the Ultra HD resolution, particularly at low bitrates. This is due to the fact that the quality of H.264 encoded video sequences increases with the rising bitrate up to the point where it reaches or even surpasses the perceived quality of H.265 sequences. Secondly, the compression efficiency of the H.265 compression standard at the Ultra HD resolution reaches the compression efficiency of both codecs at the Full HD resolution.
Indisputably, the conclusions drawn from the Analysis of Variance (ANOVA) and the graphical representation of the subjective quality evaluation results coincide. These findings could be beneficial for visual media content providers and broadcasting companies, as they indicate how to adjust video compression parameters to improve its quality. The fastest growth of perceived video quality is apparently due to an increase in bitrate. Specifically, the quality increases most rapidly until the bitrate reaches a value of approximately 5 Mbps. The analyses also revealed which combination of resolution and compression format is best used so that the resulting quality of visual content is perceived by viewers as good as possible.

4.5. Minimum Bitrate Thresholds Suggestions

Finally, Figure 10 shows the minimum bitrate thresholds at which the video sequences should be encoded to achieve good (4) or fair (3) quality. These quality thresholds are based on MOS values of used ACR method and are important for the bitrate setting of each codec to maintain a certain quality. Table 8 shows the mentioned minimum bitrates.
From Table 8, it follows that to achieve a good quality (value 4 on MOS scale), the video sequence must be coded to minimum 7.50 Mbps by both codecs for Full HD resolution and to 11.55 Mbps by H.264 codec and 9.00 Mbps by H.265 codec for Ultra HD resolution. To reach fair quality (value 3 on MOS scale), the minimum thresholds for the bitrates are 2.80 Mbps by H.264 codec and 2.60 Mbps by H.265 codec for Full HD resolution and 4.50 Mbps by H.264 codec and 2.80 by H.265 codec for Ultra HD resolution.

5. Conclusions

This paper dealt with the content impact on the perceived video quality evaluated using the subjective Absolute Category Rating (ACR) method. Eight types of video sequences with various scene content were evaluated. Two widely used video compression standards H.264/AVC and H.265/HEVC in combination with Full HD and Ultra HD resolutions, were tested. In the coding process, we selected 5 various bitrates based on our previous research, which showed that the efficiency of codecs grows nonlinearly with increasing bitrate. The number of bitrates was a compromise between the complexity and time requirements of subjective testing. In total, we created an annotated database which contains 160 different video sequences coded at constant bitrates with GOP set to half of the framerate value which is typical for video intended for transfer over a noisy communication channel. The perceived quality of the sequences was evaluated employing the subjective ACR method. The assessment was conducted in two laboratories: one situated at the University of Zilina, and the second at the VSB—Technical University in Ostrava. First, we calculated the correlation of the MOS values between both laboratories using the Pearson correlation coefficient (PCC) and the Root Mean Square Error (RMSE). The correlation proved to be considerably high. After that, we described the impact of the bitrate on video quality depending on scene content defined by Spatial (SI) and Temporal information (TI). The results showed that even if the sequences with low SI and TI values reach better MOS than the sequences with higher SI and TI values, these two parameters are not sufficient for scene description, and this domain should be the subject of further research. Subsequently, we described the impact of bitrate on video quality depending on codec and resolution. Based on the results, we concluded that the employment of the H.265 codec for compression of Full HD sequences is inessential, as the people did not observe any significant differences. Furthermore, we stated that the compression efficiency of the H.265 codec by the Ultra HD resolution reaches the compression efficiency of both codecs by the Full HD resolution. We also applied the ANOVA to verify what stemmed from the graphical representation of the subjective evaluation results. Finally, we determined the minimum bitrate thresholds at which the video sequences at both resolutions retain good and fair subjectively perceived quality.

Author Contributions

Conceptualization, A.H. and M.U.; methodology, J.B. and L.S.; validation, J.B., L.S. and M.U.; formal analysis, A.H. and J.B.; investigation, J.B. and L.S.; resources, M.U.; data curation, J.B. and L.S.; writing—original draft preparation, A.H. and M.U.; writing—review and editing, A.H., L.S. and M.U.; visualization, L.S. and M.U; project administration, M.U. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Slovak Research and Development Agency under the project PP-COVID-20-0100: DOLORES.AI: The pandemic guard system.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses or interpretation of data; in the writing of the manuscript, or in the decision to publish the result.

References

  1. Damer, N.; Grebe, J.H.; Chen, C.; Boutros, F.; Kirchbuchner, F.; Kuijper, A. The Effect of Wearing a Mask on Face Recognition Performance: An Exploratory Study. In Proceedings of the 2020 International Conference of the Biometrics Special Interest Group (BIOSIG), Darmstadt, Germany, 16–18 September 2020; pp. 1–6. [Google Scholar]
  2. Joshi, A.S.; Joshi, S.S.; Kanahasabai, G.; Kapil, R.; Gupta, S. Deep Learning Framework to Detect Face Masks from Video Footage. In Proceedings of the 2020 12th International Conference on Computational Intelligence and Communication Networks (CICN), Bhimtal, India, 25–26 September 2020. [Google Scholar]
  3. Tan, W.; Liu, J. Application of Face Recognition in Tracing COVID-19 Fever Patients and Close Contacts. In Proceedings of the 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA, 14–17 December 2020. [Google Scholar]
  4. Seshadrinathan, K.; Soundararajan, R.; Bovik, A.C.; Cormack, L.K. Study of Subjective and Objective Quality Assessment of Video. IEEE Trans. Image Process. 2010, 19, 1427–1441. [Google Scholar] [CrossRef] [PubMed]
  5. Seshadrinathan, K.; Soundararajan, R.; Bovik, A.C.; Cormack, L.K. A subjective study to evaluate video quality assessment algorithms. In Human Vision and Electronic Imaging XV; Rogowitz, B.E., Pappas, T.N., Eds.; SPIE: Bellingham, WA, USA, 2010. [Google Scholar]
  6. Moorthy, A.K.; Choi, L.K.; Bovik, A.C.; de Veciana, G. Video Quality Assessment on Mobile Devices: Subjective, Behavioral and Objective Studies. IEEE J. Sel. Top. Signal Process. 2012, 6, 652–671. [Google Scholar] [CrossRef]
  7. Moorthy, A.K.; Choi, L.K.; Veciana, G.; Bovik, A. Subjective Analysis of Video Quality on Mobile Devices. In Proceedings of the 6th International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Scottsdale, AZ, USA, 19–20 January 2012; Volume 720, pp. 1–6. [Google Scholar]
  8. Chen, C.; Choi, L.K.; de Veciana, G.; Caramanis, C.; Heath, R.W.; Bovik, A.C. A dynamic system model of time-varying subjective quality of video streams over HTTP. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013. [Google Scholar]
  9. Chen, C.; Zhu, X.; de Veciana, G.; Bovik, A.C.; Heath, R.W. Adaptive video transmission with subjective quality constraints. In Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France, 27–30 October 2014. [Google Scholar]
  10. Chen, C.; Choi, L.K.; de Veciana, G.; Caramanis, C.; Heath, R.W.; Bovik, A.C. Modeling the Time—Varying Subjective Quality of HTTP Video Streams With Rate Adaptations. IEEE Trans. Image Process. 2014, 23, 2206–2221. [Google Scholar] [CrossRef] [PubMed]
  11. Chen, C.; Zhu, X.; de Veciana, G.; Bovik, A.C.; Heath, R.W. Rate Adaptation and Admission Control for Video Transmission With Subjective Quality Constraints. IEEE J. Sel. Top. Signal Process. 2015, 9, 22–36. [Google Scholar] [CrossRef]
  12. Choi, L.K.; Cormack, L.K.; Bovik, A.C. On the visibility of flicker distortions in naturalistic videos. In Proceedings of the 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX), Klagenfurt am Wörthersee, Austria, 3–5 July 2013. [Google Scholar]
  13. Choi, L.K.; Cormack, L.K.; Bovik, A.C. Motion silencing of flicker distortions on naturalistic videos. Signal Process. Image Commun. 2015, 39, 328–341. [Google Scholar] [CrossRef]
  14. Choi, L.K.; Cormack, L.K.; Bovik, A.C. Eccentricity effect of motion silencing on naturalistic videos. In Proceedings of the 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, FL, USA, 14–16 December 2015. [Google Scholar]
  15. Choi, L.K.; Cormack, L.K.; Bovik, A.C. LIVE Flicker Video Database. 2015. Available online: http://live.ece.utexas.edu/research/quality/live_flicker_video.html (accessed on 18 April 2021).
  16. Ghadiyaram, D.; Bovik, A.C.; Yeganeh, H.; Kordasiewicz, R.; Gallant, M. Study of the effects of stalling events on the quality of experience of mobile streaming videos. In Proceedings of the 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Atlanta, GA, USA, 3–5 December 2014. [Google Scholar]
  17. Ghadiyaram, D.; Bovik, A.; Yeganeh, H.; Kordasiewicz, R.; Gallant, M. LIVE Mobile Stall Video Database, 2016. Available online: http://live.ece.utexas.edu/research/LIVEStallStudy/index.html (accessed on 18 April 2021).
  18. Ghadiyaram, D.; Pan, J.; Bovik, A.C.; Moorthy, A.K.; Panda, P.; Yang, K.C. In-Capture Mobile Video Distortions: A Study of Subjective Behavior and Objective Algorithms. IEEE Trans. Circuits Syst. Video Technol. 2018, 28, 2061–2077. [Google Scholar] [CrossRef]
  19. Ghadiyaram, D.; Pan, J.; Bovik, A.C.; Moorthy, A.K.; Panda, P.; Yang, K.C. LIVE-Qualcomm Mobile In-Capture Video Quality Database, 2017. Available online: http://live.ece.utexas.edu/research/incaptureDatabase/index.html (accessed on 18 April 2021).
  20. Bampis, C.G.; Li, Z.; Moorthy, A.K.; Katsavounidis, I.; Aaron, A.; Bovik, A.C. Study of Temporal Effects on Subjective Video Quality of Experience. IEEE Trans. Image Process. 2017, 26, 5217–5231. [Google Scholar] [CrossRef] [PubMed]
  21. Bampis, C.G.; Li, Z.; Moorthy, A.K.; Katsavounidis, I.; Aaron, A.; Bovik, A.C. LIVE Netflix Video Quality of Experience Database, 2016. Available online: http://live.ece.utexas.edu/research/LIVE_NFLXStudy/nflx_index.html (accessed on 18 April 2021).
  22. Ghadiyaram, D.; Pan, J.; Bovik, A.C. A Subjective and Objective Study of Stalling Events in Mobile Streaming Videos. IEEE Trans. Circuits Syst. Video Technol. 2019, 29, 183–197. [Google Scholar] [CrossRef]
  23. Ghadiyaram, D.; Pan, J.; Bovik, A. LIVE Mobile Stall Video Database-II, 2017. Available online: http://live.ece.utexas.edu/research/LIVEStallStudy/index.html (accessed on 18 April 2021).
  24. Bampis, C.G.; Li, Z.; Katsavounidis, I.; Huang, T.Y.; Ekanadham, C.; Bovik, A. Towards Perceptually Optimized End-to-end Adaptive Video Streaming. arXiv 2018, arXiv:1808.03898. [Google Scholar]
  25. Sinno, Z.; Bovik, A.C. Large Scale Subjective Video Quality Study. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018. [Google Scholar]
  26. Sinno, Z.; Bovik, A.C. Large-Scale Study of Perceptual Video Quality. IEEE Trans. Image Process. 2019, 28, 612–627. [Google Scholar] [CrossRef] [PubMed]
  27. Sinno, Z.; Bovik, A. LIVE Video Quality Challenge Database, 2018. Available online: http://live.ece.utexas.edu/research/LIVEVQC/index.html (accessed on 18 April 2021).
  28. VQEG Database. 2010. Available online: https://www.its.bldrdoc.gov/vqeg/projects/frtv-phase-i/frtv-phase-i.aspx (accessed on 18 April 2021).
  29. Zhang, F.; Moss, F.M.; Baddeley, R.; Bull, D.R. BVI-HD: A Video Quality Database for HEVC Compressed and Texture Synthesized Content. IEEE Trans. Multimed. 2018, 20, 2620–2630. [Google Scholar] [CrossRef]
  30. Papadopoulos, M.A.; Zhang, F.; Agrafiotis, D.; Bull, D. A video texture database for perceptual compression and quality assessment. In Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 27–30 September 2015. [Google Scholar]
  31. Mackin, A.; Zhang, F.; Bull, D.R. A study of subjective video quality at various frame rates. In Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 27–30 September 2015. [Google Scholar]
  32. Rao, R.R.R.; Goring, S.; Robitza, W.; Feiten, B.; Raake, A. AVT-VQDB-UHD-1: A Large Scale Video Quality Database for UHD-1. In Proceedings of the 2019 IEEE International Symposium on Multimedia (ISM), San Diego, CA, USA, 9–11 December 2019. [Google Scholar]
  33. Mercat, A.; Viitanen, M.; Vanne, J. UVG dataset. In Proceedings of the 11th ACM Multimedia Systems Conference, Istanbul, Turkey, 28 September–1 October 2020. [Google Scholar]
  34. Song, L.; Tang, X.; Zhang, W.; Yang, X.; Xia, P. The SJTU 4K video sequence dataset. In Proceedings of the 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX), Klagenfurt am Wörthersee, Austria, 3–5 July 2013. [Google Scholar]
  35. IVP Database. 2011. Available online: http://ivp.ee.cuhk.edu.hk/index.shtml (accessed on 18 April 2021).
  36. Péchard, S.; Pépion, R.; Le Callet, P. Suitable Methodology in Subjective Video Quality Assessment: A Resolution Dependent Paradigm. In Proceedings of the International Workshop on Image Media Quality and Its Applications, IMQA2008, Kyoto, Japan, 5–6 September 2008; Volume 6. [Google Scholar]
  37. The Konstanz Natural Video Database. Available online: http://database.mmsp-kn.de/konvid-1k-database.html (accessed on 18 April 2021).
  38. Hosu, V.; Hahn, F.; Jenadeleh, M.; Lin, H.; Men, H.; Sziranyi, T.; Li, S.; Saupe, D. The Konstanz Natural Video Database (KoNViD-1k). In Proceedings of the 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX), Erfurt, Germany, 31 May– 2 June 2017; pp. 1–6. [Google Scholar]
  39. Lin, J.Y.; Song, R.; Wu, C.H.; Liu, T.; Wang, H.; Kuo, C.C.J. MCL-V: A streaming video quality assessment database. J. Vis. Commun. Image Represent. 2015, 30, 1–9. [Google Scholar] [CrossRef]
  40. Wang, H.; Gan, W.; Hu, S.; Lin, J.Y.; Jin, L.; Song, L.; Wang, P.; Katsavounidis, I.; Aaron, A.; Kuo, C.C.J. MCL-JCV: A JND-based H 264/AVC video quality assessment dataset. In Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25–28 September 2016. [Google Scholar]
  41. Lee, J.S.; Simone, F.D.; Ramzan, N.; Zhao, Z.; Kurutepe, E.; Sikora, T.; Ostermann, J.; Izquierdo, E.; Ebrahimi, T. Subjective evaluation of scalable video coding for content distribution. In Proceedings of the International Conference on Multimedia—MM ’10, Firenze, Italy, 25–29 October 2010; pp. 65–72. [Google Scholar]
  42. Lee, J.S.; Simone, F.D.; Ebrahimi, T. Subjective Quality Evaluation via Paired Comparison: Application to Scalable Video Coding. IEEE Trans. Multimed. 2011, 13, 882–893. [Google Scholar] [CrossRef]
  43. Paudyal, P.; Battisti, F.; Carli, M. A Study On The Effects of Quality of Service Parameters on Perceived Video Quality. In Proceedings of the 5th European Workshop on Visual Information Processing, EUVIP 2014, Paris, France, 10–12 December 2014. [Google Scholar]
  44. Keimel, C.; Habigt, J.; Habigt, T.; Rothbucher, M.; Diepold, K. Visual quality of current coding technologies at high definition IPTV bitrates. In Proceedings of the 2010 IEEE International Workshop on Multimedia Signal Processing, Saint-Malo, France, 4–6 October 2010. [Google Scholar]
  45. Keimel, C.; Redl, A.; Diepold, K. The TUM high definition video datasets. In Proceedings of the 2012 Fourth International Workshop on Quality of Multimedia Experience, Melbourne, VIC, Australia, 5–7 July 2012. [Google Scholar]
  46. Řeřábek, M.; Ebrahimi, T. Comparison of compression efficiency between HEVC/H.265 and VP9 based on subjective assessments. In Applications of Digital Image Processing XXXVII; Tescher, A.G., Ed.; SPIE: Bellingham, WA, USA, 2014. [Google Scholar]
  47. Ramzan, N.; Pervez, Z.; Amira, A. Quality of experience evaluation of H 265/MPEG-HEVC and VP9 comparison efficiency. In Proceedings of the 2014 26th International Conference on Microelectronics (ICM), Doha, Qatar, 14–17 December 2014. [Google Scholar]
  48. Bienik, J.; Uhrina, M.; Vaculik, M.; Mizdos, T. Perceived Quality of Full HD Video—Subjective Quality Assessment. Adv. Electr. Electron. Eng. 2016, 14. [Google Scholar] [CrossRef]
  49. Xu, J.; Jiang, X. Assessment of subjective video quality on 4K Ultra High Definition videos. In Proceedings of the 2015 8th International Congress on Image and Signal Processing (CISP), Shenyang, China, 14–16 October 2015. [Google Scholar]
  50. Herrou, G.; Hamidouche, W.; Ducloux, X. HDR video quality evaluation of HEVC and VP9 codecs. In Proceedings of the 2016 Picture Coding Symposium (PCS), Nuremberg, Germany, 4–7 December 2016. [Google Scholar]
  51. Dumic, E.; Grgic, S.; Sakic, K.; Frank, D. Subjective quality assessment of H 265 versus H.264 Video Coding for High-Definition Video Systems. In Proceedings of the 2015 13th International Conference on Telecommunications (ConTEL), Graz, Austria, 13–15 July 2015. [Google Scholar]
  52. Milovanovic, D.; Milicevic, Z.; Bojkovic, Z. MPEG video deployment in digital television: HEVC vs AVC codec performance study. In Proceedings of the 2013 11th International Conference on Telecommunications in Modern Satellite, Cable and Broadcasting Services (TELSIKS), Nis, Serbia, 16–19 October 2013. [Google Scholar]
  53. Sotelo, R.; Joskowicz, J.; Anedda, M.; Murroni, M.; Giusto, D.D. Subjective video quality assessments for 4K UHDTV. In Proceedings of the 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Cagliari, Italy, 7–9 June 2017. [Google Scholar]
  54. Kufa, J.; Polak, L.; Kratochvil, T. HEVC/H 265 vs. VP9 for Full HD and UHD video: Is there any difference in QoE? In Proceedings of the 2016 International Symposium ELMAR, Zadar, Croatia, 12–14 September 2016. [Google Scholar]
  55. Deep, V.; Elarabi, T. HEVC/H 265 vs. VP9 state-of-the-art video coding comparison for HD and UHD applications. In Proceedings of the 2017 IEEE 30th Canadian Conference on Electrical and Computer Engineering (CCECE), Windsor, ON, Canada, 30 April–3 May 2017. [Google Scholar]
  56. Akyazi, P.; Ebrahimi, T. Comparison of Compression Efficiency between HEVC/H 265, VP9 and AV1 based on Subjective Quality Assessments. In Proceedings of the 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX), Cagliari, Italy, 29 May–1 June 2018. [Google Scholar]
  57. Zhu, Y.; Song, L.; Xie, R.; Zhang, W. SJTU 4K video subjective quality dataset for content adaptive bit rate estimation without encoding. In Proceedings of the 2016 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Nara, Japan, 1–3 June 2016. [Google Scholar]
  58. ITU-R. Recommendation ITU-R BT.500-14—Subjective Video Quality Assessment Methods for Multimedia Applications. 2019. Available online: https://www.itu.int/rec/R-REC-BT.500 (accessed on 18 April 2021).
  59. Mitsu. Mitsu Tool. Available online: http://vq.kt.agh.edu.pl/metrics.html (accessed on 18 April 2021).
  60. Uhrina, M.; Bienik, J.; Mizdos, T. QoE on H 264 and H.265: Crowdsourcing versus Laboratory Testing. In Proceedings of the 2020 30th International Conference Radioelektronika (RADIOELEKTRONIKA), Bratislava, Slovakia, 15–16 April 2020; Volume 30, pp. 1–6. [Google Scholar]
  61. FFmpeg. A Complete, Cross-Platform Solution to Record, Convert and Stream Audio and Video ffmpeg.org. Available online: https://www.ffmpeg.org (accessed on 18 April 2021).
  62. Uhrina, M.; Bienik, J.; Vaculik, M. Coding efficiency of HEVC/H 265 and VP9 compression standards for high resolutions. In Proceedings of the 2016 26th International Conference Radioelektronika (RADIOELEKTRONIKA), Kosice, Slovakia, 19–20 April 2016. [Google Scholar]
  63. ITU-T. Recommendation ITU-T P.910—Subjective Video Quality Assessment Methods for Multimedia Applications. 2008. Available online: https://www.itu.int/rec/T-REC-P.910-200804-I/en (accessed on 18 April 2021).
  64. ITU-T. Recommendation ITU-T P.1401—Methods, Metrics and Procedures for Statistical Evaluation, Qualification and Comparison of Objective Quality Prediction Models. 2020. Available online: https://www.itu.int/rec/T-REC-P.1401-202001-I/en (accessed on 18 April 2021).
  65. Judd, C.M.; McClelland, G.H.; Ryan, C.S. Data Analysis: A Model Comparison Approach to Regression, ANOVA, and Beyond; Routledge: Oxfordshire, UK, 2017. [Google Scholar]
Sensors 21 02872 g001
Figure 1. Printscreens of used test sequences. Reprinted with permission from [60], Copyright 2021, Uhrina.
Figure 1. Printscreens of used test sequences. Reprinted with permission from [60], Copyright 2021, Uhrina.
Sensors 21 02872 g001
Sensors 21 02872 g002
Figure 2. Spatial Information (SI) and Temporal Information (TI) diagram of used test sequences. Reprinted with permission from [60], Copyright 2021, Uhrina.
Figure 2. Spatial Information (SI) and Temporal Information (TI) diagram of used test sequences. Reprinted with permission from [60], Copyright 2021, Uhrina.
Sensors 21 02872 g002
Sensors 21 02872 g003
Figure 3. Process of preparing the test sequences: chroma subsampling, bit depth, and resolution changing.
Figure 3. Process of preparing the test sequences: chroma subsampling, bit depth, and resolution changing.
Sensors 21 02872 g003
Sensors 21 02872 g004
Figure 4. Complete process of coding and assessing the video quality.
Figure 4. Complete process of coding and assessing the video quality.
Sensors 21 02872 g004
Sensors 21 02872 g005
Figure 5. Comparison of Mean Opinion Score (MOS) values obtained from different laboratories. Each spot represents MOS values for corresponding codec, resolution, and test sequence
Figure 5. Comparison of Mean Opinion Score (MOS) values obtained from different laboratories. Each spot represents MOS values for corresponding codec, resolution, and test sequence
Sensors 21 02872 g005
Sensors 21 02872 g006
Figure 6. Comparison of MOS values obtained from different laboratories. Each spot represents averaged MOS values from particular test sequences for corresponding codec and resolution.
Figure 6. Comparison of MOS values obtained from different laboratories. Each spot represents averaged MOS values from particular test sequences for corresponding codec and resolution.
Sensors 21 02872 g006
Sensors 21 02872 g007
Figure 7. Bitrate impact on the perceived video quality (defined by the MOS score with associated Confidence Interval (CI)) depending on codec and resolution for both laboratories independently. Each curve represents MOS values for each type of used test sequence.
Figure 7. Bitrate impact on the perceived video quality (defined by the MOS score with associated Confidence Interval (CI)) depending on codec and resolution for both laboratories independently. Each curve represents MOS values for each type of used test sequence.
Sensors 21 02872 g007
Sensors 21 02872 g008
Figure 8. Bitrate impact on the perceived video quality (defined by the MOS score with associated CI) depending on the codec and resolution for both laboratories jointly. Each curve represents averaged MOS values from both laboratories for each type of used test sequence.
Figure 8. Bitrate impact on the perceived video quality (defined by the MOS score with associated CI) depending on the codec and resolution for both laboratories jointly. Each curve represents averaged MOS values from both laboratories for each type of used test sequence.
Sensors 21 02872 g008
Sensors 21 02872 g009
Figure 9. Bitrate impact on the perceived video quality (defined by the MOS score with associated CI) depending on used test sequence. Each curve represents averaged MOS values from both laboratories for corresponding codec and resolution.
Figure 9. Bitrate impact on the perceived video quality (defined by the MOS score with associated CI) depending on used test sequence. Each curve represents averaged MOS values from both laboratories for corresponding codec and resolution.
Sensors 21 02872 g009
Sensors 21 02872 g010
Figure 10. Bitrate impact on the perceived video quality (defined by the MOS score with associated CI). Each curve represents averaged MOS values from both laboratories for corresponding codec and resolution—average MOS score.
Figure 10. Bitrate impact on the perceived video quality (defined by the MOS score with associated CI). Each curve represents averaged MOS values from both laboratories for corresponding codec and resolution—average MOS score.
Sensors 21 02872 g010
Table 1. Parameters of test sequences.
Table 1. Parameters of test sequences.
ResolutionChroma SubsamplingBit DepthAspect RatioFramerate [fps]Length [Seconds]
3840 × 2160 (UHD)4:4:410 bits per channel16:93010
Table 2. Characteristics of the test sequences.
Table 2. Characteristics of the test sequences.
Test SequenceDescriptionTest SequenceDescription
Bund Nightscapeis a video sequence portraying the above view of a night city crossed by a busy road next the river. The time-lapse video is captured from a high angle with a steady camera as one extreme long shot. The scene is relatively static, except for the accelerated movement of cars driving on the road, people passing by, flags waving in the wind and flashing lights.Marathonis a video sequence picturing a large group of people in colorful apparel running a race on an asphalt road on a rainy day. The sequence was filmed from bird’s eye perspective with almost no camera movement as a very long shot. The scene is rather dynamic, given almost the entire frame is filled by running marathon participants and raindrops falling on the wet road.
Campfire Partyis a night time video sequence depicting a group of people posing for a photograph behind a large campfire. The long shot is captured by a stationary camera, which zooms in slightly at the end of the video. The motion in the scene is caused mainly by a flashing fire in the foreground and a woman who briefly runs out of and back into the shot.Runnersis a video sequence that captures athletes running on a tree lined road in a cloudy weather. The racers in the very long shot are approaching the stationary camera, which is positioned approximately at their eye level. The scene contains a considerable amount of motion caused by rushing contestants and by the wind in the treetops.
Construction Fieldis a very still video sequence capturing construction equipment in the middle of a building site during excavation work. A hand-held camera was used to film the very long shot from a high angle. The only moving objects in the scene are an excavator digging a foundation pit and people slowly walking in the background.Tall Buildingsis a video sequence portraying the tallest skyscrapers and busy intersections in Shanghai, with a grand river in the background. The video was captured from a bird’s eye view using a camera that slowly pans to take a panoramic extreme long shot. The movement in the scene is primarily a result of the panning motion of the camera and partially of the cars driving fast at a deep distance.
Fountainsis a video sequence focused on several fountains in the left of a housing estate with multiple trees and apartment buildings in the background. The video is captured by a static camera as a long shot. All the motion in the scene can be attributed to water gushing from the fountain jets and droplets evaporating into the air.Woodis a video sequence picturing a tall forest during a sunny autumn day. The video was filmed from a low angle as a long shot with a camera performing a moderately fast panning motion. All the movement in the scene can be attributed to the camera pan and the resulting change in the angle of the sunlight rays incident on the lens.
Table 3. Types of used displays.
Table 3. Types of used displays.
Type of AssessmentType of Display
UNIZA − FHDSamsung LE40C750R2W FHD
UNIZA − UHDSamsung U24E590D UHD
VSB − FHD + UHD24” Dell P2415Q UHD
Table 4. Statistical characteristic of the observers.
Table 4. Statistical characteristic of the observers.
UniversityResolutionNumber of MenNumber of WomenAverage Age
UNIZAFHD25524
UNIZAUHD21922
VSBFHD + UHD151525
UNIZA + VSBFHD + UHD612924
Table 5. Correlation of MOS score between the laboratories.
Table 5. Correlation of MOS score between the laboratories.
Pearson CCRMSE
FHD-H.2640.970.30
FHD-H.2650.990.31
UHD-H.2641.000.10
UHD-H.2650.980.23
Table 6. Three-way Analysis of Variance (ANOVA) using video codec as a criterion.
Table 6. Three-way Analysis of Variance (ANOVA) using video codec as a criterion.
H.264
Source of VariationSum of SquaresDegrees of FreedomMean SquareF-Valuep-Value
Bitrate (X1)2541.954635.488982.840
Scene Type (X2)134.03719.14829.610
Resolution (X3)106.681106.682164.990
X1*X2106.58283.8025.890
X1*X334.2448.56113.240
X2*X312.1671.7372.690.009
Error1518.1823480.647
Total4453.832399
H.265
Bitrate (X1)1875.054468.764669.560
Scene Type (X2)90.96712.99418.560
Resolution (X3)0.1210.120.17 0.6784
X1*X288.31283.1544.510
X1*X37.9641.992.840.0229
X2*X330.6574.3796.250
Error1643.852348
Total3736.912399
Table 7. Three-way ANOVA using video resolution as a criterion.
Table 7. Three-way ANOVA using video resolution as a criterion.
Full HD
Source of VariationSum of SquaresDegrees of FreedomMean SquareF-Valuep-Value
Bitrate (X1)2210.044552.509806.210
Scene Type (X2)82.43711.77617.180
Compression Standard (X3)0.0110.0070.01 0.9214
X1*X279.43282.8374.140
X1*X311.1142.7794.050.0028
X2*X316.2572.3223.390.0013
Error1609.1323480.685
Total4008.42399
Ultra HD
Bitrate (X1)2186.064546.515842.850
Scene Type (X2)156.8722.434.550
Compression Standard (X3)112.231112.234173.090
X1*X2145.9285.2118.040
X1*X352412.99920.050
X2*X312.3171.7592.710.0084
Error1522.4823480.648
Total4187.782399
Table 8. Minimum bitrate thresholds to achieve good (4) and fair (3) video quality.
Table 8. Minimum bitrate thresholds to achieve good (4) and fair (3) video quality.
MOS ScaleFHD-8bUHD-8b
H.264H.265H.264H.265
Good (4)7.50 Mbps7.50 Mbps11.55 Mbps9.00 Mbps
Fair (3)2.80 Mbps2.60 Mbps4.50 Mbps2.80 Mbps
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Uhrina, M.; Holesova, A.; Bienik, J.; Sevcik, L. Impact of Scene Content on High Resolution Video Quality. Sensors 2021, 21, 2872. https://doi.org/10.3390/s21082872

AMA Style

Uhrina M, Holesova A, Bienik J, Sevcik L. Impact of Scene Content on High Resolution Video Quality. Sensors. 2021; 21(8):2872. https://doi.org/10.3390/s21082872

Chicago/Turabian Style

Uhrina, Miroslav, Anna Holesova, Juraj Bienik, and Lukas Sevcik. 2021. "Impact of Scene Content on High Resolution Video Quality" Sensors 21, no. 8: 2872. https://doi.org/10.3390/s21082872

APA Style

Uhrina, M., Holesova, A., Bienik, J., & Sevcik, L. (2021). Impact of Scene Content on High Resolution Video Quality. Sensors, 21(8), 2872. https://doi.org/10.3390/s21082872

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop