Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications
Round 1
Reviewer 1 Report
The goal of this paper, as exposed by the authors, is to evaluate the use of VMAF in a different type of applications. The Introduction: 1.1 State-of-the-Art section (from 19 to 63) seems to be identical with [22] reference. Using a plagiarism detection software, I identify similarities with existing sources 30% (12%: “Boni García, Micael Gallego, Francisco Gortázar, Antonia Bertolino. Understanding and estimating quality of experience in WebRTC applications, Computing, 2018"). Please clarify the differences of the current manuscript.
The introduction sections include also the related work. Maybe it's better to write it separately. Related work section must contain actual works in the field, and should be improved with a comparative study between them.
The experimental data of the paper is relatively full. It's interesting to consider tests that include congestion or traffic constraint and different QoS scenarios.
Update the Reference section, introducing new interesting MDPI articles recently published. The authors should clearly clarify the contributions for this paper.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The paper presents a software tool to demonstrate that VMAF can be used to evaluate the video quality of WebRTC communications.
Althought the development of the tool, as well as the experiments performed, are explained in detail and appropriately, I think the details provided are not relevant enough to justify a journal publication. This detail is ok for a technical report, but not for a research dissemination. Indeed, the conclusion obtained is that VMAF and VIFp correlate with subjective measure better than SSIM and PSNR. But this result is not new, as it had already been proved in previous works.
Moreover, the authors justify the tests because WebRTC is a different use case than the one for which VMAF was developed. However, they do not test some of the possible problems, as audiovisual desynchronization.
Finally, in order to validate the results, I believe more than one sequence should be used, considereing different types of scenarios.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The paper has been improved sufficiently in order to be accepted for publication.
