Assessment of QoE for Video and Audio in WebRTC Applications Using Full-Reference Models

Round 1

Reviewer 1 Report

Dear authors, I appreciate your work and I can imagine how much effort you had invested. I provide you my remarks which I made during reading your article:

1/ However, WebRTC is signaling agnostic ...
/* At this point, I realized, that authors have missed related works about SIP. The WebRTC architectural model takes its inspiration from the SIP trapezoid and a lot of work has been done in the direction WebRTC-SIP-HTML5. Authors should include also a part with related work.
the inspiration is in this article https://ieeexplore.ieee.org/document/6495759

2/ Tab. 1, p.4
Pls. consider also MSE, it's a quite simple approach, nevertheless often used.

3/ Nevertheless, we did not carry experimentation based on delay because this parameter affects the end-to-end latency but not the media quality computed with FR algorithms.
- I understand, however, the delay affects a conversational quality significantly, so you should mention that it's valid for listening quality, for the case the described approach is correct.

4/ Regarding audio, we use the official implementations for PESQ, ViSQOL, and POLQA.
- Which implementation of POLQA have you used? On the provided link is not freely available code. You should explain in the article if you adopted an existing implementation or did you use a professional SW?

5/ How do you explain Fig. 4 and 5 in the right down in the case of POLQA? JItter is increasing and quality also. It seems that there is an effect of play-out buffer and the relation between audio coding, the size of the buffer and jitter. Probably an adaptive play-out buffer was used and it can happen in some circumstances, it will depend on the adaptive algorithm especially. The question is whether the situation is repeatable.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Overall, I found the paper to be well written and much of it to be well described. I felt confident that the authors performed the analyses carefully. However, I found some issues the authors did not talk about. I ask that the authors explicitly address each of my comments in their response. Therefore, I recommend that a minor revision is warranted. I explain my concerns in more detail below.

The related work section lists a significant number of related works, but there are too few conclusions drawn. What is learned from the cited work, how did it influence the study described in this paper?

I am wondering why the authors did not look for other forms of participant recruitment to reach different demographics. Nevertheless, that should be mentioned as this would have greatly impacted the outcome of this study.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf