# A Sign of Things to Come: Predicting the Perception of Above-the-Fold Time in Web Browsing

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### Research Questions

- Is it possible to develop a robust objective ATF metric with a consistent estimation behavior for both animated and non-animated content?
- Is it possible to integrate the proposed metric with SI to passively estimate the perceived performance of interactive web applications?

## 2. Background

#### 2.1. Structured Similarity Index Measure

## 3. Plausibly Complete Time (PCT)

#### 3.1. Algorithm

#### 3.1.1. Step 1: Preparation

#### 3.1.2. Step 2: Computation

- SSIM quality map matrix ($ssimMatrix$) is computed (see step 2.1 in Figure 4) based on the current video frame (i) and the next frame ($i+1$): $\mathrm{ssimMatrix}=\mathrm{SSIM}(F\left[i\right],F[i+1])$. The $ssimMatrix$ and video frame share the same dimension ($133\times 200$). Each value of the ($ssimMatri{x}^{(x,y)}$) represents the score of the similarity between the two corresponding pixels of two consecutive images (Section 2.1). As explained in Section 2.1, a score of 1 represents a pixel-wise 100% match, and an SSIM value of less than 1 shows a discrepancy between the two sets of input data.
- By applying a binary mask to the $ssimMatrix$, a binary matrix ($binaryMatrix$) is computed to allow the $i\mathrm{th}$ frame of the accumulator matrix to be calculated (see step 2.2 in Figure 4). If the $ssimMatri{x}^{(x,y)}$ is 1 (100% similarity), the corresponding value of $binaryMatrix$ will be 0. For any other value, $binaryMatri{x}^{(x,y)}$ will be set to 1. As a result, $binaryMatrix$ represents the pixels that have been changed between the two video frames, regardless of the degree of similarity.
- Finally, a new binary accumulator matrix ($c{m}_{i}$) is computed (see step 2.3 in Figure 4). It is achieved by performing a logical OR ($\left|\right|$) between $c{m}_{i-1}$ and the $binaryMatrix$ ($c{m}_{i}=c{m}_{i-1}\phantom{\rule{0.277778em}{0ex}}\left|\right|\phantom{\rule{0.277778em}{0ex}}binaryMatrix$). The computed ($c{m}_{i}$) represents the number of pixels that have been changed at the current iteration.

#### 3.1.3. Step 3: Estimation

Algorithm 1: Plausibly Complete Time (PCT) |

## 4. Evaluation

- ATF time estimation.
- Its influence on the result of SI for different content types.
- PCT in the wild. The term “in the wild” is used to refer to unseen data collected by other researchers from globally accessible websites, under an undefined network condition and is not a synthetic dataset.
- The effect of using PCT on QoE estimation models.

#### 4.1. PCT and the Estimation of ATF Time

#### 4.2. The Influence of PCT on SpeedIndex (SI)

#### 4.3. Impact of PCT on QoE Estimation

#### 4.4. PCT in the Wild

#### 4.5. Limitations and Future Enhancements

## 5. Discussion on SI, PCT, and the User’s Interactions

## 6. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Miller, M. Cloud Computing: Web-Based Applications that Change the Way you Work and Collaborate Online; Que Publishing: Indianapolis, IN, USA, 2008. [Google Scholar]
- Miller, M. B2B digital Marketing: Using the Web to Market Directly to Businesses; Que Publishing: Indianapolis, IN, USA, 2012. [Google Scholar]
- Alreshoodi, M.; Woods, J. Survey on QoE/QoS correlation models for multimedia services. Int. J. Distrib. Parallel Syst.
**2013**, 4, 53. [Google Scholar] [CrossRef] - Jahromi, H.Z.; Hines, A.; Delaney, D.T. Towards Application-Aware Networking: ML-Based End-to-End Application KPI/QoE Metrics Characterization in SDN. Available online: https://researchrepository.ucd.ie/rest/bitstreams/39947/retrieve (accessed on 12 February 2021).
- Callet, P.L.; Möller, S.; Perkis, A.E. Qualinet White Paper on Definitions of Quality of Experience. Available online: https://hal.archives-ouvertes.fr/hal-00977812/document (accessed on 12 February 2021).
- da Hora, D.N.; Asrese, A.S.; Christophides, V.; Teixeira, R.; Rossi, D. Narrowing the Gap between QoS Metrics and Web QoE Using Above-the-Fold Metrics. Available online: https://hal.inria.fr/hal-01677260/document (accessed on 12 February 2021).
- Fiedler, M.; Hossfeld, T.; Tran-Gia, P. A generic quantitative relationship between quality of experience and quality of service. IEEE Netw.
**2010**, 24, 36–41. [Google Scholar] [CrossRef] [Green Version] - Egger, S.; Reichl, P.; Hossfeld, T.; Schatz, R. “Time is bandwidth”? Narrowing the gap between subjective time perception and Quality of Experience. In Proceedings of the 2012 IEEE International Conference on Communications (ICC), Ottawa, ON, Canada, 10–15 June 2012. [Google Scholar]
- SpeedIndex (SI)—Google Developers. 2018. Available online: https://web.dev/speed-index/ (accessed on 10 December 2019).
- Jahromi, H.Z.; Delaney, D.T.; Hines, A. How Crisp is the Crease? A Subjective Study on Web Browsing Perception of Above-The-Fold. In Proceedings of the 2020 6th IEEE Conference on Network Softwarization (NetSoft), Ghent, Belgium, 29 June–3 July 2020. [Google Scholar]
- Hossfeld, T.; Metzger, F.; Rossi, D. Speed index: Relating the industrial standard for user perceived web performance to Web QoE. In Proceedings of the 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX), Sardinia, Italy, 29–31 May 2018. [Google Scholar]
- Brutlag, J.; Abrams, Z.; Meenan, P. Above the fold time: Measuring web page performance visually. In Proceedings of the Velocity, the Web Performance and Operations conference, Santa Clara, CA, USA, 14–16 June 2011. [Google Scholar]
- Saverimoutou, A.; Mathieu, B.; Vaton, S. Web browsing measurements: An above-the-fold browser-based technique. In Proceedings of the 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), Vienna, Austria, 2–5 July 2018. [Google Scholar]
- Da Hora, D.; Rossi, D.; Christophides, V.; Teixeira, R. A practical method for measuring Web above-the-fold time. In Proceedings of the ACM SIGCOMM 2018 Conference on Posters and Demos, Budapest, Hungary, 20–25 August 2018. [Google Scholar]
- Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process.
**2004**, 13, 600–612. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Hore, A.; Ziou, D. Image quality metrics: PSNR vs. SSIM. In Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey, 23–26 August 2010. [Google Scholar]
- McFadden, S.B.; Ward, P.A. Selecting the proper window for SSIM. In Proceedings of the Image Quality and System Performance IX, Burlingame, CA, USA, 24 January 2012. [Google Scholar]
- Nilsson, J.; Akenine-Möller, T. Understanding Ssim. Available online: https://arxiv.org/pdf/2006.13846.pdf (accessed on 12 February 2021).
- Grondin, S. Processing time between visual events. Arq. Bras. Oftalmol.
**2003**, 66, 96–104. [Google Scholar] [CrossRef] - Jahromi, H.Z.; Rooney, B.; Delaney, D.T.; Hines, A. Establishing Waiting Time Thresholds in Interactive Web Mapping Applications for Network QoE Management. In Proceedings of the 30th Irish Signals and Systems Conference (ISSC 2019), Maynooth, Ireland, 17–18 June 2019. [Google Scholar]
- Dehaene, S. The neural basis of the Weber–Fechner law: A logarithmic mental number line. Trends Cognit. Sci.
**2003**, 7, 145–147. [Google Scholar] [CrossRef] - Bahr, G.S.; Ford, R.A. How and why pop-ups don’t work: Pop-up prompted eye movements, user affect and decision making. Comput. Hum. Behav.
**2011**, 27, 776–783. [Google Scholar] [CrossRef] - Cara, C.; Dumitrașciuc, L.F. GDPR consent pop-ups. How are we thinking about them? An Elaboration Likelihood perspective. J. Int. Bus. Manag.
**2021**, 4, 01–10. [Google Scholar] - Hong, W.; Thong, J.Y.; Tam, K.Y. How do Web users respond to non-banner-ads animation? The effects of task type and user experience. J. Am. Soc. Inf. Sci. Technol.
**2007**, 58, 1467–1482. [Google Scholar] [CrossRef] - Liu, W.; Liang, X.; Liu, F. The Effect of Webpage Complexity and Banner Animation on Banner Effectiveness in a Free Browsing Task. Int. J. Hum. Comput. Interact.
**2019**, 35, 1192–1202. [Google Scholar] [CrossRef] - Jahromi, H.Z.; Delaney, D.T.; Hines, A. Beyond First Impressions: Estimating Quality of Experience for Interactive Web Applications. Available online: https://ieeexplore.ieee.org/ielx7/6287639/8948470/09027906.pdf (accessed on 12 February 2021).

**Figure 1.**The loading process of a website from time that the URL is requested until the time that the website is completely loaded.

**Figure 2.**Illustration of the Visually Complete (VC) time (x-axis) and progress (y-axis) with example page rendering. SpeedIndex (SI) calculates the area above-the-curve from page request time, $t=0$, until the time at which the Above-the-fold (ATF) is 100% VC (i.e., ATF time). VC progress usually occurs before the Page Load Time (PLT) event. However, depending on the content, network, and computational conditions, ATF and PLT may occur simultaneously.

**Figure 3.**Prolonged ATF time estimation for websites with animated content. This is an example of a website where ATF estimation is overestimated due to animated components in the webpage. The grey area between the green and red lines is included in the ATF and SI estimation as a result of the animation changing. However, the ATF for the page should be considered complete at the green line time, not the end of the video. This sample is visualized based on the data collected in Reference [10] for harveynorman.ie under 3 Mbps bandwidth condition.

**Figure 4.**Visual illustration of the major steps of the Plausibly Complete Time (PCT) estimation. In order to improve the visual clarity of the figure, the dimensions of the matrices are reduced.

**Figure 6.**The Spearman Rank-Order Correlation Coefficient (SROCC) between PCT, Perceived ATF, Objective ATF, and PLT.

**Figure 7.**The relationship on estimation of ATF time. The left subplot shows the relationship between PCT (y-axis) and perceived ATF (x-axis). The middle subplot illustrates the relationship between objective ATF (y-axis) and perceived ATF (x-axis). The right subplot demonstrates the relationship between PCT (y-axis) and the Objective ATF (x-axis).

**Figure 8.**The influence of PCT on SpeedIndex (SI). The (

**left**) subplot represents the estimation of SI using PCT (y-axis) and its tight positive correlation with the corresponding SI computed based on ground truth perceived ATF (x-axis). The (

**right**) subplot shows the relationship between SI estimated based on the objective ATF (y-axis) and perceived ATF (x-axis).

**Figure 9.**Using PCT to estimate ATF for use in the Waiting time and its QoE evaluation on a linear ACR scale is Logarithmic (WQL) and IQX Quality of Experience (QoE) models. The (

**left**) subplot shows a high positive linear correlation between QoE estimation of WQL using PCT (y-axis) and the ground truth perceived ATF (x-axis). The (

**right**) subplot represents a positive correlation between QoE estimation of IQX using PCT (y-axis) and perceived ATF time (x-axis). Comparing these results with those presented in Reference [10] shows that PCT has positively improved QoE estimation for IQX and comparable results for WQL.

**Figure 10.**The performance evaluation of PCT in the wild. The (

**left**) subplot represents positive linear relationship and some degree of agreement on ATF estimation between PCT (x-axis) and the Objective ATF (y-axis). The (

**right**) subplot shows a tight positive linear relationship between SI estimated based on PCT (x-axis) and SI estimated using objective ATF. Since the majority of the test cases are the website without animations, the result confirms that computing SI using PCT yield a similar result as the SI computed based on state-of-the-art ATF metric. A post-analysis verification shows that the majority of the outliers are related to the websites with animations, pop-ups, or late loading contents (see Figure 11).

**Figure 11.**PCT estimation on a public data-set. In the (

**left**) column of subplots, the blue lines represents the cumulative visual progress of PCT, purple bars are the Similarity Index Measure (SSIM) scores between two consecutive frames, the dotted green line is the PCT frame time, and the solid red line shows the objective ATF frame. The (

**middle**) column shows the frames associated with PCT and the (

**right**) column shows the objective ATF frames.

**Figure 12.**VC progress calculation for an entire web mapping session. The blue line shows the visual progress over time. While the sky blue vertical line is the time that objective ATF occurred, the purple vertical lines show the user’s subsequent interactions.

**Figure 13.**An example illustration of SI measured in accordance with iLT time (user’s interactions) for an entire web mapping session. Cumulative SI (CU SI) is referring to a cumulative SI that measures SI from time 0 until each user’s interaction. CU SI demonstrates how the result of SI can artificially increase, if the subsequent interactions gets included in the computation. However, if the estimation gets split, guided by iLT (based on the user interaction), iSI can be computed, which can accurately represent a speed estimation of SI associated with each user’s interaction.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Jahromi, H.Z.; Delaney, D.; Hines, A.
A Sign of Things to Come: Predicting the Perception of Above-the-Fold Time in Web Browsing. *Future Internet* **2021**, *13*, 50.
https://doi.org/10.3390/fi13020050

**AMA Style**

Jahromi HZ, Delaney D, Hines A.
A Sign of Things to Come: Predicting the Perception of Above-the-Fold Time in Web Browsing. *Future Internet*. 2021; 13(2):50.
https://doi.org/10.3390/fi13020050

**Chicago/Turabian Style**

Jahromi, Hamed Z., Declan Delaney, and Andrew Hines.
2021. "A Sign of Things to Come: Predicting the Perception of Above-the-Fold Time in Web Browsing" *Future Internet* 13, no. 2: 50.
https://doi.org/10.3390/fi13020050