Media Multitasking Scale: Validation Study with Portuguese Adolescents

Abstract

The increasing presence of digital media has amplified the need to study media multitasking (both multiple media activities simultaneously and using media while doing non-media activities). Media Multitasking Scale (MMS) has been used to assess this phenomenon, but it is yet to be validated for a Portuguese population. This study analysed the validity of a Portuguese Version of the MMS (n = 171 Portuguese adolescents) based on inter-item correlations and confirmatory factor analysis. The results suggest changes in the original version to reflect more recent digital media tendencies for a better fit to the Portuguese adolescents’ sample. Additionally, results showed that Multitasking with Media and Non-Media and Concentration Without Multitasking were partially predictors of mental health problems (predictive validity), highlighting the scale’s utility in assessing media multitasking behaviours and their impact on psychological well-being.

1. Introduction

Digital Media is continually expanding, especially among adolescents who are regularly exposed to a variety of devices and activities online (e.g., Livingstone et al. 2011; Smahel et al. 2020). The proliferation of digital media use, together with its progressive flexibility and portability, led to a phenomenon called media multitasking. Initially described as engaging in more than one media activity at the same time, either on a single device or across multiple devices (Foehr 2006; Pilotta et al. 2004; Roberts and Foehr 2008), the evolution of this kind of behaviour has resulted into a broader definition. Nowadays, two types of media multitasking can be identified: engaging in multiple media activities simultaneously (e.g., scrolling on social media while watching a movie) and using media while doing non-media activities (e.g., watching a video while eating) (Baumgartner et al. 2014; Jeong and Hwang 2012; Wallis 2010).

Media multitasking has raised concerns about its negative impact on children and adolescents’ development. Research has emphasised the impact of media multitasking on youths’ functioning, especially on cognitive control abilities (e.g., Kong et al. 2023; Luo et al. 2020b, 2021; Ophir et al. 2009; Wiradhany and Nieuwenstein 2017), academic performance (e.g., Kokoç 2021; Uncapher et al. 2017), and psychological well-being (e.g., Becker et al. 2013; Jin et al. 2023; Luo et al. 2020a; Sanbonmatsu et al. 2013). A literature review by van der Schuur et al. (2015) also found small to moderate negative relationships between media multitasking and these three domains. However, some other studies examining the impact of media multitasking have produced mixed results and questioned whether this relationship may also be reverse (Parry and le Roux 2021; Shih 2013; Uncapher et al. 2017; van der Schuur et al. 2020).

Some measures have been developed to assess media multitasking. The Media Multitasking Index (MMI; Ophir et al. 2009) has been one of the primary and most frequent measures used, although researchers across time have been making modifications on the digital media activities it assesses (e.g., Alzahabi and Becker 2013; Becker et al. 2013; Lui and Wong 2012; Pea et al. 2012; Ralph et al. 2014; Sanbonmatsu et al. 2013). This instrument is a self-report measure that quantifies how frequently individuals engage in simultaneous media use, based on the proportion of time spent multitasking across different media types. Other measures, such as Smartphone Multitasking (SMM; Lim and Shim 2016); and the Media Multitasking Measure—short (Baumgartner et al. 2017), are also commonly used to assess this type of behaviour. The SMM assesses the frequency and nature of using multiple smartphone functions simultaneously (e.g., texting while watching videos or browsing social media while chatting) (Lim and Shim 2016). The Media Multitasking Measure—short is a brief measure for assessing nine types of media activities (e.g., watching TV, sending messages) and the simultaneous engagement in these activities (Baumgartner et al. 2017).

Some limitations on media multitasking assessment have been identified, such as (a) most of the existing instruments do not consider media multitasking behaviours that involve both media and non-media activities (e.g., Baumgartner et al. 2014); (b) some media activities’ combinations included in the measures are not aligned with today’s youth digital experiences; (c) the complexity of the questionnaires (i.e., number of activity combinations; response measures) may turn them unfriendly (e.g., Segijn et al. 2019); and (d) most measures still present low levels of construct validity or internal consistency (Luo et al. 2018).

Considering the limitations mentioned above, Luo et al. (2018) developed the Media Multitasking Scale (MMS) to assess media multitasking behaviour among adolescents. The MMS was developed considering the operational definition of media multitasking by incorporating both media and non-media activities, with support from prior research (e.g., Baumgartner et al. 2014; Lim and Shim 2016). It included items related to individuals’ focus on media or non-media activities without being distracted by media, and was demonstrated to be a reliable tool considering the results of the original scale and its sub-dimensions that showed good internal consistency (Luo et al. 2018, 2022). Overall, the MMS was demonstrated to be a user-friendly, promising conceptual and psychometric property.

To our knowledge, no questionnaire has been developed or validated in Portugal to comprehensively assess the media multitasking construct in adolescents. This paper details the adaptation and validation process of the Media Multitasking Scale by Luo et al. (2018) and presents an initial study of the MMS’s psychometric characteristics in Portugal, focusing on its application within a sample of adolescents.

2. Materials and Methods

2.1. Participants

The sample consisted of 171 adolescents from two Portuguese schools (one public school and one private school). Participants’ ages ranged from 11 to 16 years old (M = 13.7; SD = 1.1). Of these, 53.2% identified as female, 45.0% as male, 0.6% as non-binary, and 1.2% preferred not to say. The participants were in seventh (28.2%), eighth (16.8%), or ninth (55.1%) grades.

2.2. Procedure

Before data collection, the Media Multitasking Scale was translated, and the items were adapted following the guidelines for translating and adapting items (Erkut 2010). After that, a think-aloud procedure was conducted for a readability check. This moment involved four students, aged between 13 and 14 years old. All items were presented individually, and participants were asked to comment based on item interpretation and rephrasing suggestions, if needed. All comments were registered, and minor changes were done in accordance.

The study followed ethical guidelines with all participants and their legal guardians providing informed consent. An online survey was created on the Qualtrics platform. Data was collected in a classroom setting with the presence of a member of the research team. To access the questionnaire, a QR code was projected for the participants to scan with their smartphones. In case of difficulties, paper copies were provided. Media Multitasking Scale items were randomised on the online platform to overcome order bias. Data were collected in May and June 2023.

2.3. Measures

2.3.1. Sociodemographic Questionnaire

The protocol included a sociodemographic questionnaire, collecting information about age, gender, and school year.

2.3.2. Media Multitasking Scale

The Media Multitasking Scale (MMS; Luo et al. 2018) is a self-report questionnaire that aims to measure media multitasking behaviour among adolescents. The original version includes 14 items presented on a five-point Likert scale (1 = Never, 2 = Seldom, 3 = Sometimes, 4 = Often, and 5 = Always). It consists of three subscales: Multitasking Across Two Media (MAM, five items, α in the original version = 0.82); Multitasking with Media and Non-Media (MMNM, four items, α in the original version = 0.64); and Concentration Without Multitasking (CWM, five items, α in the original version = 0.81). The latter subscale has reversed items, with higher scores meaning higher concentration.

For the Portuguese version of the scale, two items were added to the Multitasking with Media and Non-Media (MMNM) dimension: item 15 (“While studying, I am also on digital media”) and item 16 (“While trying to fall asleep, I am also on digital media”). These items were added considering the increased use and new trends post the COVID-19 pandemic and research outcomes on the impact of digital media use in non-media activities (Boniel-Nissim et al. 2023; Jin et al. 2023; Kokoç 2021). Also, items 1–6, 8, and 9 were adapted with minor aspects, considering cultural features applied to the Portuguese population (see Appendix A).

2.3.3. Brief Problem Monitor—Youth

The Brief Problem Monitor (BPM-Y; Achenbach et al. 2011; Portuguese adaptation by Project MIPA-Mobile 2015) is a brief rating form for monitoring children’s functioning. Each item (e.g., “I am inattentive or easily distracted”) was rated 0 = Not True, 1 = Somewhat True, or 2 = Very True. The youths’ version includes 19 items allocated in three dimensions: Attention Problems (Portuguese validation study—α normative sample = 0.72; α clinical sample = 0.70), Internalizing Problems (Portuguese validation study—α normative sample = 0.68; α clinical sample = 0.70), and Externalizing Problems (Portuguese validation study—α normative sample = 0.64; α clinical sample = 0.70). The sum of all the items provides the Total Problems Score (Portuguese validation study—α normative sample = 0.79; α clinical sample = 0.79). Higher scores suggest poorer mental health.

2.4. Data Analysis and Preliminary Steps

The first step consisted of assessing whether the collected sample was sufficient for the analyses undertaken. According to Westland (2010), the minimum sample required to conduct structural equation modeling can be calculated using the following formula: N ≥ 50r² − 450r + 1100, in which r refers to the number of items divided by the number of latent factors. Thus, the minimum sample for conducting the confirmatory factorial analysis (14 items/three factors, thus r = 4.67) was 89 participants. The second step consisted of analysing the following assumptions: absence of missing data, normality distribution, and non multicollinearity (cf. Marôco 2014) for all variables of the study.

Then, the analyses for validating the MMS to a Portuguese sample were conducted. First, inter-item correlations were analysed. Second, a Confirmatory Factor Analysis (CFA) was conducted, and the internal consistency of factors was calculated using Cronbach’s Alpha. The original factorial structure was also compared with our proposed version of the scale (including the two items added; cf. “Measures section”). Finally, MMS dimensions’ predictive validity was tested using hierarchical linear regression.

3. Results

3.1. Preliminary Analyses

Regarding preliminary analyses, the assumptions regarding the absence of missing data, normality distribution, and no multicollinearity (cf. Marôco 2014) were all met. Results are presented in

Table 1. Descriptive statistics for MMS variables (n = 171).

	min	max	M	SD	sk	ku	VIF
MMS-Item 1	1	5	2.13	1.17	0.74	−0.33	1.75
MMS-Item 2	1	5	2.13	1.16	0.81	−0.18	1.79
MMS-Item 3	1	5	2.58	1.30	0.17	−1.23	1.59
MMS-Item 4	1	5	2.56	1.12	0.14	−0.84	1.46
MMS-Item 5	1	5	2.46	1.31	0.35	−1.10	1.64
MMS-Item 6	1	5	2.33	1.18	0.56	−0.67	1.12
MMS-Item 7	1	5	3.18	1.48	−0.31	−1.34	1.05
MMS-Item 8	1	5	1.93	0.92	1.05	1.06	1.22
MMS-Item 9	1	5	1.65	0.86	1.37	1.85	1.24
MMS-Item 10	1	5	3.82	1.16	−0.86	−0.16	1.10
MMS-Item 11	1	5	4.18	1.02	−1.14	0.55	1.30
MMS-Item 12	1	5	3.53	1.23	−0.48	−0.68	1.14
MMS-Item 13	1	5	4.20	0.96	−1.28	1.31	1.41
MMS-Item 14	1	5	4.13	1.09	−1.33	1.19	1.39
MMS-Item 15	1	5	2.00	1.08	0.93	0.02	1.26
MMS-Item 16	1	5	2.52	1.42	0.42	−1.17	1.17

. In what regards normal distribution, Kline’s (2015) suggestion of skewness between −3 and 3, and kurtosis between -10 and 10 was followed. Considering such standards, results showed that no severe deviations from normality were found in this sample (−1.33 > sk < 1.37; −1.34 > ku < 1.85) and, therefore, the assumption was met. Finally, VIF coefficients were analysed for checking the multicollinearity assumption (multicollinearity assumed if VIF > 5; cf. Marôco 2014). All VIF coefficients were below 1.79.

3.2. Original Version of MMS

Inter-item correlations among items belonging to the same factor in the original scale were conducted. Results are summarized in

Table 2. Inter-item Correlations for Each Factor.

	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.	11.	12.	13.
Multitasking Across Two Media (MAM)
Item 1
Item 2	0.65 ***
Item 3	0.04	0.03
Item 4	0.10	0.14	0.49 ***
Item 5	0.17 *	0.20 **	0.55 ***	0.49 ***
Multitasking with Media and Non-Media (MMNM)
Item 6
Item 7						0.13
Item 8						0.24 **	0.09
Item 9						0.20 *	0.13	0.31 ***
Concentration Without Multitasking (CWM)
Item 10
Item 11										0.17 *
Item 12										0.20 *	0.23 **
Item 13										0.24 **	0.40 ***	0.25 ***
Item 14										0.15	0.40 ***	0.28 ***	0.45 ***

* p < 0.050, ** p < 0.010, *** p < 0.001.

.

Regarding the MAM factor, items 1 and 2 were not correlated with items 3 and 4 and, in the MMNM factor, item 7 was not correlated with any other item of the factor. In the CWM factor, items 10 and 14 were not correlated among each other.

A CFA using AMOS (v.28) by IBM^® was conducted. For each factor, one trajectory was fixed—the criteria used consisted of fixing the trajectory to the item with higher factor loading considering the original scale (cf. Luo et al. 2018). The initial results showed a poor fit of the scale structure to the data: $X^2\left(74\right)$= 199.209; p < 0.001; $X^2/\mathrm{df}$= 2.692; CFI = 0.710, PCFI = 0.578; RMSEA = 0.100 (90% C.I. [0.083; 0.117], pclose < 0.001); SRMR = 0.0989. However, once modification indexes (>11) were considered and three items belonging to the same factor were correlated among one another (items 3, 4 and 5, MAM factor), the model showed an excellent fit to the data: $X^2\left(71\right)$= 80.092; p = 0.215; $X^2/\mathrm{df}$= 1.128; CFI = 0.979, PCFI = 0.764; RMSEA = 0.027 (90% C.I. [0.000; 0.054], pclose = 0.911); SRMR = 0.059. Regression weights are displayed in

Table 3. Path Coefficients of the Confirmatory Factor Analysis.

	b	SE	p	$\mathit{\beta }$
Item 1 -> MAM	1.00			0.733
Item 2 -> MAM	1.20	0.177	<0.001	0.888
Item 3 -> MAM	0.07	0.127	0.573	0.047
Item 4 -> MAM	0.22	0.110	0.046	0.167
Item 5 -> MAM	0.35	0.128	0.006	0.233
Item 6 -> MMNM	0.83	0.228	<0.001	0.369
Item 7 -> MMNM	0.34	0.262	0.200	0.120
Item 8 -> MMNM	0.93	0.197	<0.001	0.533
Item 9 -> MMNM	1.00			0.618
Item 10 -> CWM	0.58	0.173	<0.001	0.311
Item 11 -> CWM	0.99	0.172	<0.001	0.606
Item 12 -> CWM	0.83	0.190	<0.001	0.421
Item 13 -> CWM	1.00			0.654
Item 14 -> CWM	1.16	0.193	<0.001	0.663

MAM = Multitasking Across wo Media; MMNM = Multitasking with Media and Non-Media; CWM = Concentration Without Multitasking.

.

Table 3 showed that items 3 (MAM) and 7 (MMNM) did not significantly contribute to the respective factors and, therefore, its removal was considered. The remaining items were positively related to their respective factor, as expected. The MAM and MMNM factors were also positively correlated among each other (r = 0.66, p < 0.001) and both negatively correlated with the CWM (CWM <-> MAM: r = −0.40, p < 0.001; CWM <-> MMNM: r = −0.64, p < 0.001), consistently with the original instrument. The internal consistency of the three factors was analysed using Cronbach’s Alpha. The MAM (α = 0.67) and CWM (α = 0.66) factors presented internal consistency coefficients close to acceptable values. The same did not happen with MMNM factor (α = 0.43). Considering the latter result on internal consistency, the authors decided to adjust the MMNM dimension.

3.3. 15-Item Version of MMS

Considering the previous results, item 7 was removed, and two items were added to the MMNM dimension, with permission of the instrument’s authors (see Figure 1). The CFA of the 15-item version of the scale showed a good fit to the data: $X^2\left(84\right)$= 102.952; p = 0.078; $X^2/\mathrm{df}$= 1.226; CFI = 0.963, PCFI = 0.771; RMSEA = 0.036 (90% C.I. [0.000; 0.058], pclose = 0.828); SRMR = 0.062. When comparing this structure with the original one, the 15-item was slightly inferior [∆$X^2\left(13\right)$= 22.86, p = 0.043]; however, the internal consistency of the MMNM dimension improved considerably (α = 0.62).

Predictive Validity of the 15-Item Version of MMS

In order to test the predictive validity, a hierarchical multiple regression was conducted using the 15-item version. The model included MAM, MMNM, CWM, controlling gender, and age as the covariates, with mental health problems (BPM-Y) as the dependent variable.

Overall, the results showed that the model was significant F(5,159) = 11.39, p < 0.001, R² = 0.26. Both MMNM and CWM significantly predicted mental health problems (b = 0.19, t = 2.46, p = 0.015, and b = −0.32, t = 4.45, p < 0.001, respectively). More specifically, the more participants engage in multitasking with media and non-media activities and the lower their ability to concentrate without multitasking, the higher the frequency and intensity of behaviours related to mental health problems. MAM did not show significant associations (b = 0.10, t = 2.46, p = 0.153). Overall, when gender and age were included in the model, the variables explained 26.4% of the variance.

4. Discussion

Due to the constant presence of digital media in daily life, the need to have a comprehensive measure for media multitasking cannot be overstated. Media multitasking behaviours have been studied in the past decade, and mixed results on its impact empower researchers to relook to media multitasking measures (Segijn et al. 2019; van der Schuur et al. 2015). Thus, the present study aimed to validate the Media Multitasking Scale (Luo et al. 2018), and the results support the psychometric validity of using the Media Multitasking Scale for Portuguese adolescents to measure media multitasking behaviours.

First, we consider the adaptation of the original version. CFA was conducted with acceptable results. The internal consistency of the MMS factors ranged from low to acceptable. Then, item 7 (MMNM factor) was removed due to not being correlated with any other item of the factor nor significantly contributing to the respective factor. One possible explanation can be the formulation of the remaining items of MMNM factor that considers digital media generally, and understanding how adolescents use digital media nowadays, it may be difficult for them to isolate listening to music from other digital activities (e.g., social media) or just by checking the notifications received while studying.

Also, the two new items were added to the MMNM factor considering literature review on the prevalence and impact of digital media use while studying and sleeping (Boniel-Nissim et al. 2023; Jin et al. 2023; Kokoç 2021), especially considering this age range. One of the items added—“using digital media while studying”—had previously been present in the primary exploratory version of the original Media Multitasking Scale questionnaire and removed due to Principal Component Analysis results and statistic criteria (Luo et al. 2018).

The MAM and CWM factors present close to acceptable values for internal consistency coefficients. The MMNM factor was the one showing lower levels of internal consistency, which is in line with the results from the scale development (Luo et al. 2018). It can be explained by the possible different interpretations of the use of digital media. For instance, on the one hand, it can be perceived as a negative activity (e.g., using digital media while eating can be considered a negative distractor); and, on the other hand, as a positive activity (e.g., using digital media while talking to someone in person can be considered a positive social interaction). Therefore, although the items of the MMNM factor are related to one another (which justifies being aggregated into one factor), they might not measure the same construct.

Results from predictive validity showed that MMNM and CWM partially predict mental health problems, after controlling gender and age. These results are consistent with the literature on the negative impact of digital media use and media multitasking on mental health problems (e.g., Becker et al. 2013; O’Reilly 2020). MMNM and CWM factors consider daily non-media activities (e.g., eating, sleeping, walking, being with friends), which may be related to the negative psychological outcomes.

5. Conclusions

This validation of the Media Multitasking Scale suggests that it is a valid measure for assessing media multitasking behaviours in Portugal. However, some limitations should be noted. First, the unbalanced sample regarding school grade, since ninth grade (~55%) students could have resulted in biases in the analysis of age differences. Second, the instrument only assessed media multitasking through a self-reported questionnaire and could be important to add media device data to the analysis. Future studies should address and overcome the limitations identified above but also contribute to the consolidation of this instrument. Studies are needed for other relevant validity and reliability analysis.

In conclusion, our study has shown promising results regarding the psychometric properties of the Portuguese version of the MMS as a comprehensive measure of media multitasking behaviour. The MMS may be used in future research to improve knowledge in this field, strengthening research into the characterisation of the use and further impact of digital media. Also in practice, the MMS can be used as a screening tool for deeper understanding of specific media multitasking behaviours.