The methodological design used for the present research corresponds to a descriptive study and, more specifically, to a survey study based on administration of a Likert-type scale on nomophobia to a representative sample of university students.
2.3. Instrument for the Collection of Information and Reliability and Validity
For the collection of information, we developed a Likert-type ad hoc scale of nomophobia through the selection/adaptation of items from other previously standardised instruments that deal with this topic. This instrument is formed by diverse demographic variables of the students, these being: gender, age, degree title and degree year. Further, it includes 28 items with 4 response categories: never, sometimes, frequently and always. Reliability and validity have been considered since the classical test theory (CTT) due to the inability to calculate an item response theory (IRT) model (e.g., 2 parameters, graded response model, nominal response model and modified graded response model). This difficulty in the adjustment of the item response theory (IRT) model to noncognitive tests is consistent with what we found in other researches. One possible explanation for this difficulty is that this type of research has so far been developed mainly on tests designed with CTT and not with ITR, which makes it more difficult to adjust [
42].
CTT analysis obtained corrected item-total correlations of
r > 0.20 for all items, apart from items 13 and 26. This indicates good general discrimination of the items making up the scale (George and Mallery [
43]). However, with the aim of consolidating the results obtained through the CTT, we also submitted the data obtained from the scale to an IRT analysis. To explain in detail, we developed a graded response model (GRM) using the program Stata v.15 (StataCorp, Spring, TX, USA). The GRM is an extension of the Logistic Model of two paremeters (LM2p) and it enables us to identify the ability of items to discriminate between levels of the latent trait Samejima [
44]. It is held constant, with item difficulty being established at each “step” of the item or when a response moves from one response category to another. In other words, for a 4-point response scale, as found in the present case, we would have
k-1 steps (b parameters), given that participant responses can move from (a) 1 to 2, (b) 2 to 3, or (c) 3 to 4. Thus,
k is the number of response options for a given item. In this way, the model would have 3 b parameters or threshold parameters. The model is structured in terms of cumulative probabilities and differences between accumulated differences. The probability that a participant selects response category
k or higher is given by:
According to the criteria described by Baker [
45], the results obtained (not presented here due to space limitations) indicated that estimated values for discrimination parameters (
a) suggested between acceptable (
a > 0.65) and high (
a > 0.1.34) discrimination for most items, with values of between 1.03 a 1.24 being achieved. Very high (
a > 1.69) discrimination was also shown only in the case of items 1, 12, 21, 22, 27 and 28. Further, only three of the 28 items overall obtained values
a < 0.65 (items 13, 18 and 26). As can be seen, these outcomes largely coincided with the CTT results obtained. Thus, we can conclude that the items were generally able to discriminate between students with lower vs. higher mayor nomophobia when discrimination criteria were applied. With regards to the threshold parameters (
b), these ranged between −2.01 (
b3 item 6) and 12.98 (
b3 item 26), although the majority of values were between −2 and 2. This enabled accurate measurements of the latent trait of nomophobia to be taken within a large range. The distance between
bk values was large for 28 of the considered substances. All categories emerged as most likely for some aspect of the measured trait. This appears appropriate for this scale, given that selecting a response option of 4 or 3 requires a much higher level of nomophobia in comparison to 1 or 2. This indicates that the information provided by this scale was more accurate at higher levels of the trait.
We contemplated reliability in terms of internal consistency (given that we included only a single administration of the measurement instrument) through analysis of Cronbach’s alpha and obtained a value of α = 0.88. This can be considered a high value, which denotes that high reliability is enjoyed by the scale [
46,
47].
In addition, we calculated the value of Cronbach’s alpha for the whole scale should each one of the 28 items be individually eliminated. In this sense, values of the Cronbach’s alpha coefficients for the scale were seen not only to fail to improve the obtained value but actually make it worse (all values below α < 0.88) should the scale items constituting the instrument be individually eliminated. This means that the items contemplated by the present study are necessary and should be maintained by the instrument, given that their suppression would provoke a reduction in the consistency level of the scale [
43].
In addition, we also calculated the composite reliability, as well as the average variance extracted (AVE) of each of the factors using the jamovi programme [
48] and Excel spreadsheet. For this purpose, we relied on the standardised factor loadings of each item with its reference factor, as well as the residual covariances.
The results obtained are shown in the
Table 1:
As can be seen in the immediately preceding table, the composite reliability values are all equal to or greater than 0.71, while the AVE values are equal to or greater than 0.34. In the case, of composite reliability, values greater than 0.70 are preferable [
49], and all the values obtained meet that condition. Moreover, for average variance extracted values, at least, equal to or greater than 0.5 is acceptable [
50]. In our study, that condition is met, except in factor 1.
With regards to validity, we estimated content validity and concurrent criterion validity. Content validity of the instrument sought to guarantee that the items making up the instrument measure the construct that they were developed to measure; in other words, that the instrument measures what it is supposed to measure. In order to certify this validity, we constructed a scale based on three previously standardised instruments. In accordance with this, we have to outline that 8 of the instrument items were taken/translated from the screening for new addictions questionnaire (DENA) [
51]. Another 8 of the questionnaire items were previously utilised by Ordoñez et al. [
52], and, finally, the 12 remaining items were acquired from the instrument described by Roberts [
53]. This left a total of 28 items that were scalar in nature.
We also considered concurrent criterion validity whose usefulness resides in determining the degree of correlation that individually exists between each scale item and the overall scale itself. For this purpose, we implemented the corrected item-total coefficient correlation. The results obtained in this respect indicated the cases in which
r < 0.30 (items 20 and 28). For this reason, we can agree with the contributions of Dawson [
54] and Salkind and Frey [
55], which found that the items individually measured the same thing as the overall scale and, in addition, in the same direction (all of the correlations were positive). Thus, we can assert that the scale demonstrates concurrent criterion validity.
Finally, we estimated the construct validity of the scale. For this purpose, we submitted the 28 items of the scale to the exploratory factor analysis, using the quantitative analysis program SPSS v.25. The estimates used to provide a starting point will now be introduced. Extraction method used: principal component with Kaiser Criterion (λ ≥ 1). Rotation considered: Varimax.
With regards to the statistics prior to development of the factor analysis, a determinant correlation matrix value of |A| = 0.0000935 was achieved. This is to say, close to 0 but not null. This shows that the correlation matrix is not a single matrix and that the linear equations associated to the matrix could have a solution. On the other hand, the value of the Kaiser-Meyer-Olkin (KMO) measure of global adequacy was 0.871. According to Fabrigar and Wegener [
56] and Holmes Finch [
57], this value can be considered worthy. Measures of individual sampling adequacy (MSA) obtained values of ≥0.80 for all cases, this also being higher than the minimum value of MSA = 0.5 [
58]. These results endorse the suggestion that the items’ bivariate correlations are more important than the outcomes obtained for the partial correlations. In addition, results for the Bartlett test for sphericity obtained a value of χ
2 = 4497.882 (df = 378;
p = 0.000). This means that the correlation matrix obtained is not an identity matrix; in other words, it is not a matrix where the correlations found in the diagonal are perfect (between the items themselves) and are null elsewhere in the square.
In relation to the results of the exploratory factor analysis implemented, we highlighted the presence of 8 factors that, when taken together, explain σ
2 = 64.45%. As we will see later, and following the considerations of Carmines and Zeller [
59] and Reckase [
60], the first factor extracted does not explain at least 20% of the variance explained by the factorial solution, which is why we cannot consider the presence of unidimensionality on the scale used; however, it can be considered the main factor of the factorial model obtained. The commonalities obtained pertaining to each one of the variables of the scale are all greater than h
2 = 0.5. This denotes that each and every one of the commonalities obtained good representation in the resulting factor solution. With regards to the final factor solution, we should highlight that the factor loadings were saturated at r > 0.35 (at least 10% of explained σ
2, practical significance criterion).
Given these precedents, we contemplated a first factor comprised of 9 items (explained σ
2 = 14.01%) that seemed to have in common something we call prominence; that is, a behaviour that becomes important when it is deeply integrated into the daily life of surveyed participants. It coincided with factor 2 pertaining to abuse and difficulty in controlling abuse from the scale presented by Meyers et al. [
58].
We also had a second factor comprised of 6 items (explained σ
2 = 9.96%), all of which had in common something we can refer to as abstinence symptoms. It coincided with factor 1 (not being able to communicate oneself) from the scale described by Yildirim and Correia [
26].
We also found a third factor composed of 4 items (explained σ
2 = 9.30%) related with tolerance of mobile device use. As with the second factor, the third also coincided with factor 1, pertaining to tolerance and abstinence from the scale of Meyers et al. [
58].
The fourth factor, for its part, was also organized according to 4 items (explained σ
2 = 7.05%), all of which had something to do with what we can refer to as comfort and euphoria resulting from mobile device use (coincident with factor 4, pertaining to giving up comfort from the scale described by Yildirim and Correia [
26]).
The fifth factor was constructed of 3 items (explained σ2 = 5.64%) that maintained an association with conflicts generated by mobile device use in the family setting. The sixth factor was configured by 2 items (explained σ2 = 5.52%) that were linked to conflicts produced by mobile device use in the sphere of friendships.
The seventh factor was configured by a single item (explained σ
2 = 5.43%) and was similarly related with conflicts created by mobile phone use at a social level. Finally, an eighth factor was presented that was also formed by a single item (explained σ
2 = 4.47%). This had something to do with engagement in dangerous behaviours and personal integrity, such as mobile device use whilst driving a vehicle. Factors 5, 6, 7 and 8 from our scale could be seen as being reflected in factor 3 from the scale described by Meyers et al. [
58], which globally referred to problems brought about by excessive mobile device use.
However, in order to demonstrate that the questionnaire worked similarly on males and females with measurement invariance analyses, we calculated a confirmatory factor analysis (CFA) by the jamovi program through the adjustment for maximum likelihood (ML). The results obtained about fit measures of CFA female vs. male are shown in the
Table 2:
Confirmatory factor analyses conducted with male and female samples supported an eight-factor structure. In this structure, the obtained factors presented generally satisfactory standardised factor saturations (not presented here due to space limitations), which further provided strong evidence of the existence of factor invariance between studied male and female university students. In this respect, we should highlight that, in relation to CFA, measures of fit can be generally considered for both genders, whilst also appearing to be well-adjusted in both genders. Firstly, upon examination of the incremental fit indices of the comparative fit index (CFI) and the non-normed fix index or Tucker–Lewis index (TLI), values of 0.90–0.95 were obtained for both genders. For this reason, a reasonable fit was considered [
61]. Secondly, if we consider the absolute fit measure of the root mean square error of approximation (RMSEA), we can see that both genders obtained higher values than that dictated by the cutt-point criteria of 0.06 advised by Hu and Bentler [
62]. Nonetheless, given that this value has a known statistical distribution, confidence intervals can also be calculated (lower-upper) from the level of 90%. In this way, if the lower interval is above 0.05 and the upper value lower than 0.08, we can consider the fit to be reasonably good [
63]. Finally, if we consider the parsimony-based fit measure of the Bayesian information criterion (BIC), according to Raftery [
64], we can conclude that both considered models are well-adjusted. This is shown through the fact that BIC values >10 denote consistent fit, and, in our particular case, these values are well above this limit.