Construction of an Instrument for the Evaluation of the Effects of Information and Communication Technologies among Young People

Abstract

The aim of this paper is to investigate the issue of access to Information and Communication Technologies (ICT) at younger ages, which is leading to dependency on mobile phones, video games, and compulsive aimless internet surfing—an issue that schools have been increasingly seeking to tackle. With the appearance of emerging technologies, and not forgetting those already established, an instrument is required that will adapt to new casuistry and help to design intervention programmes in accordance with present and future patterns of use, abuse, and addiction. Studies such as the one proposed here will provide data about the profile of this population in order to improve programmes and influence the ICT policies rolled out by central and local governments. The chief aim of this paper is to construct and validate an instrument capable of evaluating problems experienced by young people in relation to technology use, abuse, and addiction within the programmes developed in Spain. The research design used in this study is mixed empirical, non-experimental, and sequential in nature in three stages: interviews conducted with 11 prevention professionals, group of 11 experts and pilot group of 30 participants in indicated prevention programmes. The findings of the study indicate that the instrument fulfills the parameters established to be considered a systematic empirically sustainable instrument, since the young population needs to identify these patterns in order to understand and prevent risk behaviours associated with their use.

1. Introduction

The technological developments experienced over the past decade have brought forth a wide range of devices, applications, and tools designed for recreation, communication, and the services sector, becoming essential activities that facilitate everyday life. However, in some cases, their use has fuelled different types of repetitive, addictive or abusive behaviours, and this phenomenon has been studied within various disciplines such as psychology, education, and sociology [1,2,3,4,5].

Some studies have shown that access to Information and Communication Technologies (ICT) at increasingly young ages can make subjects more prone to inappropriate usage, triggering dependency on mobile phones, video games, compulsive aimless internet surfing, social media, or instant messaging apps.

As noted by Matalí, García, Marín and Pardo [6], when technology changes from being a means to an end in itself, we become subject to a situation of dependency, and even addiction. Great social alarm has been generated about the addiction to new technologies in adolescents, which is reflected in various studies [7,8,9,10,11,12,13]. When use of such technology is high but controlled, we talk about problematic usage, taking as a basis the data provided by the Pfizer Foundation’s study conducted in 2009 [14], which indicated that 98% of young Spanish people aged 11–20 used the internet, and of these, around 3%–6% spent an average of 1.5 hours a day online. Along these lines, the report published in 2015 by the Organisation for Economic Co-operation and Development (OECD) [15] showed that Spanish students spent an average of 2 hours 20 minutes a day online, a much higher figure than advised. Spain’s National Institute of Statistics [16] signalled that 95% of Spanish teenagers, ranging in age from 10 to 15, went online every day. In any case, when dealing with digital or internet devices and/or the tools that require them in order to be used, the term techno-addiction can be used [17].

Adolescence is characterised by a prioritisation of immediate gratification and living in the present. Teenagers believe they are invincible, and they are interested in new stimulating and risky experiences, which makes them more susceptible to engaging in high-risk behaviours [18,19]. They have difficulty controlling their impulses, they are easily influenced by the media and advertising, and drug taking during adolescence can be linked with increased internet use or video gaming [20].

Various risk factors are related to maladaptive or problematic usages of ICTs [21,22,23].

The results of Chen, Ho and Lwin’s [24] research revealed that risky information and communications technology (ICT) use, moral disengagement, social norms, and traditional bullying perpetration were the main predictors of cyberbullying perpetration, while risky ICT use and traditional bullying victimization were the major contributors of cyberbullying victimization (Gossip, Cyber-baiting, Happy Slapping…).

In order to examine these phenomena, a variety of instruments has been developed to measure the prevalence, abuse or problematic use of ICT among teenagers and students, largely associated with web surfing, the use of mobile phones, the consumption of television, and video-gaming [25,26]. López-Fenández, Freixa-Blanxart and Honrubia-Serrano [27] reviewed the available scales for assessing problematic internet use and to validate a new scale about the Problematic Internet Entertainment Use for Adolescents. However, with the appearance of emerging technologies, not forgetting the established ones, an instrument is required that will adapt to new casuistry and help to design intervention programs in accordance with present and future patterns of use, abuse, and addiction [17,22,23].

For more than 20 years, Proyecto Hombre has been helping families who ask for support with regard to problems affecting their children. It runs indicated prevention programs for young people. According to the European Monitoring Centre for Drugs [28], Indicated Prevention involves identifying and intervening with young people who present indicators that are highly correlated with the risk of developing problems related with drug-taking and other risk behaviours over the course of their lives, or who present early drug consumption patterns. Hence, the aim is not only to prevent young people from taking drugs in the first place, but also to prevent the development of dependency, decrease the frequency of use, or prevent progression towards more harmful consumption patterns or risk behaviours.

The main goal of this paper is to construct and validate an instrument, to be used with young people, in order to evaluate potential problems with regard to technology use, abuse, and addiction, in indicated prevention programmes run by Proyecto Hombre Association throughout Spain.

2. Materials and Methods

The research design used is mixed empirical, nonexperimental and sequential in nature. It will explore relationships by associating and comparing data groups [29]. This design is rooted in the basic premise of a prior exploration, since there are no standardized measurement instruments available or a compendium of theoretically justified variables, prioritizing the compilation of qualitative data as the preliminary stage [30]. It will also help to improve research processes and products [31], providing quantifiable and contextual information [32] and allowing for the triangulation of results, the complementary nature of the phenomena studied, the discovery of singularities based on the profiles encountered, the sequencing of instruments designed, and the expansion of the study as we move through each of the stages [33].

Since this is a multi-stage study, the different stages designed to respond to the initially formulated research aim are as follows:

• Stage 1 (Designing a first draft based on the information provided by in-depth interviews conducted with prevention professionals): In order to compile information to construct the instrument aimed at young people involved in indicated prevention programs run by the Proyecto Hombre Association, capable of evaluating possible technology use, abuse or addiction problems; initially, we sought to conduct semi-structured interviews with the programme officers, in order to compile information about the real needs of the professional team working directly with the target study group. Based on the analysis of this information, a first draft of the instrument was developed.
• Stage 2 (Results of the procedure for validating the instruments by means of expert opinion): The tool developed in the previous stage was submitted for validation to a group of experts in the key issues of this project, gathering from them agreement indices for each of the evaluation elements proposed. The result of this activity provided a consensus of responses used to develop a second draft of the tool.
• Stage 3 (Experimental application of the instrument to a pilot group): Having taken on board the recommendations of the experts consulted, the next stage that allowed us to develop the definitive instrument involved applying said instrument to a pilot group. The aim was to identify the reliability and validity of the measures, considering whether the questions were appropriate, whether the wording of the items was correct and comprehensible, whether the questions were of the right length, whether the answers were correctly categorized, whether there was any resistance or psychological barriers or rejection towards any of the questions, whether they were ordered logically, and whether the duration of the questionnaire was acceptable to respondents. This process gave shape to the definitive instrument.

3. Results

Below we set out the results of the different stages of this study, showing the process by which, the instrument was progressively constructed.

3.1. Stage 1: Designing a First Draft Based on the Information Provided by In-depth Interviews Conducted with Prevention Professionals

The aim of this first stage is to establish the core issues surrounding information and communication technology use, abuse, and addiction among the young people who take part in the indicated prevention programmes run by the Proyecto Hombre Association. Based on this evidence, the foundations can be laid to build an instrument to evaluate this problem systematically and empirically.

Semi-structured interviews were used to access this information, based on 13 questions designed to collate the following data:

• Technological devices used by minors and young people within the indicated prevention programme.
• Activities or tasks they carry out using these devices.
• Knowledge possessed by programme officers about technology use, problematic use and addiction.
• Indicators used to determine technology use, problematic use and addiction in the personal, family, social, education, and occupational or work dimensions.
• Reasons that lead to use, problematic use, or addiction.
• Profile of indicated prevention programme users.
• Attitude of the family to the consumption of technology.
• Family information required to determine the problem.
• Other necessary information.

The team of indicated prevention programme officers from the Proyecto Hombre Association who took part in this study was 11 women and 4 men from Alicante, Asturias, the Canary Islands, Malaga and Valladolid. With an average age of 38, and 9 years of professional experience on average, most of these professionals are trained in psychology, although some of them are also trained in primary education teaching or social work.

The interviews were incorporated into the qualitative analysis programme NVivo 11, using the interview questions as key elements when constituting the category tree shown in

Table 1. Category tree “Perceptions of Proyecto Hombre officers”.

Category	f	%	Subcategory	f	%
1. Devices used by users of the indicated prevention programme	57	8	Video console	14	2
			Computer	11	2
			Tablet	9	1
			Mobile phone	15	2
			Television	8	1
2. Actions or tasks carried out using these devices	73	10	Online gambling	3	0
			Information searches	4	1
			Broadcast channels (YouTube)	14	2
			TV and film channels	5	1
			Listening to music	4	1
			Photos	1	0
			Online games	9	1
			Phone calls	2	0
			Instant messaging	12	2
			Social media	14	2
			Smart TV	2	0
			Video console	3	0
3. Technology use, problematic use and addiction	42	6	Technology addiction	15	2
			Not determined in the dimensions described	1	0
			Use	12	2
			Problematic use	14	2
4. Personal dimension indicators	87	12	Addiction	29	4
			Not determined in the dimensions described	14	2
			Use	22	3
			Problematic use	22	3
5. Family dimension indicators	70	9	Addiction	17	2
			Not classified in the dimensions described	16	2
			Use	17	2
			Problematic use	20	3
6. Social dimension indicators	69	9	Addiction	22	3
			Not classified in the dimensions described	9	1
			Use	18	2
			Problematic use	20	3
7. Education dimension indicators	63	8	Addiction	15	2
			Not classified in the dimensions described	14	2
			Use	17	2
			Problematic use	17	2
8. Occupational or work dimension indicators	61	8	Addiction	18	2
			Not classified in the dimensions described	11	2
			Use	16	2
			Problematic use	16	2
9. Reasons	60	8	Addiction	14	2
			Not classified in the dimensions described	13	2
			Use	20	3
			Problematic use	13	2
10. Technology use profile in prevention programmes	54	7	Technology use profile in prevention programmes	54	7
11. Family attitude	16	2	Family attitude	16	2
12. Family information required to specify the problem	62	8	Family information required to specify the problem	62	8
13. Information required to define the problem	28	4	Information required to define the problem	28	4
Total	742	100	Total	742	100

.

Eight percent of the coded text is classified in the category “Devices used by the users”. Ten percent of the text pertains to actions or tasks carried out when using the devices. Regarding technology use, problematic use and addiction, 6% of references were coded in this category. The weighting of indicators for the different dimensions (personal, social, family, education, occupational or work) is between 8% and 12%. Seven percent of the coded text is classified in the category “Profile of technology use in prevention programmes” and 8% in the “Family information required to specific the problem”. Furthermore, the two categories with the lowest percentage weighting are related with family attitude, at 2%, and other information required to define the problem, at 4% of the coded references.

The instrument derived from this first stage is made up of a total of 10 analytical dimensions, 46 assessment elements, and 122 items. It is accompanied by a first round of elements not accounted for, the purpose of which is to identify the person providing information, and one last question about final observations referring to the tool.

Differentiation between elements and items stems from the fact that the evaluation elements are understood as units of observation that configure each analytical dimension, and the items are analytical units established within the evaluation elements as scaled and compulsory multiple-choice items.

The instrument is applied by means of a personal interview between indicated prevention programme officers and the user offering the information.

3.2. Stage 2: Results of the Procedure for Validating the Instruments by Means of Expert Opinion

The end product of the previous stage had to be validated by means of a suitable methodological procedure. In this instance, expert opinion [34] was the chosen method because, rather than being expressed quantitatively using an index or coefficient, it is estimated by means of a generally subjective or intersubjective opinion given by experts in the field. The purpose of this technique is to collate the opinions of people whose academic or professional background reflects their capacity to give evidence or critical assessments of the object of study [35], which enhances the validity of the content studied by seeking rational consensus [36].

When setting up this group, the selection criteria used brought together academics and professionals from the field of drug addiction prevention, ICT specialists, as well as experts in the design of instruments. A total of 10 people took part in the expert panel, as shown in

Table 2. Description of the group of experts.

Number	Sex	Area of Work
1	Woman	National Prevention Committee, Proyecto Hombre Association
2	Woman	National Prevention Committee, Proyecto Hombre Association
3	Woman	National Prevention Committee, Proyecto Hombre Association
4	Man	Indicated Prevention Group (Proyecto Hombre Association)
5	Woman	Indicated Prevention Group (Proyecto Hombre Association)
6	Man	Indicated Prevention Group (Proyecto Hombre Association)
7	Man	University of Salamanca
8	Woman	Libertador Experimental Pedagogical University, Venezuela
9	Woman	National Polytechnic Institute, Mexico
10	Woman	National Plan on Drugs

.

According to Skjong and Wentworth and Escobar [37] and Cuervo [35], the analysis process involved e-mailing the group an invitation to complete a validation protocol for the two instruments generated in the previous stage. The procedure entailed evaluating each of the elements of the tool aimed at users, assigning a score between 1 and 5 (1 indicating the minimum score and 5 the maximum score), in accordance with the following criteria:

• Breadth of the content: fit of the question wording so that there is no redundancy and it is consistent with the response options.
• Congruency: linkage and coherence of the items that make up the instrument.
• Pertinence: correspondence between the content of the item and the dimension in which it will be used.
• Precision: rigorousness with which words have been used when formulating each item in order to express what said item aims to measure.
• Clarity: accuracy of the wording of each item, ensuring it is clear and easy to understand.

Once the ten experts had given their general opinion when validating the instrument designed to be used with young users of indicated prevention programmes, we observed that all the parameters defined had scored highly (see

Table 3. General evaluation of the instrument designed for users.

Evaluation Criteria	Mean	SD
Breadth of content	4.13	0.835
Congruency	4.13	0.641
Pertinence	4.14	0.690
Precision	3.88	0.641
Clarity	4.13	0.354

).

A breakdown of the evaluation data for each of the elements that make up the instrument confirmed this result. Furthermore, the consensus obtained between the groups of expert judges was high according to Aiken’s V coefficient for the five stipulated criteria (V > 0.50) (see

Table 4. Evaluation of the elements of the instrument designed for users

Element	Breadth			Congruence			Pertinence			Precision			Clarity
	Mean	SD	V	Mean	SD	V	Mean	SD	V	Mean	SD	V	Mean	SD	V
1	3.67	1.581	0.733	3.89	1.269	0.911	3.89	1.054	0.911	3.56	1.333	0.711	4.11	1.054	0.800
2	4.44	1.014	0.888	4.67	0.500	0.866	4.67	0.500	0.866	3.67	1.323	0.822	3.89	1.364	0.755
3	4.44	0.527	0.888	4.56	0.527	0.866	4.56	0.527	0.911	4.67	0.500	0.577	4.56	0.527	0.822
4	4.00	1.000	0.800	4.33	0.707	0.888	4.33	0.707	0.688	3.89	1.269	0.666	4.22	0.972	0.800
5	4.56	1.014	0.911	4.33	1.118	0.733	4.33	1.000	0.711	3.56	1.590	0.777	3.89	1.364	0.755
6	4.22	1.302	0.844	4.62	0.744	0.822	4.75	0.463	0.866	4.25	1.389	0.777	4.37	1.188	0.911
7	3.44	0.882	0.688	3.67	1.323	0.844	3.44	1.424	0.866	2.89	1.453	0.800	2.78	1.563	0.888
8	4.25	1.165	0.822	4.25	1.165	0.844	4.00	1.309	0.866	3.38	1.768	0.733	3.50	1.604	0.800
9	4.00	1.323	0.800	4.22	0.972	0.844	4.33	1.000	0.866	3.89	1.054	0.711	3.44	1.333	0.755
10	4.22	0.972	0.844	4.22	0.833	0.844	4.33	0.707	0.777	3.89	1.167	0.777	4.33	0.866	0.800
11	4.11	1.269	0.822	4.22	1.302	0.755	4.33	1.000	0.866	4.00	1.000	0.755	4.44	0.527	0.800
12	4.22	0.972	0.844	4.22	0.833	0.866	4.33	0.707	0.866	3.67	1.414	0.844	4.00	1.000	0.844
13	3.78	1.302	0.755	3.78	1.302	0.866	3.89	1.269	0.822	3.56	1.236	0.844	3.88	0.991	0.844
14	4.33	1.000	0.866	4.33	1.000	0.844	4.33	0.866	0.822	3.89	1.453	0.844	4.00	1.500	0.933
15	4.33	1.000	0.866	4.33	1.000	0.844	4.33	0.866	0.822	3.89	1.453	0.733	4.00	1.500	0.933
16	4.22	1.302	0.844	4.22	1.302	0.844	4.11	1.364	0.822	3.78	1.641	0.866	4.00	1.500	0.822
17	4.22	1.302	0.844	4.22	1.302	0.844	4.11	1.364	0.777	4.22	1.302	0.911	4.22	1.302	0.777
18	4.22	1.302	0.844	4.22	1.302	0.800	4.11	1.364	0.866	4.22	1.302	0.800	4.22	1.302	0.911
19	4.22	1.302	0.844	4.22	1.302	0.822	4.11	1.364	0.822	4.22	1.302	0.577	4.22	1.302	0.844
20	4.00	1.323	0.800	4.00	1.323	0.966	3.89	1.364	0.755	3.67	1.658	0.733	3.67	1.658	0.777
21	4.33	0.707	0.866	4.25	0.707	0.800	4.33	0.707	0.777	4.50	0.535	0.711	4.33	1.000	0.844
22	4.44	0.726	0.888	4.33	0.866	0.733	4.11	1.167	0.800	4.56	0.527	0.911	4.33	1.000	0.555
23	4.25	1.035	0.822	4.13	1.126	0.755	3.88	1.356	0.800	4.13	1.356	0.933	3.75	1.488	0.688
24	4.00	1.225	0.800	3.67	1.581	0.822	3.89	1.453	0.911	2.89	1.453	0.911	3.33	1.323	0.688
25	4.00	1.323	0.800	3.78	1.302	0.933	4.00	1.323	0.933	3.67	1.323	0.844	3.78	1.302	0.866
26	4.00	1.323	0.800	4.43	0.787	0.933	4.00	1.323	0.888	3.56	1.424	0.777	3.67	1.323	0.888
27	4.67	0.500	0.933	4.67	0.500	0.888	4.75	0.463	0.911	4.56	0.726	0.800	4.78	0.441	0.800
28	4.88	0.354	0.933	4.88	0.354	0.866	4.88	0.354	0.844	4.88	0.354	0.755	4.75	0.463	0.755
29	4.63	0.518	0.888	4.63	0.518	0.844	4.63	0.518	0.822	4.75	0.463	0.822	4.63	0.744	0.800
30	4.56	0.527	0.911	4.33	0.707	0.822	4.56	0.527	0.800	4.22	1.093	0.800	4.22	0.972	0.800
31	4.22	0.972	0.844	4.22	0.972	0.844	4.22	0.972	0.844	3.89	1.269	0.755	4.00	1.225	0.800
32	4.11	1.054	0.844	4.11	1.054	0.844	4.11	1.054	0.866	4.00	1.323	0.911	3.78	1.481	0.844
33	4.22	0.972	0.844	4.22	0.972	0.844	4.00	1.225	0.933	3.78	1.394	0.888	4.11	1.269	0.844
34	4.22	1.093	0.844	4.22	1.093	0.800	4.22	1.093	0.911	4.11	1.364	0.800	4.00	1.323	0.844
35	4.22	0.972	0.933	4.22	0.972	0.933	4.22	0.972	0.911	4.00	1.225	0.755	4.11	1.269	0.733
36	4.22	1.093	0.911	4.00	1.118	0.911	4.50	0.756	0.911	3.78	1.302	0.800	3.67	1.225	0.866
37	4.67	0.500	0.800	4.67	0.500	0.866	4.67	0.500	0.922	4.56	0.527	0.800	4.56	0.726	0.866
38	4.56	0.726	0.822	4.56	0.726	0.888	4.56	0.726	0.911	4.44	0.726	0.844	4.44	0.726	0.733
39	4.13	0.835	0.844	4.33	0.707	0.844	4.56	0.527	0.888	4.00	0.707	0.844	4.11	0.928	0.666
40	4.11	0.601	0.888	4.44	0.527	0.888	4.56	0.527	0.933	3.78	0.972	0.933	4.00	1.000	0.755
41	4.22	0.833	0.888	4.22	0.833	0.911	4.25	1.035	0.955	4.13	0.991	0.933	4.00	0.926	0.733
42	4.44	0.527	0.888	4.44	0.527	0.933	4.56	0.726	0.866	4.00	1.118	0.822	4.11	1.054	0.955
43	4.44	0.527	0.955	4.56	0.726	0.955	4.44	1.014	0.711	4.22	0.972	0.777	4.00	1.000	0.911
44	4.44	0.527	0.888	4.67	0.500	0.955	4.67	0.500	0.733	4.22	0.833	0.911	4.33	0.707	0.888
45	4.78	0.441	0.777	4.78	0.441	0.777	4.78	0.441	0.933	4.67	0.500	0.844	4.56	0.527	0.844
46	4.44	1.014	0.933	4.78	0.441	0.933	4.33	1.323	0.777	4.67	0.500	0.777	4.67	0.500	0.800

).

The output from this second stage was a tool comprising the 10 analytical dimensions from the first draft, increasing the evaluation elements to 50 and the number of items to 138. It is accompanied by a first round of elements not previously taken into consideration, the purpose of which is to identify the user who is providing information, and one final question relating to final observations about the tool.

3.3. Stage 3: Experimental Application of the Instrument to a Pilot Group

Having incorporated the recommendations of the experts to create the second draft of the instrument, the next stage of definitive construction was the experimental application to a pilot group of users participating in indicated prevention programmes run by Proyecto Hombre. The aim of this pilot test was to evaluate the consistency of the instrument (properties of the scale and its constituent elements) and its appropriateness to the object of measurement.

To study the psychometric properties of the instrument, the following analytical procedures were applied to the dimensions that, in their wording, incorporate elements configured by items from the scaled evaluation (the dimensions Description of family sphere and Availability of digital devices in the home are made up of elements of choice and are not eligible for validation):

• Internal Consistency Analysis, in the sense of endowing the items with significance, in other words, ensuring that each of them measures a portion of the trait or characteristic studied. To this end, Cronbach’s Alpha coefficient was used [38].
• Analysis of the discrimination capacity of the elements to reinforce the one-dimensional nature of the test. Student’s t-test was applied to the mean values of the established groups, indication of validity endorsed by García, Gil y Rodríguez [39].

The 30 people chosen for the pilot study are taking part in indicated prevention programmes at six centres run by Proyecto Hombre (Asturias, Catalonia, Madrid, Malaga, Melilla and Murcia) in a similar proportion in each of them (17.7%). 86.7% are male, and the remaining 13.3% are female. The age of the participants ranged from 14 to 22, and the majority of them had completed secondary education (73.3%). This number of people is valid to carry out this phase of the study according to McMillan and Schumacher [40].

Looking at the evaluation of the elements in the “Personal sphere” dimension, we see that the total value for Cronbach’s Alpha in the scale (0.505) represents an acceptable correlation [41], an acceptable level of stability in the responses; hence, this part of the instrument presents signs of guaranteed reliability.

The discriminatory power of all the items in the test reinforces its one-dimensionality. To carry out this procedure, three closed items were chosen with ordinal response choices (response scale from 1 to 5) so that the sum total was recoded into three groups (Low, Medium and High):

• 1 = Low group (minimum value, percentile 33): (6, 16)
• 2 = Medium group (percentile 34, percentile 66): (17, 23)
• 3 = High group (percentile 67, maximum value): (24, 35)

By applying Student’s t-test for independent samples, we were able to establish the existence or non-existence of statistical difference (n.s. = 0.05) between the groups that score low and high in the selected elements. The results obtained using this test based on the 6 items belonging to this dimension and present in

Table 5. Discriminatory power of the items in the dimension ‘description of the personal sphere’.

Items	Lower Mean	Upper Mean	t	p	Discriminatory?
I feel frustrated when I have limitations with technology	2.90	5.63	−2.203	0.043	Yes
I lie about the actual amount of time I spend on technology	1.70	4.13	−2.362	0.031	Yes
I prefer to interact with others through technology	2.40	3.63	−1.487	0.157	No
I escape reality through the use of technology	1.90	6.13	−1.487	0.157	No
I seek refuge in technology because I feel alone	1.30	3.63	−4.581	0.001	Yes
I can’t control the use of technology in my everyday life	2.50	6.25	−3.676	0.002	Yes

show that 67% of the element possesses acceptable statistical discriminatory power, which indicates acceptable levels of validity in this dimension. The nondiscriminatory elements were maintained on account of their relevance in the instrument and in accordance with the suggestions made by the programme officers.

The second dimension analysed, “ICT consumption habits”, presented a low level of reliability, reflected in Cronbach’s alpha coefficient, 0.27. Furthermore, discriminatory power revealed that only 25% of items have a valid discriminatory power (see

Table 6. Discriminatory power of the items in the dimension ‘Information and Communication Technologies consumption habits’.

Items	Lower Mean	Upper Mean	t	p	Discriminatory?
User’s tendency to watch broadcast channels and participate in comments	2.00	5.00	−2.183	0.049	Yes
User’s tendency to have a broadcast channel or to upload random videos onto the Web	2.90	3.00	−0.060	0.953	No
User’s tendency to participate in or look at Social Media	6.00	8.11	−1.767	0.095	No
User’s tendency to participate in instant messaging	8.20	7.44	0.459	0.652	No
User’s tendency to play online through a video console	3.80	5.78	−1.277	0.219	No
User’s tendency to play online through other devices	3.10	6.44	−2.149	0.046	Yes
User’s tendency to watch series and film channels	5.70	7.67	−1.422	0.173	No
User’s tendency to look for information on the Net	5.80	7.89	−1.641	0.119	No
User’s tendency to listen to music on the Net	7.80	9.33	−1.291	0.214	No
User’s tendency to gamble online	1.00	2.44	−1.386	0.203	No
User’s tendency to shop online	1.50	3.78	−2.109	0.062	No
User’s tendency to consume sexual content online	1.50	5.11	−2.658	0.023	Yes

). These results concluded that the dimension lacked acceptable reliability and validity criteria, and so we followed the recommendations of the prevention programme officers, by eliminating elements without discriminatory power and incorporating 13 new elements that would ensure the clarity and pertinence of the elements in the dimension according to the characteristics of the target population.

Analysis of the 14 scaled items of the dimension “User’s reasons for consuming ICT” revealed a guarantee of stability in the measure they offer, reflected in the Cronbach Alpha coefficient of 0.58. The application of the item discrimination test provided data that supported the observations proposed by the prevention programme officers participating in the study, since only 43% of items offered acceptable validity (see

Table 7. Discriminatory power of the items in the dimension ‘Reasons for consuming Information and Communication Technologies’.

Items	Lower Mean	Upper Mean	t	p	Discriminatory?
Meeting new people	4.78	7.00	−1.785	0.093	No
Contacting acquaintances	7.11	8.89	−1.985	0.065	No
Family means of communication	3.22	7.88	−3.008	0.008	Yes
Setting up groups according to context	4.78	9.11	−3.706	0.002	Yes
Searching for information	2.78	6.78	−3.034	0.011	Yes
Disconnecting	2.67	4.00	0.393	0.393	No
Technological update	7.11	7.00	0.096	0.925	No
Immediate satisfaction	5.56	4.56	0.716	0.484	No
Personal recognition	3.00	6.67	−2.750	0.014	Yes
Repeat behaviours	2.22	5.56	−2.375	0.030	Yes
Social pressure	3.44	3.33	0.092	0.928	No
Escaping from reality	1.56	3.00	−1.729	0.103	No
Rebelling against authority	2.11	5.11	−2.252	0.044	Yes
Other option	2.67	3.67	−0.986	0.339	No

). The relevance of the contents processed did not lead to the suppression of any element but did lead to a redrafting of items where the discrimination was confusing.

A Cronbach Alpha coefficient of 0.828 guaranteed the reliability of the 10 elements that make up the dimension “User’s emotional management”. Furthermore, the analytical test to establish the validity of these questions indicated that 70% offered valid content, leading us to revise the wording of three items (see

Table 8. Discriminatory power of the items in the dimension ‘Emotional management’.

Items	Lower Mean	Upper Mean	t	p	Discriminatory?
Use of technology due to a feeling of dissatisfaction with interpersonal relations	1.22	2.88	−1.917	0.092	No
Use of technology because they help to relate with others	2.33	7.00	−5.484	0.000	Yes
Use of technology due to fear of facing reality	1.33	2.88	−1.872	0.097	No
Use of technology due to fear of being socially excluded	1.11	2.25	−1.551	0.163	No
Use of technology as a means of coping with shyness	1.44	3.88	−2.477	0.032	Yes
Use of technology due to fear of not being up to date	1.00	3.25	−1.865	0.046	Yes
Irritability over control over number of hours of technology use	1.89	9.00	−7.794	0.000	Yes
Displays of aggression when number of hours of technology use are controlled	1.11	7.00	−9.688	0.000	Yes
Fear of being without devices	1.67	7.25	−7.081	0.000	Yes
Use of technology due to fear of losing control over what is happening around them	1.33	5.0	−3.254	0.012	Yes

).

The reliability of the scaled items in the dimension “ICT in the family setting” when the subject is living in the family home is very high (Cronbach Alpha = 0.839). Sixty percent of these items possess an acceptable discriminatory power, which validates their inclusion in the instrument (see

Table 9. Discriminatory power of the items in the dimension ‘Information and Communication Technologies in the family settings-Effects of the use of technologies on cohabitation’.

Items	Lower Mean	Upper Mean	t	p	Discriminatory?
The user does not speak with their father/guardian	1.25	3.71	−2.353	0.052	No
The user fights with their father/guardian	1.00	4.86	−3.911	0.008	Yes
The user is aggressive and rude towards their father/guardian	1.13	3.29	−2.853	0.027	Yes
The user does not speak with their mother/guardian	1.25	6.00	−2.803	0.030	Yes
The user fights with their mother/guardian	1.75	7.00	−4.020	0.002	Yes
The user is aggressive and rude towards their mother/guardian	1.00	4.57	−2.946	0.026	Yes
The user does not speak with their sibling(s)	1.25	4.00	−1.832	0.114	No
The user fights with their sibling(s)	1.00	3.43	−2.497	0.047	Yes
The user is aggressive and rude towards their sibling(s)	1.00	3.00	−2.103	0.080	No
The user does not participate in family activities	1.25	4.57	−2.265	0.062	No

). Four of these items required further work based on the suggestions of the programme officers, who suggested that they should not be removed from the instrument but instead the wording needed to be revised to make them more understandable.

The eight elements that make up the dimension “ICT in the social setting”, once the relevant reliability test had been applied, contributed a global value of 0.890, clearly demonstrating their metric consistency. Furthermore, the application of the validity test showed that 100% of these items possess discriminatory power (see

Table 10. Discriminatory power of the items in the dimension ‘Information and Communication Technologies in the family settings-Effects of technology use on cohabitation’.

Items	Lower Mean	Upper Mean	t	p	Discriminatory?
The user shows more interest in virtual than physical relationships	1.17	2.60	−3.523	0.004	Yes
The user enjoys virtual relations more than physical relationships	1.00	3.50	−2.748	0.023	Yes
The user invests more time in virtual relationships than physical ones	1.00	5.40	−7.333	0.000	Yes
The user shows their emotions more easily in the virtual world than in the physical world	1.42	6.00	−5.144	0.000	Yes
The user says what they think more easily in the virtual world than in the physical world	1.92	5.90	−3.584	0.004	Yes
The user pays more attention to virtual conversations than ones in the physical world	1.25	4.80	−4.260	0.002	Yes
The user avoids conflicts by relating with others virtually	1.17	3.10	−2.543	0.031	Yes
The user prefers to relate with others who share their love of technology	1.50	6.40	−4.420	0.001	Yes

).

The internal consistency of the nine scaled elements that describe the dimension “ICT in the education setting” was 0.880, scientifically guaranteeing their reliability. The validity of these items, measured by means of the corresponding item discriminatory test, showed that 89% measure the construct covered by this dimension, with just one of the items requiring revision. Furthermore, it should be noted that, at the request of the programme officers, a new element was incorporated into this dimension on account of its relevance for the subject studied (see

Table 11. Discriminatory power of the items in the dimension ‘Information and Communication Technologies in the family settings-Effects of technology in the education setting’.

Items	Lower Mean	Upper Mean	t	p	Discriminatory?
Breaks school rules regarding the use of technology and has on occasion been told off because of this	3.57	6.67	−1.950	0.075	No
Grades have gone down	1.00	5.14	−3.023	0.023	Yes
Finds it harder to concentrate on studies	2.57	7.71	−4.129	0.001	Yes
Finds it hard to pay attention in class	1.71	6.43	−3.200	0.008	Yes
Finds it hard to memorise concepts	1.00	5.86	−4.250	0.005	Yes
Doesn’t do homework	1.86	7.57	−6.124	0.000	Yes
Skips class	1.14	4.29	−2.465	0.048	Yes
Doesn’t want to study anymore	1.00	5.43	−3.175	0.019	Yes
Doesn’t interact with classmates	1.29	4.86	−2.933	0.013	Yes

).

Finally, the eight elements that make up the dimension “ICT in the work setting” possess a high degree of reliability, as reflected in the Cronbach Alpha coefficient, 0.769, although in contrast these are not items that possess an acceptable level of discriminatory power, and accordingly all of them needed to be revised (see

Table 12. Discriminatory power of the items in the dimension ‘Information and Communication Technologies in the work setting’.

Items	Lower Mean	Upper Mean	t	p	Discriminatory?
Skips work	1.00	1.50	−1.633	0.178	No
Stops doing job/can’t manage to do what they are supposed to	1.00	1.50	−1.633	0.178	No
Arrives at work late	1.00	4.00	−1.500	0.374	No
Is tired	1.00	8.00	−1.000	0.500	No
Has conflicts with colleagues	1.00	4.00	−1.500	0.374	No
Has conflicts with boss(es)	1.00	4.00	−1.500	0.374	No
Has had a workplace accident	1.00	1.50	−1.633	0.178	No
Doesn’t want to go to work	1.00	6.00	−1.250	0.430	No

). However, it should be noted that only 5% of these prevention programmes participants are engaged in the employment market; hence, the results obtained understandable. Accordingly, the decision was made to maintain the elements as originally formulated.

Following the same analytical dynamic as in the previous stages, the result of this third stage was a tool in the format of a personalised interview administered by prevention programme officers, made up of 10 analytical dimensions, 50 evaluation elements, and 156 items (see

Table 13. Description of the definitive instrument aimed at programme users.

Dimension	Draft 1		Draft 2		Draft 3
	Elements	Items	Elements	Items	Elements	Items
1. Description of personal sphere	9	14	9	14	9	25
2. Description of family sphere	17	17	17	17	17	17
3. Availability of digital devices in the home	1	1	1	1	1	1
4. ICT consumption habits	12	22	12	23	12	30
5. Reasons for consuming ICT	1	16	3	18	3	17
6. Description of emotional management	1	10	1	10	1	10
7. ICT in the family setting	2	16	4	30	4	30
8. ICT in the social setting	1	9	1	8	1	8
9. ICT in the education setting	1	8	1	9	1	10
10. ICT in the work setting	1	9	1	8	1	8
Total	46	122	50	138	50	156

). As indicated previously, it is supplemented by a first round of elements, the purpose of which is to identify the user who is providing the information, along with an observations section in each dimension, and one final question related to final observations about the tool, all of which are not included in the total number of elements that make up in the final instrument (see Appendix A).

4. Discussion

Following the results set out regarding the construction and validation of the instrument created to identify technology use, abuse or addiction among the young people taking part in indicated prevention programmes run by the Proyecto Hombre Association throughout Spain, we can confirm that the instrument fulfills the parameters established to be classed as a systematic and empirically sustainable instrument, since the youth population needs to identify these patterns in order to understand and prevent risk behaviours associated with technology usage. In turn, the instrument must differentiate between the applications used and the use being made of them, in order to understand whether minors are using or abusing them [25]. This instrument adds to the contributions of López-Fenández, Freixa-Blanxart and Honrubia-Serrano’s internet abuse scale for adolescents [27], detailing the devices used, the habits and reasons for consumption and the effects of use in the family, professional and emotional dimensions. There are direct connections to the instrument designed by Chen, Ho and Lwin [24] in the emotional and social effects, but the possible situations of bullying that arise with the use of these devices are not specified. Also, the scale designed by Peris, Maganto and Garaigordobil [42], focused on the use of social networks, prioritizes elements that are worked on in the instrument designed here such as the social use of devices and nomophobia.

The Jiménez, Alvarado and Llopis [43] instrument assesses the usefulness of ICT in the work of university students, the emotions that their use generates and the feelings of frustration that their absence generates. Understood by these elements as predictors of ICT addiction, the instrument referred to in this article considers those aspects in the adolescent population participating in the indicated prevention programmes. More than 28% of university students display risk behaviours with regard to the use of technology, and other variables need to be taken into consideration such as personality, family setting, and peer group [44]. The study conducted by [45] establishes that 90.6% of the population engages in controlled use of the internet, and just 9.4% have frequent problems.

However, the specificity of the adolescents attended in the indicated prevention programs for drug use and other risky behaviors and their relationship to the problematic use of ICT [46] and other behavioral problems or mismanagement of impulsivity [47] make it necessary to create an instrument adapted to new casuistic. Hence, this instrument facilitates the identification of use, abuse, and addiction profiles and their relationship with the technologies associated with these patterns.

5. Conclusions

By means of a mixed, nonsequential experimental study, three stages were developed with a view to achieving the aim of this research.

First, conducting semi-structured interviews with 15 programme officers working at Proyecto Hombre has shed light on risk factors in the family, education, and social dimensions, which can lead to problematic or addictive behaviour with regard to technological devices. This information was used to consolidate an initial draft of the evaluation instrument, comprising a total of 10 analytical dimensions, 46 evaluation elements, and 122 items.

Then, a panel of 10 experts in the evaluation of technology use, abuse, and addiction took part in the project. The results of the consensus analysis conducted allowed us to reshape the instrument into 10 analytical dimensions, 50 evaluation elements, and 138 items.

The third stage in the design of this tool involved the experimental application of this instrument to a pilot group of 30 users. The aim of this stage was to identify the reliability and validity indices of the instrument. Analysis of the information provided in this stage gave rise to the definitive instrument, made up of 10 analytical dimensions, 50 evaluation elements and 156 items. These dimensions are: description of the personal sphere, description of the family sphere, availability of digital devices in the home, ITC consumption habits, reasons for consuming ICT, description of emotional management, ICT in the family setting, ICT in the social setting, ICT in the education setting, and ICT in the work setting. The instrument was applied by means of a personalised interview conducted by prevention programme officers with users of the programme who provided the information.