2.2. Data Collection Instruments
Artificial Intelligence Usage Questionnaire: The AI questionnaire was used to investigate the usage of AI among adolescents, referring to the Mobile Phone Use Questionnaire written by Wang and his teammate [
31]. The AI questionnaire consists of three dimensions, including the frequency of usage, the intention and attitude toward using AI, and the feelings after using AI. The items of the frequency of usage include “
Whether AI is used for learning”, “
How many days do you spend on learning with AI per week”, and “
How many hours do you spend on learning with AI every day”. The items of the intention and attitude toward using AI include “
Why do you use AI for learning”; “
Attitude of your family towards you learning with AI”. The items of the feeling after using include “
You think it is helpful for you to use AI for learning”, “
You find yourself spending more and more time on AIEd”, “
You will feel insecure and anxious if you study without AI”, and “
How interesting do you think it is to learn with AI”. There are 9 items in total.
Social Adaptability: This scale, which was adopted from Zheng [
32] includes 20 items that measure five dimensions of social adaptation, including peer relationships, self-management, academic performance, obedience, and willingness to express. Then, participants should respond “
agree” by choosing “1” or “
uncertain” by choosing “2” or “
disagree” by choosing “3”. The questionnaire had high validity and reliability in this study (Cronbach’s alpha = 0.80).
Interpersonal Relationship: We adopted the 28 items from Zheng [
32] to measure interpersonal relationships. One sample was that “
I can’t concentrate on listening to others”. Then, participants should respond “
yes” or “
no” to each item. The scale had high validity and reliability in this study (Cronbach’s α = 0.88).
Interparental Relation: A 9-item scale was adopted to measure adolescents’ perception of interparental relations [
33]. To complete this subscale, the participants were asked to answer 9 items that concerned their perceptions of interparental relations in their families. Students should respond to each item from 1 (strongly disagree) to 5 (strongly agree). A total score ranges from 9 to 45 with higher scores indicating closer relation. The scale had high validity and reliability in this study (Cronbach’s alpha = 0.92).
Teacher–student Relation: We adopted the 7-item scale to measure teacher–student relation [
34]. One sample is “
Teacher always plays favorites”. Students were required to respond from 1 (strongly disagree) to 5 (strongly agree). A total score ranges from 7 to 35 with higher scores indicating closer relation. The reliability and validity of the scale have been well documented (Cronbach’s alpha = 0.91).
Peer Relation: Adolescents’ peer relation was measured using the Chinese version of the Peer Relation Questionnaire [
35](Chen and Zhu, 1997). All items were assessed using a six-point scale (1 = “strongly disagree,” 6 = “strongly agree”). For each participant, his score for all 18 items was determined, with higher scores showing higher levels of peer relationships. For the current study, the measure demonstrated excellent reliability (Cronbach’s alpha = 0.83).
Loneliness: We used the UCLA Loneliness Scale [
36] to assess participants’ loneliness. Participants were required to rate on a 4-point Likert scale. The 4-point scale was ranging from 1 (strongly disagree) to 4 (strongly agree). Internal consistency in the current sample was adequate (α = 0.91).
Impulsivity: S-UPPS-P scale [
37] is a 20-item scale that measures five dimensions of impulsivity, including negative urgency, positive urgency, programmatic, perseverance and sensation seeking. Each item was rated on a scale from 1 (strongly disagree) to 4 (strongly agree). The scale had suitable validity and reliability in this study (Cronbach’s alpha = 0.65).
Academic Emotion: A 18-item Academic Emotions Questionnaire [
38] was adopted to measure two dimensions of social adaptation. All 18 items were rated using a five-point scale (1 = “strongly disagree,” 5 = “strongly agree”). For each participant, his total score for all 18 items was determined, with higher scores indicating higher academic anxiety and academic boredom. For the current study, the measure demonstrated good reliability (α = 0.92).
Emotion Regulation Strategies: Adolescents’ emotional regulation strategy was measured using the Chinese version of the Emotion Regulation Strategies Questionnaire [
39]. A total of 10 items were assessed using a seven-point scale (1 = “strongly disagree,” 7 = “strongly agree”). For the current study, the measure demonstrated high reliability (Cronbach’s alpha = 0.86 for cognitive reappraisal sub-scale; Cronbach’s alpha = 0.64 for expression inhibition sub-scale).
Empathy: Basic Empathy Scale [
40] was used to measure the variable of empathy in this study. There are 20 items on the scale, including two dimensions: cognitive empathy and emotional empathy. All items were rated using a five-point scale (1 = “strongly disagree,” 5 = “strongly agree”). For each participant, their total score for all 20 items was determined. The higher the score, the stronger the empathy. In this study, the measure demonstrated good reliability (α = 0.83).
This study is exploratory, and machine learning models are more suitable for exploratory research than traditional regression models. There are three factors contributing to our decision that utilize machine learning to give a deep analysis. First of all, utilizing random forest (RF) to conduct a regression model can maintain the accuracy of the study for RF is insensible to missing data. In addition, RF can also demonstrate the importance of different potential predictors. What is more, RF can also better handle the analysis of multiple variables. Therefore, to assess the difference in social adaptability between AI group and non-AI group, the partial dependence plot (PDP), a machine learning method, is employed to make a preliminary analysis. Partial dependence is a library for visualizing input–output relationships of machine learning models, which can measure the prediction change when changing one or more input features. However, PDP is plotted out based on machine learning model. In this study, RF method was also adopted. The RF method provides an ensemble learning method for classification, operates by constructing numerous decision trees, and produces the best result of classification based on the combination of individual trees. Random decision forests are able to correct the habit of decision trees overfitting to their training set.
The following is a description of the RF method executed with the Python Sklearn classification method.
Given a training set X ={x1,…, xn} with responses Y= {y1,…, yn}, random samples are selected (B times), with their replacements from the training set, and are used to train the decision trees:
For b = 1,…, B:
Sample, with replacement, B training examples from {X, Y}; call these {Xb, Yb}.
Train regression tree fb on {Xb, Yb}.
After training, a prediction for unseen sample x′ can be made by averaging the predictions from all the trained individual regression trees on x:
Based on RF model, we can conduct PDP by: