Application of Psychometric Approach for ASD Evaluation in Russian 3–4-Year-Olds

Abstract

Background: Autistic spectrum disorder (ASD) is a significant socio-biological problem due to its wide prevalence and negative outcomes. In the current study, we aimed to develop an autism scale for early and accurate differentiation of 3- to 4-year-olds at risk for ASD since there is no systematic monitoring of young children in Russia yet. Methods: The total sample (N = 324) included 116 children with ASD, 131 children without ASD (healthy controls), and 77 children with developmental delay (DD). An online survey of specialists working with children was conducted based on a specially designed autism questionnaire consisting of 85 multiple-choice tasks distributed across 12 domains. Initially, each child was assessed by 434 items using a dichotomous scale (0 = no, 1 = yes). Factor and discriminant analyses were performed to identify a compact set of subscales that most accurately and with sufficient reliability predicted whether a child belongs to the ASD group. Results: As a result, four subscales were obtained: Sensorics, Emotions, Hyperactivity, and Communication. The high discriminability of the subscales in distinguishing the ASD group from the non-ASD group was revealed (accuracy 85.5–87.0%). Overall, the obtained subscales meet psychometric requirements and allow for creating an online screening system for wide application.

1. Introduction

Autistic spectrum disorder (ASD), by its classic definition, is a neurodevelopmental disorder characterized by a triad of symptoms such as impairment of social interaction, difficulties in social skills acquisition connected with qualitative impairment of communication, and restricted and repetitive ways of behaviors, interests, and activities [1]. However, in recent years, the focus has been shifted to two core domains: social communication and restricted behavior [2]. In addition, the development of children with ASD is often burdened by sensory disintegration and limitations in information processing [3], cognitive impairment [4], psychosomatic symptoms [5], increased anxiety level [6], and behavioral disorders, such as food refusal, self-injury, and aggression [7]. The variability of clinical and behavioral symptoms depends on the child’s age, cognitive abilities, and verbal functions [8]. In recent years, there has been a significant increase in the number of children with ASD, so that approximately 1 in 54 children has been identified with ASD, according to the data provided by CDC’s Autism and Developmental Disabilities Monitoring (ADDM) Network [9]. It is common for children with ASD to have regression in language or social skills, which happens most typically between 18 and 24 months of age and affects approximately one-quarter of children with ASD [10]. The early identification and evaluation of autism are of fundamental social importance since the success of subsequent socialization and adaptation of children with ASD depends directly on the time of diagnosis [11]. However, the assessment is complicated by the fact that the diagnostic criteria are diverse and heterogeneous and have to include both social and non-social domains of child development [12]. Despite multiple attempts to identify the common cause of ASD symptoms, the question remains unanswered. Furthermore, ASD shares multiple symptoms with developmental delay, childhood schizophrenia, intellectual disability, and other conditions, especially at the early preschool age, which complicates differential diagnosis and early intervention [13]. According to a large survey conducted in European countries [14], the mean age for ASD assessment is 3 years and 5 months (SD = 13.42). In contrast to Western countries, Russia does not carry out systematic monitoring of youngsters’ mental development. In fact, Russian children first come into the focus of specialists’ attention when they start attending preschool institutions at the age of 3 years. At the same time, some preschoolers are home-schooled and assessed for the first time at the age of 6–7 years when they start attending primary school. Herewith, there is no detailed epidemiological data for Russia on ASD yet, and it is hard to estimate the mean age of diagnosis. Though according to the pilot epidemiological screening of the risk for ASD and other mental disorders, conducted by the Ministry of Health of the Russian Federation from 2014 to 2016 in the largest regions of the country (according to the criteria of ICD-10 F84.0-F84.8), the incidence of ASD in children under 2 years old was 0.5:1000 and in children under 4 years old was 1.8:1000 [15]. Another complication for such data collection is the lack of diagnostic methods adopted for the Russian population.

Various tools have been developed to identify symptoms and determine the severity of ASD in children. Among the most frequently applied are Childhood Autism Rating Scale (CARS), Modified Checklist for Autism in Toddlers (M-CHAT), Autism Diagnostic Interview (ADI), and Autism Diagnostic Interview–Revised (ADI–R), and Autism Diagnostic Observation Scale (ADOS). CARS consists of 14 domains aimed to assess signs of autistic behaviors, while the 15th domain rates general impressions of autism [16]. CARS has its limitations since it appears to give false-positive results, as was noticed by C. Lord [17]. M-CHAT has been created to evaluate symptoms of autism in toddlers and includes parents’ questionnaires and interviews [18]. Among the known disadvantages of the method are high false positive and false negative rates in children with sensory impairments [19]. ADI and its latest version ADI-R are considered the most accurate diagnostic tools, covering core domains of ASD, that allow collecting data by interviewing a parent or a caregiver of the patient [20]. The main drawback of this tool is that it takes approximately two hours to conduct the data collection. Additionally, the obtained scores are particularly influenced by age, IQ, and language level of the child, and thus it may be challenging to evaluate the severity of ASD symptoms [21]. ADOS is a standardized semi-structured tool, allowing assessment of social interactions as well as the development of isolated skills in children of different ages and speech development [22]. There are also some limitations in this tool application, including costliness of a standardized set of toys, absence of cross-validation for sensitivity and specificity, and presence of false positive and false negative results [23]. Early intervention programs for children with ASD have proven their efficacy in numerous studies. One of the most effective early intervention programs is the Early Start Denver Model (ESDM) [24]. ESDM is designed for children from 3 to 4 years old and continues until they turn 4–5. This emphasizes the importance of early and accurate ASD assessment.

Since the criteria of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) do not describe the structure of autistic symptoms accurately enough and existing diagnostic tools have some known limitations, more research is needed to capture the structure of ASD in early childhood and identify the predictors. There is a gap in the literature concerning factor structure of autistic symptomatology, understanding of which is crucial for learning mechanisms underlying ASD. Although the research is scarce and its results are equivocal, the latter have been nevertheless critical to the interpretation of ASD patterns. For example, a five-factor structure of ASD was obtained where three factors were connected with social communication impairment (emotion recognition, social avoidance, and interpersonal relatedness), and two factors were linked to repetitive behavior (insistence on sameness and repetitive mannerisms) [25]. Another five-factor structure of ASD consisted of restricted and repetitive behavior and interests, shaking and nodding, and three factors representing deficits in social interaction and communication [26]. In another study, a four-factor structure was obtained, consisting of social relationships, nonverbal communication/socialization, verbal communication, and restricted interests/insistence on sameness [27]. It is obvious that obtained factor structures of ASD correspond to the two-factor structure in the updated DSM-5 but also bring up some new dimensions in understanding of ASD.

The purpose of the current study is to develop an Autism Scale to evaluate ASD risks in 3–4-year-old Russian children by identifying a factor structure of ASD. The current study differs from previous ones in that the factor structure of ASD is identified not by using a ready-made questionnaire but rather by revealing a factor structure of a diverse set of items connected with ASD symptoms. The rationale for focusing on 3–4-year-olds is that the assessment of 3-year-olds compared to 2-year-olds shows greater predictive validity [28], and that is usually the age when children start receiving the diagnosis, although some characteristics like speech fluency could push that timeline forward [14]. Another reason for choosing this age group is that children in Russia usually start attending preschool institutions at this age, and if they have any ASD symptoms, it becomes more noticeable in comparison to typically developing peers to parents and educational specialists.

2. Materials and Methods

2.1. Materials

An autism questionnaire was developed especially for this study. The tasks’ content (situations, areas of activity, behavioral pattern, and possible signs of ASD) was obtained by interviewing experienced specialists engaged in psychological and pedagogical support of children in ordinary and special preschool institutions. They selected the signs of ASD that are key for assessment by observing the child’s behavior and interviewing his/her parents. As a result, an array of distinctive signs was collected, including well-known ones used in CASD and ADOS, as well as features specific for the Russian sample. Additionally, to analyze the possibilities of differential diagnosis, the survey included the signs of developmental delay (DD) identified in our previous studies [29,30,31]. The questionnaire tasks were grouped into 12 distinct domains. Each domain had within 1 to 14 tasks, resulting in 85 tasks in total. All domains are described in

Table 1. The domains of autism questionnaire.

Code	Number of Tasks	Domain Name
I	1	Child’s interests (diversity, focus, sustainability of interests)
E	5	Emotions (features of emotional reactions, smile, mood changes, other people’s emotions understanding)
S	14	Speech (the presence and features of speech, vocalizations; nonverbal communication: facial expressions, gestures, pictures; the use of varying complexity speech structures, intonations, speech understanding)
L	12	Social interaction (the nature of interaction with other people, eye contact, demonstration of objects of interest, attracting attention, the presence of anxiety, attachment, communication with relatives)
M	4	Communication (ways to communicate his/her needs to others, refuse to do something, communication with family and strangers)
G	6	Play (attitude to different types of play, difficulty levels, independence in the play, interaction with other children and adults in the play)
F	4	Self-care and independence (independence in everyday life, including dressing, eating).
B	13	Behavior (typical reactions and patterns, self-control in various situations)
C	12	Cognitive functions (characteristics of attention, memory, skills acquisition, perception, reactions to various stimuli)
P	1	Physical development (level of physical activity, agility).
N	11	Sensory processes (features of responses to certain stimuli of different modality: auditory, visual, tactile, vestibular, proprioception, features of tonic regulation).
H	2	Health (anamnesis)

.

2.2. Justification of the Choice of the Domains

The selected domains are described in detail in the literature sources. The deficit in social interaction is the core symptom of ASD and is mentioned in every publication on this problem [32]. Impairment of communicative skills acquisition is closely related to symptoms of impaired social interaction. Abnormalities of language development, comparatively slower rates of gestures attainment, decrease in diversity of gestures and facial expressions are well-recognized signs of ASD [33]. Herewith, the majority of children with ASD have difficulty mastering language skills [34]. Even those who have phrasal or fluent speech still demonstrate such abnormalities as increased pitch variability [35], monotone intonation, atypical speed and volume of speech [36], deficits in prosody processing [37], echolalia [38], mistakes in using prepositions [39], and delays in syntactic knowledge acquisition [40]. Children with ASD exhibit deficits in various types of play, especially social ones based on imitation skills [41]. Furthermore, restricted interests, repetitive motor movements, and insistence on sameness are persistent in ASD and considered to be the key symptoms with high prevalence and severity [42] as well as stability over time [43]. The data on emotional regulation in young children with ASD is limited, and research usually focuses on older high functioning individuals. Nonetheless, children with ASD have difficulties in recognizing emotions from facial expressions [44], are in a risk group for being diagnosed with anxiety and mood disorders [45], and have less adaptive emotion regulation profile in cases of Asperger’s syndrome and high functioning autism [46]. In addition, atypical behavioral patterns and reactions are the most common sources of parental stress [47] and one of the most staggering obstacles in an autistic child’s education [48]. Aggression towards oneself [49] and others [50], insistence on nonfunctional rituals and activities, noncompliance to behavior rules, and tantrums [50] are common among patients with ASD and cause more psychological familial distress than core autistic symptoms [47]. The level of intellectual development in children with ASD is a strong predictor of social and adaptive functioning in adult life [51] and is associated with restricted interests and repetitive behavior [52], sensory processing [53], language growth [54], atypical behavior and expressive language [50]. Sensory symptoms in infancy are associated with a higher risk of developing ASD [55]. At the same time, these symptoms are not always present in children with ASD and can be observed in other developmental disorders [56]. Nevertheless, sensory processing abnormalities significantly contribute to the development of social, cognitive, and behavioral skills, deficits in which determine autistic disorders [57,58]. The ability to take care of one’s own basic needs is fundamental for independent functioning and starts developing early in life. This domain remains little explored compared to others in preschoolers with ASD. A significant correlation has been demonstrated between sensory avoidance, over-responsiveness to sensory stimuli, fine motor skills, and self-care skills regardless of children’s cognitive performance [59]. Several studies have shown that prenatal maternal stress is associated with a higher risk for ASD [60]. Anamnestic data concerning somatic diseases is also an important factor since children with ASD are more than twice as likely to develop ear infections and otitis-related complications [61].

2.3. Procedure

In each task, the specialist was asked to select either one, several, or none of the items (statements) concerning the presence of certain ASD signs that characterize a child. Here is an example:

B28. Describe a child’s attitude to changes.

• B2801. The child experiences stress due to changes (in daily routine, walking routes).
• B2802. The child has difficulties in shifting his/her attention (for example, between activities).
• B2803. When the child chooses an activity, he/she has an urgent need to complete the work he/she has begun.
• B2804. The child prefers rituals (goes to bed under a certain scenario, drinks only from his/her favorite cup, dresses only in a specific order, walks only a certain route).
• B2805. The child is very picky about food, has a limited range of favorite dishes, constantly demands to be given the same food, has requirements for food shape, color, consistency, design.

Thus, for each child, values were recorded for 434 items (0 = no, 1 = yes). Item designation format was the following: the letter assigns the task to one of the listed domains (see Table 1), the next two digits are a task’s number, and the last two digits are an item’s number.

The autism questionnaire was designed as an online survey on the website of the online testing system lnd-spb.ru. After the completion of data collection, the questionnaire was switched into a demo mode for the possibility of demonstrating it to new participants and working together on a new version of the questionnaire for the next stage of the study (the current version works without saving the entered data at https://lnd-spb.ru/go/m505-demo-mask).

Data collection was carried out by 17 experienced specialists engaged in psychological and pedagogical support of children in specialized and ordinary preschool institutions of St. Petersburg and Omsk. These specialists have collaborated with the main executors of this project in the past as part of research or training groups. The project executors invited specialists to take part in the online survey by email, providing a link to the survey website and indicating which groups of children should be examined. The assignment to the groups of ASD, DD, or healthy controls was carried out as a part of children’s examinations after receiving written parental consent or based on a previously obtained opinion from other specialists (e.g., presented upon admission to a preschool educational institution). Thus, the data was collected for children who had already been classified by specialists as belonging to either ASD, DD, or healthy controls before the study began.

The specialists obtained data in the process of personal sessions with a child while consulting his/her parents on a regular basis. After attaining the necessary information, the specialists entered the data on a child into the online questionnaire out of the consultation time. The data was collected anonymously. Only a child’s research code, answers to questions, assignment to a group (ASD, DD, and healthy controls), birth date, and gender were registered in the database. The data collection was carried out from 23 August 2020 to 25 October 2020 on 324 children of 3 to 4 years old.

2.4. Sample

The minimal sample size required for the assessment of test performance was calculated with a formula suggested by Arkin and Wachtel [62] for constructing confidence intervals of a specific length:

$$N_D=\frac{Z_{\alpha /2}^{2}p\left(1-p\right)}{r^2}$$

(1)

where N_D is a number of clinical cases (e.g., diagnosed with ASD), Z_α/2 is z-value corresponding to a chosen confidence interval, p is a predefined level of sensitivity or specificity, and r is desired width of one-half of a chosen confidence interval. Initially, we aimed for the sensitivity of 80% and specificity of 85%. Z_α/2 value was chosen for 95% confidence interval (z = 1.96), whereas r was chosen as 10%. As a result, sensitivity calculations yielded a minimal sample size of 62 clinical cases, and specificity calculations ended with comparable 49 cases. Given that we planned the ratio of children with ASD to those without ASD to be 1:2, the total sample size had to be no less than 186.

In total, 214 boys and 108 girls aged 3 to 4 years (evenly represented in the age range from 1065 to 1824 days) were examined. The distribution of the sample by age and diagnosis is presented in

Table 2. Distribution of the sample by age and diagnosis.

Diagnosis	3-Year-Olds	4-Year-Olds	Total
ASD Count	49	67	116
%	42.2%	57.8%	100%
Norm Count	86	45	131
%	65.6%	34.4%	100%
DD Count	29	48	77
%	37.7%	62.3%	100%
Total Count	164	160	324
%	50.6%	49.4%	100%

. Supplementary Materials can be found at https://lnd-spb.ru/go/m505-demo-mask, data supporting reported results can be found at https://www.researchgate.net/publication/35301423.

2.5. Data Analysis

Data analysis was carried out for the following purposes: (a) identification of a compact set of subscales that have sufficient reliability and most accurately predict the child’s belonging to the ASD group; (b) determination of the relative contribution of the subscales to grouping on ASD/non-ASD; (c) cross-validation of the prediction model by constructing a discriminant model on a sample of 3-year-olds to test its effectiveness on a sample of 4-year-olds, and vice versa; (d) development of Autism Scale and testing its effectiveness. Statistical analysis was performed using R version 3.6.2 (R Core Team. 2019. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria) and IBM SPSS Statistics 26 version (IBM Corp. Released 2019. IBM SPSS Statistics for Windows, Version 26.0. Armonk, NY, USA: IBM Corp.).

3. Results

3.1. A Preliminary Selection of Items and Formation of the Subscales

The main steps of preliminary data analysis are depicted in Figure 1. The purpose of this stage of the analysis was to identify a factor structure that meets the following requirements: (a) the factors provide high accuracy in identifying the ASD group; (b) each item is included in only one factor with a factor loading of at least |0.4|; (c) each factor includes a set of items that provide sufficiently high internal consistency reliability (Cronbach’s alpha is not lower than 0.7); (d) each factor has a distinct meaningful interpretation. Firstly, out of 434 items, we selected 409 for which any response option (yes/no) in the ASD group had a frequency of less than 95%. Then, for each item, the discrimination value was calculated as the absolute value of the phi coefficient of this item and the grouping variable ASD/non-ASD. Three hundred and eleven items with a phi coefficient of at least 0.11 (p < 0.05) were selected. To further reduce the number of items, we used Principal Component Analysis (PCA) with equamax rotation and Kaiser normalization. The calculated components served as independent variables for Discriminant analysis (DA); the grouping variable was ASD/non-ASD. The step-by-step procedure of DA was used, with p = 0.05 for F-deletion and p = 0.1 for F-inclusion. At the first step, we used a threshold eigenvalue greater than one and identified 80 components that explained 78% of the total variance. According to DA results, 48 components were excluded. For the next step, 32 components and 219 items were specified. These steps (PCA–DA cycle) were repeated until 12 components with 190 items were left. After that, all items having at least one component loading higher than |0.4| were retained, resulting in 170 items in total. According to the scree plot of eigenvalues after PCA of 170 items, the “break” point was observed at the fifth eigenvalue. Therefore, in the next step, six components were set, which were calculated after the rotation. DA showed that five components were at F-removal p < 0.001, and the sixth one, which explained the maximum variance, was at F-removal p = 0.015. Furthermore, for the sixth component, the standardized coefficient of the discriminant function was 2.5–3 times less than for the other components. Although this component included 64 items with loadings higher than 0.4, 56 of them were excluded due to low discriminative coefficients (≤0.35). For the remaining 114 items, an alpha factor analysis was applied with the assignment of five factors and equamax rotation. After that, all items not included in any factor with the loading of at least |0.35| were removed, leaving 102 items. Four out of five factors encompassed items similar in meaning and had a fairly clear interpretation. From the factor with an unclear interpretation, 18 items with discrimination lower than 0.35 were removed. Regarding the remaining 84 items, the alpha factor analysis with equamax rotation and number of factors fixed to four was repeatedly applied, followed by the removal of items according to the following criteria: (1) items are included in the most informative factor but have lower discrimination than items included in this factor with loadings of at least |0.4|; (2) items are “not similar” in meaning to other items included in this factor. Combining these criteria, we obtained a four-factor structure with 43 items.

At the final stage, we conducted an exploratory factor analysis (EFA) on 43 items to determine the underlying factor structure in order to group items into subscales. The analysis was done in R 3.6.2 using packages psych, GPArotation, and EGAnet [63,64]. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.88, indicating that data were appropriate for factor analysis. The number of factors was identified by parallel analysis (PA), which is among the most robust methods for such purposes [65,66]. More specifically, we employed PA with tetrachoric correlations using PCA and decided on a number of factors by the 95th percentile value. Under such conditions, PA was considered to perform optimally given the results of previous Monte-Carlo simulations [67]. We also applied exploratory graph analysis (EGA) with the triangulated maximally filtered graph algorithm to support our decision on the number of factors to extract. Although EGA is a relatively new technique, prior simulations have already demonstrated its high accuracy in establishing data dimensionality [64,68]. As a result, PA suggested four or five factors, whereas EGA suggested four factors. Hence, we decided to look at both four- and five-factor models.

EFA was performed using weighted least squares (WLS) on tetrachoric correlations with Crawford–Ferguson oblique rotation (κ = 0.06; see Sass and Schmitt [69]). We chose the WLS estimation method rather than WLSMV (weighted least squares means and variance adjusted) because the latter failed to converge due to technical problems. At the same time, we admit that WLS could yield suboptimal results in our case, given that its functioning depends heavily on both sample size and model complexity [70,71]. As for the rotation criterion, we preferred Crawford–Ferguson family because it allowed greater flexibility for result optimization. To enable a more parsimonious solution, we put emphasis on minimizing variable complexity and thus considered smaller kappa values (i.e., closer to zero). Due to difficulties in interpreting the five-factor solution, the four-factor solution was retained. The factors included 40 items, the distribution of positive responses to which is presented in

Table 3. Distribution of positive responses on 40 items for ASD and non-ASD samples.

Item	ASD (n = 116)		Non-ASD (n = 208)		Total (n = 324)
	Count	%	Count	%	Count	%
L8201	25	21.6	4	1.9	29	9.0
L8303	58	50.0	8	3.8	66	20.4
L1501	39	33.6	5	2.4	44	13.6
G2101	48	41.4	4	1.9	52	16.0
L8210	26	22.4	7	3.4	33	10.2
M1601	36	31.0	16	7.7	52	16.0
M1702	53	45.7	20	9.6	73	22.5
C3201	36	31.0	5	2.4	41	12.7
S5301	25	21.6	88	42.3	113	34.9
M7205	14	12.1	136	65.4	150	46.3
E7403	20	17.2	138	66.3	158	48.8
E7401	72	62.1	28	13.5	100	30.9
L0903	19	16.4	150	72.1	169	52.2
L1003	17	14.7	154	74.0	171	52.8
M7201	54	46.6	40	19.2	94	29.0
L1304	53	45.7	47	22.6	100	30.9
S5404	37	31.9	15	7.2	52	16.0
L8202	24	20.7	17	8.2	41	12.7
L0902	51	44.0	36	17.3	87	26.9
E0302	45	38.8	10	4.8	55	17.0
B2707	40	34.5	7	3.4	47	14.5
B2804	48	41.4	13	6.3	61	18.8
N4601	69	59.5	34	16.3	103	31.8
B2805	63	54.3	19	9.1	82	25.3
B6501	54	46.6	22	10.6	76	23.5
F2303	29	25.0	30	14.4	59	18.2
N4205	54	46.6	17	8.2	71	21.9
N4602	37	31.9	25	12.0	62	19.1
C3304	50	43.1	25	12.0	75	23.1
I0103	44	37.9	9	4.3	53	16.4
B2502	20	17.2	17	8.2	37	11.4
B6201	44	37.9	25	12.0	69	21.3
P3905	27	23.3	21	10.1	48	14.8
N6901	36	31.0	41	19.7	77	23.8
B2901	15	12.9	13	6.3	28	8.6
B2501	48	41.4	39	18.8	87	26.9
B6202	46	39.7	37	17.8	83	25.6
B2603	23	19.8	7	3.4	30	9.3
B2503	39	33.6	38	18.3	77	23.8
N7102	36	31.0	22	10.6	58	17.9

.

The main results of this stage of analysis, with items grouped into subscales (S1–S4), are presented in

Table 4. Pattern coefficients for four-factor solution (EFA) and reliability estimated by Cronbach’s alpha.

Rotated Pattern Coefficients of 40 Items (n = 324), α = 0.909, 60.87% of Variance	PC 1
Factor 1 (S1 3): Communication (9 items; 15.75%, α = 0.848)
L8201. Practically the child does not communicate.	0.963
L8210. The child almost never responds when addressed; never initiates contact with an adult	0.789
S5301 2. The child speaks, but the grammatical structure of his/her speech is broken	0.783
G2101. The child plays by him/herself, aloof, does not allow other people into his play neither adults nor children	0.680
M1601. The child has no appeal to other people. He/she does not communicate his/her needs, tries to take everything on his/her own, or uses strategies typical to young children (crying). What the child “asks” for becomes clear when the cry stops.	0.671
L1501. The child does not imitate the actions of other people: he/she is busy with his/her own business, and does not pay attention to people.	0.663
L8303. The child does not respond to questions addressed to him/her	0.649
M1702. The child does not verbally express his/her refusals. This can be understood from his/her gestures (pushes away, shakes his/her head) or facial expressions, or vocalizations	0.488
C3201. It seems that the child does not pay attention to the surroundings he/she wanders around, takes objects aimlessly, does not focus on them, and immediately throws them, sometimes behind his back	0.478
Factor 2 (S2 3): Emotions (11 items; 15.58%, α = 0.859)
L8202. The child communicates little (no more than 10–15 min a day), spends more time on his own	0.753
M7201. The child uses pointing and communicative gestures. The child addresses others only when he/she needs something	0.700
S5404. The child’s speech is imitative (echolalia): he/she repeats the words of other people, but does not use speech for communication	0.662
M7205 2. The child easily talks about his/her needs. Uses words, phrases, gestures, and facial expressions.	0.624
L1304. The child has difficulty in making friends	0.622
E7401. The child has difficulty in recognizing other people’s emotions and responding to those emotions	0.582
E7403 2. When watching a cartoon, the child understands what is happening on the screen and emotionally reacts appropriately in the same way in familiar situations.	0.548
L0903 2. The child immediately looks at the person who is addressing him	0.542
L0902. To make the child look into the eyes of someone who speaks to him/her, it is needed to ask him/her to do so (for example, “look at me”), but he/she does not show a desire to look into the eyes by him/herself	0.529
E0302. The child’s smile is delayed, not related to the adult’s smile	0.527
L1003 2. Showing something to another person, the child places the object so that it can be viewed and checks whether the person sees what he/she is being shown	0.494
Factor 3 (S3 3): Sensorics (10 items; 15.03%, α = 0.824)
F2303. The child is rebellious he/she demands to get changed into the clothes that he/she prefers	0.832
B2804. The child prefers rituals (goes to bed under a certain scenario, drinks only from his/her favorite cup, dresses only in a specific order, walks only a certain route)	0.716
B2707. The child has unusual fears, such as fear of elevators, stairs, toilets, balconies, vacuum cleaners, etc.	0.714
N4602. The child’s mouth and the space around the mouth are hypersensitive (difficulties in teeth brushing, speech therapy massage, visiting a dentist)	0.673
B6501. The child is overly attached to certain objects (blanket, toy, clothing). If a favorite item is lost, he/she is worried, may get hysterical	0.635
C3304. The child has well developed sensory-motor skills (for example, can do puzzles, plays blocks, good with electronic devices, understands how to deal with various mechanisms)	0.614
B2805. The child is very picky about food, has a limited range of favorite dishes, constantly demands to be given the same food, has requirements for food shape, color, consistency, design	0.597
N4601. The child is picky/sensitive to certain food textures (for example, pieces in mashed potatoes or porridge). “Inappropriate” food is disgusting for him/her
N4205. The child likes to watch the lights turn on and off, the doors open and close, the wheels turn, the fan spins, the blinds open and close, shiny objects, pages flickering when flipping through, etc.	0.574
I0103. The child retains an unusually long interest in certain objects toys, ropes, balls, stones, plugs and lids, car wheels, toy parts, etc.	0.476
Factor 4 (S4 3): Hyperactivity (10 items; 14.51%, α = 0.817)
B2502. Can’t play quietly, being inadequately noisy.	0.823
N6901. There are too many unnecessary movements in the child’s activity, the child is fussy, hyperactive	0.789
B2901. The child is aggressive, pugnacious, prone to physical aggression against animals and other people	0.783
P3905. The child can’t sit still, leaves his seat in the classroom or elsewhere, jumps up, and wanders around	0.734
B6201. Loses self-control, is prone to emotional outbursts.	0.718
B2501. The child does not know how to wait, is not able to stand in queues or wait for his/her turn	0.672
B6202. The child strives to achieve his/her goal, easily “loses his/her temper”	0.658
B2603. The child is often angry and irritable	0.626
N7102. The child can’t sit still. He swings his body or shakes his head; chooses swings on the playground	0.617
B2503. The child is challenging to control. His/her behavior depends on external stimuli—“I run everywhere I look” (chaotic behavior)	0.591

¹—pattern coefficients; ²—inverted items; ³—the subscales’ designation used below in the text.

. Five items included in the factors with negative loadings were inverted beforehand.

Each factor (subscale) demonstrated sufficient reliability, with all values of Cronbach’s alpha being higher than 0.80. Each item was included in only one subscale with a factor loading of at least |0.4|. The factors were named in accordance with the items included in them (see Table 4). Factor 1 was named Communication because it included items related mostly to verbal and non-verbal communication and social skills (e.g., “The child does not respond to questions addressed to him/her”). Factor 2 was labeled Emotions since most items in the factor referred to the child’s ability to establish emotional contact, express their own emotions, and decode emotions of others in the context of social interaction (e.g., “The child’s smile is delayed, not related to the adult’s smile”). Factor 3 contained items that were related to the child’s sensitivity to a particular sensory modality and behavioral features associated with sensory disintegration (e.g., “The child has unusual fears, such as fear of elevators, stairs, toilets, balconies, vacuum cleaners, etc.”). Thus, it was named Sensorics. Factor 4 was labeled Hyperactivity because all items included in this factor were linked with excessive activity, motor disinhibition, and restless behavior (e.g., “The child cannot sit still, leaves his seat in the classroom or elsewhere, jumps up and wanders around”). Further, the subscales values were calculated for each child as the sum of items, included in the corresponding factor: S1 (Communication), S2 (Emotions), S3 (Sensorics), S4 (Hyperactivity), and SS (the sum of these four subscales’ values).

3.2. Discriminability and Relative Contribution of the Subscales for Predicting ASD/Non-ASD Group Membership

DA was applied to determine the effectiveness of ASD/non-ASD classification based on four subscales (S1–S4) and child’s age (in days). Age was included in the analysis to test the assumption that it can influence group membership, as we found earlier in the classification of healthy controls and DD groups [26,27,28]. The stepwise method was used to check the relative contribution of independent variables and to determine the significance level of F-to-enter/remove for each of these variables. At the fourth step of the analysis, the program included all four subscales with a significance of F-to-remove no more than 0.05 and excluded age with a significance of F-to-enter equal to 0.907. The standardized coefficients of the discriminant function are given in

Table 5. Standardized canonical discriminant function coefficients.

Variables	Coefficients
Communication (S1)	0.424
Emotions (S2)	0.435
Sensorics (S3)	0.536
Hyperactivity (S4)	0.208

(the centroid of the function for the ASD group is at the positive pole, while for the non-ASD group, it is at the negative pole).

Thus, all four subscales made a significant contribution to the prediction of belonging to the ASD/non-ASD group, and there was no need to take age into account. The largest relative contribution was made by Sensorics, while the smallest was made by Hyperactivity. The higher the value for each of the four subscales, the higher the probability of belonging to the ASD group.

Table 6. Classification results (Discriminant analysis).

Diagnosis			Predicted Group Membership		Total
			ASD	non-ASD
Original a	Count	ASD	98	18	116
		non-ASD	21	187	208
	%	ASD	84.5	15.5	100.0
		non-ASD	10.1	89.9	100.0
Cross-validated b	Count	ASD	98	18	116
		non-ASD	21	187	208
	%	ASD	84.5	15.5	100.0
		non-ASD	10.1	89.9	100.0

^a 88.0% of original grouped cases were correctly classified, ^b Cross-validation was done only for those cases in the analysis; in cross-validation, each case was classified by the functions derived from all cases other than that case; 88.0% of cross-validated grouped cases were correctly classified.

shows the ratio of the actual classification of cases into groups of ASD/non-ASD predicted by the discriminant function.

Thus, using a discriminant function with four predictors (S1–S4) to classify children into two groups, the expected prediction accuracy was 88.0% (sensitivity of 84.5% and specificity of 89.9%). However, we had to make sure that such a model will be suitable for different parts of the sample that differ in gender and age. To do this, predictive cross-validation of this discriminant model was performed.

3.3. Cross-Validation of the Discriminant Prediction Model

Cross-validation was carried out four times: two times for age groups (3 and 4 years) and two times for boys and girls. Each time, the DA was carried out by selecting one group for which the discriminant prediction model was built, and this model was used to classify the non-selected observations.

The results of age-adjusted cross-validation are shown in

Table 7. Cross-validation based on the child’s age.

Group				Predicted Group Membership		Total
				ASD	non-ASD
Cases Selected a (3-year-olds)	Original	Count	ASD	42	7	49
			non-ASD	12	103	115
		%	ASD	85.7	14.3	100.0
			non-ASD	10.4	89.6	100.0
Cases Not Selected b (4-year-olds)	Original	Count	ASD	59	8	67
			non-ASD	20	73	93
		%	ASD	88.1	11.9	100.0
			non-ASD	21.5	78.5	100.0
Group				Predicted Group Membership		Total
				ASD	non-ASD
Cases Selected c (4-year-olds)	Original	Count	ASD	55	12	67
			non-ASD	8	85	93
		%	ASD	82.1	17.9	100.0
			non-ASD	8.6	91.4	100.0
Cases Not Selected d (3-year-olds)	Original	Count	ASD	37	12	49
			non-ASD	7	108	115
		%	ASD	75.5	24.5	100.0
			non-ASD	6.1	93.9	100.0

^a 88.4% of selected original grouped cases were correctly classified, ^b 82.5% of unselected original grouped cases were correctly classified, ^c 87.5% of selected original grouped cases were correctly classified, ^d 88.4% of unselected original grouped cases were correctly classified.

. The results showed that the 3-year-old prediction model correctly assigned ASD/non-ASD group membership for 4-year-olds with 82.5% accuracy, and the 4-year-old prediction model correctly assigned ASD/non-ASD group membership for 3-year-olds with 88.4% accuracy.

The results of cross-validation based on the child’s gender are presented in

Table 8. Cross-validation based on the child’s gender.

Group				Predicted Group Membership		Total
				ASD	non-ASD
Cases Selected a (Boys)	Original	Count	ASD	73	16	89
			non-ASD	14	111	125
		%	ASD	82.0	18.0	100.0
			non-ASD	11.2	88.8	100.0
Cases Not Selected b (Girls)	Original	Count	ASD	21	6	27
			non-ASD	3	80	83
		%	ASD	77.8	22.2	100.0
			non-ASD	3.6	96.4	100.0
Group				Predicted Group Membership		Total
				ASD	non-ASD
Cases Selected a (Girls)	Original	Count	ASD	23	4	27
			non-ASD	5	76	81
		%	ASD	85.2	14.8	100.0
			non-ASD	6.2	93.8	100.0
Cases Not Selected b (Boys)	Original	Count	ASD	78	11	89
			non-ASD	24	103	127
		%	ASD	87.6	12.4	100.0
			non-ASD	18.9	81.1	100.0

^a 86.0% of selected original grouped cases were correctly classified, ^b 91.8% of unselected original grouped cases were correctly classified, ^c 91.7% of selected original grouped cases were correctly classified, ^d 83.8% of unselected original grouped cases were correctly classified.

. The boys’ prediction model assigned girls to ASD/non-ASD groups with an accuracy of 91.8%, and the girls’ prediction model assigned boys to these groups with an accuracy of 83.8%.

In all four cases, the prediction model with the inclusion of four predictors (S1–S4) demonstrated cross-predictive validity of at least 82.5%.

3.4. Forming Autism Scale and Testing Its Effectiveness

When constructing the questionnaire, it is necessary to take into account that the four subscales make a different relative contribution to the grouping on ASD/non-ASD. In order to do this, we applied DA to the entire sample to calculate the unstandardized coefficients of the discriminant function. We used these coefficients to calculate discriminant scores (DS) for each child. The equation for calculating discriminant scores (DS) was as the following:

DS_i = −1.803 + 0.229 × S1_i + 0.175 × S2_i + 0.268 × S3_i + 0.094 × S4_i

Descriptive statistics for each subscale (S1–S4), summary scale (SS), and discriminant scores (DS) are shown in

Table 9. Descriptive statistics for each subscale (S1–S4), summary scale (SS), and discriminant scores (DS) ¹.

Variable	Group	N	Mean	Std. Deviation	Skewness	Kurtosis	Cohen’s d
Communication (S1)	ASD	116	3.552	2.632	0.421	−0.953	1.429
	non-ASD	208	0.909	1.214	3.527	16.843
	Total	324	1.855	2.241	1.589	1.644
Emotions (S2)	ASD	116	6.293	2.377	−0.312	−0.443	1.667
	non-ASD	208	2.149	2.545	1.226	0.879
	Total	324	3.633	3.181	0.425	−1.034
Sensorics (S3)	ASD	116	4.207	2.762	0.525	−0.767	1.622
	non-ASD	208	0.966	1.405	1.769	3.095
	Total	324	2.127	2.530	1.351	1.115
Hyperactivity (S4)	ASD	116	2.879	2.530	0.813	−0.108	0.735
	non-ASD	208	1.250	2.023	2.165	4.915
	Total	324	1.833	2.348	1.463	1.586
SS	ASD	116	16.931	5.881	−0.250	−0.132	2.265
	non-ASD	208	5.274	4.689	1.292	1.277
	Total	324	9.448	7.598	0.620	−0.741
DS	ASD	116	1.510	1.225	−0.084	−0.340	2.353
	non-ASD	208	−0.842	0.849	1.495	2.120
	Total	324	0.000	1.508	0.773	−0.475

¹ The statistical significance of Student’s t-test for all comparisons was p < 0.0001 (with the Benjamini–Hochberg correction for multiple comparisons).

.

The effect size (Cohen’s d) of the mean differences confirmed the previously obtained result concerning the contribution of the subscales (S1–S4) to classification accuracy: the largest effect sizes for Emotions (S2) and Sensorics (S3) were followed by Communication (S1) and Hyperactivity (S4; in descending order). The effect size for DS was only slightly higher than the effect size for SS. At the same time, the distribution of DS yielded skewness and kurtosis values close to the normal distribution. Therefore, the DS distribution for the ASD sample was used to establish test norms for the Autism Scale. Figure 2 shows the DS distributions for ASD and non-ASD samples. Reference lines indicate the lower (yellow) and upper (red) limits of critical DS values for assigning a child to the ASD or non-ASD group.

Due to the fact that the DS distribution for the ASD sample was close to a normal distribution, the test norms for Autism Scale were formed by dividing the entire DS range into equal intervals with a step of 0.25. Thus, a 20-point scale was formed. Receiver operating characteristic (ROC) analysis was performed to evaluate the effectiveness of the ASD/non-ASD group belonging forecast. The SS and S1–S4 subscales were compared. The ROC curves are shown in Figure 3, and the Area Under the Curves is shown in

Table 10. Area Under the Curves for the Autism Scale (Scale), summary scale (SS), and for each subscale (S1–S4).

Test Result Variable(s)	Area	Std. Error a	Asymptotic Sig. b	Asymptotic 95% Confidence Intervals
				Lower Bound	Upper Bound
Scale	0.931	0.015	0.000	0.902	0.961
SS	0.928	0.014	0.000	0.899	0.956
S1	0.819	0.028	0.000	0.765	0.873
S2	0.873	0.019	0.000	0.835	0.911
S3	0.869	0.020	0.000	0.829	0.909
S4	0.720	0.030	0.000	0.662	0.778

^a Under the nonparametric assumption, ^b Null hypothesis: true area = 0.5.

. The areas under the ROC curves were almost identical for SS and S1–S4 subscales. However, as can be seen in Figure 3, in the most important sensitivity range of 0.8 ± 0.1, subscales provided noticeably higher specificity than SS.

Table 11. Test norms for the Autism Scale (Scale) and main statistics for each subscale (S1–S4) and summary scale (SS).

Scale	DS a (Upper Bound)	Sensitivity	Specificity	Medians for Intervals
				SS	S1	S2	S3	S4
1	−1.00	0.974	0.553	2.0	1.0	0.0	0.0	0.0
2	−0.75	0.966	0.654	6.0	0.0	2.0	1.0	2.0
3	−0.50	0.922	0.750	7.0	1.0	3.0	2.0	2.0
4	−0.25	0.905	0.788	8.0	0.0	4.0	2.0	1.5
5	0.00	0.888	0.856	9.0	1.0	4.5	2.0	1.0
6	0.25	0.862	0.899	11.0	1.0	3.5	2.5	4.0
7	0.50	0.810	0.918	12.0	1.0	5.0	2.5	2.5
8	0.75	0.733	0.933	13.0	2.0	6.0	2.0	2.5
9	1.00	0.672	0.952	15.0	2.0	7.0	2.0	2.0
10	1.25	0.612	0.962	15.0	3.0	6.0	4.0	2.0
11	1.50	0.491	0.976	16.0	4.0	7.0	2.0	1.0
12	1.75	0.431	0.981	18.5	4.5	5.5	3.5	5.0
13	2.00	0.328	0.990	19.0	4.0	6.0	4.0	3.0
14	2.25	0.284	0.995	20.5	1.5	7.5	7.5	3.0
15	2.50	0.224	1.000	19.5	4.0	8.0	5.5	1.5
16	2.75	0.147	1.000	21.0	5.0	9.0	6.0	3.0
17	3.00	0.129	1.000	26.5	8.5	7.0	3.0	8.0
18	3.25	0.095	1.000	23.0	7.0	8.0	7.0	1.0
19	3.50	0.052	1.000	24.0	7.0	9.0	6.0	2.0
20	>3.50	0.000	1.000	25.5	6.5	8.0	9.5	2.0

^a—Discriminant Scores.

shows the test standards for the Autism Scale (Scale), as well as sensitivity and specificity for each of the 20 values of the scale.

The most important values for the ASD diagnosis are probably Scale = 5 (Sensitivity = 0.888; Specificity = 0.856) and Scale = 6 (Sensitivity = 0.862; Specificity = 0.899), which correspond to the DS range above −0.25 to 0.25 (see Table 11). Diagnostic accuracy for these Scale values are shown in

Table 12. Indicators of test norms effectiveness.

Statistic	Scale = 5		Scale = 6
	Value	95% CI	Value	95% CI
Sensitivity	88.8%	81.60% to 93.90%	86.21%	78.57% to 91.91%
Specificity	85.58%	80.05% to 90.05%	89.90%	84.98% to 93.64%
Accuracy	86.73%	82.54% to 90.23%	88.58%	84.60% to 91.83%

.

4. Discussion

This study is the next stage of our long-term research based on a psychometric approach in constructing screening tools for the early detection of developmental problems in Russian preschoolers. We should emphasize that the Autism Scale is a diagnostic tool for assessing the risk for ASD but not for making a diagnosis of ASD. It is a screening tool aimed at timely detection of autistic symptoms in children aged 3–4 years who need targeted attention from specialists and a more detailed examination. This age group was chosen because the vast majority of children in Russia—unlike children in most Western countries—become the focus of specialists’ attention only when they begin to attend preschool institutions at the age of three. While developing the Autism Scale, we clearly followed the recommendations of compliance with accepted psychometric standards in terms of specificity, sensibility, and accuracy [72], which made up in our case 86.7% and higher. We carried out our research on a moderate sample size (N = 324) and used an initially wide range of test tasks, ensuring the strictly identical process of data collection for each participant in order to avoid common biases in the development of screening scales.

All subscales in Autism Scale significantly contributed to the prediction of a child’s belonging to the ASD/non-ASD group. At the same time, the 40 items included in the Autism Scale represent all 12 initially formed domains and should be considered the most valuable ASD symptoms for prognosis. Furthermore, it was found that the greatest relative contribution to the ASD prediction was made by the Emotions subscale (S2; child’s ability to establish emotional contact, express his/her own emotions, and decode emotions of others in the context of social interaction), followed in descending order of the contribution by Sensorics subscale (S1; child’s sensitivity in a particular sensory modality and behavioral features associated with sensory disintegration), Communication subscale (S4; verbal and non-verbal communication and social skills), and Hyperactivity subscale (S3; excessive movements, motor disinhibition, and restless behavior). Importantly, the obtained factor structure of ASD corresponds to the updated DSM-5. More concretely, Emotions, Communication, and Sensorics subscales refer to the core ASD symptoms according to the DSM-5. Sensorics factor is connected with repetitive, stereotyped behaviors because it is well-known that shared neurobiological mechanisms may underlie hyperresponsive sensory symptoms and repetitive behaviors in children with ASD [73]. Meanwhile, the obtained Hyperactivity factor is beyond the scope of ASD core symptoms. Interestingly, one of the previous studies has found similar results with a five-factor structure of ASD, where factors such as restricted behavior and interests coupled with deficits in social interaction and communication went along with a factor of shaking and nodding, though the data were received on the sample aged between 3 and 23 years [26].

The higher the value for each of the four subscales, the higher the probability of belonging to the ASD group. The presence of almost a quarter of the 40 ASD symptoms (items) does not yet allow us to confidently state the presence of ASD in a child. Thus, the subscales of the Autism Scale are not vectors of autism itself but the directions in which children with autism differ from children without autism. This is a consequence of the fact that the subscales were developed on a mixed sample of children with and without ASD in such a way that they most accurately distinguished these particular groups.

One of the two most important subscales in predicting belonging to the ASD group was Emotions (S2), which is connected with limitations in social communication. Most studies in the field emphasize the impairment of social interaction in autistic children [13,17,43]. Additionally, it is a challenging task for children with ASD to recognize the facial expressions of others and to respond appropriately to others’ facial expressions of emotion [44], as well as to express their own emotions and develop empathy [74]. Thus, the obtained result is noteworthy as an early symptom of the autistic spectrum. The second most substantial contribution to predicting belonging to the ASD group was made by Sensorics (S1). This result was quite expected since most of the repetitive behaviors and narrowly focused intense stereotypical interests in children with ASD might be explained due to the sensory disintegration, which is a common issue [3,47,53]. Yet the pattern of sensory disintegration in each autistic child is whimsically unique [50]. Since impaired social interaction and difficulty in developing social skills is one of the core symptoms of autism, the obtained result on the contribution of Communication (S4) was not surprising at all. It is widely known that children with ASD experience significant difficulties in developing all social skills, including passive speech perception and initiating speech utterance [34], imitation skills [41], playing with others [75], etc. At the same time, the subscale Hyperactivity (S3), despite its smallest contribution, also deserves attention. Although autism research rarely focused on hyperactivity, it was found before that children with ASD experience elevated levels of hyperactivity/impulsivity [76]. Meanwhile, hyperactivity in children with ASD may indicate a biochemical imbalance with a predominance of excitatory amino acids as a neurochemical foundation for stereotypical behavior, aggression, and auto-aggression [77]. Apparently, hyperactivity can be considered an additional aggravating symptom that worsens the prognosis and indicates the need for pharmacotherapy.

Notably, the identified structure of ASD differs significantly from that of DD obtained for children of the same age, in which the most significant contribution was made by logical reasoning, motor development, and general awareness [31]. This once again highlights the unique position that the problem of autism occupies, as well as the huge difference between this neurodevelopment disorder and other developmental problems and the importance of timely diagnosis of ASD symptoms.

5. Conclusions

Our study is the first attempt to develop a screening scale for rapid assessment of ASD risk in Russian 3–4 year-olds. The main result of the study is the development of a 20-point Autism Scale, which has a fairly high accuracy of prognosis (more than 85%) that significantly exceeds the accuracy of existing screenings in other countries. This scale encompasses 40 symptoms of autism grouped under four vectors of its manifestation, in which three vectors—Emotions, Sensorics, and Communication—correspond to core autistic symptoms according to the DSM-5, and the fourth one—Hyperactivity—is an additional factor that may be prognostically unfavorable and needs further research. The Autism Scale will serve as a foundation for the development of an online screening system that allows quick identification of the ASD risk group for further clarification of the diagnosis. The practical application of this online system in the future will allow us to expand the number of children being examined and significantly improve the screening scales we have been developing.

6. Limitations

The present study has two major limitations. First, the Autism Scale and its subscales are not, in fact, vectors of ASD. They have been developed on a mixed sample of children with and without ASD and therefore reflect the vectors by which these groups diverge the most. Building a vector model of ASD is a part of our immediate plans that will require a significant increase in the sample size of children with ASD. Second, some children assigned by specialists to the DD group showed a large number of ASD symptoms, whereas some children from the ASD group showed only 1–3 out of 40 autistic symptoms included in the scale. This, of course, reduced the accuracy of the developed scale. Thus, one of the problems of the developed scale is the differentiation of children with ASD from children belonging to the DD group. We plan to solve this problem as the number of children being examined increases over time alongside an analysis of prediction errors.