Design and Validation of Rule-Based Expert System by Using Kinect V2 for Real-Time Athlete Support

Abstract

In sports and rehabilitation processes where isotonic movements such as bodybuilding are performed, it is vital for individuals to be able to correct the wrong movements instantly by monitoring the trainings simultaneously, and to be able to train healthily and away from the risks of injury. For this purpose, we designed a new real-time athlete support system using Kinect V2 and Expert System. Lateral raise (LR) and dumbbell shoulder press (DSP) movements were selected as examples to be modeled in the system. Kinect V2 was used to obtain angle and distance changes in the shoulder, elbow, wrist, hip, knee, and ankle during movements in these movement models designed. For the rule base of Expert System developed according to these models, a 2⁸-state rule table was designed, and 12 main rules were determined that could be used for both actions. In the sample trainings, it was observed that the decisions made by the system had 89% accuracy in DSP training and 82% accuracy in LR training. In addition, the developed system has been tested by 10 participants (25.8 ± 5.47 years; 74.69 ± 14.81 kg; 173.5 ± 9.52 cm) in DSP and LR training for four weeks. At the end of this period and according to the results of paired t-test analysis (p < 0.05) starting from the first week, it was observed that the participants trained more accurately and that they enhanced their motions by 58.08 ± 11.32% in LR training and 54.84 ± 12.72% in DSP training.

1. Introduction

Simultaneous monitoring of movements in the training process in sports such as bodybuilding and weightlifting is a necessity for athletes to train healthily. This allows the athlete to train effectively without the risk of injury until motor skills related to movement develop. Athletes and trainers are looking for ways to train most effectively without causing injury. This process is carried out within the framework of training programs as in all sport branches. Visual training programs designed for more accurate and safe training use visual perception and motor tasks based on it [1]. By using these programs, potential injury risks may decrease and sensory processes and motor movements may become faster and more accurate [2].

In the literature, there are studies about the optimization of sports technique and the use of more effective training mechanisms and computerized vision and human–machine interaction in sports on rehabilitation processes [3]. The methods developed for this purpose are generally divided into two groups as observational and direct measurement [4]. Data collection in the observation method is based on subjective observations or on a simple estimation of angles provided from videos or images [5]. This method does not restrict the movements of the athletes and does not disturb the athlete [6]. However, it is costly to place multiple cameras for information in suitable locations where visibility is not obstructed in gyms [7]. In the direct measurement method, data is collected through sensors attached to the body of the athlete. There are three main methods in this group: optical, mechanical, and magnetic [8,9,10].

In an optical system, the correct placement of markers is crucial. The fact that a marker is misplaced, slipped, or not seen by the system before training can lead to incorrect results in the monitoring process [11,12]. Magnetic and mechanical systems can restrict the movement of the athlete due to the need for cabling [13,14]. Mechanical systems have to be calibrated frequently because of sweating or bumping during training [15]. These methods cannot frequently be used in a real gym environment due to the time required for calibration—setup time and preparation (sensor placement, installation, cabling, etc.).

An alternative to all this is Kinect, a second generation RGB-D type sensor (2014) which is notable for its ease of setup and usage. This sensor includes an RGB camera, an infrared camera, and a microphone mechanism. Kinect uses its own proprietary algorithm to create 3D movements to enable monitoring movements without the need for any special equipment or markers [16]. Studies on the ability to monitor body movements during static and dynamic balance tests of Kinect have shown that this sensor has a wider accuracy range compared to video movements capture systems [16,17]. In comparison with systems with gold standards (≥99% accuracy), it is stated that this sensor could be used in biomedical, sports, and rehabilitation fields [18,19].

The methods used in the literature to monitor and evaluate human movement are divided into two parts: template-based and pose-based [20,21,22]. In template-based methods, a movement or movement group is recorded and analyzed sequentially to form a model [23,24]. The monitored movements are used to define the movement or activity through this model. In this approach, results are obtained depending on whether the monitored movement is defined by the template or not. The pose-based approach does not require specific examples and trained models [25,26]. Instead, a key dynamic set of rules for the movement is defined, indicating how much the performed movement with the given feedback is similar to the one that is defined [27,28]. When these methods are used in the gym environment, they give information about what the movement is, but cannot give any correction data about how the distortions in movement can be corrected.

When the studies using Kinect in the field of sports and rehabilitation were examined, it was observed that in the pose monitoring studies of Cervantes et al., movement distortion was shown only in the respective joints by comparing the results they obtained during training with the pose template they created before and finding the movement matching rate [29], that Khazaeli et al., gave feedback by manually examining the results of the movement follow-up after training [30], and that Elaoud et al., performed kinematic analyses of handball throwing poses after training [31]. Furthermore, Scherer et al., investigate the performance monitoring and user satisfaction through the similarity of movement with the pose monitoring-based system they had developed for supervised training at home [32]. However, it has been observed that all these systems were unable to give users simultaneous feedback on how to train more effectively and efficiently by correcting the wrong movement.

In their studies in the field of sports and rehabilitation for hemophilia patients, Mateo et al., analyzed and conducted performance analysis on similarity with Kinect through dynamic time warping (DTW) by examining movements affecting single and multiple joints through posing follow-up [33], Urturi et al., designed a system for wheelchair users and combined physical skills with entertainment [34], Yu et al., analyzed user performance through movement similarity by processing pose data from tai chi exercises obtained with Kinect through DTW [35] while, on the other hand, Su aimed to help knee rehabilitation with a bike racing game he developed using Kinect [36]. However, it was seen that these systems were limited only to patients in the area where the study was conducted or with similarity in movement, and that they are not suitable in terms of movement accuracy for use in sports gym environment and weight training with isotonic movements. In an another remarkable study, a recurrent self-organization structure made by Parisi et al., was examined for three different weightlifting exercises and it attempted to give simultaneous feedback to the user [37]. However, also in this study, the effect of the system on physical–cognitive development with its ability to increase training efficiency was not tested for long-term use.

In addition to these studies, there are also systems that perform posture and pose analysis by using Kinect in the treatment of Parkinson by reaching the shapes and in the treatment of balance disorder by following walking on a straight line [38,39,40]; also in these studies, however, it was shown that the abovementioned deficiencies were still present.

When all these mentioned studies are examined, the 10 main features that should be considered in an athlete support system can be listed as follows. These are:

• Simultaneously work (S);
• Determination of movement accuracy (MA);
• Giving feedback for movement correction (F);
• Skeleton extraction (SE);
• Usability in training (UT);
• Usability in rehabilitation (UR);
• Being oriented towards physical ability (PA);
• Being oriented towards cognitive ability (CA);
• To be able to make pose analysis (POA);
• The ability to perform movement model analysis (MMA).

All these features can be simultaneously used during training or rehabilitation processes; in particular, it may be useful for beginner athletes to perform healthy training in sports such as body building and weightlifting, to perform an instant performance evaluation until the physical and cognitive skills of the athlete related to movement develop, to prevent the injury process by intervening during the training when necessary, or to increase the efficiency of rehabilitation processes.

Based on these elements, a new athlete support system is proposed in our study using Kinect V2 and Expert System as shown in Figure 1.

In addition, a comparative summary of these studies and the characteristics of our study are presented in

Table 1. Comparison of the Features of Presented System with Other Systems. S, simultaneously work; MA, determination of movement accuracy; F, giving feedback for movement correction; SE, skeleton extraction; UT, usability in training; UR, usability in rehabilitation; PA, being oriented towards physical ability; CA, being oriented towards cognitive ability; POA, to be able to make pose analysis; MMA, the ability to perform movement model analysis.

Systems	S	MA	F	SE	UT	UR	PA	CA	POA	MMA
[29]	√	√	X	√	√	√	√	X	√	X
[30]	X	√	X	√	√	X	√	X	√	√
[31]	X	√	X	√	√	X	√	X	√	X
[32]	√	√	X	√	√	X	√	X	√	X
[33]	√	√	X	√	√	√	√	√	√	√
[34]	√	X	X	√	X	√	√	√	√	X
[35]	√	√	X	√	√	X	√	√	√	√
[36]	√	√	X	√	X	√	√	X	√	X
[37]	√	X	X	√	√	√	X	X	√	√
[38,39,40]	√	X	X	√	X	√	X	X	√	√
Proposed System	√	√	√	√	√	√	√	√	√	√

.

In the proposed system, the rule-based Expert System, which uses the calculated angle and distance data from Kinect data according to the developed movement model, primarily determines the accuracy of athletes’ movements and distortions in movements. It then gives the user simultaneous feedback that tells them how to correct the movement. In addition, in the simultaneous monitoring and feedback process, data such as the joint angle and distance associated with training are presented to the athlete to provide more accurate training.

The tests of the proposed system were conducted under the supervision of a certified trainer in DSP and LR training with 10 volunteer participants. The reason for choosing DSP and LR movements is that both movements are the most commonly used movements in sports training and rehabilitation processes and affect the shoulder area of the upper extremity where the most injuries are experienced [41]. In these trainings, the feedback given by the system was recorded and the decision-making adequacy of the system was examined by comparing it with the recommendations of the trainer.

In addition, participants were asked to use the system for four weeks in DSP and LR trainings. Within the period, they have benefited from the outputs and recommendations of the system in their trainings. During the process, the average displacement data of the related joints was recorded in each training performed. Thus, the change in the performance of the participants and the effect of the system on physical–cognitive development were analyzed through this data as stated in [42].

2. Materials and Methods

2.1. Isotonic Movements

Isotonic movements are movements in which muscle tendons are shortened for the purpose of creating movement. Any movement from weightlifting to swimming or squats is in this category [43]. In sports, isotonic training consists of isotonic movements where the most force is applied to a particular muscle or muscle group to increase muscle mass or performance in general [44]. Isotonic movements form the basis of many training protocols because most human activity and athletic performance require movements like these. In addition, muscle endurance can also be increased through these movements [44,45].

2.2. Data Analysis

The data analysis first uses Excel and then the Statistical Package for Social Sciences (SPSS version 25—Trial; SPSS Inc.; Chicago, IL, USA). The training efficiency and the effect of the system on athlete development were done through paired t-test, independent t-test, and effect size as stated in [46,47,48]. The paired t-test compares the averages of a variable’s values observed in two different states. These two situations usually occur through the analysis of the data obtained from the stages of a method to be applied [49]. Thus, the aim was to examine the weekly physical ability development as stated in [50]. The results obtained from paired t-test are evaluated based on the p value and the effect size (Cohen’s d for correlation between two variables) [51] as stated in [42,52]. Independent t-test and effect size were used to examine cognitive ability development by examining changes in joints as weekly pairs from the beginning to the end of the trial period as stated in [53]. Independent t-test determines whether there is a statistically significant difference between the means in two unrelated groups using p value and the test of Levene [54].

In the SPSS package program, the p value is given by the abbreviation Sig. (2-tailed) and refers to the abbreviation for relevance. The smaller the p value, the greater the evidence for statistically significant difference. It is stated that there is a significant difference if the value is between 0.01 and 0.05, a highly significant difference if it is between 0.001 and 0.01, and a very highly significant difference if it is smaller than 0.001. In our study, the significance was accepted as p < 0.05 as stated in [55]. Effect size (d) refers to the difference between the means of two events or groups [56]. It is stated that there is a weak effect size if the value is less than 0.2, a moderate effect size if it is between 0.2 and 0.5, a large effect size if it is between 0.5 and 0.8, and a strong effect size if it is greater than 0.8.

2.3. Angles between Joints and Kinect V2

In our system, we used Kinect V2, consisting of an RGB camera capable of shooting at 30 frames per second (1920 × 1080 pixels), a depth sensor (512 × 424 pixels) and a microphone. The operation of the depth sensor in Kinect works according to the 3D depth detection principle by sending IR (infrared) rays to the object in front of it and measuring the time taken for the reflected rays to return (time of flight) [57,58]. The 3D depth is calculated with the help of an algorithm belonging to Kinect running in the background and thus makes it possible to follow the skeleton of the users [59]. As shown in Figure 2, the data from here were used to express the 3D positions of the joints by vectors.

The depth of any pixel in each depth data frame obtained from Kinect can be converted to three-dimensional coordinates according to the principle of triangulation (1). In this way, the 3D coordinate of any joint is obtained with the information obtained from the sensor.

$$\begin{array}{l}X=\frac{(x_p-p_h/2)-\tan ({\theta }_h/2)}{p_h/2}\ast z_p\\ Y=\frac{(p_p-y_p)-\tan ({\theta }_v/2)}{p_v/2}\ast z_p\\ Z=z_p\end{array}$$

(1)

Equation (1) x_p refers to the horizontal coordinate of a pixel in the depth image, y_p to the vertical coordinate of a pixel in the depth image, z_p to the depth value of a pixel in a given coordinate, p_h to the total number of pixels in the horizontal direction, p_v to the total number of pixels in the vertical direction, θ_h to the horizontal angle of view of the IR camera and θ_v refers to the vertical angle of view of the IR camera. Based on Equation (1), vectors were created for the joints by finding their coordinates and the dot product called multiplication [60] was carried out. Mathematically, dot product is the process that takes two vectors as a value and returns a scalar value as the result. If we explain the example shown in Figure 2, presented in dot product (2) are the non-zero A vectors formed between the wrist–elbow and the non-zero B vectors formed between the elbow–shoulder, and the two vectors are identified as A = <A₁, A₂, A₃, …, A_n> and B = <B₁, B₂, B₃, …, B_n>.

$$A\ast B={\displaystyle \sum _{i=1}^n{A}_i\ast B_i}$$

(2)

Here, A = a₁i + a₂j + a₃k and B = b₁i + b₂j + b₃k are defined in the R³ coordinate system. With the data obtained from Equation (2), the angle Θ between these vectors is found using Equation (3).

$$\Theta =\arccos \left(\frac{A\ast B}{\left|A\right|\ast \left|B\right|}\right)$$

(3)

In the continuation, as shown in Equation (4), the distance d between the joints is calculated with the data obtained for each joint coordinate by using Equations (2) and (3).

$$d=\sqrt{{\displaystyle \sum _{i=1}^N{\left(A_i-B_i\right)}^2}}$$

(4)

2.4. Modeling of DSP and LR Movements

The methods used to process the data obtained during training are usually focused on the similarity of pose and movement. Expert poses are used to obtain the most accurate movement, but during these and also during different weights in the gym environment, there is the occurrence of either differences in movement between the experts or due to the fact that one movement which was performed cannot be performed exactly the same again [61]. In addition, an injury can occur during the performance of training exercises until the movement reaches its final form [62]. Therefore, the main thing to note when obtaining the training movements used in the evaluation is that instead of analyzing the final form of the obtained pose or movement the process from the beginning to the end of the movement can be modeled in a way that does not lead to injury.

For this purpose, movement models were designed for isotonic DSP and LR movements by consulting the opinions of faculty member of Physical Education and Sports Teaching (Karamanoğlu Mehmetbey University) and a certified trainer (Antalya Sport Center). In these models, the changes in angles of shoulder (A2 and A3), elbow (A1 and A4), wrist (A13 and A14), hip (A7 and A8), knee (A9 and A10), ankle (A11 and A12), and spine (A5) are obtained through the data of Kinect during performing the movements shown in Figure 3.

In the movement models, it is defined how the movements should be performed and warnings are given for correction when the movement is incorrect.

If the model created for LR training is explained as shown in Figure 3a,

• You should be standing upright (Equation (5)) with a dumbbell in each hand in the range of 170–180° (A5) [63]:

  $$R_1=\{\begin{array}{c}1,170\le \mathrm{A}5<180\\ 0,\mathrm{Otherwise}\end{array}$$

  (5)
• The distance between the feet (A11–A12) should be slightly narrower than shoulder width (A2–A3) and slightly more than the width of the hip (A7–A8) (Equation (6)) [63,64]:

  $$R_2=\{\begin{array}{c}1,{\mathrm{distance}}_{\mathrm{A}11-\mathrm{A}12}\le {\mathrm{distance}}_{\mathrm{A}2-\mathrm{A}3}{\mathrm{And}\mathrm{distance}}_{\mathrm{A}7-\mathrm{A}8}\le {\mathrm{distance}}_{\mathrm{A}11-\mathrm{A}12}\\ 0,\mathrm{Otherwise}\end{array}$$

  (6)
• Arms should be opened at the elbows (A1 and A4) in the range of 140–160° (Equation ((7)) [65]:

  $$R_3=\{\begin{array}{c}1,140\le \mathrm{A}1\le 160\mathrm{and}140\le \mathrm{A}4\le 160\\ 0,\mathrm{Otherwise}\end{array}$$

  (7)
• The arms should be raised up sideways until the elbows (A1 and A4) form an angle of 15–30° at shoulder level (A2 and A3) (Equation ((8)) [63,65]:

  $$R_4=\{\begin{array}{c}1,15\le {\theta }_{A1-A2}\le 30and15\le {\theta }_{A3-A4}\le 30\\ 0,\mathrm{Otherwise}\end{array}$$

  (8)
• At the last point of movement, the wrists (A13 and A14) should be high enough to create 10–30° with the elbows (A1 and A4) (Equation ((9)) [65,66]:

  $$R_5=\{\begin{array}{c}1,10\le {\theta }_{A1-A13}\le 30and10\le {\theta }_{A4-A14}\le 30\\ 0,\mathrm{Otherwise}\end{array}$$

  (9)

If the model created for DSP training is explained as shown in Figure 3b,

• You should be positioned upright when the movement is made by sitting, at a 90° angle (A5) and when the movement is made by standing, at 170–180° angle (Equation ((5)) [63];
• Elbows (A1 and A4) should have a vertical angle of 90–100° on the arms (Equation ((10)) [67,68]:

  $$R_6=\{\begin{array}{c}1,90\le \mathrm{A}1\le 100\mathrm{and}90\le \mathrm{A}4\le 100\\ 0,\mathrm{Otherwise}\end{array}$$

  (10)
• Dumbbells should lie in a vertical line, arms should lie on top of the head and be pushed almost until the dumbbells reach the point of touch (Equation (11)) [66,67]:

  $$R_7=\{\begin{array}{c}1,{\mathrm{distance}}_{\mathrm{A}13-\mathrm{A}14}\le {\mathrm{distance}}_{\mathrm{A}2-\mathrm{A}3}{\mathrm{And}\mathrm{distance}}_{\mathrm{A}7-\mathrm{A}8}\le {\mathrm{distance}}_{\mathrm{A}13-\mathrm{A}14}\\ 0,\mathrm{Otherwise}\end{array}$$

  (11)
• Elbow (A1–A4) angles should be 160–170° (Equation (12)) to maintain tension in muscles [67,69]:

  $$R_8=\{\begin{array}{c}1,160\le \mathrm{A}1\le 170\mathrm{and}160\le \mathrm{A}4\le 170\\ 0,\mathrm{Otherwise}\end{array}$$

  (12)

When the movement is performed correctly, the user gets a “Good Posture” message on the screen as positive feedback as stated in [70], and when it is done incorrectly, the user gets as feedback the message shown in

Table 2. Simultaneous Warnings Given to User.

Warning No.	Warnings
1	Please stand upright
2	Feet should be slightly narrower than the shoulder
3	The angle of the arm with the elbow is small
4	The elbows should not be below the shoulder line, please choose lighter weights
5	The wrist should not be below the elbow line
6	The elbows should be positioned at right angles
7	The elbows and shoulders are not aligned, please push up the weights or choose lighter weights
8	Raise your elbows
9	Good Posture

.

2.5. Rule-Based Expert System Design

The Expert System is a decision support software created by copying one or more individual judgement capabilities and decision-making processes encountered in specific areas [71]. The information and logical inference mechanism used by this program is modeled according to the information and logical inference mechanism of the person or persons who are experts in the field [72]. In addition, the decision structures that will be used in the modeling process must have unquestionably accuracy. Thus, users can be guided by Expert System in accordance with their needs and wishes. One of the most important parts of expert systems, the information database, is created and updated as decision rules according to the knowledge of the person or persons who are experts in the field [73]. By creating these rules, an “If–Then” structure is used. These structures are expressed as follows:

If (one or more condition = True) then (outcome/result)

Multiple conditions can be used when creating this structure. These conditions are linked using the terms “And” and “Or” depending on the situation. As an example, if two conditions are checked, both conditions are verified using the expression “And”. However, if only one of the two conditions is sufficient, the expression used is “Or”. An example algorithm of a sports area can be expressed as follows (Algorithm 1).

Algorithm 1: Example algorithm of a sports area

Input: Joint Angles J = (Elbow angle1, Elbow angle2, Shoulder angle1, Shoulder angle2, Spine angle) Output: Accuracy of Movement

If (Elbow Angle Difference ≤ 10°) and (Shoulder Angle Difference ≤ 15°) and (Spine Angle ≥ 170°)    Then      “Correct Movement”    Else      “Wrong Movement”    End If

The combined movement models developed for this purpose are shown in

Table 3. Accuracy Table for DSP and LR.

Case.	R1	R2	R3	R4	R5	R6	R7	R8	Y
0	0	0	0	0	0	0	0	0	0
1	0	0	0	0	0	0	0	1	0
2	0	0	0	0	0	0	1	0	0
…	…	…	…	…	…	…	…	…	…
7	0	0	0	0	0	1	1	1	1
…	…	…	…	…	…	…	…	…	…
135	1	0	0	0	1	1	1	1	2
…	…	…	…	…	…	…	…	…	…
248	1	1	1	1	1	0	0	0	1
…	…	…	…	…	…	…	…	…	…

with an accuracy table consisting of 8 rules (R1–R8) and 256 conditions according to the change in the angle of movement for use in Expert System in the study. In this table, Y is defined as output values, “1” as correct LR movement, “2” as correct DSP movements, and “0” as incorrect movements. This process has been realized by determining the other rules that can be used jointly according to the execution of the movement and combining the main rules of the movement.

In Table 3, to create a rule base that can be used for both movements, the cases where the rule is provided are marked with “1”, the cases where the relevant rule is not provided are marked with “0”, and the cases where the relevant rule is unimportant according to the action made are marked with “X”.

In order to provide movement correction information depending on the movement flow, the warnings given in Table 2 according to the correct movement and the incorrect movements that can be corrected are assigned as output states. Hence, a table of accuracy that can work according to the flow of movement is obtained and presented in

Table 4. Simplified Rule Base for DSP and LR.

Case.	R1	R2	R3	R4	R5	R6	R7	R8	Y
…	…	…	…	…	…	…	…	…	…
31	0	0	0	X	X	1	1	1	2
…	…	…	…	…	…	…	…	…	…
199	1	X	0	0	1	1	1	1	2
…	…	…	…	…	…	…	…	…	…
249	1	1	1	1	1	0	0	X	6
…	…	…	…	…	…	…	…	…	…

.

Shown in

Table 5. Sample Output Values of Main Conditions.

Y	Output Cases	Rules Conditions
		Q1	Q2	Q3	Q4	Q5	Q6	Q7	Q8
1	1xxx1111	1	x	x	x	1	1	1	1
2	111x1x11	1	1	1	x	1	x	1	1

are the two example rules together with the warnings (Y), exit states, and rule states (Q1–Q8) given by the system through determining the 12 main rules that can be used for both training in this table.

2.6. System Architecture

The movement data in our system during training is obtained through Kinect via a USB 3.0 connection. The data received are in the form of depth data. The system is written in C# language. The computer in which the system is run has a i7 2.4 GHz processor, 12 GB of RAM, 1 TB HDD, and a 4 GB external graphics card.

Skeletal inference is performed as shown in Figure 4a by processing with SDK 2.0, which is part of Kinect that we use in the system and can be downloaded from Microsoft’s official website.

Users do their training at a distance of 2.5 m from Kinect, which is placed on a tripod 0.8–1.2 m high from the ground, as described in [33,59]. In addition, joint angles and inter-joint distances are calculated using the data obtained from skeletal extraction (see Section 2.3). The obtained angle and distance values are sent to Expert System as input values over the running memory.

The Expert System, which we develop according to the movement model, consists of 7 sections (Figure 4b). These are expert interface, Expert System rule base, system database, running memory, inference module, knowledge base editor, and user interface.

Rules can be added to the expert system through the expert interface. In the interface shown in Figure 5a, experts can enter the angle, distances, states of accuracy, and warnings about the training system. The created rules are made available in Expert System by converting them into “If–Then” structures in the knowledge base editor before they are added to the rule base, as shown in Section 2.5.

This process is done by compiling the converted “If–Then” structures in the interface as a dynamic-link library (DLL) both before and during running [74,75]. The rules created here are classified according to the type of movement and displayed first in the rule base and then sent to the database. On the other hand, during usage, the rules are retrieved from the database as DLL and transferred to the rule base [76], then processed in the inference engine.

The inference engine processes by taking rules from the running memory and rule base on which training data of the user is arranged for use. The inference engine performs three main functions: matching, selecting, and executing. Based on the data from the inference engine, it looks for a match in the rules on the rule base. This matching is done in two ways as forward and backward chaining in Expert System [77]. In terms of ease of use in our system, the forward chaining model is preferred as used in [78]. It executes the specified operations by selecting rule or rules about matching (creating movement accuracy and movement correction data) as shown in Figure 6.

Figure 6 illustrates the work of the inference engine in a case where the rules 2–3, 4–5, and 7–8 are active. Inference engine processes rule according to their relationship to each other (availability in the DSP and/or LR movement, rule scope). This process consists of two parts. The first part is the exclusion of rules by using OR procedure in the rules used for the same movement. In this process, warning selection is made by considering which rule is more comprehensive. In the second part, AND procedure is performed in rules used for different movements and a warning is extracted. In this process, a warning selection is made for both rules by considering which rules contain more common features. By applying the same operations to the warnings, the warning selection is made, and the inference engine concludes.

The inferences obtained here are sent to the user interface. The user interface is in use for movement selection, for determining the countdown time before training, and to give the angle data related to the training and the inferences obtained from the Expert System to the user (Figure 5b). With the help of this interface, the user can choose the training and the countdown period before training. This selected information is used for the selection of training through sending of it to the inference engine. The information provided by this interface to the user are the results obtained from the training data and inference engine. After all the parts we have explained have been integrated together, users were asked to practice using the weights system, and the obtained data are presented in the Results and Discussion section.

2.7. Participants and Setup

A total of 5 men and 5 women (with at least 2 years of experience in strength training) voluntarily participated in our study. Our study was carried out under the supervision of a certified bodybuilding trainer in Antalya Sports Centre. Participants were informed about the content of our study and a signed consent form was obtained from all of them. The number, age, gender, weight, and height information of the participants are shown in

Table 6. Information about gender, age, height and weight of the participants.

Participant No.	Age	Gender	Weight (kg)	Height (cm)
1	21	Male	80	163
2	25	Male	82.3	178
3	29	Male	87	180
4	33	Male	85	177
5	37	Male	104.6	193
6	27	Female	70	180
7	22	Female	62	160
8	24	Female	60	172
9	19	Female	58	167
10	21	Female	58	165

.

3. Results and Discussion

3.1. Kinect V2 Data Obtained from the Trainings

An example of the data obtained by using the system is shown through the marking of each repetition over the results of participants 3 and 6. Figure 7a shows the right and left shoulder angles as well as differences in these angles and spine angles of participant number 3 in LR training when using a 12.5 kg dumbbell.

When the figure is examined, distortions are observed at the waist angle during the movement in Rep2, Rep3, and Rep5. When we consulted our faculty member about this situation, it was seen that the support needed for movement was taken from the back which, as mentioned in [79], may cause neck and spine diseases in later stages by causing excessive cervical spine bending. Furthermore, the shoulder angles are within the acceptable distance in all repetitions (see Section 2.5), the difference in elbow angles is that the left elbow is not sufficiently open which, as stated, has been observed to cause damage and injury to the joints by distorting the shape of the movement due to the high weight used [80].

Figure 7b shows the data of participant number 6 obtained during DSP training using a 25 kg dumbbell. When the figure is examined, it is seen that in the process of lifting the weight up in Rep1 and Rep4, by taking the necessary support from the back, there is a distortion of the waist angle and the movement has been performed incorrectly, as indicated in [81]. In Rep2, Rep3, and Rep8, it is observed that the angle of movement is within acceptable limits (see Section 2.5) and that the movement is performed correctly. In this case, the system gives the user the message “Good Posture”. It is observed that in Rep1, Rep4, Rep5, Rep6, Rep7, Rep8, Rep9, and Rep10, the right elbow angle is less than the left during weight reduction, and when we consult our faculty member, it is seen that this situation can cause stress and injuries in the rotator cuff muscle group by disrupting the balance of movement as stated in [82,83].

The same situation was seen in the difference between elbow angles in Rep7, Rep 9, and Rep10 during weightlifting. These data, obtained from the system and related to the movement of the user, can also be followed by the user via a monitor. Some of the data obtained from the system belonging to other users are presented in Figure 8.

3.2. Decisions of Rule-Based Expert System

As mentioned in the Introduction, the accuracy of the data obtained in the systems examined in other studies was based on either movement similarity or similarity of pose. However, in our study [84,85,86], since differences can occur even at different weights in isotonic movements made with weights in the gym environment, the accuracy of the decisions made by the system was compared with the decisions of the certified coach. For this purpose, the participants were asked to perform DSP and LR exercises for one set of 10 repetitions (50%–90% load, one set of 10 reps), as in [87]. The decisions of the system and of the trainer were made by recording and comparing a total of 200 decisions in both training sessions. In this comparison, the coach was asked to make his decisions as good, moderate, and bad, as stated in [37]. All decisions obtained in this comparison are presented in

Table 7. Comparison of the System and Trainer Decisions.

	Part. No.	Decision Type	Reps
			1	2	3	4	5	6	7	8	9	10
DSP	1	System	√	√	X	X	√	X	√	√	√	√
		Sys. Warnings	9	9	3	3	9	4	9	9	9	9
		Trainer	O	√	X	X	√	X	√	√	O	O
	2	System	√	√	√	√	√	X	X	X	√	√
		Sys. Warnings	9	9	9	9	9	6	8	1	9	9
		Trainer	√	√	√	√	√	O	X	X	O	√
	3	System	√	√	√	X	X	√	X	X	√	X
		Sys. Warnings	9	9	9	7	2	9	1	1	9	1
		Trainer	√	√	O	√	X	√	X	√	O	X
	4	System	X	X	√	√	X	√	√	X	√	X
		Sys. Warnings	6	4	9	9	3	9	9	3	9	1
		Trainer	X	√	O	√	O	O	√	X	O	X
	5	System	√	√	√	√	√	√	X	X	X	X
		Sys. Warnings	9	9	9	9	9	9	1	8	4	1
		Trainer	√	√	√	√	√	√	X	X	O	X
	6	System	X	√	√	X	√	X	√	X	X	√
		Sys. Warnings	6	9	9	6	9	8	9	4	3	7
		Trainer	X	O	√	X	√	X	√	√	√	O
	7	System	√	√	X	√	√	X	√	√	√	X
		Sys. Warnings	9	9	1	9	9	3	9	9	9	8
		Trainer	√	√	X	√	√	X	O	O	√	O
	8	System	X	X	√	√	X	X	√	√	X	X
		Sys. Warnings	1	1	9	9	3	3	9	9	3	1
		Trainer	X	X	O	O	X	√	√	√	X	X
	9	System	X	X	√	√	√	√	X	√	√	√
		Sys. Warnings	8	8	9	9	9	9	4	9	9	9
		Trainer	X	√	√	√	√	√	√	√	√	√
	10	System	√	√	√	√	√	√	√	√	√	X
		Sys. Warnings	9	9	9	9	9	9	9	9	9	1
		Trainer	√	√	√	√	√	√	√	√	√	X
LR	1	System	√	√	X	√	√	√	X	X	√	√
		Sys. Warnings	9	9	1	9	9	9	8	3	9	9
		Trainer	√	√	√	√	√	√	X	√	√	√
	2	System	√	√	√	√	X	X	√	√	√	√
		Sys. Warnings	9	9	9	9	1	5	9	9	9	9
		Trainer	√	√	O	O	X	X	√	√	√	√
	3	System	X	X	√	X	X	X	√	√	√	X
		Sys. Warnings	5	5	9	3	7	7	9	9	9	3
		Trainer	√	√	√	√	X	X	X	√	√	√
	4	System	√	√	X	X	√	√	X	X	X	X
		Sys. Warnings	9	9	1	3	9	9	7	6	1	5
		Trainer	√	√	X	X	√	√	O	X	X	X
	5	System	√	√	√	√	√	√	√	√	X	X
		Sys. Warnings	9	9	9	9	9	9	9	9	1	1
		Trainer	X	X	√	√	√	√	O	X	X	O
	6	System	√	√	X	√	√	X	X	√	X	√
		Sys. Warnings	9	9	5	9	9	1	1	9	7	9
		Trainer	√	√	X	√	√	X	X	O	√	√
	7	System	X	X	X	√	√	√	X	√	√	√
		Sys. Warnings	6	1	1	9	9	9	5	9	9	9
		Trainer	X	X	O	√	√	O	X	√	√	√
	8	System	√	√	X	X	√	√	√	√	√	X
		Sys. Warnings	9	9	6	7	9	9	9	9	9	5
		Trainer	√	√	√	X	√	√	√	√	O	X
	9	System	√	√	√	X	X	O	X	√	√	√
		Sys. Warnings	9	9	9	5	1	9	1	9	9	9
		Trainer	√	√	√	√	√	√	X	√	√	√
	10	System	√	X	X	√	√	X	X	√	√	X
		Sys. Warnings	9	5	5	9	9	1	7	9	9	5
		Trainer	√	X	√	√	O	O	X	√	√	X

X: bad movement; √: good movement; O: moderate movement. For system warnings, see Table 3.

on the basis of both Expert System (with system warnings) and trainer.

In order to facilitate the procedures, good and moderate movements of the participants in the trainer evaluations were accepted as the correct movements as in [88,89]. In Table 7, the decisions taken by the system are matched by the decisions of the trainer by 89% in DSP training, 82% in LR training, and 85.5% in total. In addition, the results obtained from the system for LR training have the same achievements as the results obtained in [33]. This suggests that the system is suitable for use in the home-gym environment, as stated for the sports field [90] and in the rehabilitation area [91].

3.3. The Effect of the System on Physical and Cognitive Development

In order to examine the effect of the system on physical and cognitive development, participants were asked to perform LR and DSP trainings (4 sets, 8–12 reps, and 70%–90% load) using the system for four weeks. Glenohumeral joints (right and left shoulder) and spine displacement values of the participants were recorded via the system during the trainings. The recorded data are shown in Figure 9 as the amount of position change in the joints (cm) by type of training, participant number, and joint type. The data obtained in Figure 9 are presented in Appendix A as

Table A1. Weekly Joint Displacement of Participants.

Training Type	Part. No.	Joint Names and Weeks
		GJLweek1	GJLweek2	GJLweek3	GJLweek4	GJRweek1	GJRweek2	GJRweek3	GJRweek4	Spweek1	Spweek2	Spweek3	Spweek4
DSP	1	6.92	5.84	4.93	3.65	6.8	6.01	4.24	3.52	4.25	3.92	1.95	1.1
	2	9.56	9.01	6.58	5.23	10.25	8.15	7.56	6.48	6.08	5.24	5.14	4.95
	3	10.02	8.11	5.66	4.66	11.88	7.94	5.22	4.37	12.58	10.33	6.62	5.87
	4	3.25	3.2	1.86	1.1	2.81	2.22	1.35	1.27	4.11	3.27	2.55	2
	5	3.78	3.61	1.66	0.8	2.94	2.85	2.01	1.6	3.31	3.08	2.82	1.2
	6	3.55	2.48	2.14	1.2	3.68	2.74	1.93	0.9	2.92	1.8	1.2	0.8
	7	3.62	2.93	2.11	1.7	3.26	2.88	2.2	1.57	3.11	2.52	2.35	1.3
	8	3.76	3.15	2.56	1.9	3.82	3.15	2.17	1.37	2.85	2.31	2.1	1.15
	9	3.13	2.62	2.4	1.66	3.27	2.84	2.35	2.04	2.77	2.28	1.65	1.27
	10	6.21	4.64	3.96	2.54	5.63	4.87	3.58	2.71	4.66	4.01	3.42	2.61
LR	1	6.25	4.35	3.48	3.12	5.21	4.66	3.32	2.96	15.22	11.64	8.77	6.4
	2	7.13	6.27	5.4	4.41	7.02	5.74	4.31	3.96	10.1	7.72	7.26	6.21
	3	5.11	4.24	2.88	2.02	5.42	4.25	2.88	1.97	8.6	5.53	5.21	3.3
	4	8.91	8.14	5.22	3.1	9.82	6.76	4.88	3.24	12.47	10.66	8.99	5.11
	5	4.9	4.77	2.95	2.64	4.92	3.11	2.56	2.57	6.24	5.01	3.58	2.99
	6	7.44	5.28	2.9	1.8	8.12	5.12	3.2	1.6	6.88	3.24	1.55	1.3
	7	6.95	6.08	4.57	2.23	6.82	4.91	2.85	3.12	5.56	4.51	2.64	1.58
	8	4.75	3.84	2.98	1.79	4.68	3.94	2.69	2.18	4.21	3.26	2.49	1.77
	9	5.29	4.83	3.29	2.15	5.35	4.52	3.41	2.23	4.86	4.19	2.86	2.07
	10	6.2	5.47	4.37	3.53	6.3	5.13	3.68	2.64	6.87	4.62	3.01	2.27

DSP: dumbbell shoulder press, LR: lateral raise, GL: glenohumeral left joint, GR: glenohumeral right joint, S: spine, GL1: glenohumeral left joint week 1, GR1: glenohumeral right joint week 1, S1: spine week 1, GR2: glenohumeral right joint week 2, S2: spine week 2, GL2: glenohumeral left joint week 2, GR3: glenohumeral right joint week 3, S3: spine week 3, GL3: glenohumeral left joint week 3, GR4: glenohumeral right joint week 4, S4: spine week 4, GL4: glenohumeral left joint week 4.

.

When Figure 9—which is showing the average weekly displacement of glenohumeral joints and spine affected by DSP and LR training—was examined, it was observed that users started to do their movements using only the related muscles without using momentum, and minimized the risk of injury in this process as stated in [85]. Furthermore, in these data, an average enhancement of 54.84% ± 12.72% in DSP training and 58.08% ± 11.32% in LR training is observed. This indicates that each participant has performed their training more stably at the end of the system usage process, as stated in [92]. A detailed analysis of these recovery rates is shown in

Table 8. Improvement Rates Achieved at the End of System Usage Process. GJL% means glenohumeral left joint % improvement; GJR% means glenohumeral right joint % improvement; S% means spine % improvement.

			Participant No.
			1	2	3	4	5	6	7	8	9	10
Improvement According to Training Type	DSP	GJL%	47.25	45.29	53.49	66.15	78.84	66.20	53.04	49.47	46.96	59.10
		GJR%	48.24	36.78	63.22	54.80	45.58	75.54	51.84	64.14	37.61	51.87
		S%	74.12	18.59	53.34	51.34	63.75	72.60	58.20	59.65	54.15	43.99
	LR	GJL%	50.08	38.15	60.47	65.21	46.12	75.81	67.91	62.32	59.36	43.06
		GJR%	43.19	43.59	63.65	67.01	47.76	80.30	54.25	53.42	58.32	58.10
		S%	57.95	38.51	61.63	59.02	52.08	81.10	71.58	57.96	57.41	66.96

.

The data shown in Figure 9 were analyzed using SPSS via the paired t-test according to the significance criterion p < 0.05, and the results are presented in

Table 9. Paired t-test Results (CRITERIA = CI (0.9500)) of the Four-Week Training Period. DSP: dumbbell shoulder press, LR: lateral raise, GL: glenohumeral left joint, GR: glenohumeral right joint, S: spine, GL1: glenohumeral left joint week 1, GR1: glenohumeral right joint week 1, S1: spine week 1, GR2: glenohumeral right joint week 2, S2: spine week 2, GL2: glenohumeral left joint week 2, GR3: glenohumeral right joint week 3, S3: spine week 3, GL3: glenohumeral left joint week 3, GR4: glenohumeral right joint week 4, S4: spine week 4, GL4: glenohumeral left joint week 4.

Pairs No.	Pair Groups	Paired Differences					t	df	Sig, (2-Tailed)	Effect Size (d)
		Mean	Std. Dev	Std. Err. Mean	95% Confidence Interval of the Difference
					Lower	Upper
Pair 1	DSP_GJL1–DSP_GJL2	0.82100	0.58936	0.18637	0.39940	1.24260	4.405	9	0.002	0.328
Pair 2	DSP_GJL1–DSP_GJL3	1.99400	1.04455	0.33032	1.24677	2.74123	6.037	9	0.000	0.884
Pair 3	DSP_GJL1–DSP_GJL4	2.93600	1.23701	0.39118	2.05110	3.82090	7.506	9	0.000	1.351
Pair 4	DSP_GJR1–DSP_GJR2	1.06900	1.14143	0.36095	0.25247	1.88553	2.962	9	0.016	0.383
Pair 5	DSP_GJR1–DSP_GJR3	2.17300	1.69575	0.53624	0.95994	3.38606	4.052	9	0.003	0.814
Pair 6	DSP_GJR1–DSP_GJR4	2.85100	1.85241	0.58578	1.52586	4.17614	4.867	9	0.001	1.093
Pair 7	DSP_S1–DSP_S2	0.78800	0.57569	0.18205	0.37618	1.19982	4.329	9	0.002	0.288
Pair 8	DSP_S1–DSP_S3	1.68400	1.59516	0.50443	0.54289	2.82511	3.338	9	0.009	0.698
Pair 9	DSP_S1–DSP_S4	2.43900	1.59002	0.50281	1.30157	3.57643	4.851	9	0.001	0.998
Pair 10	LR_GJL1–LR_GJL2	0.96600	0.61281	0.19379	0.52762	1.40438	4.985	9	0.001	0.744
Pair 11	LR_GJL1–LR_GJL3	2.48900	0.93475	0.29560	1.82032	3.15768	8.420	9	0.000	2.114
Pair 12	LR_GJL1–LR_GJL4	3.61400	1.28388	0.40600	2.69556	4.53244	8.901	9	0.000	3.225
Pair 13	LR_GJR1–LR_GJR2	1.55200	0.88919	0.28119	0.91591	2.18809	5.519	9	0.000	1.150
Pair 14	LR_GJR1–LR_GJR3	2.98800	1.18392	0.37439	2.14108	3.83492	7.981	9	0.000	2.364
Pair 15	LR_GJR1–LR_GJR4	3.71900	1.57827	0.49909	2.58998	4.84802	7.452	9	0.000	2.972
Pair 16	LR_S1–LR_S2	2.06300	1.10056	0.34803	1.27571	2.85029	5.928	9	0.000	0.630
Pair 17	LR_S1–LR_S3	1.40200	0.74800	0.23654	0.86691	1.93709	5.927	9	0.000	0.442
Pair 18	LR_S1–LR_S4	4.80100	2.03101	0.64226	3.34810	6.25390	7.475	9	0.000	1.686

. In this table, the results of the paired t-test analysis of the weekly joint displacement are shown in pairs (1–2, 1–3, 1–4) according to the training of the participants, as available. In addition, d values are added to the last column of the table.

The results of the analysis indicate a significant change in physical ability (p < 0.005) in DSP (Pair 1–9) and LR (Pair 10–18) training using the system from the first week. This change continues throughout the process. When the results were examined in terms of effect size, weak and medium effects were observed in the pair 1, pair 4, and pair 7 data in the comparison of the first and second weeks of DSP training, whereas large and strong effects were observed in the other pairs starting from the second week. In LR training, only medium and large effects were observed in pair 10, pair 16, and pair 17, while the remaining pairs had strong effects. These results support a significant change in physical ability according to the p value.

To examine the cognitive development during the training process, an independent t-test was applied to the left glenohumeral joint, right glenohumeral joint, and spine data of the 1st and 4th weeks. The results obtained from this analysis are shown in

Table 10. Paired t-Test Results (CRITERIA = CI (0.9500)) of the Four-Week Training Period. DSP: dumbbell shoulder press, LR: lateral raise, S: spine, GJL: glenohumeral left joint, GJR: glenohumeral right joint, week 1–4: joint data of week 1 and week 4.

Independent Samples Test
		Levene’s Test for Equality of Variances		t-Test for Equality of Means
		F	Sig.	t	df	Sig. (2-Tailed)	Mean Diff.	Std. Err. Diff.	95% Conf. Interval of the Difference
									Lower	Upper	Effect Size
DSP GJL Week 1–4	Eq. var. assum.	4.531	0.047	3.022	18	0.007	2.93600	0.97154	0.89488	4.97712	0.95564
	Eq. var. not as.			3.022	14.514	0.009	2.93600	0.97154	0.85916	5.01284	0.95564
DSP GJR Week 1–4	Eq. var. assum.	3.458	0.079	2.444	18	0.025	2.85100	1.16635	0.40058	5.30142	0.772861
	Eq. var. not as.			2.444	13.827	0.029	2.85100	1.16635	0.34649	5.35551	0.772861
DSP S Week 1–4	Eq. var. assum.	0.439	0.516	2.232	18	0.039	2.43900	1.09288	0.14295	4.73505	0.70582
	Eq. var. not as.			2.232	14.679	0.042	2.43900	1.09288	0.10513	4.77287	0.70582
LR GJL Week 1–4	Eq. var. assum.	1.741	0.204	7.211	18	0.000	3.61400	0.50118	2.56106	4.66694	2.280318
	Eq. var. not as.			7.211	15.317	0.000	3.61400	0.50118	2.54768	4.68032	2.280318
LR GJR Week 1–4	Eq. var. assum.	5.064	0.037	6.645	18	0.000	3.71900	0.55968	2.54316	4.89484	2.101334
	Eq. var. not as.			6.645	12.178	0.000	3.71900	0.55968	2.50154	4.93646	2.101334
LR S Week 1–4	Eq. var. assum.	3.168	0.092	3.771	18	0.001	4.80100	1.27326	2.12598	7.47602	1.192495
	Eq. var. not as.			3.771	13.899	0.002	4.80100	1.27326	2.06826	7.53374	1.192495

Eq. var. assum.: Equal variances assumed; Eq. var. not as.: Equal variances not assumed.

with d values.

Table 10 is examined primarily according to the Sig. value (Levene’s test). If this value is less than 0.05, the variances are not homogeneous. In this case, the equal variances not assumed line is used. However, if the Sig. value is greater than 0.05, it is decided that the variances are homogeneous. The p and d values (Sig. (2-tailed)) used in these cases are shown in the table as underlined.

When these values were analyzed for DSP movement, p values of the glenohumeral left joint, spine, and glenohumeral right joint indicate a significant change in cognitive abilities before and after the process (p < 0.05). The same situation is observed in the p values of LR movement, indicating a significant change in cognitive abilities before and after the process. Moreover, these results are supported by the d values shown in pair 3, pair 6, pair 9, pair 12, pair 15, and pair 18 of Table 9 as in [42,56].

These situations in Table 9 and Table 10 are an indication that in addition to making the movements more stable, the use of the system makes training more accurate, so physical and cognitive abilities related to the movements are increased as stated in [93,94,95].

4. Conclusions

In this study, a new athlete support system designed by using Kinect V2 and Expert System was proposed to monitor and improve athlete training. In the design process of the rule base used by the system, movement models for DSP and LR have been created by consulting the opinions of experts. The Expert System rule table design was made according to the created movement models. On the designed rule base, a 2⁸-element rule table for DSP and LR movements was examined, and the main conditions that can be used in both movements were determined. During the study, Expert System makes decisions by processing the change of elbow, wrist, knee, hip, and shoulder joint data according to the rule base it contains.

The test of the system was carried out by 10 participants in a real gym environment. The first of the obtained data concern the decision-making ability of Expert System. The fact that the decisions of the system match those of the human expert in 85.5% of cases shows that the system can be used in gyms which lack trainers, by people who want to do sports at home, or by athletes working alone. However, if the system is to be used in movements other than DSP or LR, then a movement model for those movements and a rule design for the Expert System depending on this model are required.

A four-week planned study was conducted to assess the effect of the proposed system on the training process. The paired t-test and effect size analysis results of this study indicate a significant change in physical abilities (p < 0.05 and d > 0.5) from the first week according to joint changes in training, and shows that this change continues throughout the process.

The independent t-test analysis results of the glenohumeral joints and spine data of the 1st and 4th weeks indicates a significant change in cognitive abilities (p < 0.05) between the beginning and the end of training period, and these results are supported by the physical ability results.

In training efficiency, it is observed that there is an average enhancement of 54.84% ± 12.72% in DSP training and 58.08% ± 11.32% in LR training. This indicates that by using proposed system, each participant has performed their training more stably.

These results suggest that the designed athlete support system can be used in gyms, homes, and physiotherapy centers as a low-cost training support system to enhance the quality of training and reduce the risk of injury until the motor skills of new athletes develop.

As previously explained, the effect of the proposed system on physical and cognitive development was tested with data from 10 volunteers with at least two years of sports experience. The statistical limitations of our study can be seen as a limited number of participants and the resulting effect size values. It is possible to overcome these limitations with a study that will examine the physical and cognitive effects of the proposed system through more participants using individuals with different age, gender, sports expertise, sports experience, or degree of disability. Based on this study, a separate study is planned to examine these limitations and issues.

5. Data Availability

The data used to support the findings of this study are included within the article.