Web Applications for Teaching the Respiratory System: Content Validation

Abstract

The subject of respiratory mechanics has complex characteristics, functions, and interactions that can be difficult to understand in training and medical education contexts. As such, education strategies based on computational simulations comprise useful tools, but their application in the medical area requires stricter validation processes. This paper shows a statistical and a Delphi validation for two modules of a web application used for respiratory system learning: (I) “Anatomy and Physiology” and (II) “Work of Breathing Indexes”. For statistical validation, population and individual analyses were made using a database of healthy men to compare experimental and model-predicted data. For both modules, the predicted values followed the trend marked by the experimental data in the population analysis, while in the individual analysis, the predicted errors were 9.54% and 25.38% for maximal tidal volume and airflow, respectively, and 6.55%, 9.33%, and 11.77% for rapid shallow breathing index, work of breathing, and maximal inspiratory pressure, respectively. For the Delphi validation, an average higher than 4 was obtained after health professionals evaluated both modules from 1 to 5. In conclusion, both modules are good tools for respiratory system learning processes. The studied parameters behaved consistently with the expressions that describe ventilatory dynamics and were correlated with experimental data; furthermore, they had great acceptance by specialists.

1. Introduction

The respiratory system can be described in terms of mechanical properties, including airway resistance (R_aw), elastic recoil forces of the lungs, and chest wall compliance (C_L and C_cw, respectively). During ventilation, respiratory muscles contract, stretching the chest wall’s elastic tissues and the lungs to displace inelastic tissues and move air towards the airways [1]. Changes in these mechanical properties can make breathing difficult. Therefore, diseases that either reduce the compliance or increase the resistance of the respiratory system, or both, such as restrictive diseases (e.g., pulmonary edema and pneumonia) and obstructive diseases (e.g., asthma and chronic bronchitis), increase the work of breathing in response to changes in mechanical properties and increments of metabolic requirements [2].

Mechanical ventilation (MV) is usually used when respiratory dysfunction produces abnormalities in gas exchange or when the work of breathing is increased [3]. During MV, the effort exerted by respiratory muscles is determined by the patient’s ventilatory mechanics and mechanical ventilator settings, which, in spontaneous ventilation, is mainly defined by parameters such as positive pressure at the end of expiration (PEEP) and pressure support (PS). A proper configuration of the mechanical ventilator allows for decreasing the work of breathing and does not generate an unnecessary burden on the patient [4].

Some ventilation indexes have been proposed to assess the effect of MV. Three of them are widely used, owing to their ease of application in daily clinical practice: the work of breathing (WOB) index, which describes the energy required to achieve ventilation [5]; rapid shallow breathing index (RSBI), proposed by Yang and Tobin in [6] to quantify the degree of shallow breathing, defined as the ratio of respiratory frequency (RR) to tidal volume (V_T); and maximal inspiratory pressure (PI_max), described as a measure of the strength of inspiratory muscles, primarily the diaphragm, which allows for the assessment of ventilatory failure, restrictive lung disease, and respiratory muscle strength [7].

Maintaining airflow within the pulmonary airways requires a pressure gradient that falls along the direction of flow, the magnitude of which is determined by the flow rate, the frictional resistance to flow, and the elastic recoil of the lungs and the chest wall; the respiratory muscles must overcome those loads for ventilation [8,9]. In fact, like any other mechanism involving fluid dynamics, the respiratory system responds according to thermodynamics and fluid mechanics laws [10], including the equation of air motion. This expression relates the pressure in the system to the different values of volume and airflow as a function of time and the system’s mechanical characteristics (elastance and resistance), so all the respiratory mechanics can be described [11].

Due to the relationship between the dynamics of the respiratory system and classical physics, understanding respiratory behavior may be challenging for some regular students. Moreover, there are medical, scientific, and educational interests to promote medical students’ training in the interaction of mechanical properties in the ventilatory process to determine an accurate and effective treatment with timely diagnostics for some respiratory pathologies.

Some studies [12,13] highlight the advantages of implementing communication and information technologies in the teaching–learning process. For medical training, students approve in silico didactic strategies that facilitate their learning through intuitive and interactive tools.

Simulators and web apps based on mathematical models are the most common strategy currently used by medical schools, so a validation must be made to certify the data output’s accuracy. Sargent [14] defines a model as valid for a set of experimental conditions if the model’s accuracy is within its acceptable range of accuracy, which is the required model accuracy for its intended purpose. According to Hillston [15], there are three approaches to model validation, and any combination of them may be applied according to the characteristics of the system: expert intuition, or how carefully the model inspects the output and its behavior; real system measurements, where a comparison with a real system can be made; and theoretical analysis, to prove if operational laws coincide with the model’s outputs.

However, many of the models and simulators that have been developed for the study of the respiratory system [16,17,18] do not expose a validation that demonstrates the accuracy and precision of the involved systems, hampering their credibility.

This article presents the validation of the modules “Anatomy and Physiology” and “Work of Breathing Indexes” from a web application for teaching the respiratory system (https://healthsimlab.com/, accessed on 19 April 2020). The validation has been carried out based on information collected in databases (DB) of healthy subjects under changes in their respiratory mechanics properties. According to experts, the application has also been validated on a systematic examination of conceptual abstractions by observation and measurement of responses. In addition to providing valuable information about the respiratory system, the modules promote students’ self learning and facilitates their approach to abstract topics.

2. Materials and Methods

2.1. Application and Modules’ Description

The application developed is structured by modules and sections based on mathematical models covering a specific topic. From them, students will be able to make predictions of ventilatory waveforms such as airflow, volume, and pressure signals, and respiratory effort indexes such as WOB, RSBI, and PI_max. At the same time, they can make changes to the controls, according to the model’s parameters, as shown in Figure 1.

2.1.1. Anatomy and Physiology Module

The “Anatomy and Physiology” module has been developed to evaluate the respiratory response to changes in a patient’s mechanics and breathing pattern. It comprises a ventilation section (see Figure 2) that, in turn, is divided into four sub-sections: (I) equation of motion, in which the respiratory system equation of air motion is presented, with a detailed description of each of the terms that compose it; (II) anatomy, which shows graphs referring to ventilatory mechanics, respiratory muscles, and the alveolar ventilatory model—each representation contains points with explanatory information; (III) ventilatory signals, which displays airflow, tidal volume, pleural and muscular pressure waveforms, and pressure–volume and airflow–volume loops in a breath cycle; and (IV) controls, which are the components that allow for modifcations to the variables related to a patient’s effort and mechanical properties, e.g., muscle effort pressure (P_musc), respiratory rate (RR), lung compliance (C), and airway resistance (R). From these controls, users can interact with the behavior of the ventilatory curves after making changes to every one of them, according to their relationship, through the equation of air motion (Equation (1)):

$$P_{musc}\left(t\right)+P_{aw}\left(t\right)=R\dot{V}\left(t\right)+C^{-1}{V}_T\left(t\right),$$

(1)

where $\dot{V}$ is the inspired airflow, and V_T represents the tidal volume. In this case, restrictive and obstructive pathologies can be simulated by changing the R and C values, so students can interactively learn about ventilatory behavior in these situations.

2.1.2. Work of Breathing Indexes Module

The “Work of Breathing Indexes” module has been developed to evaluate the patient’s WOB under spontaneous MV with pressure support (pressure support ventilation, PSV). The main section, called the work of breathing assessment, is composed of four sub-sections: (I) assessment of dyspnea, in which a description of dyspnea and the mechanisms used to determine it are presented; (II) a Campbell diagram, which allows for visualizations of the volume–pressure loop; (III) ventilatory indexes, which display the RSBI, WOB, and PI_max index values; and (IV) controls, to modify the variables related to a patient’s effort, mechanical properties, and MV parameters (see Figure 3). This module aims to predict the level of muscular effort of a simulated patient under MV through the controls arranged in the interface. Likewise, the module allows for visualizations of when the subject is exposed to high respiratory work by marking thresholds, according to the literature [4,6,19], so the student can come to a conclusion in this regard.

2.2. Validation Process

A comparative study was performed to evaluate the model prediction capability between experimental (information obtained from subjects under different conditions) and predicted (information obtained from models’ simulators) data, following both a population and an individual approach. In the first case, we evaluated the adjustment between the experimental and simulated trend data using a set of fixed input values. In the latter case, we estimated the similarity between the experimental and simulated data using the mechanic’s pattern from each subject as the input values. The information was compared in both cases using linear trends and box plots. Figure 4 presents a concise diagram of the input parameters and the output values from the models, and the variables used to compare with the experimental data.

2.2.1. Anatomy and Physiology Module

Description of the Database

A proprietary database of 41 healthy male subjects (age 26.49 ± 5.21, weight 73.87 ± 9.62 kg, size 1.73 ± 0.05 m, and BMI 28.89 ± 2.96 kg/m${}^2$) was used to quantify the performance and the scope of the models involved. Subjects underwent an obstructive maneuver, wherein three levels of obstruction were applied to obtain, as a result, three modifications in R and, with that, changes in the ventilatory signals (see Equation (1)).

Table 1. Median and interquartile ranges of ventilatory mechanics data for obstructive DB.

Ventilatory Mechanics	Values
R (cmH${}_2$O/L/s)	13.86 (9.06–21.33)
C (mL/cmH${}_2$O)	73.26 (57.62–88.33)
Pmusc (cmH${}_2$O)	15.78 (12.45–21.81)
RR (bpm)	13.58 (10.50–17.38)

summarizes the obtained ventilatory mechanics. The sensor types and resistors used for the registration of the DB, the inclusion criteria, the ethical approval, and other acquisition aspects are detailed in the work carried out by the GIBIC research group in [20].

Analysis

Inspiratory airflow amplitude ($\dot{V}$) and maximum tidal volume (V_Tmax) variables were selected for the analysis because they allow for knowledge of both the airflow limitations and the lung capacity in a normal respiratory cycle without forced inspiration maneuvers.

For the population analysis, changes in $\dot{V}$ and V_Tmax were assessed after variations of R or C (see Figure 4). For that, fixed values of P_musc, RR, and C or R (depending on the case) were defined. Values for R and C were fixed as equal to the first (Q1), second (Q2), and third (Q3) experimental quartiles (see values in Table 1), and labeled as Test A, Test B, and Test C, respectively, to represent the data variability. Muscle effort values, such as P_musc and RR, were established considering the normal values for an adult in 8 cmH${}_2$O and 12 bpm, respectively [21,22], to limit the analysis to changes in the ventilatory mechanics parameters (R and C).

The models’ prediction capability was evaluated by comparing the linear trends between the experimental and predicted values of the assessed variables, $\dot{V}$ and V_Tmax.

On the other hand, the individual analysis assesses the model performance on a subject-to-subject basis. For that, the prediction error between the experimental and model-predicted data for both $\dot{V}$ and V_Tmax was evaluated using Equation (2). For experimental data, $\dot{V}$ and V_Tmax were measured in equal and stationary epochs and, for model simulation, the following inputs were considered: P_musc, RR, R, and C, specific to each subject.

$$\%E=\frac{100|AppValue-RealValue|}{RealValue}$$

(2)

2.2.2. Work of Breathing Indexes Module

Description of the Database

A proprietary database comprising 35 healthy men (age 26.60 ± 5.40, weight 73.12 ± 9.15 kg, size 1.72 ± 0.05 m, and BMI 24.58 ± 2.76 kg/m${}^2$) was used to validate the models involved in the “Work of Breathing Indexes” module. Volunteers were involved in two trials of PEEP and PS variations during non-invasive spontaneous ventilation. In Test D, the PEEP was changed from 0 to 10 cmH${}_2$O, with increments of 2 cmH${}_2$O every 3 minutes, while the PS was kept at 0 cmH${}_2$O. In Test E, the PS was varied from 0 to 10 cmH${}_2$O with increments of 2 cmH${}_2$O every 3 min, while the PEEP was preserved at 0 cmH${}_2$O. Each trial consisted of 210 records, 6 per subject.

Table 2. The median and interquartile range of the MV parameters and ventilatory mechanics of the subjects in Test D and E in DB PS/PEEP.

Parameter	Test D	Test E
PEEP (cmH${}_2$O)	0, 2, 4, 6, 8, 10	0
PS (cmH${}_2$O)	0	0, 2, 4, 6, 8, 10
I:E	1 : 0.83 (0.73–0.92)	1 : 0.83 (0.75–0.93)
R (cmH${}_2$O/L/s)	9.67 (7.62–12.57)	12.27 (9.11–15.01)
C (mL/cmH${}_2$O)	99.68 (80.00–139.42)	74.99 (61.20–89.97)
Pmusc (cmH${}_2$O)	12.32 (9.07–16.12)	11.58 (8.65–15.48)
RR (bpm)	15.28 (12.17–17.40)	15.98 (13.26–19.24)

summarizes the ventilatory mechanics and MV parameters of both tests. The registration protocol, inclusion criteria, and ethical approval are detailed in the work carried out by the GIBIC research group in [4].

Analysis

For both the population analysis and the individual analysis (see Figure 4), the WOB, RSBI, and PI_max indexes were compared. They were calculated from information extracted from the DBs, following the expressions in Equations (3)–(5), and averaging their values for at least five respiratory cycles.

The WOB was calculated as the area under the Campbell diagram curve, which relates the pressure exerted through the respiratory system with the resulting changes in volume [23]. For this, Equation (3) was used for a respiratory cycle:

$$WOB=\frac{1}{V_T}{\int }_{t_0}^t\mathrm{P}\mathrm{musc}\left(t\right)*\dot{V}\left(t\right)dt.$$

(3)

The RSBI was calculated by Equation (4), according to Yang and Tobin’s proposal [6]:

$$RSBI=\frac{RR}{V_T}.$$

(4)

The PI_max was calculated according to the American Thoracic Society/European Respiratory Society, ATS/ERS [24], as the maximum pressure that a subject can generate in the mouth during inspiration, as shown in Equation (5), where t₀ and t correspond to the onset and final inspiration times:

$$\mathrm{PI}{}_{\max }=max\left\{{\left.P_{aw}\left(t\right)\right|}_{t_0}^t\right\}.$$

(5)

For the population analysis, the trend and the behavior of the indexes predicted by the models were compared to the data obtained from subjects in the DBs and analyzed. The experimental data were grouped according to their PEEP and PS levels for Tests D and E, respectively. To obtain the predicted data (WOB, RSBI, and PI_max), the median values of the muscular effort (RR and P_musc), ventilatory mechanics (R and C), and I:E variables were used as the input parameters for the models, according to Table 2. Corresponding PEEP and PS values were used for each level.

Regarding the individual analysis, predicted values of the WOB, RSBI, and PI_max were obtained using experimental values of RR, P_musc, R, C, and I:E of each subject as the module’s inputs. Then, the prediction error was calculated as the average difference between the experimental data (obtained from each record) and simulated data (see Equation (2)).

2.2.3. Delphi Validation

The content validation of the application corresponded to health professionals’ evaluations, with experience in the areas of anatomy, physiology, and the pathophysiology of the respiratory system. The inclusion criteria for selecting participants were focused on medical specialty (anesthesiology and intensive care medicine) and experience in the clinical field.

The aspects evaluated were focused on both the theoretical solidity and graphic quality of the content presented. The theoretical solidity aims to assess the relevance, sufficiency, and veracity of the concepts and clinical cases presented in the modules. In contrast, the graphic quality seeks to evaluate the definition, ordering, and veracity of the images (X-rays) and other visual resources (anatomical representations and charts).

The instruments built for the validation consisted of questionnaires based on the Likert scale (1 to 5), in which statements related to each of the contents displayed in the modules were presented. Every statement aimed to evaluate the aforementioned aspects and collect related comments, opinions, or suggestions. The instruments were designed following the structure of the modules, i.e., the statements were grouped according to their sections and sub-sections.

The built-in instruments were available to evaluators on a website specifically designed for validation.

The validation was carried out by eight evaluators who were contextualized about the procedure before obtaining their informed consent. Virtual sessions were scheduled to record the data, which at least two researchers supported. Their role included guiding the evaluators regarding the use of the website, and collecting comments, opinions, and suggestions.

2.3. Statistical Techniques

In both applications, the Lilliefors test was implemented to characterize the distribution of the groups. Once the free distribution of the data was corroborated, the Kruskal–Wallis nonparametric test was used in conjunction with the Tukey method to detect statistically significant differences between each pair of datasets, with a significance level of p < 0.05. The data was processed and computationally tabulated in MATLAB (MathWorks, R2018a).

3. Results

3.1. Anatomy and Physiology Module

3.1.1. Statistical Analysis

Population Analysis

Figure 5 shows the results obtained after the R and C variations using the values presented in Table 1 and with a RR of 12 bpm and P_musc of 8 cmH${}_2$O. Results are shown as the difference ($\Delta$) between the parameter and each volunteer’s biologically nominal value (i.e., the subject’s R and C values without applying any maneuvers).

Although there is high dispersion in the population information of the DBs, after a qualitative inspection of the curves, the predicted values followed the expected trend, which was checked by verifying the slope value in each trial.

Table 3. Median, interquartile range, and slope for each dataset in the population analysis for the “Anatomy and Physiology” module.

Curve	Test	Slope	Median	Interquartile Range	Data Units
$\Delta \dot{V}$ vs. $\Delta R$	Exp. data	−0.01	−0.12	(−0.28–0.00)	L/s
	A *	−0.05	−0.44	(−0.85–−0.23)
	B *	−0.05	−0.42	(−0.81–−0.22)
	C *	−0.04	−0.40	(−0.77–−0.21)
$\Delta V_{Tmax}$ vs. $\Delta R$	Exp. data	−3.89	−28.87	(−101.08–111.33)	mL
	A	−5.85	−43.17	(−80.78–−12.06)
	B *	−8.89	−72.53	(−131.00–−25.20)
	C *	−11.80	−105.58	(−181.39–−41.55)
$\Delta \dot{V}$ vs. $\Delta C$	Exp. data	0.00	0.11	(−0.12–0.28)	L/s
	A *	0.00	−0.11	(−0.17–−0.06)
	B *	0.00	−0.08	(−0.12–−0.04)
	C *	0.00	−0.05	(−0.07–−0.03)
$\Delta V_{Tmax}$ vs. $\Delta C$	Exp. data	1.53	8.86	(−75.35–136.12)	mL
	A *	4.55	170.45	(85.88–290.27)
	B *	3.05	144.09	(73.11–248.05)
	C	1.82	114.28	(55.40–185.60)

* Cases where statistically significant differences were found by the Kruskal-Wallis test.

shows the median values, the interquartile ranges, and the slopes of the experimental and predicted data (Tests A, B, and C) for each curve.

Individual Analysis

A sample with 122 data points was selected to estimate the model’s predictive capacity. The $\dot{V}$ and V_Tmax were obtained for the experimental data using the subjects’ corresponding values of R, C, P_musc, and RR. Figure 6 shows the experimental and predicted data distribution for comparative purposes. According to the results of the Kruskal–Wallis test, no statistically significant differences were found between each pair of data (p-value of 0.81 and 0.09 for $\dot{V}$ and V_Tmax, respectively).

In graphic (I) of Figure 6, the V_Tmax prediction describes high similarity in the data distribution; furthermore, the median values differ by 2.85% and the interquartile range differs by 5.93%, which is considered a good prediction. In graphic (II), for $\dot{V}$, differences between the experimental and predicted median data were similar (2.85% of error). Although $\dot{V}$ shows differences in the data distribution, it did not show any statistical difference.

Table 4. Tabulated errors between experimental and predicted data for the “Anatomy and Physiology” module.

Parameter	RR Condition	Median (%)	Interquartile Range (%)	n (Subjects)
VTmax	RR ≥ 12 bpm	9.26	(4.72–17.68)	77
	RR < 12 bpm	10.38	(5.07–19.57)	45
	Total	9.54	(4.80–18.18)	122
$\dot{V}$	RR ≥ 12 bpm	31.01	(14.78–65.57)	77
	RR < 12 bpm	22.62	(12.54–37.52)	45
	Total	25.38	(13.15–51.35)	122

reports the median and the interquartile range corresponding to the dispersion of the error calculated with Equation (2) for the total data, and separated according to the RR of each record. The errors obtained for $\dot{V}$ show that the model’s predictive power increases when the subject’s RR is lower than 12 bpm (errors remain smaller than 38%). In the case of V_Tmax, the total error shows a reasonable degree of prediction of the model due to the similarity and lower dispersion of the data shown in Figure 6(I), so there is no direct dependence with the RR. No case had errors greater than 20%.

3.1.2. Delphi Validation

Table 5. Delphi validation results of the “Anatomy and Physiology” module, organized by sections and sub-sections and scored from 1 to 5.

Section	Sub-Section	Score	n (Evaluated Items)
Ventilation	Equation of motion	4.00 (3.95–4.30)	6
	Anatomy	4.60 (4.20–4.80)	7
	Ventilatory signals	4.80 (4.60–5.00)	26
	Controls	4.50 (4.25–4.80)	8

presents the Delphi validation results regarding the sub-sections of the module, scored from 1 to 5. The scores obtained show a general assessment of the content as adequate, mainly highlighting the excellent result for the ventilatory signals and the possibility of improving the equation of air motion learning.

The Delphi validation results regarding the evaluated aspects are presented in

Table 6. Delphi validation results of the “Anatomy and Physiology” model, organized by aspects evaluated and scored from 1 to 5.

Content Type	Aspect	Score	n (Evaluated Items)
Theoretical	Relevance	4.30 (4.05–4.85)	4
	Sufficience	4.80 (4.15–4.80)	14
	Veracity	4.60 (4.50–4.85)	6

. Regarding this classification, a good and similar assessment is generally evidenced for all aspects, mainly highlighting the perceived sufficiency.

3.2. Work of Breathing Indexes Module

3.2.1. Statistical Analysis

Population AnalysisFigure 7 presents the distribution of the RSBI, WOB, and PI_max values obtained from six records at different values of PEEP and PS of 35 male volunteers. In addition, the RSBI, WOB, and PI_max values predicted by the model are reported. The Figure shows that the response follows the trend marked by the population data as the pressure increases in spontaneous ventilation. In both Tests D and E, the work of breathing and the Tobin index predicted by the model are within the first and third quartiles. Furthermore, in some cases, including the PI_max, some values are highly approximate to the median and, despite the dispersion of the experimental data, the models show very good fitting.

Individual Analysis

A total of 420 records were used to elaborate the comparative graphs of Figure 8, taking, as input parameters, the information of ventilatory mechanics, muscular effort, and the relationship between the inspiration and expiration time, measured in each variation of PEEP or PS, according to the test. The medians differ by 0.58 breaths/min/L (2.81% error), 0.02 J/L (2.52% error), and 0.40 cmH${}_2$O (6.02% error) for the RSBI, WOB, and, PI_max, respectively, which suggests a good level of prediction. The median, the interquartile range from the predicted error distribution (calculated with Equation (2)), and the p-value between experimental predicted values are reported in

Table 7. Percentage of error and p-values between experimental and predicted data for each of the evaluated indexes of the “Work of Breathing Indexes” module.

Index	p-Value	Median (%)	Interquartile Range (%)
RSBI	0.43	6.55	(3.04–11.49)
WOB	0.43	9.33	(3.84–18.40)
PImax	0.32	11.77	(5.31–23.00)

.

3.2.2. Delphi Validation

The Delphi validation results regarding the sections and sub-sections of the application are presented in

Table 8. Delphi validation results of the “Work of Breathing Indexes” module, organized by modules and sub-modules and scored from 1 to 5.

Section	Sub-Section	Score	n (Evaluated Items)
Work Indexes	Assessment of dyspnea	5.00 (4.60–5.00)	40
	Campbell diagram	5.00 (4.50–5.00)	17
	Ventilatory indexes	5.00 (4.80–5.00)	43
Obstructive Pattern	Clinical evaluation	3.80 (3.30–4.30)	15
	Pulmonary function charts and ventilatory monitoring	4.50 (4.00–5.00)	17
	Ventilatory loops	5.00 (3.80–5.00)	7
	Diagnostic tests	4.80 (4.50–5.00)	35
Restrictive Pattern	Clinical evaluation	4.40 (3.58–4.88)	12
	Pulmonary function charts and ventilatory monitoring	4.80 (4.00–5.00)	15
	Ventilatory loops	5.00 (4.80–5.00)	7
	Diagnostic tests	4.15 (3.50–4.68)	12
Mixed Pattern	Clinical evaluation	4.30 (4.00–4.80)	16
	Pulmonary function charts and ventilatory monitoring	5.00 (4.30–5.00)	15
	Ventilatory loops	5.00 (4.80–5.00)	7
	Diagnostic tests	5.00 (2.80–5.00)	18

. The scores for all the modules show, in general, an adequate perception of their contents, but the obstructive pattern must be improved regarding the clinical evaluation.

The Delphi validation results regarding the evaluated aspects are presented in

Table 9. Delphi validation results of the “Work of Breathing Indexes” model, organized by aspects evaluated and scored from 1 to 5.

Content-Type	Aspect	Score	n (Evaluated Items)
	Relevance	4.75 (4.31–5.00)	28
Theoretical	Sufficience	4.25 (3.75–4.75)	31
	Veracity	4.75 (4.00–5.00)	147
	Definition	4.80	2
Graphic	Ordering	5.00 (4.50–5.00)	3
	Veracity	5.00 (4.75–5.00)	65

. For this classification, the results also show generally high scores, mainly regarding the graphic content.

4. Discussion

Modeling breathing dynamics is key to understanding the progression of many diseases, human performance, and the development of life support systems [25]. Guaranteed mathematical model outputs are required to avoid compromising derivative studies, so quantitative and qualitative research must be conducted. Real predicted comparison data was used for recent related studies [26,27], proving that it is a valid methodology for the respiratory system’s model statistical analysis. In addition, Chan [28] proves that Delphi validation techniques provide qualitative value information for validating the gathered and reviewed data.

4.1. Anatomy and Physiology Module

4.1.1. Statistical Analysis

The elastic and resistive properties of the respiratory system affect the volume and airflow since they intervene in the time constant $\tau$, defined as the product between R and C that determines the rate of change of $\dot{V}$ and V when they are evaluated as a function of the ventilation time [29]. In addition, according to the equation of air motion, there is also a linear relationship between these variables, which is why they are the parameters that provide relevant information for evaluating ventilatory signals. This section presents an individual and a population analysis of the response of the mathematical models of the “Anatomy and Physiology” module, after variations in R and C, proving that it responds theoretically with the expressions above and expectedly at the physiological level.

Population Analysis

According to Figure 5, it is found that, regardless of the data dispersion, the linear regression of Tests A, B, and C follow the trend shown by the experimental data in each of the curves, suggesting a good prediction by the model in all cases.

As the airway resistance increases—represented mainly by the upper airway and the trachea, since they have a smaller cross-sectional area concerning the total area product of the lower branches—the gas flow that crosses the tracheobronchial tree decreases as a consequence of frictional losses [4]. This relationship is observed in Tests A, B, and C of Figure 5(I), so the model has a good response in the maximum flow value after changes in R. The small values on the slopes are due to the low rate of fluid loss. However, although the experimental data, as expected, also has a negative slope, its value is much lower than the model responses. This population’s tendency to maintain constant flow after changes in resistance may be a product of the natural response of the healthy subject maintaining the necessary flow for ventilation, since their respiratory muscles allow it (PI_max and RR were those of each subject), unlike what happens in subjects with obstructive pathologies, such as EPOC [30,31]. This fact supports the statistically significant differences of the experimental data with Tests A, B, and C.

The magnitude of resistance to airflow also affects lung volume, as shown in Figure 5(II). When the lungs are inflated, the airways tend to open, reducing their resistance [32], so they are inversely proportional. Authors such as Briscoe et al. [33] have compared changes in airway resistance with changes in lung volume at different degrees of lung inflation using plethysmography in healthy men, women, and children, showing that the relationship between both variables is maintained regardless of physical characteristics. These results show the model responds according to expectations with V_Tmax due to its trend. In addition, the ratio of the slopes indicates a good approximation, as their values increase by a factor of three to compliance increments; see Figure 5(IV). This relation is because the compliance of the respiratory system is usually calculated as the relationship between changes in lung volume and airway pressure [34], so an increase in C results in an increase in volume, as verified by Suter et al. [35]. Therefore, considering the trends and slope values in Figure 5(IV), it is found that the model’s response is validated due to the volume behavior found in the DB, and at a theoretical level, is very similar, i.e., it follows the same trend.

To achieve flow, there must be a pressure difference between two points to overcome the frictional forces. Although the lung tissue’s elastic properties and the rib cage’s structures offer some degree of resistance, approximately 90% of the total resistance of the respiratory system is constituted by that exerted by the airways [36]. The elastic properties and their associated impedance do not represent a parameter that directly affects the airflow through the system. Tests A, B, and C in Figure 5(III) show that the model considers the losses caused by the impedance associated with the elastic and inertial load. However, according to the magnitude of the slopes of each trend (approximately zero in all cases), it is found that the changes in flow are minor as compliance increases, but greater as resistance becomes higher, which is theoretically consistent. The slope of the experimental data reflects the insignificant changes that the airflow undergoes as compliance increases, since in normal conditions, respiratory muscles easily overcome elastic resistance [29]. This airflow behavior is proved by statistically significant differences between experimental data and Tests A, B, and C.

Individual Analysis

The results obtained for $\dot{V}$ and V_Tmax show a good level of prediction due to the similarity between medians and the non-existence of statistically significant differences between the experimental and predicted values, as shown in Figure 6. However, the equations involved in the model do not consider that the respiratory cycle can be altered by reflexes that arise in the lungs, airways, and the cardiovascular system as a result of respiratory control exerted by the nervous system [37]. This reflex mechanism against variations in mechanical parameters, the error related to the experimental data due to electromyography techniques and obstructive maneuvers for signal acquisition, and the respiratory mechanics’ data variability reported in [20] all justify the errors presented in Table 4.

In the V_Tmax case (see Figure 6(I)), the predicted values do not exceed the normal inspiratory capacity, which is reported around 2500 mL [1], demonstrating that the application is compatible at a physiological level. Moreover, there are no over-peaks when physiologically consistent R, C, P_musc, and RR values are entered. Studies by Nicolò et al. [38,39] suggest no direct relationship between respiratory rate and volume since ventilatory control responds under different stimuli in each case. Hence, the error is independent, as is presented in the first row of Table 4 after changes in the RR.

On the other hand, the amplitude of the predicted flow presents more significant variability with tendencies to high flows than the information collected in the DB (see Figure 6(II)), as result of the increase in RR, as evidenced in Table 4. The way RR is interpreted in mathematical models defines breathing dynamics and brings critical information for breathing characterization, so the parameter must be effectively evaluated [25].

In healthy subjects, when there is a difference from the normal breathing pattern, neuronal activity is modified, and appropriate changes in ventilation occur [40] because respiratory muscles allow them under normal conditions. When an obstructive disease occurs, such as asthma, subjects respond by increasing the RR by constantly maintaining minute ventilation—and, therefore, flow (in fluid dynamics, flow is defined as the mathematical derivative of volume over time [41], to which both ventilatory parameters are related)—to maintain ventilatory demand [42]. Therefore, using a database of healthy men is a limitation for this validation. Expanding the database with obstructive subjects will show more details and allow us to support the model validation.

However, when there is no significant increase in RR by the healthy subjects analyzed, the pair of experimental predicted values does not present a high error range because the closed-loop control does not intervene in the ventilatory variables.

Although a RR greater than 12 rpm is a limiting factor for the prediction due to the resulting high values that differ from the experimental data, in obstructive patients submitted to MV, a flow whose inspiratory peak is high is usually used to decrease the inspiratory time and increase the expiratory one, reducing the risk of auto-PEEP [34]. Thus, it can be deduced that the model responds adequately to physiological considerations where neuronal control is not involved, as previously mentioned.

4.1.2. Delphi Validation

The results obtained for this module can be considered satisfactory because the scores for each sub-section are greater than or equal to 4.00. The lowest score was obtained for the equation of air motion sub-section. The recommendations for improvement involve incorporating the units in the equation, examples, and a further description of the terms. Regarding the evaluation based on the type of content, the scores also exceeded 4.00. The lowest score was obtained for the relevance aspect and is related to the need for greater accuracy of the descriptions. The associated recommendations focus on the greater detail of the main concepts.

4.2. Work of Breathing Indexes Model

4.2.1. Statistical Analysis

The main objective of MV is to increase the gas exchange and reduce the respiratory effort, or the respiratory muscles’ load, produced by variations in R or C due to restrictive or obstructive pathologies [43]. This load can be measured by indexes related to weaning success, such as RSBI, WOB, and PI_max.

Population Analysis

The relation between changes in the MV setting and the RSBI, WOB and, PI_max indexes was depicted in Figure 7. The changes assessed were PEEP and PS increments, so as to know the response to changes in respiratory muscles’ loads in a population of healthy men and the response of the predictive model.

PEEP is defined as the positive end-expiratory pressure, and it is detected as an offset in the pressure measured in the mouth. In physiological terms, PEEP is used for alveolar recruitment, raising C and lung capacity [44]. Increases in PEEP improve the gas exchange because they make more alveolar units available [45]. Pelosi et al. [46] analyzed changes in C and the level of oxygenation of healthy and obese subjects under PEEP stimuli of 0 cmH${}_2$O and 10 cmH${}_2$O. They found considerable oxygenation improvements and C increments only in PEEP of 10 cmH${}_2$O and pathological situations, concluding that PEEP does not have a relevant effect when it comes to subjects without obstructive, restrictive, or cardiopulmonary problems. Therefore, both the experimental and predicted RSBI and WOB values (Figure 7(I),(II), respectively) do not present a marked effect as pressure increases.

PS can be remarkably effective in reducing patient effort, avoiding respiratory distress, and can offer a comfortable ventilator support to many patients [47]. Studies with healthy subjects have shown the PS effect for WOB [48] and RSBI decreases [49], just as the results for the predicted and experimental data in Figure 7(IV),(V) suggest.

The PI_max increases proportionally under PEEP (Figure 7(III)) and PS (Figure 7(VI)) stimuli, although it does not exceed the muscular overexertion threshold product of the healthy condition of the subjects. In both cases, the increases in each group of data are equivalent to the added pressure value with respect to to the basal state (PEEP and PS = 0 cmH${}_2$O), due to the fact that the mechanical ventilator applies both pressures, increasing pressure at the mouth (P_ao) and consequently increasing airway pressure [24,50,51].

Individual Analysis

The similarities in both the experimental and predicted distribution data of Figure 8, the non-existence of statistically significant differences between both values, and the low percentage of error shown in Table 7 (less than 23%) prove the good level of prediction by the models involved, including the non-dependence on input parameters as long as they are consistent at a physiological level.

The index with the highest error percentage was the PI_max, which may be a product of the difficulty in the measurement reported by the ATS/ERS and confirmed by some studies [24,52]. There is a dependence on the participants’ complete collaboration.

4.2.2. Delphi Validation

The results obtained as a function of the sections and sub-sections of the application generally showed high scores, with the “Work Indexes” and “Mixed Pattern” sections standing out as those with the highest median scores. The lowest score was obtained for the “Obstructive Pattern” section, and was related to its clinical evaluation. From the evaluators’ comments, it is identified that the case needs to be modified by another with more appropriate variable values and diagnostic tools.

For the “Restrictive Pattern” section, some of the evaluators agree that the information of the round presented for the clinical case may be confusing and incomplete. This fact is reflected in the score obtained for the clinical evaluation sub-section (4.40 (3.58–4.88)); concerning the diagnostic tests sub-section, the evaluators consider that the values given by arterial blood gas do not correspond to a patient with restrictive failure.

Finally, the “Mixed Pattern” section achieved good scores for all its sub-sections. However, for the diagnostic tests sub-section, some evaluators consider that arterial blood gas results do not correspond to a patient with a mixed pattern, which is why it is recommended to carry out a conduct review of these values.

Concerning the classification by evaluated aspects, given that the scores obtained in median exceeded 4.25, it can be considered an adequate perception of the evaluators. The graphic content for this module stands out for the best scores, mainly concerning the aspects of order and veracity. The lowest score was obtained regarding the sufficiency of the theoretical content and is related to selecting complete and better-defined clinical cases.

5. Conclusions

In this article, the statistical validation of the ventilatory signals in the “Anatomy and Physiology” module and the ventilatory indexes in the “Work of Breathing” module, in conjunction with Delphi validations, is presented. All the methods demonstrate the accuracy and precision of the involved systems, which validates the application for respiratory system teaching.

For both modules, the population analysis showed that the models responded as expected in physiological terms for the studied variables ($\dot{V}$ and V_Tmax in the “Anatomy and Physiology” module, and RSBI, WOB and PI_max in the “Work of Breathing” module). Flow and volume behave consistently with the expressions that describe ventilatory dynamics in healthy subjects after variations in R and C, without including compensations made by the nervous control system when RR increases, which makes it a limiting parameter; furthermore, the ventilatory indexes follow the expected trend according to the population response where PEEP and PS are increased. On the other hand, the individual analysis showed that both applications have a low percentage of error between the experimental and predicted data pair when they are analyzed under consistent physiological conditions.

As with any other validation study, increasing the DB, including women and children, could yield more information regarding the faculties of the models involved to achieve a significant decrease in the limitations and guarantee that the result is independent of the anthropometric characteristics of the simulated subject.

The results obtained from the Delphi validation evidenced an adequate concept of the content of the modules. The scores obtained demonstrated, in both a general and specific way for the modules and aspects evaluated, a good conception by specialists.

The main improvement options were identified for the ventilation section of the “Anatomy and Physiology” module, and the “Obstructive Pattern” sub-module of the “Work of Breathing Indexes” module. The reasons and improvements in both cases are mainly related to more detailed descriptions, including examples, and better-defined clinical cases.