Optimizing Mineralization of Bioprinted Bone Utilizing Type-2 Fuzzy Systems

Abstract

Bioprinting is an emerging tissue engineering method used to generate cell-laden scaffolds with high spatial resolution. Bioprinting parameters, such as pressure, nozzle size, and speed, highly influence the quality of the bioprinted construct. Moreover, cell suspension density and other critical biological parameters directly impact the biological function. Therefore, an approximation model that can be used to find the optimal bioprinting parameter settings for bioprinted constructs is highly desirable. Here, we propose a type-2 fuzzy model to handle the uncertainty and imprecision in the approximation model. Specifically, we focus on the biological parameters, such as the culture period, that can be used to maximize the output value (mineralization volume 21.8 mm³ with the same culture period of 21 days). We have also implemented a type-1 fuzzy model and compared the results with the proposed type-2 fuzzy model using two levels of uncertainty. We hypothesize that the type-2 fuzzy model may be preferred in biological systems due to the inherent vagueness and imprecision of the input data. Our numerical results confirm this hypothesis. More specifically, the type-2 fuzzy model with a high uncertainty boundary (30%) is superior to type-1 and type-2 fuzzy systems with low uncertainty boundaries in the overall output approximation error for bone bioprinting inputs.

1. Introduction

Additive manufacturing is a process by which three-dimensional objects are generated by material deposition in sequential layers [1,2]. Bioprinting is an emerging field of additive manufacturing in which bioactive scaffolds can be quickly generated by depositing layers of cell-laden biocompatible materials, such as collagen or other hydrogels. After specifying the exact geometry of the construct, a G-code containing the extrusion path and parameters is generated to direct fabrication by one of several commercially available desktop bioprinters. Indeed, the ability to place cells in biologically relevant scaffold materials with a high spatial resolution has made bioprinting a popular fabrication method for tissue engineering [3,4].

In addition to the specific geometry of the bioprinted construct, the parameters used to perform the bioprinting procedure itself will have significant effects on the final properties of the model. Therefore, it is essential to fully characterize and optimize the bioprinting parameters (e.g., print speed or bioink viscosity) necessary to reach the desired outputs, such as high cell viability, appropriate cell function, and necessary mechanical properties [5]. For example, increasing the nozzle size on the bioprinter decreases the shear stress placed on the biomaterial during extrusion, which results in increased cell viability and reduced print resolution [6]. Therefore, determining the optimal print parameters is imperative for success in bioprinting. As a result, several studies have been performed in the field of bioprinting optimization, such as optimization of a solid model for 3D bioprinting, bioink optimization, and bioprinting parameter selection [7,8].

Determining printability using universal parameters is challenging due to the variability of customized bioinks. Moreover, this complexity is increased in bioinks with multiple extracellular matrix (ECM) components that affect the rheological properties and can result in non-linear relationships between input and output parameters [9,10]. The results from our previous study suggest that the computational approach is useful for optimizing printing parameters and will improve reproducibility across diverse bioinks as well as provide an objective characterization of bioprinting precision for newly formulated bioinks. In this paper, we investigate the computational optimization of the ECM components in bioprinting.

Nonetheless, there is a significant degree of imprecision and uncertainty inherent in bioprinting optimization. A potential approach to handling this issue, which arises from normal biological variation, is through the implementation of approximation systems based on computational methods [11]. In recent years, systems biology has become a critical multidisciplinary research area between computer science and biology. Studies in this field aim to develop computational models of biological processes, which requires both a robust dynamic model and well a large dataset of experimental results.

Developing a dynamic model is challenging and requires well-characterized control parameters to approximate laboratory experiments’ outcomes. Nonetheless, recent studies have observed that computational optimization algorithms can effectively approximate output parameters using either a deterministic or stochastic biological model. A partial list of the approaches employed to this end includes meta-heuristic, evolutionary, global optimization, genetic programming, simulated annealing, simplex, ant-colony, fuzzy genetic hybrid system, and multi-objective optimization. In-silico models have helped reproducibility and quality in the in-vitro experiments significantly in tissue engineering [12,13].

Here, we have developed a quantitative model of a biological system using the fuzzy system approach, which is a potential solution for overcoming uncertainty in an experimental dataset. In fact, previous studies have shown that the accuracy of a fuzzy system approach is the same as the deterministic mathematical approach (ordinary differential equations) for the same kinetic dataset. Moreover, fuzzy systems can be utilized to find the qualitative system response when a quantitative dataset is not available [14]. Theoretical fuzzy models have been used in decision making systems to predict biomechanical properties as stress/strain of a bone structure in a biological process [15].

Fuzzy logic is an extended model of standard logic. In standard logic, truth values can only be either completely false or completely true (with degrees of truth equal to 0 or 1, respectively), whereas, in fuzzy logic, values can have a degree of truth between 0 and 1. This generalization provides a mathematical framework to move from discrete to continuous values. In other words, in contrast to sets in classical logic, a fuzzy set is a set without a crisp boundary. For instance, if the reference set X is a Universe of discourse for elements x, the fuzzy set A is defined as:

$$A=\{ \left(x, {\mu }_A\left(x\right)\right)|x\in X\}$$

where ${\mu }_A\left(x\right)$ is called the Membership Function (MF) for the fuzzy set A. The MF maps each element of the Universe set X to a grade between 0 and 1, i.e., a membership of 0 means that the associated element is not included, whereas a membership of 1 means an element is fully included.

A fuzzy rule-based system is a modeling framework that uses the above fuzzy set theory along with a set of “if-then” rules where the antecedents and consequents are fuzzy logic propositions. This rule-based fuzzy system is used for modelling the inputs and their relationships with the output variables. A type-1 Fuzzy System (T1 FS) is a framework consisting of weighted rules, membership functions, and a fuzzy inference system. This system takes the crisp data (fuzzy singletons) or fuzzy inputs and generates fuzzy outputs based on the given if-then rules. A method of defuzzification is then used to extract a crisp value inferred from the fuzzy model.

A type-2 fuzzy system (T2 FS) is also similar to its type-1 counterpart, but includes a type-reducer and defuzzifier, which generate a type-1 fuzzy set output and then a crisp number, respectively. T2 FS have been widely applied to a variety of problems where handling uncertainty is critical, including decision making, function approximation, and data preprocessing [16,17,18]. One example is a model with noisy and uncertain training data: in this model, uncertainty exists in the antecedent and consequents. The level of uncertainty and information regarding it can be used in mathematical modeling of antecedents and consequents. Moreover, non-quantitative data is often disseminated using words that convey an indistinct level of certainty [19]. T2 FS has the capability to grade these linguistic representations into membership functions, which shows a more robust algorithm rather than T1 FS in inferencing input data. However, the computational cost of T2 FS is higher than the T1 FS [19].

The main objective of this study is to implement fuzzy systems to model and find the optimal bioprinting parameters. We hypothesize that the implementation of a T2 FS would reduce the error in the output in biological systems with inherent uncertainty in both the inputs and outputs.

To directly test this hypothesis, in Section 2 we implement type-1 and type-2 fuzzy logic systems with the experimental dataset. We then evaluate and compare the performance of each for use in the approximation of bioprinting output in Section 3. Section 4 describes how fuzzy systems can optimize the bioprinting and biological parameters.

Type-2 Fuzzy System

The concept of a Type-2 fuzzy set is an extension of the ordinary type-1 fuzzy set, as originally introduced by Zadeh [20]. In contrast to a T1 FS, Type-2 fuzzy sets have grades of membership that are themselves fuzzy both in primary and secondary memberships. A primary membership is the same as a type-1 membership that maps each element to a grade between 0 and 1. Relative to each primary membership, there is a secondary membership (a grade between 0 and 1) that describes the uncertainty in defining the primary membership using a fuzzy set construct.

The general diagram of a T2 Mamdani fuzzy logic system is shown in Figure 1, including the fuzzification, fuzzy inference, Karnik–Mendel type reduction, and defuzzification steps. Considering the crisp inputs (n inputs) and one output:

$$x_1\in X_1, \dots , x_n\in X_n \mathrm{and} y\in Y,$$

the k-th ($k=1, \dots , K$) rule in Mamdani T2 FS is expressed as:

$$R^k:if x_1 is {\tilde{F}}_1^k and\dots and x_n is {\tilde{F}}_n^k, then y is {\tilde{G}}^k$$

where ${\tilde{F}}_1^k$ and ${\tilde{G}}^k$ are type-2 fuzzy sets. In this system, rules represent the fuzzy relations between multiple dimensional input space $X\triangleq X_1\times \dots \times X_n$ and output space $Y$. The definitions below are paraphrased from the Mendel and Liang article in T2 FS [21]:

Definition 1.

(Footprint of Uncertainty of a Type-2 Membership Function): Uncertainty in the region between the upper and lower boundaries of a Type-2 membership function is called the footprint of uncertainty. It is the union of all primary membership grades.

Definition 2.

(Upper and Lower MFs): A type-1 fuzzy upper and lower boundary MFs for the Footprint of Uncertainty, and (FOU) of an interval type-2 MF. The upper and lower bounds of the region are the maximum and minimum membership grades of FOU (Figure 1-fuzzification [21]).

The over and under bars show the upper and lower MFs, respectively. The membership function of ${\tilde{F}}_1^k$ is represented as:

$${\mu }_{{\tilde{F}}_1^k}\left(x_k\right)={{\displaystyle \int }}_{w^l\in \left[{\underset{\_}{\mu }}_{{\tilde{F}}_1^k}\left(x_k\right),{\overline{\mu }}_{{\tilde{F}}_1^k}\left(x_k\right)\right]}1/{\omega }^l$$

(1)

The upper and lower boundaries on a Gaussian primary function with an uncertain standard deviation is represented below. In this representation, the Gaussian primary MF has a fixed mean $m_k^l$ and an uncertain standard deviation ${\sigma }_k^lϵ\left[{\sigma }_{k1}^l, {\sigma }_{k2}^l\right]$:

$${\mu }_k^l\left(x_k\right)=\exp [-\frac{1}{2}{\left(\frac{x_k-m_k^l}{{\sigma }_k^l}\right)}^2]$$

(2)

where

$$\begin{gathered} k \in \left( 1, \dots , number of antecedents\right) \\ l\in \left( 1, \dots , number of rules\right) \end{gathered}$$

The upper and lower MF of ${\mu }_k^l\left(x_k\right)$ are:

$$\begin{gathered} {\overline{\mu }}_k^l\left(x_k\right)=\mathcal{N}\left(m_k^l,{\sigma }_{k2}^l;x_k\right)\hspace{1em}\hspace{1em}\mathrm{upper} \mathrm{MF} \\ {\underset{\_}{\mu }}_k^l\left(x_k\right)=\mathcal{N}\left(m_k^l,{\sigma }_{k1}^l;x_k\right)\hspace{1em}\hspace{1em}\mathrm{lower} \mathrm{MF} \end{gathered}$$

In the interval T2 non-singleton fuzzy system with type-2 fuzzification and minimum or product t-norm, the output fuzzy set is represented as:

$${\mu }_{\tilde{B}}\left(y\right)={{\displaystyle \int }}_{b\in \left[\left[{\underset{\_}{f}}^1\star {\underset{\_}{\mu }}_{{\tilde{G}}^1}\left(y\right)\right]\vee \dots \vee \left[{\underset{\_}{f}}^N\star {\underset{\_}{\mu }}_{{\tilde{G}}^N}\left(y\right)\right],\left[ {\overline{f}}^1\star {\overline{\mu }}_{{\tilde{G}}^1}\left(y\right)\right]\vee \dots \vee \left[{\overline{f}}^N\star {\overline{\mu }}_{{\tilde{G}}^N}\left(y\right)\right]\right]}\frac{1}{b}$$

(3)

In (3), $\underset{\_}{f}$ and $\overline{f}$ are the result of the input and antecedent operations (Figure 1-Inference engine), based on the value of $x_k$, in which the supremum occurs as ${\underset{\_}{x}}_{k,max}^l$ and ${\overline{x}}_{k,max}^l$:

$${\overline{f}}_k^l={\overline{\mu }}_{{\tilde{X}}_k}\left({\overline{x}}_{k,max}^l\right)\star {\overline{\mu }}_{{\tilde{F}}_k^l}\left({\overline{x}}_{k,max}^l\right)$$

(4)

$${\underset{\_}{f}}_k^l={\underset{\_}{\mu }}_{{\tilde{X}}_k}\left({\underset{\_}{x}}_{k,max}^l\right)\star {\underset{\_}{\mu }}_{{\tilde{F}}_k}\left({\underset{\_}{x}}_{k,max}^l\right)$$

(5)

In the algorithm above, we follow the following steps to obtain the crisp output: (1) fuzzification, (2) fuzzy inference, (3) type-reduction, and (4) defuzzification. The result of crisp input and antecedents is an interval type-1 fuzzy set, defined by the lower and upper MF ${\underset{\_}{f}}_{ }^{ }$ and ${\overline{f}}_{ }^{ }$, respectively. Referring to (3), the fired output value, which is the combined output consequent set ${\mu }_{\tilde{B}}\left(y\right),$ is computed. The type-2 fuzzy system is computationally intensive to implement. A potential solution is a type-reduction method, as proposed by Kendrick and Mendel for the type-1 defuzzification method for reducing the type-2 to type-1 fuzzy system [21].

Various forms of type-reduction algorithms are proposed as centroid and center-of-sets [22]. In our method, we have used the center-of-set type-reduction method (Figure 1-type reduction):

$$y_l=\frac{{{\displaystyle \sum }}_{i=1}^M{f}_l^i{y}_l^i}{{{\displaystyle \sum }}_{i=1}^M{f}_l^i} \mathrm{and} y_r=\frac{{{\displaystyle \sum }}_{i=1}^M{f}_r^i{y}_r^i}{{{\displaystyle \sum }}_{i=1}^M{f}_r^i},$$

(6)

where the maximum and minimum value of $y_{ }$ are $y_r$ and $y_l$, respectively; $y_r$ and $y_l$ depend only on a mixture of ${\underset{\_}{f}}_{ }^{ }$ and ${\overline{f}}_{ }^{ }$ values, because $f^i\in F^i=[ {\underset{\_}{f}}_{ }^{ }$, ${\overline{f}}_{ }^{ }]$. Due to $y$ being an interval non-convex set, we defuzzify it by using the average of $y_r$ and $y_l$ (Figure 1-defuzzification).

$$f\left(x\right)=\frac{y_r+y_l}{2}=d,$$

(7)

where d is the defuzzified output in the above formula.

2. Methods

2.1. Fuzzy Inference Engine

The first step to develop this approximation model is to make a fuzzy inference system, such as the Mamdani or Sugeno systems. For this study, we chose the Mamdani inference engine due to the intuitive interpretable nature of its rule-base inferencing.

The main difference between Mamdani-type and Sugeno-type Fuzzy Inference System (FIS) is the method of how the output result is obtained. In Mamdani FIS, the crisp result is obtained through defuzzification of the rules. However, Sugeno FIS uses a weighted average of the rules to compute the crisp output. Moreover, Mamdani FIS can be applied to both “Multiple Input, Single Output” (MISO) and “Multiple Input, Multiple Output” (MIMO) systems, which is advantageous for biologic systems that frequently have multiple outputs. It is important to note that Sugeno-type systems can be used for MISO systems.

2.2. Membership Function Variables and Parameterization

We obtained optimization data for the variables in our model from a previously published experimental study [23]. As shown in Figure 2, the bioprinting approximation system has two inputs, which are the number of cells (millions per milliliter) and the culture period (days), and one output variable, which is the mineralized volume of the bioprinted construct (mm3). The prior knowledge defines the membership function values and If-Then rules in the T2 fuzzy model as shown in Figure 2.

The mineralized volume of the bioprinted bone was calculated based on registered microCT scans using previously established methods [24]. Here, microCT imaging was performed after 7, 14, 21, 28, 35, and 42 days in culture [23].

Next, the membership functions (MFs) were defined. MFs are distributed evenly by dividing the full input data range by the number of MFs, which is the input labels shown in

Table 1. Input Bioprinting data.

Culture Period (Days)	Cell Suspension Density (Million Cells /Milliliter)	Mineralization Volume (mm3)
7	0	0
7	1.67	0.1
7	5	0.1
7	15	0.2
14	0	0
14	1.67	1
14	5	3
14	15	12
21	0	4
21	1.67	14
21	5	21
21	15	24

and Figure 3. The membership function is a 2D curve (type-1 fuzzy) that describes the variables’ degree of membership to a fuzzy set, using a value between 0 and 1. Membership functions are used in a fuzzification process to convert the crisp values to fuzzy values [25]. Membership functions can be implemented using a variety of functions, such as triangular, Gaussian, or gamma. Here, we chose the Gaussian membership function due to the similarity of this function with many biological processes. In general, Gaussian membership functions are popular because of their smoothness and concise notation [26]. MFs designed with Gaussian forms are modified by tuning the standard deviation and their mean values, as shown in

Table 2. Gaussian membership functions range [Sigma, Mean].

Linguistic Variable	Culture Period (Days)	Cell Suspension Density (Million Cells /Milliliter)	Mineralization Volume (mm3)
Low	[0.5011 6.78]	[1.981 1]	[1.54 0.1]
Medium	[2.109 14]	[1.981 5.667]	[3.38 8.11]
High	[2.476 21.03]	[1.981 10.33]	[3.384 16.03]
Very High	-	[1.981 15]	[3.384 24]

. Table 1 indicates the fact that low mineralization volume and close data points in a 7-day culturing period compared to the 14- and 21-day periods results in a smaller standard deviation than the two other MFs.

The membership functions based on the input value representation (Table 1) are plotted in Figure 3. The Type-1 membership functions were converted to Type-2 with a prescribed 20 and 30 percent uncertainty using the Fuzzy Logic System toolbox (Matlab). The lower and upper boundary of the MFs type-2 FS with 0.2 lag are plotted in Figure 3.

2.3. If-Then Rules Establishment

The next step in designing a fuzzy system is to define the fuzzy IF-THEN rules. As shown in Table 2, we utilized previously published results from an experimental bioprinting study [23]. The model rules are shown in

Table 3. Type-1 and Type-2 Fuzzy system rules.

Rule	Culture Period (Days)	Cell Suspension Density (Million Cells/Milliliter)	Mineralization Volume (mm3)
1	Low	Low	Low
2	Low	Medium	Low
3	Low	High	Low
4	Low	Very High	Low
5	Medium	Low	Low
6	Medium	Medium	Medium
7	Medium	High	Medium
8	Medium	Very High	High
9	High	Low	Medium
10	High	Medium	High
11	High	High	Very High
12	High	Very High	Very High

.

2.4. Fuzzy Inference Process

Finally, we implemented the type-1 fuzzy inference process using the following procedure:

• Fuzzification of the input variables;
• Application of the fuzzy operator (AND) in the antecedent;
• Implication from the antecedent to the consequent;
• Aggregation of the consequent across the rules;
• Defuzzification.

Next, the type-2 fuzzy system is implemented using the following process:

• Fuzzification of the input variables;
• Application of the fuzzy operator (AND) in the antecedent;
• Convert T1 MF to T2 MF with 0.2 and 0.3 lag;
• Implication from the antecedent to the consequent;
• Aggregation of the consequent across the rules;
• T2 FS to T1 FS type reduction by Karnik–Mendel;
• Defuzzification.

In the above process, we use “Minimum” for AND (Step 2), “Minimum” for Implication (Step 4), “Maximum” for Aggregation (Step 5), and “Centroid” for defuzzification (Step 7). We used the “Centroid” defuzzification because it has the highest correlation in defuzzification methods in comparison to the other methods, such as bisector, mean of maximum, and largest of maximum for non-interval data, as previously described [22]. The 3D surface of the implemented type-1 fuzzy and type-2 fuzzy (20% and 30% uncertainty) rules based on the two inputs and one output is plotted in Figure 4.

3. Results

Type-1 and type-2 fuzzy systems were implemented as described using input data from a previous experimental study [23].

Table 4. Accuracy of Type-1 and Type-2 (20% and 30% uncertainty) Fuzzy Systems.

Culture Period (Days)	Cell Suspension Density (Million Cells /Milliliter)	Mineralization Volume (mm3)
		Experimental Values (Actual)	Zero Uncertainty (T1 FS)	20% Uncertainty (T2 FS)	30% Uncertainty (T2 FS)
7	1.67	0.1	1.84	1.32	1.2
7	5	0.1	2.04	2.1	2.73
7	15	0.2	1.75	1.32	1.31
14	1.67	1	7.6	5.8	4.0
14	5	3	8.28	8.21	8.21
14	15	12	15.57	15.58	15.87
21	1.67	14	9.43	8.89	8.9
21	5	21	14.9	15.4	15.45
21	15	24	21.2	21.81	21.89
RMSE		N/A	3.6	3.79	3.5

shows the T1 and T2 FS approximated values. In these tables, in addition to the experimental input data, the type-2 fuzzy model has an additional input, which is the uncertainty boundary (either 20% or 30%). This uncertainty boundary is used as an input to compare the mineralization volume output with different noise levels in the input data.

First, we generated approximated values at each of the input value combinations available (Table 4). A graphical visualization of these values has been generated as a 3D surface, for the T1 FS (Figure 4a), 20% T2 FS (Figure 4b), and 30% T2 FS (Figure 4c). Next, we compared the experimental output value (mineralization volume) to the approximated values in each of the three fuzzy systems (Table 4). Here, we calculated the root mean square error (RMSE) between the experimental and approximated values with the same input values.

The equation of the measurement is given as follows (8):

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^N{\left(x_i-{\overline{x}}_i\right)}^2}{N}}$$

(8)

Compared to the T1 FS, we observed that the 20% T2 FS increases the overall error of the approximation (+5.3%). In contrast, we observed that the 30% T2 FS decreases the overall error of the approximation (−2.8%), relative to the T1 FS.

4. Discussion

Bioprinting research is a time-consuming and expensive process requiring the use of cells that may be difficult to source followed by a lengthy culture period. Finally, quantification of the suitability of the bioprint itself can be challenging. As a result, it is difficult to exhaustively optimize a bioprinted construct experimentally. Here, we have demonstrated that approximating bioprinting output parameters using fuzzy systems based on input variables is a viable approach to accelerate research, reduce experimental costs, and improve outcomes. Furthermore, our fuzzy model approach can be redesigned with additional input and output variables, qualitative results, or expert knowledge using linguistic rules.

In this study, we implement a fuzzy logic-based model using both type-1 and type-2 fuzzy systems to compare the results in handling the uncertainty associated with the bioprinting process. This uncertainty can arise from noisy input data or imperfect expert knowledge. Using experimental data, we have demonstrated that the implemented fuzzy logic can convert the discrete crisp input data to fuzzy sets to achieve a continuum data surface with high accuracy.

Additional work may be necessary to determine the specific level of uncertainty necessary to achieve the highest accuracy. Moreover, further experimental work is required to scrutinize the accuracy of this model. Nonetheless, since the increased range of uncertainty may increase the overfitting variance, we used 30% percent uncertainty level as the highest boundary to alleviate the bias–variance tradeoff [27]. In this experiment, the results with two different uncertainty percentages indicate a high correlation with experimental data. Furthermore, the 30% type-2 fuzzy model can accommodate more imprecision with higher accuracy. In contrast, the 20% type-2 fuzzy model could not provide higher accuracy relative to the type-1 fuzzy model.

We have provided 3D surfaces generated by the fuzzy rules (e.g., Figure 4) as an intuitive tool to help researchers design new studies for the experimental optimization of bioprinted constructs. For example, researchers should avoid performing new experiments in “flat” areas of the 3D model. In our study, the 3D surface illustrates a relatively flat area for moderate cell suspension densities and culture periods of around 14 days. Therefore, if a researcher wishes to maximize mineralization volume with a low number of cells (e.g., 5 × 10⁶ cells/mL), they may prefer to increase their culture time to 21 days rather than attempt to triple their cell number in order to traverse the flat area of the surface. In total, implementation of this system may help researchers optimize their study design to eliminate unnecessary experimentation.

We also note that the 3D surfaces generated for the type-2 fuzzy models (Figure 4b,c) are both qualitatively smoother than the surface generated using the type-1 fuzzy system (Figure 4a). In particular, we observe a sharp edge around a culture period of 8 days (Figure 4a). This smoothness in Figure 4b,c is likely to result in higher accuracy in approximated values, as biological mathematical models ought to have a smooth transmission when increasing the input values.