Assisting the Human Embryo Viability Assessment by Deep Learning for In Vitro Fertilization

Abstract

The increasing global infertility rate is a matter of significant concern. In vitro fertilization (IVF) significantly minimizes infertility by providing an alternative clinical means of becoming pregnant. The success of IVF mainly depends on the assessment and analysis of human blastocyst components such as the blastocoel (BC), zona pellucida (ZP), inner cell mass (ICM), and trophectoderm (TE). Embryologists perform a morphological assessment of the blastocyst components for the selection of potential embryos to be used in the IVF process. Manual assessment of blastocyst components is time-consuming, subjective, and prone to errors. Therefore, artificial intelligence (AI)-based methods are highly desirable for enhancing the success rate and efficiency of IVF. In this study, a novel feature-supplementation-based blastocyst segmentation network (FSBS-Net) has been developed to deliver higher segmentation accuracy for blastocyst components with less computational overhead compared with state-of-the-art methods. FSBS-Net uses an effective feature supplementation mechanism along with ascending channel convolutional blocks to accurately detect the pixels of the blastocyst components with minimal spatial loss. The proposed method was evaluated using an open database for human blastocyst component segmentation, and it outperformed state-of-the-art methods in terms of both segmentation accuracy and computational efficiency. FSBS-Net segmented the BC, ZP, ICM, TE, and background with intersections over union (IoU) values of 89.15, 85.80, 85.55, 80.17, and 95.61%, respectively. In addition, FSBS-Net achieved a mean IoU for all categories of 87.26% with only 2.01 million trainable parameters. The experimental results demonstrate that the proposed method could be very helpful in assisting embryologists in the morphological assessment of human blastocyst components.

1. Introduction

The infertility rate among couples is rapidly increasing worldwide. According to recent survey findings, more than 80 million couples experience infertility [1]. In some African populations, the infertility rate has approached an alarming figure of 50% [2]. Similarly, a large population (approximately 12–16%) in Canada also suffers from infertility. Infertility not only affects human generations but also triggers anxiety and psychological complications, which are the source of many other diseases [3]. Therefore, alternative methods for clinical pregnancy are highly desirable in the prevailing situation. In vitro fertilization (IVF) is one of the main alternatives for overcoming fertility problems adopted by the masses. The external combination of sperm with eggs through medical intervention is referred to as assisted reproductive technology (ART). IVF is a part of ART that significantly contributes to resolving infertility problems. Previous studies have reported over eight million successful IVF cases since 1978. The success rate of IVF is primarily based on embryo selection. Embryologists usually assess embryo viability based on morphological structure and appearance. Selected potential embryos are transferred to the uterus for further IVF. Usually, the embryo becomes a blastocyst on its fifth day [3]. The first attempt at embryo transfer for ICVF was made in 1997 using an artificial neural network (ANN) [4]. IVF-based treatments are performed more than one million times a year globally [5]. However, only 33.1% of successful IVF cases were reported in Canada in 2017 [6]. In IVF, embryologists combine eggs with human sperm and select viable embryos for transfer back to the uterus. Embryos are maintained in a controlled environment for a few days until they develop into blastocysts. Potential blastocysts (embryos) are selected based on morphological assessments. In the initial developmental stage of IVF, multiple embryos were selected and transferred to the uterus, resulting in multiple pregnancy issues. Subsequently, only a few embryos are now selected to avoid multiple pregnancies. Selection of potential embryos is the most critical task in IVF. Embryologists conduct embryo selection manually, which is a subjective, error-prone, and resource-intensive procedure [6]. Therefore, it is desirable to combine existing assessment methods with AI-based methods to increase the success rate of IVF. Although IVF has gained popularity in recent decades, potential in vitro risks need to be assessed using the latest approaches, such as the new methodologies suggested by Magurany et al. (2023) [7].

AI has a proven record in computer-aided diagnosis and clinical decision support systems [8,9]. Deep-learning-based methods provide several intelligent solutions for effectively dealing with different diseases and medical conditions [10,11]. Recent studies indicate that health professionals exhibit positive behavior toward accepting AI-based frameworks in the diagnostic industry [12]. Similarly, Calisto et al., (2022) conducted a brief study on the acceptance, adoption, and trust of technology-based methods for medical imaging. This study also analyzed the risks and security concerns associated with technological solutions in the medical sector [13]. Additionally, IVF using AI has many ethical implications that should be discussed openly. As suggested in [14], both bioethics and AI ethics (in combination) should be considered to further promote IVF using AI. According to one study, the IVF success rate ranges from 4 to 40%, depending on several parameters, such as age and medical conditions. AI is considered the main candidate for promoting automated IVF. ANNs also play a vital role in IVF; however, they have several limitations and flaws in terms of objectivity and training. In their current form, ANNs cannot replace embryologists; however, they can assist embryologists in supporting clinical decisions [15]. Two firms, from Sweden and Australia, working on IVF, have employed AI-based software to assist in embryo selection procedures; however, these are experimental setups and are in the pipeline for clinical validation [16]. Specifically, segmentation algorithms have substantially aided in the automatic analysis and quantification of various diseases [17]. AI has also helped address infertility by providing several automatic solutions [6,18]. Embryologists confirm the viability of the blastocysts by assessing their potential components: the trophectoderm (TE), blastocoel (BC), zona pellucida (ZP), and inner cell mass (ICM). Figure 1 shows an example of a blastocyst and its corresponding ground-truth image.

The morphological assessment of TE is crucial because of its direct correlation with blastocyst feasibility. TE is mainly responsible for coating cells, which consequently produce fluid, thereby assisting in the formation of the placenta [19]. The creation of BC takes place when an embryo becomes a blastocyst on the fifth day and is one of the potential fluid cavity-based components in the blastocyst [20]. Subsequently, in the ovary, the complete oocyte is encapsulated in the protective layer of the ZP. The ZP plays a key role in the egg and sperm binding processes within the blastocysts. Additionally, the thickness of the ZP decreases with embryo maturity [21]. The ICM is the mass of the cluster and is responsible for fetal structuring. The ICM is usually located at a corner inside the BC [22]. All these potential blastocyst components are strongly correlated with blastocyst viability testing in IVF. However, the aforementioned blastocyst components are typically assessed using time-lapse imaging [23]. The manual observation and examination of blastocysts can suffer significantly from inter-and intraobserver variability. Many studies have emphasized blastocyst assessment to increase the acceptance and success of IVF. During the early cleavage stage, monitoring and examining the TE and ICM are very important, whereas BC can be used to check blastocyst feasibility when the embryo is filled with liquid. AI-based methods, such as convolutional neural networks (CNNs), can increase the effectiveness and efficiency of this evaluation.

Several methods have been proposed as automatic solutions for promoting IVF through blastocyst component assessment. However, the availability of limited data for validating these frameworks remains a challenge. Encouraging improvements in the IVF success rate will motivate health professionals to collect more data for experimentation. Currently, data augmentation is primarily used to limit data availability [3]. Both deep learning and basic image processing-based methods have contributed to enhancing the success rate of IVF. Basic image-processing-based methods suffer from the selection of thresholds, parameters, and features. Although deep feature-based methods provide a satisfactory performance, they typically suffer from extensive computational overhead. In a study [24], a basic image-processing framework was employed for automatic segmentation of TE and ICM components. This method uses texture characteristics and biological properties to detect the desired classes (TE and ICM) using a histogram. Similarly, another study [25] proposed noninvasive imaging and daily zygote image assessment using a particle filter approach for IVF. Filho et al. [26] performed microscopic blastocyst grading by segmenting blastocyst components such as the ZP, TE, and ICM. The inner and outer boundaries of the ZP were detected using ellipse fitting and thresholding, respectively. The TE and ICM were detected using a level-set-based approach. Another study [27] presented an automated method for segmenting the TE region of the embryo. This method employed a level-set algorithm for segmentation, whereas pre- and post-processing were used for refined segmentation.

In learning-based approaches, one study [28] utilized a stacked U-Net framework with a dilation mechanism to segment the ICM region. They improved the accuracy of the segmentation by optimizing the design through the selection of an appropriate dilation factor, kernel size, and model depth. Similarly, Bori et al. (2021) employed an ANN for embryological assessments and used contrast adjustments and normalization for preprocessing. Moreover, the Hough transform was used to segment the image, whereas region and texture analyses were performed using an ANN to measure the components [29]. Rad et al. [6] employed another deep feature-based architecture to segment potential components from blastocyst images. On the encoder side, ResNet-50 was used as the backbone, whereas the cascading of atrous pyramid pooling was used to attain features with different scales. The same researchers also studied a boosting model using an ensemble approach to detect ZP components in blastocyst images. They used patches as input to the model, and a refinement approach based on self-supervision was employed for performance enhancement [30]. Kheradmand et al. (2016) proposed a CNN-based approach for the automatic detection of potential blastocyst components. Blastocyst components were detected using edge detection with preprocessing. Preprocessing was applied to the compressed domain images to extract multiple features. Block variance, edge, inner ZP boundary, and blastocyst center were the main features considered for pre-processing. Block variance was computed to assess the uniformity of blastocyst regions. A schematic technique was employed to classify the edge orientation in each block. Similarly, a process pipeline was developed to detect the inner boundary of the ZP component. Subsequently, the blastocyst center was extracted by averaging the block positions in the inner boundary. Information on the blastocyst center can help with data generalization [31]. Similarly, another method [32] used a deep learning-based model to analyze blastocysts using time-lapse data. In this study, an AI platform with an optimized U-Net architecture was employed to apply semantic segmentation to time-lapse image files. According to this study, the U-Net can be optimized using an AI platform.

In another study [33], a deep-feature-based framework was employed to segment the pixels of the ICM area. A fully convolutional neural network (FCNN) was used to segment the ICM, and the boundary of the ZP was used to remove background pixels. Wang et al. [34] employed a deep-learning model with a VGG-16 network to classify blastocyst images. Moreover, the classification accuracy was improved by ensembling VGG-16 and MobileNetV2. Subsequently, another study [35] proposed an inception U-Net design for segmenting the TE components from blastocysts. In this method, synthetic images were created using a generative method, whereas atrous convolution was used in the proposed design for performance enhancement.

Previous methods typically considered a single class of blastocysts, and most architectures were computationally expensive, requiring a large number of parameters. In addition, blastocyst segmentation is challenging because of the extensive variation in illumination and the high similarity in terms of gray levels among classes. Moreover, the structure of the TE was similar to that of the ICM. All the aforementioned problems make accurate pixel-wise segmentation challenging. Conventional image-processing methods are usually incapable of addressing these challenges. Therefore, deep learning-based methods have been developed for satisfactory multiclass segmentation of blastocyst images. Existing deep-feature-based methods have limitations in terms of their performance and computational requirements. There are several reasons that may account for the performance limitations of existing deep learning methods. Many segmentation methods based on the U-Net or SegNet architectures suffer from vanishing gradient problems and exhibit performance limitations, particularly for minor classes. Similarly, many segmentation networks use excessively small final feature map sizes because of several pooling operations, and consequently suffer from segmenting small-sized objects in the image. Failing to fuse or aggregate low-level image information with high-level information can also be one of the reasons for performance degradation. Moreover, many methods require preprocessing techniques, including, but not limited to, patch-based, artifact-removal, contrast adjustment and normalization, feature extraction, and transformation-based schemes, to achieve satisfactory performance [29,36]. Using this method, we developed an independent network capable of providing accurate blastocyst segmentation without excessive computational or preprocessing requirements. Additionally, the proposed method provides multiclass (BC, TE, ICM, and ZP) segmentation, which aids in the overall decision to confirm blastocyst viability. FSBS-Net can assist embryologists by providing segmented blastocyst components for automatic visual assessment; however, the final decision regarding blastocyst quality and viability is made by the embryologist. Viable blastocysts can be transferred to the ovaries for IVF.

The key contributions of this study are summarized as follows:

• We developed a novel architecture, a feature supplementation-based blastocyst segmentation network (FSBS-Net), for the multiclass segmentation of blastocyst images to confirm blastocyst viability for successful IVF.
• FSBS-Net uses effective feature supplementation (FS) mechanisms at different scales and stages of the network to enhance the blastocyst segmentation performance. In addition, FSBS-Net uses an ascending channel convolutional block (ACCB), which applies ascending channels to the convolved initial potential spatial information and transfers them to the deep stage of the network to reduce spatial loss.
• The proposed method (FSBS-Net) was evaluated using a publicly available blastocyst segmentation database. FSBS-Net exhibited promising performance, requiring a small number of parameters (2.01 million).

The remainder of this paper is organized as follows. Section 2 and Section 3 present the methodology and the results, respectively. A discussion is presented in Section 4, and the conclusions of this study are summarized in Section 5.

2. Material and Methods

2.1. Database

We used the same publicly available dataset as in [24], and to the best of our knowledge, this is the only available open blastocyst segmentation dataset. Because we used an open database, only data division according to the data split suggested by the dataset provider and resizing of the training images were performed for data curation using the MATLAB framework [37]. This database contains 235 Hoffman Modulation Contrast (HMC) blastocyst images. Images were collected from different patients at the Pacific Center for Reproduction (Canada). Images were collected from 2012 to 2016. Image selection was performed randomly, focusing mainly on the ICM and TE components. In addition, blastocyst images were pixel-level annotated by expert embryologists from the dataset provider to create ground-truth images [24]. For a fair comparison, we followed the same official data-splitting protocol used in many studies [6]. We used 200 images (85%) for training and 35 images (15%) for testing. A sample image, along with the corresponding ground-truth image from the blastocyst database, is shown in Figure 1.

2.2. Method

2.2.1. Summary of Proposed Method

An overview of the proposed FSBS-Net is presented in Figure 2. The blastocyst images were fed into the network after resizing the training images for efficient training. The proposed method uses FS at different stages of the network to enhance segmentation accuracy for the prediction of blastocyst components. The grayscale values of the blastocyst components were quite close to each other, making pixel-wise segmentation challenging. FS at different stages and scales overcame this problem and promoted accurate prediction of all blastocyst components. Similarly, the ACCB takes the initial potential information and applies a convolutional operation with ascending channels to minimize the spatial loss. Many existing methods use a single class of blastocyst images, which may be insufficient to confirm blastocyst viability. The proposed FSBS-Net considers all blastocyst classes and generates a multiclass prediction mask. The pixel-level prediction mask is compared with the ground-truth image to assess the performance of the proposed method. In Figure 2, the blastocyst-detected components of ICM, TE, ZP, and BC are shown in blue, red, green, and yellow, respectively.

2.2.2. FSBS-Net Architecture and Working

FSBS-Net is a novel architecture developed for multiclass semantic segmentation of blastocyst images. The pixel-wise segmented blastocyst image helps embryologists understand and quantify the valuable components (TE, BC, ZP, and ICM) of the blastocyst. The proposed method was trained using resized images (400 × 400 pixels) for fast training. The training split was augmented to increase the number of training samples for effective training of the network. No pretraining or weight migration was used to keep the framework less complex and independent. A trained network was applied to the unseen test images, and a prediction mask was generated at the output of the network. This prediction mask is multiclass; therefore, each blastocyst component was simultaneously segmented pixel-wise. Segmentation of blastocyst images can help embryologists analyze blastocyst viability for successful IVF. The architectural design of the FSBS-Net is illustrated in Figure 3.

Images were provided to the network using an image input layer. The input features were passed through a convolutional pipeline consisting of four convolutional layers. In CNNs, the early layers of the network contain initial potential information, and supplementation with this information can help improve the prediction accuracy. Therefore, the initial potential information was supplemented with features from the convolutional pipeline through an identity skip connection. After processing through a convolutional layer, the supplemented features were downsampled using a strided convolutional layer (Strided-Conv). The downsampled spatial features were passed through a second convolutional pipeline with four convolutional layers for refined feature extraction. The spatial dimension of the features was further reduced using another strided convolution followed by two convolutional layers. Finally, the feature map size of the input feature was reduced using the same layer type, and the final feature map size was provided for feature upsampling. Transposed convolutional (Tra-Conv) layers were used to increase the spatial dimensions of the features. The final feature map size was upsampled using back-to-back tra-conv operations, and the upsampled features were fed to FS-2.

FSBS-Net uses ACCB to apply ascending channels to the initial potential information. ACCB is based on strided-Conv along with four convolutional layers with ascending channels. Strided-Conv initially reduces the spatial dimension, and ascending channels from four channels to 64 channels in a doubling sequence are applied to the initial potential information. The ACCB transfers these ascending channel convolutional features to FS-2. In FS-2, the upsampled spatial information and ascending channel convolutional features were supplemented. These features were further provided to the last Tra-Conv for the final feature map size upsampling via a convolutional layer. After feature supplementation in FS-3 and FS-4 (detailed below), the final spatial features were provided for final supplementation at FS-5. The initial potential information obtained from the third layer (ReLU) of the network was processed using a convolutional pipeline comprising five convolutional layers for further feature extraction. This processed initial potential information was also transferred to FS-5. In FS-5, the processed initial potential information was supplemented with the final spatial features to improve the prediction of blastocyst components. The refined supplemented features, after passing from the final convolutional layer, were finally provided to the softmax and pixel classification layers for probability assignment and multiclass predictions in the blastocyst image. Notably, the number of convolutional layers in the deep stages of the network (where the spatial dimension is small, and the number of channels is high) was less than that in the other stages of the network. Computational efficiency is the primary concern when fewer layers are used in the deep stages of a network. Convolutional layers in the deep stage of a network require many trainable parameters. Therefore, FSBS-Net uses fewer convolutional layers to make the network less expensive and more computationally efficient.

The feature supplementation process is further explained using the schematic shown in Figure 4. The ACCB provides ascending channel convolutional features $\left({\mathrm{F}}_{\mathrm{ac}}\right)$ to be supplemented with upsampled spatial information $\left({\mathrm{I}}_{\mathrm{us}}\right)$. FS(x) performs element-wise feature supplementation and provides the supplemented features ${\mathrm{SF}}_{\mathrm{x}}$ as follows:

$${\mathrm{SF}}_{\mathrm{x}}={\mathrm{F}}_{\mathrm{ac}}+{\mathrm{I}}_{\mathrm{us}}.$$

(1)

${\mathrm{SF}}_{\mathrm{x}}$ is further upsampled and processed through the convolutional layer and becomes ${\mathrm{SF}}_{\mathrm{x}}^{*}$. Subsequently, spatial information from the identity skip path $\left({\mathrm{I}}_{\mathrm{si}}\right)$ is supplemented with ${\mathrm{SF}}_{\mathrm{x}}^{*}$. The supplemented features at FS(y) produce SF_y as follows:

$${\mathrm{SF}}_{\mathrm{y}}={\mathrm{I}}_{\mathrm{si}}+{\mathrm{SF}}_{\mathrm{x}}^{*}.$$

(2)

${\mathrm{SF}}_{\mathrm{y}}$ features are further convolved through two convolutional layers, resulting in ${\mathrm{SF}}_{\mathrm{y}}^{*}$. The initial potential information $\left({\mathrm{I}}_{\mathrm{ip}}\right)$ from the early layers of the network is further processed through two convolutional layers and becomes ${\mathrm{I}}_{\mathrm{pi}}^{*}$. At FS(z), ${\mathrm{I}}_{\mathrm{pi}}^{*}$ is supplemented with ${\mathrm{SF}}_{\mathrm{y}}^{*}$ and produces the supplemented features ${\mathrm{SF}}_{\mathrm{z}}$, as follows:

$${\mathrm{SF}}_{\mathrm{z}}={\mathrm{I}}_{\mathrm{pi}}^{*}+{\mathrm{SF}}_{\mathrm{y}}^{*}$$

(3)

Before transferring the supplemented features towards the prediction stage, the supplemented features are further convolved using another couple of convolutional layers, and these convolved supplemented features are denoted as ${\mathrm{SF}}_{\mathrm{z}}^{*}$. Supplementing features at different stages and scales enables the network to learn a variety of features, which consequently results in the effective learning of the network.

2.2.3. Experimental Details and Data Preparation

FSBS-Net was evaluated using a challenging blastocyst database for segmentation of its potential components. Training using the proposed method does not require preprocessing or ensemble steps. FSBS-Net was independently trained from scratch. Cross-entropy (CE) loss [38] was used for fast convergence and the effective training of the network. As shown in Figure 1, the blastocyst images have five classes, including the background (BG) class. Pixels of BC and BG usually dominate other classes, which transforms this scenario into a class-imbalance problem. Median frequency balancing was employed in our experiments to deal with it. Median frequency balancing in combination with weighted cross entropy (CE) loss minimizes the relative loss for well-segmented samples, whereas it enhances the focus on hard mis-segmented samples, as given in [39]. In addition, weights computed using median frequency balancing enable the network to achieve enhanced class-average accuracy [40]. In our experiments, the Adam [41] optimizer was used because of its fast convergence compared with other optimizers. Additionally, a large number of training images were created after augmentation for training purposes. The Adam optimizer is also capable of handling large amounts of data and contributing to effective training in the case of large data. The initial parameters for the learning rate, square gradient decay, and L2 regularization were set to 0.001, 0.95, and 0.0005, respectively. The training loss and accuracy plots in Figure 5 show increasing accuracy with decreasing loss. In the plots, the training loss and accuracy are shown in red and blue, respectively. In the initial iteration, the network faced a larger loss, as shown in Figure 5, and it learned the candidate classes faster to gradually reduce the loss as it learned to identify the distinct classes. As shown in Figure 5, the FSBS-Net was trained for 5001 iterations (in 21 epochs) presented on the x-axis. In the training accuracy–loss plot, the corresponding accuracy and loss data points for all iterations are shown for a detailed analysis.

The proposed FSBS-Net was developed and implemented using MATLAB 2021a [37]. The experimental work was conducted on an Intel Core i7-950 computer with 32 GB of RAM and an NVIDIA GeForce GTX 1080 GPU [42]. In the experimental work, the same data splits, experimental protocols, and evaluation methods were followed for a fair comparison with the existing methods. Early stopping and augmentation were performed to avoid overfitting the training data. Medical images require annotation by experts; therefore, the lack of availability of sufficiently annotated medical data is a common limitation. To overcome these limitations, augmentation was used to create sufficient data for effective training. The data augmentation of a sample image using some of the techniques employed is shown in Figure 6. In this study, the data augmentation involved geometric operations, including flipping, cropping, and translation, followed by resizing. A total of 3200 images were created using 200 images from the training split.

3. Results

3.1. Evaluation Measure

The trained FSBS-Net model was applied to unseen test images to evaluate the proposed method. AI-based methods have also been evaluated using several measures [43]. However, the Jaccard index (JC) is a commonly accepted measure for evaluating semantic segmentation performance [6,44]. In testing, the FSBS-Net generates a binary prediction mask for each class, providing “1” for correct predictions or otherwise “0.” The prediction results were compared with the ground truth, and the final results were computed based on the false positive (fp), true positive (tp), and false negative (fn) pixels. However, existing methods mostly rely on JC to evaluate blastocyst segmentation performance. For a more thorough evaluation, the quantitative results in terms of precision (PRE), recall (RC), and dice similarity coefficient (DSC) are presented in

Table 1. Quantitative results of FSBS-Net using other evaluation measures. (PRE: precision; RC: recall; DSC: dice similarity coefficient) (All reported results in the last six columns are percentages “%”).

Blastocyst Component	PRE	RC	DSC
ZP	97.66	94.01	92.29
TE	97.50	92.21	88.90
BC	97.30	91.20	94.06
ICM	98.73	92.63	92.0
BG	98.10	99.04	97.65
Average	97.85	93.81	92.98

[45,46]. Mathematically, the evaluation measures are expressed as follows:

$$\mathrm{JC}=\frac{\mathrm{tp}}{\mathrm{tp} + \mathrm{fp} + \mathrm{fn}};$$

(4)

$$\mathrm{DSC}=\frac{2\mathrm{tp}}{2\mathrm{tp}+\mathrm{fp}+\mathrm{fn}};$$

(5)

$$\mathrm{PR}=\frac{\mathrm{tp}}{\mathrm{tp}+\mathrm{fp}};$$

(6)

$$\mathrm{RE}=\frac{\mathrm{tp}}{\mathrm{tp}+\mathrm{fn}}.$$

(7)

3.2. Comparing FSBS-Net with Those of the State-of-the-Art Methods

In this subsection, the quantitative results produced by FSBS-Net are compared with those of the state-of-the-art methods. FSBS-Net was evaluated using multiclass blastocyst images. Hence, FSBS-Net generated five binary prediction masks. The prediction masks were compared with the ground-truth images, and the JI score was recorded for every class. In

Table 2. Comparison of quantitative results of FSBS-Net with state-of-the-art methods. (All reported results in the last six columns are percentages “%”).

Method	No. of Parameters	ZP	TE	BC	ICM	BG	Mean JC
UNet-Baseline [47]	31.03 M	79.32	75.06	79.41	79.03	94.04	81.37
TernausNet U-Net [48]	10 M	80.24	76.16	78.61	77.58	94.50	81.42
PSP-Net [49]	35 M	80.57	74.83	79.26	78.28	94.60	81.51
DeepLab V3 [50]	40 M	80.84	73.98	78.35	80.60	94.49	81.65
Blast-Net [6]	25 M	81.15	76.52	80.79	81.07	94.74	82.85
SSS-Net Residual [44]	4.04 M	82.88	77.40	88.39	84.94	96.03	85.93
SSS-Net Dense [44]	4.04 M	84.51	78.15	88.68	84.50	95.82	86.34
ECS-Net [3]	2.83 M	85.34	78.43	88.41	85.26	94.87	86.46
FSBS-Net (Proposed)	2.01 M	85.80	80.17	89.15	85.55	95.62	87.26

, the blastocyst segmentation results generated by FSBS-Net are listed for comparison with state-of-the-art methods. It is evident from Table 2 that FSBS-Net shows better performance than the other methods with a smaller number of trainable parameter requirements. FSBS-Net requires only 2.01 million trainable parameters for complete training. In addition, no complex pretraining or weight migrations were employed to obtain these results. The performance difference gained by FSBS-Net can be attributed to the feature supplementation mechanisms used in its architecture. Considering the challenges associated with the accurate semantic segmentation of blastocysts (discussed in Section 1), improving the segmentation performance for blastocyst components is highly challenging. Although ECS-Net [3] showed the second-best performance in terms of mean JC, the proposed FSBS-Net showed a better segmentation performance for all blastocyst components with a lesser requirement (0.82 million) of trainable parameters compared with ECS-Net. Many studies have reported that TE morphology and formation are important for analyzing embryos in successful IVF [3,19]. FSBS-Net showed a considerable performance improvement for TE compared to the existing methods, including ECS-Net. Better segmentation performance, even with slight improvements in individual components, can provide an overall improved visual assessment and blastocyst area-related computations to assist embryologists.

3.3. Qualitative Results Produced by FSBS-Net for Blastocyst Components Segmentation

FSBS-Net uses five channels in the final convolutional layer of the network. Each channel of the final convolutional layer referred to a class (from five classes) in the blastocyst image. The blastocyst images have five classes: ICM, BC, TE, ZP, and BG. FSBS-Net produces five prediction masks, corresponding to every single class, and assigns “1” for desired class, otherwise “0.” The qualitative results produced by FSBS-Net are shown in Figure 7. The original images, ground-truth images, and images segmented by FSBS-Net are presented in columns 1, 2, and 3 of Figure 7. The predicted pixels of a class were compared with the pixels in the ground truth; hence, a segmented image was formed based on these predictions. In Figure 7, the correctly predicted pixels (tp) of TE, ICM, ZP, BC, and BG are shown in red, blue, green, yellow, and no color (same as the original), respectively. Similarly, fp pixels are shown in black, whereas fn pixels are shown in pink. Overall, the presented qualitative segmentation results confirm the effectiveness of FSBS-Net.

A sample image with relatively poor segmentation using FSBS-Net is shown in Figure 8. The complex and similar structures of TE and ICM with indistinct boundaries in certain areas can be attributed to compromised segmentation performance. However, FSBS-Net maintained class discrimination and managed to provide a competitive performance.

The component-wise segmentation results for the blastocyst images are shown in Figure 9. Two sample images are presented for component-wise segmentation. The original images are shown in the first column of Figure 9a and the segmented components of the blastocyst images are shown in Figure 9b–e. As shown in the component-wise segmentation results, the segmented images were very close to the ground-truth images, confirming the satisfactory performance of the proposed method. In addition, component-wise visual segmentation results can be helpful when an embryologist assesses a specific component.

4. Discussion

Embryologists perform morphological and visual assessments of the blastocyst components to examine blastocyst viability during IVF. Therefore, the accurate detection of blastocyst components is crucial for increasing the success rate of IVF. Pixel-wise segmentation of blastocyst components enables embryologists to acquire the computational details of blastocyst components. Additionally, most existing methods consider a single class in the blastocyst image, whereas FSBS-Net is capable of providing multiclass segmentation. FSBS-Net provides combined segmented blastocyst images for joint assessment and delivers component-wise segmented images for individual component analysis. An assessment of the morphological properties of the blastocyst components is important for confirming the viability of blastocysts for IVF [19,20,21,22,51].

Accurate segmentation, pixelwise, can also provide the correct area for the respective blastocyst components, which can contribute to blastocyst analysis. Similarly, the location properties can be assessed based on the positions of the detected blastocyst components. The BC is one of the most potent components of blastocysts and develops on the fifth day of embryo formation. The predicted mask of the BC component can provide the morphological and computational details of the BC to support embryologists in decision making. The thickness of the ZP component is also valuable for enhancing the success rate of IVF and can be assessed by the output of our proposed method. The existing methods are typically unable to provide satisfactory segmentation accuracy with minimal trainable parameters [4]. As presented in the Results section, FSBS-Net delivered a promising performance for overall blastocyst image segmentation. FSBS-Net exhibits satisfactory segmentation accuracy without excessive computational requirements. FSBS-Net requires only 2.01 million parameters, which was the lowest number in the results listed in Table 2. High-performance segmentation provides more accurate structural, morphological, and computational details of the blastocyst components, which are directly correlated with the success of the IVF process. A few classification-based approaches have been proposed for human blastocyst selection. However, classification algorithms cannot provide pixel-level predictions and other important details such as the morphology, area, and number of pixels in each blastocyst component [52]. As mentioned in the principal findings by Arsalan et al. (2022), the areas, proportions, and locations of blastocyst components can provide significant insights for embryologists in embryo quality assessment [44]. FSBS-Net generates a multiclass prediction mask for each component. Therefore, it can accurately provide the location, area, and proportions of a blastocyst, which can help embryologists in blastocyst assessment. Similarly, Bori et al. (2021) classified the area of the blastocyst and the number of pixels in each component as important factors representing the characteristics of a blastocyst [29]. FSBS-Net provides semantic segmentation and is capable of providing visual details along with computational details, such as the area and number of pixels in each component. The results achieved by the proposed method confirm the effectiveness of our method and can accurately provide morphological details, area, position, and number of pixels for each blastocyst component. The visual results shown in Figure 7 and Figure 9 confirm the high segmentation performance of FSBS-Net, even when dealing with challenging cases (the challenges associated with blastocyst segmentation are discussed in Section 1). Therefore, the proposed method can be used to enhance the efficiency of blastocyst component assessment and it can also help to minimize the subjectivity and error-prone nature of manual assessments. However, the proposed method can assist embryologists but cannot replace them. Final quantitative assessments should be conducted by embryologists for final decisions.

Existing methods typically rely on complex pre-processing requirements. To achieve the reported performance, FSBS-Net uses a simple framework that does not require any preprocessing. In the FSBS-Net, feature supplementation mechanisms at different scales and levels of the network can be the main reason for eliminating the need for preprocessing. The initial potential spatial information is transferred from an early stage of the network to the deep and final stages for feature supplementation, resulting in improved performance. Moreover, the number of convolutional layers in and near the maximum depth of the network (where the number of channels is high) was set to be low to ensure that the network was computationally efficient.

Despite providing a satisfactory performance, the proposed method has a few limitations. The availability of medical data is a common limitation. In addition, annotation of the available data requires medical experts; therefore, sufficient annotated data are not available for effective training of the network. The proposed method (FSBS-Net) used data augmentation schemes to produce the required training data. However, we employed only geometric augmentation, which may result in performance degradation for very small datasets with few images. Moreover, to the best of our knowledge, the database used in this study is the only one available for blastocyst segmentation. In future, we intend to expand our work by collecting more data related to blastocyst segmentation.

5. Conclusions

The rapid increase in infertility rate has triggered the need for alternative means of human fertilization. In vitro fertilization (IVF) is one of the most commonly adopted methods for assisting pregnancy worldwide. However, the success rate of IVF requires significant improvement and automated blastocyst assessment can effectively fill this gap. Morphological assessment of blastocyst components (TE, ICM, ZP, and BC) can provide significant support to embryologists when analyzing the viability of blastocysts in IVF. In this study, a novel architecture, FSBS-Net, was proposed for the semantic segmentation of blastocyst components. FSBS-Net uses feature supplementation at different stages and scales of the network to enhance segmentation accuracy. Moreover, FSBS-Net uses an ascending channel mechanism through the ACCB, which takes the initial potential information and provides it to the deep stage of the network to reduce spatial loss. The proposed method was evaluated using the blastocyst database and exhibited satisfactory performance with a smaller computational overhead. The FSBS-Net uses only 2.01 parameters for its full training. FSBS-Net achieved a mean JC of 87.26% without preprocessing or weight initialization. These results suggest that the proposed method can assist embryologists in confirming blastocyst viability for successful IVF. In the future, we will explore other augmentation schemes, including morphological variations, for blastocyst segmentation tasks. Moreover, we intend to further modify the existing architecture for other medical applications, such as the segmentation of different types of cancers or lesions in the human body. Architectural changes in a network may include adding additional identity skip paths and reducing the number of convolutional layers to increase the computational efficiency of the network.