Next Article in Journal
Quantitative Protein Analysis of ZPB2, ZPB1 and ZPC in the Germinal Disc and a Non-Germinal Disc Region of the Inner Perivitelline Layer in Two Genetic Lines of Turkey Hens That Differ in Fertility
Next Article in Special Issue
Adipose Fin as a Natural “Optical Window” for Implantation of Fluorescent Sensors into Salmonid Fish
Previous Article in Journal
The Growth Performance, Nutrient Digestibility, Gut Bacteria and Bone Strength of Broilers Offered Alternative, Sustainable Diets Varying in Nutrient Specification and Phytase Dose
Previous Article in Special Issue
Acute Stress in Lesser-Spotted Catshark (Scyliorhinus canicula Linnaeus, 1758) Promotes Amino Acid Catabolism and Osmoregulatory Imbalances
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Animals
Volume 12
Issue 13
10.3390/ani12131670
animals-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Automated Cardiac Chamber Size and Cardiac Physiology Measurement in Water Fleas by U-Net and Mask RCNN Convolutional Networks

by
Ferry Saputra
1,2,†,
Ali Farhan
1,2,†,
Michael Edbert Suryanto
1,2,
Kevin Adi Kurnia
1,2,
Kelvin H.-C. Chen
3,
Ross D. Vasquez
4,
Marri Jmelou M. Roldan
5,
Jong-Chin Huang
3,*,
Yih-Kai Lin
6,* and
Chung-Der Hsiao
1,2,7,8,*
1
Department of Chemistry, Chung Yuan Christian University, Taoyuan 320314, Taiwan
2
Department of Bioscience Technology, Chung Yuan Christian University, Taoyuan 320314, Taiwan
3
Department of Applied Chemistry, National Pingtung University, Pingtung 90003, Taiwan
4
Department of Pharmacy, Research Center for Natural and Applied Sciences, University of Santo Tomas, Manila 1008, Philippines
5
Faculty of Pharmacy, The Graduate School, University of Santo Tomas, Manila 1008, Philippines
6
Department of Computer Science, National Pingtung University, Pingtung 90003, Taiwan
7
Center for Nanotechnology, Chung Yuan Christian University, Taoyuan 320314, Taiwan
8
Research Center for Aquatic Toxicology and Pharmacology, Chung Yuan Christian University, Taoyuan 320314, Taiwan
*
Authors to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Animals 2022, 12(13), 1670; https://doi.org/10.3390/ani12131670
Submission received: 29 May 2022 / Revised: 22 June 2022 / Accepted: 27 June 2022 / Published: 29 June 2022
(This article belongs to the Special Issue Applications of Physiological Tools to Assess the Welfare of Crustaceans, Mollusks and Fish)
Browse Figures
Review Reports Versions Notes

Abstract

:

Simple Summary

With the rapid development of technology, artificial intelligent become a major breakthrough that can help human with laborious job. Previously cardiac imaging in Daphnia was also suffer from laborious and tedious process to extract some information from it. Thus the aim of this study was to develop a simple artificial intelligent based method to help anyone in this field to perform analysis in fast, reliable, and less tedious manner. In this study, we compare U-Net and Mask RCNN and found out that Mask RCNN was perform better than U-Net in cardiac chamber area estimation. From this data, several parameter like heart rhythm, stroke volume, ejection fraction, fractional shortening, and cardiac output can be extracted. The validation was done by comparing the normal and Roundup exposed group and it show that Roundup can increase the stroke volume, cardiac output, and the shortening fraction of Daphnia magna.

Abstract

Water fleas are an important lower invertebrate model that are usually used for ecotoxicity studies. Contrary to mammals, the heart of a water flea has a single chamber, which is relatively big in size and with fast-beating properties. Previous cardiac chamber volume measurement methods are primarily based on ImageJ manual counting at systolic and diastolic phases which suffer from low efficiency, high variation, and tedious operation. This study provides an automated and robust pipeline for cardiac chamber size estimation by a deep learning approach. Image segmentation analysis was performed using U-Net and Mask RCNN convolutional networks on several different species of water fleas such as Moina sp., Daphnia magna, and Daphnia pulex. The results show that Mask RCNN performs better than U-Net at the segmentation of water fleas’ heart chamber in every parameter tested. The predictive model generated by Mask RCNN was further analyzed with the Cv2.fitEllipse function in OpenCV to perform a cardiac physiology assessment of Daphnia magna after challenging with the herbicide of Roundup. Significant increase in normalized stroke volume, cardiac output, and the shortening fraction was observed after Roundup exposure which suggests the possibility of heart chamber alteration after roundup exposure. Overall, the predictive Mask RCNN model established in this study provides a convenient and robust approach for cardiac chamber size and cardiac physiology measurement in water fleas for the first time. This innovative tool can offer many benefits to other research using water fleas for ecotoxicity studies.

1. Introduction

Cladocera, commonly known as water fleas, belong to the subclass Phyllopoda of the class Crustacea [1]. They are lower invertebrate animals often shaped like flat disks with small sizes ranging from 0.2 to 3 mm in length. They are widely distributed in the global freshwater ecosystem and have a vital role in the aquatic food chain as one of fish’s primary natural food sources [2]. Cladocera is an essential component of microcrustacean zooplankton, which are sensitive indicators of environmental changes. They provide early warnings by demonstrating a prompt response to environmental changes [3].
Most of the order Cladocera belongs to the suborder Anomopoda, which is principally comprised of the families Daphniidae (the Genus Daphnia) and Moinidae (Genus Moina) [1], which are the common water flea crustacea used for ecotoxicological testing [4,5,6,7]. With sufficient food and aeration, they are easily cultured in the laboratory. High availability, rapid reproduction, and economic feasibility are the reasons for selecting this animal model for a toxicity test [8]. They have high sensitivity toward various chemical pollutants found in the environment, such as metals, pesticides, and pharmaceuticals [9,10,11,12]. The Organization for Economic Cooperation and Development (OECD) has described the international guidelines of standard bioassays using freshwater cladocerans to determine the toxicity of chemicals and pollutants [13]. The ecotoxicological assessment is based on easily measurable endpoints such as lethality, growth, reproduction, immobilization, or behavior [14]. Another important parameter, cardiovascular function, is also usually used as an indicator of toxicity evaluation. A previous study displayed that the water flea’s heartbeat, cardiac output, and heartbeat regularity are significantly reduced when exposed to the pesticide imidacloprid [15]. It is plausible to detect and measure cardiovascular performance with their body transparency, which is suitable for ecotoxicity assay [16,17].
Digital imaging processing (DIP) is a well-known subject for analyzing image datasets [18]. Unlike task-based methods, deep learning (DL) comprises machine learning data processing [19]. It has efficient results, especially on larger datasets. The big data world is using DL for many of its applications. It has been considered that neural networks are significantly designed to converge computer vision programs with DL applications. The traditional approaches in DL are used for video sequencing and image processing. Digital data acquisition is a common practice in medical informatics with the help of DL. Heart chamber size measurement and volumetric behaviors are established using the DL models. Two-dimensional convolutional neural networks (CNN) are the principle neural network extension that converges pixel-based instant segmentation. Kernel slides with 2D networks convolve the layer to the previous with height and width, which is used to extract the features of the image. The Deep Convolutional Neural Networks (DCNNS) are integrated with high-level performance in image classification with soaring heights array [20].
Good classification and segmentation algorithms always help researchers find accurate test cases in the desired image. In the field of computer vision, several different methods such as U-Net [21,22], Mask RCNN [23], and YOLO [24,25] have been developed in the past decade, which put the image classification world in more conducive manners [26]. In previous studies conducted by Karatzas et al. [27], the Mask RCNN method has been tested to detect heart malformation in Daphnia. Zhao et al. also utilized the U-Net architecture to segment cardiac chamber in magnetic resonance images [28]. A further study by Dong et al. showed that U-Net could also be used for video segmentation for cardiac MRI video with state-of-the-art performance [29]. All those studies have proven that the deep learning approach was beneficial for cardiac chamber segmentation.
Previous studies measured the cardiac performance endpoints in water fleas based on image-based methods (Table 1). Multiple endpoints such as heart rate, cardiac output, ejection fraction, fractional shortening, and heartbeat regularity can be extracted. Heart size, contraction capacity, and heart shape malformation due to toxicants have also been studied [17]. All these endpoints were achieved based on the manual analysis of heart images and calculating the cardiac chamber size during heart relaxation (diastole) and contraction (systole) [30]. Therefore, cardiac physiology, as well as morphology observation, is fundamental in cardiac performance evaluation. Both quantitative and qualitative methods are essential to detect possible alterations of heart function in this animal model due to exposure to hazardous or toxic substances within the ecotoxicology field. Based on Table 1, we learned that most of the previous cardiac physiology measurement methods were conducted using manual counting or ImageJ-based methods. However, slow and complex operation processing is the major drawback that makes previous methods unsuitable for high-throughput toxicity assessment. Thus, the incorporation of deep learning will really benefit everyone in this feld of study as a fast and reliable performance can be achieved with help of machine learning.
As mentioned above, the drawback of the previous assessment method was the slow and complex operation. In addition, it is also prone to human error as most of the process was conducted by human operator. Thus, the main focus of this study was to develop a machine learning based tool to undertake the assessment of cardiac physiology in water fleas. This study will compare the performance of U-Net and Mask RCNN on cardiac image segmentation for water fleas after being trained with a video of water fleas’ heart chamber. Later the optimized convolutional networks will be used to predict the cardiac chamber size of water fleas, and from this, data several cardiac physiology parameters can be obtained. The successful establishment of this novel approach can significantly boost the throughput for cardiac physiology and toxicity studies in water fleas while also reducing the possibility of error caused by human mistake.

2. Materials and Methods

2.1. Water Flea Culture

This study used three water flea species Daphnia magna, Daphnia pulex, and Moina sp. Cultures of D. magna and D. pulex were obtained from the National Chiayi University stock center, and Moina sp. was obtained from the National Pingtung University stock center. The water fleas were kept in 10 L plastic tanks and supplied with baker yeast as food, and the temperature was maintained at ±24 °C. To maintain the healthy culture conditions, half of the old culture water was removed weekly and replaced with freshwater.

2.2. Chemical Exposure

The roundup (41% w/v) was purchased from Yih Fong Chemical Corp. (Taichung, Taiwan) and then diluted into 1000 ppm stock concentration using ddH2O and kept at 4 °C until the time of exposure. At the time of exposure, D. magna was placed into a 5 cm petri dish. The stock solution was further diluted using Daphnia culture water until the concentration of 5 ppm. The exposure was performed for 24 h. The Daphnia culture water was used to reduce the shock due to sudden water temperature change caused by the different water used for the experiment. All protocols and procedures involving Daphnia were approved by the Committee for Animal Experimentation of the Chung Yuan Christian University (Approval No. 109001, issue date 15 January 2020).

2.3. Video Acquisition

For cardiac chamber size estimation, the videos of water fleas were captured using a high-speed charged coupled device (CCD) camera (AZ Instrument, Yuyao, Taiwan), mounted onto an inverted microscope (Sunny Optical Technology, Yuyao, China). The LPlan objective lens with 20× magnification was used to capture the video at high quality. Videos were recorded for 10 s at a frame rate of 200 frames per second (fps) and later converted to 30 fps to create slow-motion videos with a total of 2000 frames using HighBest Viewer software (AZ Instrument, Taichung City, Taiwan). For the mounting solution, 3% methylcellulose was used to immobilize water fleas before recording the cardiac chamber.

2.4. Training Dataset Preparation

After recording, the video format was converted into .avi format with VirtualDub software (Available online: http://www.virtualdub.org/, accessed on 26 June 2022) and then processed using ImageJ [39]. A total of ten frames were selected from the video as the dataset and outputted in .png format at an interval of 200 frames for each video. The training dataset was prepared by marking the image extracted from the video using ImageJ. The border of the heart chamber was marked with pencil tools using ImageJ, and the marked images were saved in .png format (Figure A1). When the image set was not big enough, image augmentation was used to increase the size of the training image [40,41]. The image augmentation methods performed included cropping, flipping, grid distortion, elastic transform, optical distortion, and brightness contrasting [42]. In total, 600 images were included as a training dataset, and manual labeling was performed on the given dataset. The training dataset was used separately to train both Mask-RCNN and U-Net models. The code used for

2.5. Performance Validation

The four following parameters were used for deep learning performance evaluation: Dice coefficient, Intersection over Union (IOU), sensitivity, and specificity. Given the number of true positives (TP), false positives (FP), and false negatives (FN) in the pixel-wise classification of the predicted mask, the Dice coefficient is defined as Dice = 2 T P 2 T P + F P + F N . The IOU is defined as I O U = T P T P + F P + F N . The sensitivity (also called recall) is defined as sensitivity = T P T P + F N . The specificity is defined as specificity = T N T N + F P . Those four important parameters were collected and measured for the U-Net and Mask RCNN methods.

2.6. Volumetric Estimation of Water Flea’s Heart

In calculating the volume of the water flea heart chamber, the Open source Computer Vision (OpenCV) library was used for ellipse fitting and point detection of the long and short axis of the water flea’s heart chamber [43]. The black-white frame-by-frame image exported from Mask-RCNN was used as the basis for volumetric assessment to reduce the background noise present in the original image. The basic concept was initiated by finding contour boundaries using Python’s fit ellipse function [44]. Contour is defined as the line surrounding the water flea’s heart chamber. Contour can be presented in multiple ways as polygons and Freeman chain codes [45]. In OpenCV, contours are mostly computed using binary images as it was easier to define contrast in respective images. Once contour was computed, the next step was to fit an ellipse with a diameter range of the water flea’s heart chamber. Cv2.fitEllipse function in OpenCV was used to estimate the x and y coordinates of the ellipse diameter [46]. Finally, the short and long axes that fit the heart chamber area were extracted and exported into .csv format automatically in the respective folder (Figure A2).

2.7. Computer Hardware Requirement

The proposed experimental design was implemented using the deep learning library Pytorch [47] on a desktop computer running the Linux operating system with AMD Ryzen 9 5900X Computer Processing Unit (CPU), 128- gigabyte Random Access Memory (RAM), 64 Terrabyte hard disk storage, and RTX3090 24G VRAM Graphic Processing Unit (GPU). Although a lower computer specification could also be used for the study, a high-speed GPU card is necessary to complete the design and testing of the model faster.

2.8. Cardiac Performance Analysis

In validating this newly developed method in detecting the cardiac rhythm, the heart rate and the heart rate variability were calculated and then compared with the previously published ImageJ method [15]. In calculating the heart rate using the newly developed method, the size of the heart chamber from time to time was exported and then processed using OriginPro 2019 software (Originlab Corporation, Northampton, MA, USA). The timing of the heart muscle relaxation was extracted using the OriginPro 2019 software, and the average time interval between each relaxation was calculated. Later the heart rate per minute was calculated by dividing 60 with the average time interval. Calculating the heart rate using the method was performed by detecting the difference in brightness at the Region of Interest (ROI). In this case, the heart chamber using Time Series Analyzer V3 Plugin is available in ImageJ software (https://imagej.nih.gov/ij/plugins/time-series.html, accessed on 26 June 2022). Later the data were also processed similarly by using OriginPro 2019 software to get the timing of heart muscle relaxation. The Poincare Plot Plugin on OriginPro 2019 software was used to calculate the heart rate variability, and the sd1 and sd2 generated were noted and compared statistically. In calculating the cross-section area change, the following formula was used: E D V   a r e a − E S V   a r e a E D V   a r e a × 100 % . (EDV, end diastolic volume; ESV, end systolic volume). Other cardiac physiology parameters such as stroke volume, cardiac output, shortening fraction, and ejection fraction were calculated using the same concept and formulas described in the previous study [15].

2.9. Statistical Calculation

Statistical analysis was performed using GraphPad Prism (GraphPad Inc., La Jolla, CA, USA). Depending on the data distribution, a paired t-test or Wilcoxon test was performed to calculate the statistical significance between the method used, and a student t-test and a Mann–Whitney test was performed to calculate the significance between the control and the treatment group [48,49].

3. Results

3.1. Overview of Experimental Design and Training Dataset Preparation

In this study, the collection of water flea heartbeat videos was captured by using high-speed CCD. The high-speed CCD setup with 200 fps frame rate can capture superfast heartbeat in water fleas with less information loss [15]. Later, images were frame-by-frame outputted from the video, and the heart’s outlooking was manually labeled as a training dataset. Two predictive models of mrcnn_predict.py (source code is provided in Supplementary File S1) and unet_predict.py (source code is provided in Supplementary File S2) were established after extensively training by Mask RCNN or U-net methods, respectively. The performance of both predictive models was tested to predict the heart size (area) of the water flea. Once we got the heart area prediction data, the major and minor axis length could be extracted using the Cv2.fitEllipse function (axis.py, source code is provided in Supplementary File S3) in OpenCV. Finally, cardiac physiology endpoints such as cardiac rhythm, heartbeat regularity, heart 2D/3D area, cardiac output, stroke volume, fractional shortening, and ejection fraction, could be obtained by using the mathematic calculation function in excel (the entire experimental design is summarized in Figure 1).

3.2. Training and Validation Performance

We evaluated deep learning’s performance by comparing the dice and loss curves in the training and validation processes. In order to train our U-Net and Mask RCNN models, we prepared 540 D. magna images and their ground truth segmentation images as the dataset. In the training stage, 60 of the 540 images in the dataset were used as the validation set, and the remaining 480 images were used as the training set. Each image in the validation and training set was augmented into 15 images using image processing methods such as affine transform, rotation, flip, and deformation. Therefore, there are 7200 images for training and 900 images for verification. For training by the U-Net method, the dice coefficient increased exponentially from the first steps and reached the maximal plateau after about 4300 steps. The loss curve reached the minimal level after about 3000 steps. The dice coefficient reached the maximal plateau after about 3600 steps for validation. Similarly, the loss curve showed a minimal level of about 3600 steps (Figure 2A). For validation by the Mask RCNN method, the dice coefficient gradually increased starting from the first step and reaching the maximal plateau after about 25,000 steps. The loss curve also appeared from the first step and reached the minimal level after 40,000 steps (Figure 2B). The high dice coefficient for both training (Figure 2C) and validation (Figure 2D) processes suggests that the trained U-Net and Mask RCNN might predict heart size for D. magna with high accuracy.

3.3. Testing Process

To test whether the optimized training networks for either U-Net or Mask RCNN are suitable for heart size prediction in water fleas, we initially used 60 images from D. magna as a testing dataset. We also determined whether the optimized networks have broad application utility to predict heart chamber size for other water flea species. Four parameters in terms of Dice coefficient, IOU, sensitivity, and specificity were used for performance evaluation, given the number of true positives (TP), false positives (FP), and false negatives (FN) in the pixel-wise classification of the predicted mask. Figure 3 shows a comparison of the prediction power between Mask RCNN (Figure 3A) and U-Net (Figure 3B). The prediction results were consistent with the ground truth in three tested water fleas using Mask RCNN. On the contrary, only D. magna showed consistent results between prediction and ground truth by the U-Net method. The other two closely related water fleas of D. pulex and Moina sp. displayed inconsistent ground truth and prediction results. Therefore, Mask RCNN’s performance is better than U-Net for heart size prediction in water fleas.
Next, we conducted a detailed quantitative comparison of the heart size prediction performance between Mask RCNN and U-Net among three water fleas in terms of Dice coefficient, IOU, sensitivity, and specificity. Results showed that the optimized Mask RCNN model trained by D. magna performed well and predicted heart chamber size with good performance for D. pulex and Moina sp. based on the high score obtained from Dice coefficient, IOU, sensitivity, and specificity (Table 2). On the contrary, U-net only displayed a qualified prediction power on D. magna, and relatively low prediction power on D. pulex and Moina sp. For example, based on IOU, Mask RCNN maintained relatively high scores of 0.940 ± 0.015, 0.919 ± 0.020, and 0.874 ± 0.083 for D. magna, D. pulex, and Moina sp., respectively. For U-Net, the IOU scores sharply declined from 0.872 ± 0.070 for D. magna to 0.707 ± 0.161 for D. pulex and 0.526 ± 0.228 for Moina sp. Therefore, the optimized Mask RCNN model established in this study can be applied to water flea’s heart size prediction with high accuracy which is supported by statistical analysis.

3.4. Analysis of Heart Cardiac Size Change over Time in Three Water Fleas by Mask RCNN

Since the trained Mask RCNN networks displayed good performance on heart size prediction in three tested water fleas of D. magna, D. pulex, and Moina sp., we evaluated whether it can be applied to measure cardiac rhythm and cardiac size change over time. We first outputted a 10 s high-speed video at 200 fps frame-by-frame into the image series to reach this goal. Later, 2000 frames were output as a testing dataset to perform heart size prediction using Mask RCNN. Finally, the cardiac rhythm plot was generated by plotting heart size as the y-axis and time as the x-axis. Using this approach, we found that the trained Mask RCNN network works as a universal tool for heart size prediction and cardiac rhythm detection for all three water flea species tested in this study (Figure 4). By analyzing the heart chamber size variation over time, it was able to recapitulate heartbeat rhythm for three tested water fleas. For example, D. magna (455 bpm, pink color) and D. pulex (460 bpm, green color) were found with a similar level of heartbeat rate which was faster than that detected in Moina sp. (208 bpm, blue color) (Figure 4). Videos for heart chamber prediction of D. magna, D. pulex, and Moina sp. can be found in Videos S1–S6.

3.5. Validation of Cardiac Physiology Alterations in D. magna after Herbicide Exposure

We performed data validation for our newly developed deep learning method and previously reported ImageJ method for the next step. The result showed that no significant difference in heart rate was found in both the control (p = 0.9677) and roundup (p = 0.4982) treatment between deep learning and ImageJ-based methods (Figure 5A). However, after comparing the cross-sectional area change using our developed method and manual counting using ImageJ, we found that the deep learning method predicted heart size was significantly larger than the manual method conducted by ImageJ for both the control (p = 0.003) and Roundup treatment (p = 0.0125) (Figure 5B). We estimated they were around 6–8% bigger when the deep learning method was conducted. In the case of heart rate variability, no significant different was found in the sd1 (Control p = 0.2026 & Roundup p = 0.5144) (Figure 5C) and sd2 (Control p = 0.4458 & Roundup p = 0.3119) (Figure 5D) in both the treatment groups. These results strongly suggest that the newly developed deep learning method can give similar results to the already established ImageJ method except in the cross-sectional area change.
The last step was performed by analyzing other cardiac physiology parameters such as the stroke volume, ejection fraction, shortening fraction, and cardiac output by using the ellipse fitting from the OpenCV and point detection to get the distance of the long and short axes of the heart chamber. After getting the distance of both chambers, the calculation was performed by assuming the heart chamber has a three-dimensional ellipsoid shape, and the body size was normalized the volume to get a more accurate measurement of the heart pumping ability [15]. In this study, we observed a significant increase in the stroke volume (p < 0.0001), cardiac output (p < 0.0001), and the shortening fraction (p < 0.0078) of D. magna after incubation of 5ppm Roundup (Figure 6A,B,D). However, no significant difference was observed in the ejection fraction after incubation in 5 ppm of Roundup compared to the control, which suggests that at 5 ppm concentration, Roundup can increase the heart size of D. magna without significantly changing the pumping capability of the heart chamber.

4. Discussion

Several methods currently available in the literature may be suitable for marking the edge of the heart chamber. In this study, U-Net [21,22] and Mask RCNN [23] methods were adopted to conduct cardiac size prediction in water fleas as those two tools have been successfully reported to perform heart segmentation in humans [50,51,52,53]. We discovered that the heart chamber detection in water fleas using Mask RCNN shows superior performance in comparison to U-Net. U-net is a generic deep-learning solution for image detection and segmentation and can be used for biomedical image data analysis. U-Net uses the U-shaped network structure first to capture the features of the images and reconstruct the required partitions based on these features [22]. The U-Net architecture contains a U shape path. The image enters one end of the U shape network to go through the encoding part of the network (also called the contracting path). The encoding part is used to capture the features of the input images, and during this step, the spatial information is reduced while the feature information is increased. Then the information of the input image is sent to the decoding part (also called the expansive path). At the expansive path, the feature collected during the contracting step was combined through a high-resolution sequence of up-convolutions and concatenations to construct the segmentation image [22]. Therefore, U-Net’s method of fusing low-level and high-level image features may have a chance to mark the heart chamber’s edge successfully.
Unlike U-Net, Mask RCNN was used to solve instance segmentation problems in computer vision. Mask RCNN consists of two stages. The first step is to propose the region which contains the object based on the input image. Then in the second stage, it will predict the class of object, make the boundaries of the object, and generate a mask at pixel level based on the proposal from the first stage [54]. Compared to U-Net, which performs better on semantic segmentation, Mask RCNN performs better than U-net, for instance, in segmentation [55]. Several studies suggest that the combination of Mask RCNN and U-Net can outperform a single component only, which could be explored more to increase the accuracy of the image segmentation [56,57,58].
Our training result suggests that the Mask-RCNN model in this study performs better than the U-Net model in defining the heart chamber boundary. A similar case happened when both models were challenged to perform nuclei segmentation, which shows that Mask-RCNN has better precision than U-Net [59]. Another study also reported that U-Net yielded more false-positive results when detecting the immunofluorescence images of kidney biopsies from lupus nephritis patients than Mask-RCNN [60]. Although U-net has higher accuracy, the limitation of U-Net observed in the study was the obstruction of detection when labeled data has edge noise, thus yielding false-positive segmentation [59,61]. However, it can be said that both U-Net and Mask-RCNN models might have similar outputs if the training dataset has more viable point boundary determination. In conclusion, the accuracy and effectiveness of each model depend upon the targeted images that are being classified and segmented.
Another thing that could be observed in this study is the significant difference between manual counting using ImageJ and the Mask RCNN-based method to calculate the cross-section area change of the cardiac chamber. This might be because by using Mask RCNN, the calculation of the cross-sectional area change was based on the maximum and minimum cross-sectional area of the heart chamber from the whole cardiac cycle recorded in the video. On the contrary, the calculation was only based on randomly selected images from a few cardiac cycles for manual counting. Thus, the calculation using manual counting has a lower cross-sectional area change than the automated Mask RCNN method.
It is also worth noting that some limitations were noticed in this deep learning-based study. As the developed tool was based on detecting the edge of the heart chamber, the clarity of the heart chamber edge plays a crucial role in the accuracy of cardiac chamber prediction. In the case of Daphnia and Moina sp., the brood chamber was located right beside the heart chamber. Thus if the samples are in the breeding period, the accuracy of the tools would be compromised when the brood was positioned overlapping the heart chamber (Figure A3 & Video S7) [62]. Another type of limitation is the presence of some parasites near the heart chamber. This problem could also reduce the clarity of the heart chamber, making it harder for deep learning to analyze the heart chamber [63]. Compared to the ImageJ-based method in which the selection of ROI can be made anywhere as long as it has distinct dynamic pixel change, our tool only works accurately if the whole heart chamber can be detected. However, this problem can be overcome by selecting the water fleas with an empty brood chamber or those which are parasite free, ensuring that heart chamber edge is undisturbed.

5. Conclusions

In conclusion, it can be said that Mask-RCNN performed better than U-Net in estimating D. magna cardiac chamber size. Higher Dice coefficient, specificity, and sensitivity show that the Mask-RCNN model was superior in Daphnia cardiac chamber segmentation. Furthermore, the segmentation result could also be used to calculate several cardiac performance parameters such as heart rate, heart rate variability, and cross-sectional area change, while with the addition of the ellipse fitting function from OpenCV, more parameters such as stroke volume, cardiac output, shortening fraction, and ejection fraction could also be calculated. Several limitations such as the dependency on the heart chamber clarity and the presence of obstruction nearby the heart chamber could decrease the accuracy of the segmentation, but both problems could be solved by selecting and sorting the Daphnia before recording the heart chamber. Overall, this study suggests that deep learning could help analyze cardiac physiological parameters in Daphnia. Although in this study, fully automated analysis was not yet achieved, in the future, fully automated high throughput analysis could be achieved by incorporating essential function in several software by using our trained network as the foundation.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ani12131670/s1. File S1. Python script used to calculate heart size of water fleas by Mask-RCNN. File S2. Python script used to calculate heart size of water fleas by U-Net. File S3. Python script used to calculate the length of major and minor axis of heart in water fleas. Video S1. D. magna heartbeat video deposited at YouTube: https://www.youtube.com/watch?v=NwB-Hyf47qo (accessed on 29 July 2021). Video S2. D. pulex heartbeat video deposited at YouTube: https://www.youtube.com/watch?v=jTCGQ8yzZbM (accessed on 29 July 2021). Video S3. Moina sp. heartbeat video deposited at YouTube: https://www.youtube.com/watch?v=LjX1xQXG38I (accessed on 29 July 2021). Video S4. D. magna heartbeat video tracked with Mask RCNN deposited at YouTube: https://www.youtube.com/watch?v=DBuLshjuMTU (accessed on 29 July 2021), Video S5. D. pulex heartbeat video tracked with Mask RCNN deposited at YouTube https://www.youtube.com/watch?v=RGf5naSgcVE (accessed on 29 July 2021), Video S6. Moina sp. heartbeat video tracked with Mask RCNN deposited at YouTube https://www.youtube.com/watch?v=RVRKXvpFq74 (accessed on 29 July 2021). Video S7. Bias for heart size prediction in Moina sp. by Mask RCNN. In this video, the heart size prediction power will be interfered by embryos in the breeding chamber due to loss of sharp edge of heart. Video has been deposited at YouTube https://www.youtube.com/watch?v=MoRyMSYCNNY accessed on 29 July 2021).

Author Contributions

Conceptualization, F.S., J.-C-H., Y.-K.L. and C.-D.H.; methodology, A.F., M.E.S. and K.A.K.; software, Y.-K.L.; validation, M.E.S., K.A.K., R.D.V. and M.J.M.R.; formal analysis, K.H.-C.C. and J.-C.H.; investigation, F.S.; resources, K.H.-C.C.; writing—original draft preparation, F.S. and C.-D.H.; visualization, C.-D.H.; supervision and funding acquisition, J.-C.H. and C.-D.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Ministry of Science Technology, Taiwan, grant numbers MOST107-2622-B-033-001-CC2 and MOST108-2622-B-033-001-CC2 to C.-D.H.

Institutional Review Board Statement

All protocols and procedures involving Daphnia were approved by the Committee for Animal Experimentation of the Chung Yuan Christian University (Approval No. 109001, issue date 15 January 2020).

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.

Appendix A

Animals 12 01670 g0a1 550
Figure A1. The image of the heart chamber of D. magna before (A) and after (B) manual marked with red color line.
Figure A1. The image of the heart chamber of D. magna before (A) and after (B) manual marked with red color line.
Animals 12 01670 g0a1
Animals 12 01670 g0a2 550
Figure A2. Detection of the short (minor) and long (major) axes of the heart chamber using fit ellipse function in OpenCV. The heart images at either diastolic or systolic phases were used for long and short axis length extraction. The binary images before (left panel) and after (right panel) performing the Cv2.fitEllipse function in OpenCV were listed for side-by-side comparison. Scale bar was provided for size estimation.
Figure A2. Detection of the short (minor) and long (major) axes of the heart chamber using fit ellipse function in OpenCV. The heart images at either diastolic or systolic phases were used for long and short axis length extraction. The binary images before (left panel) and after (right panel) performing the Cv2.fitEllipse function in OpenCV were listed for side-by-side comparison. Scale bar was provided for size estimation.
Animals 12 01670 g0a2
Animals 12 01670 g0a3 550
Figure A3. The image of Moina sp. shows the limitation of the tool. The heart chamber detection accuracy was lower if the brood was presented in the brood chamber (A,B) compared to the one without brood in the chamber (C), (D–F) brood chamber.
Figure A3. The image of Moina sp. shows the limitation of the tool. The heart chamber detection accuracy was lower if the brood was presented in the brood chamber (A,B) compared to the one without brood in the chamber (C), (D–F) brood chamber.
Animals 12 01670 g0a3

References

  1. Smirnov, N.N. Physiology of the Cladocera; Academic Press: New York, NY, USA, 2017. [Google Scholar]
  2. Karuthapandi, M.; Rao, D. Cladoceran diversity, distribution and ecological significance. In Arthropod Diversity and Conservation in the Tropics and Sub-Tropics; Springer: Berlin/Heidelberg, Germany, 2016; pp. 183–196. [Google Scholar]
  3. Berta, C.; Tóthmérész, B.; Wojewódka, M.; Augustyniuk, O.; Korponai, J.; Bertalan-Balázs, B.; Nagy, A.S.; Grigorszky, I.; Gyulai, I. Community response of cladocera to trophic stress by biomanipulation in a shallow oxbow lake. Water 2019, 11, 929. [Google Scholar] [CrossRef] [Green Version]
  4. Iwai, C.B.; Somparn, A.; Noller, B. Using zooplankton, Moina micrura Kurz to evaluate the ecotoxicology of pesticides used in paddy fields of Thailand. In Pesticides in the Modern World—Risks and Benefits; INTECH Open Access Publisher: Rijeka, Croatia, 2011; pp. 267–280. [Google Scholar]
  5. Lee, S.; Jung, D.; Kho, Y.; Ji, K.; Kim, P.; Ahn, B.; Choi, K. Ecotoxicological assessment of cimetidine and determination of its potential for endocrine disruption using three test organisms: Daphnia magna, Moina macrocopa, and Danio rerio. Chemosphere 2015, 135, 208–216. [Google Scholar] [CrossRef] [PubMed]
  6. Nam, S.-H.; Shin, Y.-J.; An, Y.-J. Accelerated ecotoxicity of photoreactive nanoparticles on Moina macrocopa. Environ. Health Toxicol. 2017, 32, e2017007. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Tkaczyk, A.; Bownik, A.; Dsudka, J.; Kowal, K.; Ślaska, B. Daphnia magna model in the toxicity assessment of pharmaceuticals: A review. Sci. Total Environ. 2020, 763, 143038. [Google Scholar] [CrossRef] [PubMed]
  8. Harris, K.D.; Bartlett, N.J.; Lloyd, V.K. Daphnia as an emerging epigenetic model organism. Genet. Res. Int. 2012, 2012, 147892. [Google Scholar] [CrossRef] [Green Version]
  9. de Oliveira, L.L.D.; Antunes, S.C.; Gonçalves, F.; Rocha, O.; Nunes, B. Acute and chronic ecotoxicological effects of four pharmaceuticals drugs on cladoceran Daphnia magna. Drug Chem. Toxicol. 2016, 39, 13–21. [Google Scholar] [CrossRef]
  10. Pociecha, A.; Wojtal, A.Z.; Szarek-Gwiazda, E.; Cieplok, A.; Ciszewski, D.; Kownacki, A. Response of Cladocera fauna to heavy metal pollution, based on sediments from subsidence ponds downstream of a mine discharge (S. Poland). Water 2019, 11, 810. [Google Scholar] [CrossRef] [Green Version]
  11. Sarma, S.; Nandini, S. Review of recent ecotoxicological studies on cladocerans. J. Environ. Sci. Health Part B 2006, 41, 1417–1430. [Google Scholar] [CrossRef]
  12. Suhett, A.L.; Santangelo, J.M.; Bozelli, R.L.; Steinberg, C.E.W.; Farjalla, V.F. An overview of the contribution of studies with cladocerans to environmental stress research. Acta Limnol. Bras. 2015, 27, 145–159. [Google Scholar] [CrossRef] [Green Version]
  13. [OECD] Organization for Economic Co-operation and Development. Guideline for testing of chemicals. Daphnia sp., Acute immobilisation test. In OECD Guidel. No 202; OECD: Paris, France, 2004. [Google Scholar]
  14. Peake, B.M.; Braund, R.; Tong, A.Y.; Tremblay, L.A. Impact of pharmaceuticals on the environment. In The Life-Cycle of Pharmaceuticals in the Environment; Peake, B.M., Braund, R., Tong, A.Y.C., Tremblay, L.A., Eds.; Woodhead Publishing: Sawston, UK, 2016; pp. 109–152. [Google Scholar]
  15. Santoso, F.; Krylov, V.V.; Castillo, A.L.; Saputra, F.; Chen, H.-M.; Lai, H.-T.; Hsiao, C.-D. Cardiovascular performance measurement in water fleas by utilizing high-speed videography and imagej software and its application for pesticide toxicity assessment. Animals 2020, 10, 1587. [Google Scholar] [CrossRef]
  16. Bekker, J.M.; Krijgsman, B. Physiological investigations into the heart function of Daphnia. J. Physiol. 1951, 115, 249. [Google Scholar] [CrossRef] [PubMed]
  17. Bownik, A. Physiological endpoints in daphnid acute toxicity tests. Sci. Total Environ. 2020, 700, 134400. [Google Scholar] [CrossRef] [PubMed]
  18. Ahmed, B.; Gulliver, T.A.; Alzahir, S. Image splicing detection using mask-RCNN. Signal Image Video Process. 2020, 14, 1035–1042. [Google Scholar] [CrossRef]
  19. Pu, Y.; Gan, Z.; Henao, R.; Yuan, X.; Li, C.; Stevens, A.; Carin, L. Variational autoencoder for deep learning of images, labels and captions. Adv. Neural Inf. Process. Syst. 2016, 29, 2352–2360. [Google Scholar]
  20. Chen, L.-C.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A.L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 40, 834–848. [Google Scholar] [CrossRef] [Green Version]
  21. Falk, T.; Mai, D.; Bensch, R.; Çiçek, Ö.; Abdulkadir, A.; Marrakchi, Y.; Böhm, A.; Deubner, J.; Jäckel, Z.; Seiwald, K. U-Net: Deep learning for cell counting, detection, and morphometry. Nat. Methods 2019, 16, 67–70. [Google Scholar] [CrossRef]
  22. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image Computing and Computer-Assisted Intervention, Proceedings of the 18th International Conference, Munich, Germany, 5–9 October 2015; Springer: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
  23. Kido, S.; Hirano, Y.; Hashimoto, N. Detection and classification of lung abnormalities by use of convolutional neural network (CNN) and regions with CNN features (R-CNN). In Proceedings of the 2018 International Workshop on Advanced Image Technology (IWAIT), Chiang Mai, Thailand, 7–9 January 2018; pp. 1–4. [Google Scholar]
  24. Chang, Y.; Song, B.; Jung, C.; Huang, L. Automatic segmentation and cardiopathy classification in cardiac mri images based on deep neural networks. In Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada, 15–20 April 2018; pp. 1020–1024. [Google Scholar]
  25. Ünver, H.M.; Ayan, E. Skin lesion segmentation in dermoscopic images with combination of YOLO and grabcut algorithm. Diagnostics 2019, 9, 72. [Google Scholar] [CrossRef] [Green Version]
  26. He, P.; Zuo, L.; Zhang, C.; Zhang, Z. A value recognition algorithm for pointer meter based on improved Mask-RCNN. In Proceedings of the 2019 9th International Conference on Information Science and Technology (ICIST), Hulunbuir, China, 2–5 August 2019; pp. 108–113. [Google Scholar]
  27. Karatzas, P.; Melagraki, G.; Ellis, L.J.A.; Lynch, I.; Varsou, D.D.; Afantitis, A.; Tsoumanis, A.; Doganis, P.; Sarimveis, H. Development of deep learning models for predicting the effects of exposure to engineered nanomaterials on Daphnia Magna. Small 2020, 16, 2001080. [Google Scholar] [CrossRef]
  28. Zhao, M.; Wei, Y.; Lu, Y.; Wong, K.K. A novel U-Net approach to segment the cardiac chamber in magnetic resonance images with ghost artifacts. Comput. Methods Programs Biomed. 2020, 196, 105623. [Google Scholar] [CrossRef]
  29. Dong, S.; Zhao, J.; Zhang, M.; Shi, Z.; Deng, J.; Shi, Y.; Tian, M.; Zhuo, C. DeU-Net: Deformable U-Net for 3D Cardiac MRI Video Segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, Proceedings of the 23rd International Conference, Lima, Peru, 4–8 October 2020; Springer: Cham, Switzerland, 2020; pp. 98–107. [Google Scholar]
  30. Silva, C.E.S.; Ferreira, L.D.C.; Peixoto, L.B.; Monaco, C.G.; Gil, M.A.; Ortiz, J. Study of the myocardial contraction and relaxation velocities through Doppler tissue imaging echocardiography: A new alternative in the assessment of the segmental ventricular function. Arq. Bras. Cardiol. 2002, 78, 206–211. [Google Scholar] [CrossRef] [Green Version]
  31. Baylor, E. Cardiac pharmacology of the cladoceran, Daphnia. Biol. Bull. 1942, 83, 165–172. [Google Scholar] [CrossRef] [Green Version]
  32. Villegas-Navarro, A.; Rosas, L.E.; Reyes, J.L. The heart of Daphnia magna: Effects of four cardioactive drugs. Comp. Biochem. Physiol. Part C Toxicol. Pharmacol. 2003, 136, 127–134. [Google Scholar] [CrossRef]
  33. FernáNdez-GonzáLez, M.A.; Gonzalez-Barrientos, J.; Carter, M.J.; Ramos-Jiliberto, R. Parent-to-offspring transfer of sublethal effects of copper exposure: Metabolic rate and life-history traits of Daphnia. Rev. Chil. Hist. Nat. 2011, 84, 195–201. [Google Scholar] [CrossRef]
  34. Mishra, A.; Shukla, S.; Chopra, A. Physiological responses of heart of tailless fresh water flea Simocephalus vetulus (Crustacea-cladocera) under copper sulphate stress. CIBTech J. Zool 2016, 5, 52–59. [Google Scholar]
  35. Bownik, A. Protective effects of ectoine on physiological parameters of Daphnia magna subjected to clove oil-induced anaesthesia. Turk. J. Fish. Aquat. Sci. 2016, 16, 691–701. [Google Scholar] [CrossRef]
  36. Jeong, T.-Y.; Asselman, J.; De Schamphelaere, K.A.; Van Nieuwerburgh, F.; Deforce, D.; Kim, S.D. Effect of β-adrenergic receptor agents on cardiac structure and function and whole-body gene expression in Daphnia magna. Environ. Pollut. 2018, 241, 869–878. [Google Scholar] [CrossRef] [Green Version]
  37. Kurnia, K.A.; Saputra, F.; Roldan, M.J.M.; Castillo, A.L.; Huang, J.-C.; Chen, K.H.-C.; Lai, H.-T.; Hsiao, C.-D. Measurement of Multiple Cardiac Performance Endpoints in Daphnia and Zebrafish by Kymograph. Inventions 2021, 6, 8. [Google Scholar] [CrossRef]
  38. Farhan, A.; Kurnia, K.A.; Saputra, F.; Chen, K.H.-C.; Huang, J.-C.; Roldan, M.J.M.; Lai, Y.-H.; Hsiao, C.-D. An OpenCV-Based Approach for Automated Cardiac Rhythm Measurement in Zebrafish from Video Datasets. Biomolecules 2021, 11, 1476. [Google Scholar] [CrossRef]
  39. Rueden, C.T.; Schindelin, J.; Hiner, M.C.; DeZonia, B.E.; Walter, A.E.; Arena, E.T.; Eliceiri, K.W. ImageJ2: ImageJ for the next generation of scientific image data. BMC Bioinform. 2017, 18, 529. [Google Scholar] [CrossRef]
  40. Frid-Adar, M.; Klang, E.; Amitai, M.; Goldberger, J.; Greenspan, H. Synthetic data augmentation using GAN for improved liver lesion classification. In Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA, 4–7 April 2018; pp. 289–293. [Google Scholar]
  41. Shorten, C.; Khoshgoftaar, T.M. A survey on image data augmentation for deep learning. J. Big Data 2019, 6, 60. [Google Scholar] [CrossRef]
  42. Buslaev, A.; Iglovikov, V.I.; Khvedchenya, E.; Parinov, A.; Druzhinin, M.; Kalinin, A.A. Albumentations: Fast and flexible image augmentations. Information 2020, 11, 125. [Google Scholar] [CrossRef] [Green Version]
  43. Jin, S.; Zedong, H.; Yuan, L. Software implementation of corn grain morphology detection based on OpenCV. In Proceedings of the 2017 13th IEEE International Conference on Electronic Measurement & Instruments (ICEMI), Yangzhou, China, 20–22 October 2017; pp. 412–415. [Google Scholar]
  44. Hart, K.A.; Rimoli, J.J. MicroStructPy: A statistical microstructure mesh generator in Python. SoftwareX 2020, 12, 100595. [Google Scholar] [CrossRef]
  45. Marengoni, M.; Stringhini, D. High level computer vision using opencv. In Proceedings of the 2011 24th SIBGRAPI Conference on Graphics, Patterns, and Images Tutorials, Alagoas, Brazil, 28–30 August 2011; pp. 11–24. [Google Scholar]
  46. Zhang, Z.; Ryoo, D.; Balusek, C.; Acharya, A.; Rydmark, M.O.; Linke, D.; Gumbart, J.C. Inward-facing glycine residues create sharp turns in β-barrel membrane proteins. Biochim. Biophys. Acta (BBA)-Biomembr. 2021, 1863, 183662. [Google Scholar] [CrossRef] [PubMed]
  47. Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; Lerer, A. Automatic Differentiation in Pytorch. Available online: https://openreview.net/pdf?id=BJJsrmfCZ (accessed on 28 June 2022).
  48. MacFarland, T.W.; Yates, J.M. Introduction to Nonparametric Statistics for the Biological Sciences Using R; Springer International Publishing: Cham, Switzerland, 2016. [Google Scholar]
  49. Watt, T.A.; McCleery, R.H.; Hart, T. Introduction to Statistics for Biology; CRC Press: Boca Raton, FL, USA, 2007. [Google Scholar]
  50. Liu, T.; Tian, Y.; Zhao, S.; Huang, X.; Wang, Q. Automatic whole heart segmentation using a two-stage u-net framework and an adaptive threshold window. IEEE Access 2019, 7, 83628–83636. [Google Scholar] [CrossRef]
  51. Shu, J.-H.; Nian, F.-D.; Yu, M.-H.; Li, X. An improved mask R-CNN model for multiorgan segmentation. Math. Probl. Eng. 2020, 2020, 8351725. [Google Scholar] [CrossRef]
  52. Tong, Q.; Ning, M.; Si, W.; Liao, X.; Qin, J. 3D deeply-supervised U-net based whole heart segmentation. In International Workshop on Statistical Atlases and Computational Models of the Heart, Proceedings of the 8th International Workshop, STACOM 2017, Held in Conjunction with MICCAI 2017, Quebec City, Canada, 10–14 September 2017; Springer: Cham, Switzerland, 2017; pp. 224–232. [Google Scholar]
  53. Zhang, J.; Du, J.; Liu, H.; Hou, X.; Zhao, Y.; Ding, M. LU-NET: An Improved U-Net for ventricular segmentation. IEEE Access 2019, 7, 92539–92546. [Google Scholar] [CrossRef]
  54. He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask r-cnn. In Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969. [Google Scholar]
  55. Zhang, X.; An, G.; Liu, Y. Mask R-CNN with feature pyramid attention for instance segmentation. In Proceedings of the 2018 14th IEEE International Conference on Signal Processing (ICSP), Beijing, China, 12–16 August 2018; pp. 1194–1197. [Google Scholar]
  56. Dogan, R.O.; Dogan, H.; Bayrak, C.; Kayikcioglu, T. A Two-Phase Approach using Mask R-CNN and 3D U-Net for High-Accuracy Automatic Segmentation of Pancreas in CT Imaging. Comput. Methods Programs Biomed. 2021, 207, 106141. [Google Scholar] [CrossRef]
  57. Konopczyński, T.; Heiman, R.; Woźnicki, P.; Gniewek, P.; Duvernoy, M.-C.; Hallatschek, O.; Hesser, J. Instance Segmentation of Densely Packed Cells Using a Hybrid Model of U-Net and Mask R-CNN. In International Conference on Artificial Intelligence and Soft Computing, Proceedings of the19th International Conference, ICAISC 2020, Zakopane, Poland, 12–14 October 2020; Springer: Cham, Switzerland, 2020; pp. 626–635. [Google Scholar]
  58. Xu, Z.; Wu, Z.; Feng, J. CFUN: Combining faster R-CNN and U-net network for efficient whole heart segmentation. arXiv 2018, arXiv:1812.04914. [Google Scholar]
  59. Vuola, A.O.; Akram, S.U.; Kannala, J. Mask-RCNN and U-net ensembled for nuclei segmentation. In Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, 8–11 April 2019; pp. 208–212. [Google Scholar]
  60. Durkee, M.S.; Abraham, R.; Ai, J.; Fuhrman, J.D.; Clark, M.R.; Giger, M.L. Comparing Mask R-CNN and U-Net architectures for robust automatic segmentation of immune cells in immunofluorescence images of Lupus Nephritis biopsies. In Proceedings of the Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues XIX, Online Only, CA, USA, 6–12 March 2021; p. 116470X. [Google Scholar]
  61. Quoc, T.T.P.; Linh, T.T.; Minh, T.N.T. Comparing U-Net Convolutional Network with Mask R-CNN in Agricultural Area Segmentation on Satellite Images. In Proceedings of the 2020 7th NAFOSTED Conference on Information and Computer Science (NICS), Ho Chi Minh City, Vietnam, 26–27 November 2020; pp. 124–129. [Google Scholar]
  62. Campbell, A.K.; Wann, K.T.; Matthews, S.B. Lactose causes heart arrhythmia in the water flea Daphnia pulex. Comp. Biochem. Physiol. Part B Biochem. Mol. Biol. 2004, 139, 225–234. [Google Scholar] [CrossRef]
  63. Ebert, D. Ecology, Epidemiology, and Evolution of Parasitism in Daphnia; National Library of Medicine: Bethesda, MD, USA, 2005.
Animals 12 01670 g001 550
Figure 1. Experimental workflow for water flea heart size prediction by using a deep learning approach. Two different convolutional networks of U-Net and Mask RCNN were used for comparison.
Figure 1. Experimental workflow for water flea heart size prediction by using a deep learning approach. Two different convolutional networks of U-Net and Mask RCNN were used for comparison.
Animals 12 01670 g001
Animals 12 01670 g002 550
Figure 2. The Dice and loss curves for the U-net and Mask RCNN method for D. magna heart size prediction. The Dice and loss curve for the U-Net method at either training (A) or validation (B) process. The Dice and loss curve for the Mask RCNN method at either training (C) or validation (D) process.
Figure 2. The Dice and loss curves for the U-net and Mask RCNN method for D. magna heart size prediction. The Dice and loss curve for the U-Net method at either training (A) or validation (B) process. The Dice and loss curve for the Mask RCNN method at either training (C) or validation (D) process.
Animals 12 01670 g002
Animals 12 01670 g003 550
Figure 3. Image segmentation done using Mask RCNN (A) and U-Net (B) to predict heart size in water fleas. The white area show the predicted position of heart chamber. Three water flea species of D. magna, D. pulex, and Moina sp. were tested and the heart size for ground truth, and prediction is shown for comparison.
Figure 3. Image segmentation done using Mask RCNN (A) and U-Net (B) to predict heart size in water fleas. The white area show the predicted position of heart chamber. Three water flea species of D. magna, D. pulex, and Moina sp. were tested and the heart size for ground truth, and prediction is shown for comparison.
Animals 12 01670 g003
Animals 12 01670 g004 550
Figure 4. Use Mask RCNN to study cardiac rhythm in water fleas. Three water flea species of D. magna (pink color), D. pulex (green color), and Moina sp. (blue color) were tested, and the cardiac rhythm and heart size change dynamic can be elucidated by the Mask RCNN method.
Figure 4. Use Mask RCNN to study cardiac rhythm in water fleas. Three water flea species of D. magna (pink color), D. pulex (green color), and Moina sp. (blue color) were tested, and the cardiac rhythm and heart size change dynamic can be elucidated by the Mask RCNN method.
Animals 12 01670 g004
Animals 12 01670 g005 550
Figure 5. Comparison of cardiac physiology parameters after incubation in 5 ppm Roundup for 24 h using either the Mask RCNN or ImageJ method. The data were shown as a box and whisker plot with mean ± min to max values. The statistical significance was compared using either paired t-test (A) or Wilcoxon test (B–D) to analyze the intra-treatment result and using t-test (A) and Mann–Whitney test (B–D) to analyze the inter-treatment result (* p<0.05, *** p<0.001). The open box represented the Mask RCNN method, and the shaded box represented the ImageJ method.
Figure 5. Comparison of cardiac physiology parameters after incubation in 5 ppm Roundup for 24 h using either the Mask RCNN or ImageJ method. The data were shown as a box and whisker plot with mean ± min to max values. The statistical significance was compared using either paired t-test (A) or Wilcoxon test (B–D) to analyze the intra-treatment result and using t-test (A) and Mann–Whitney test (B–D) to analyze the inter-treatment result (* p<0.05, *** p<0.001). The open box represented the Mask RCNN method, and the shaded box represented the ImageJ method.
Animals 12 01670 g005
Animals 12 01670 g006 550
Figure 6. Comparison of cardiac physiology parameters after incubation in 5 ppm of Roundup for 24 h. The heart size was first predicted by Mask RCNN, and later the long and short axes of the cardiac chamber was predicted by OpenCV. Next, we used long and short axis lengths to assess cardiac physiology by measuring stroke volume (A), cardiac output (B), shortening fraction (C), and ejection fraction (D). The data were shown as mean ± SD, and the statistical significance was compared using an unpaired student t-test (** p<0.01, **** p<0.0001).
Figure 6. Comparison of cardiac physiology parameters after incubation in 5 ppm of Roundup for 24 h. The heart size was first predicted by Mask RCNN, and later the long and short axes of the cardiac chamber was predicted by OpenCV. Next, we used long and short axis lengths to assess cardiac physiology by measuring stroke volume (A), cardiac output (B), shortening fraction (C), and ejection fraction (D). The data were shown as mean ± SD, and the statistical significance was compared using an unpaired student t-test (** p<0.01, **** p<0.0001).
Animals 12 01670 g006
Table
Table 1. Comparison of previous cardiac physiology measurement methods in Cladocera.
Table 1. Comparison of previous cardiac physiology measurement methods in Cladocera.
ReferenceRecording InstrumentSoftware/ToolsAnimal modelObtainable Result
This studyHigh-speed CCD camera mounted to an inverted microscopeU-Net and Mask RCNN convolutional Networks D. magna, D. pulex, and Moina sp.Cross sectional area change, heart rate, stroke volume, ejection fraction, fraction shortening, cardiac output, and heartbeat regularity
[31]Spencer microscope devised with stroboscopeStroboscope or stopwatch for manual counting with the naked eyeD. magnaHeart rate
[32]Inverted microscope, digital video camera, and videotape recorder assembled to computerEchocardiographyD. magnaIrregularity of cardiac rhythm, cardiac area in systole/diastole, and beats per min.
[33]Digital camera attached to a microscopeManual counting D. pulexHeart rate
[34]Panasonic DMC-LZ8 cameraMovie maker was used to play the recording video in slow motion, then manual counting (beats/min) was conductedSimocephalus vetulusHeart rate
[35]a digital camera Nikon D3100 mounted
on a microscope. 		Tracker® softwareD. magnaHeart rate, diastole/systole heart area ratio, duration of diastole
[36]microscope (CKX41SF, Olympus) equipped with a digital cameraGOM player and ImageJ softwareD. magnaHeart size, contraction capacity, and heart rate
[27]Nikon stereomicroscope, model SMZ800 Digital Sight, fitted with a D5-Fi2 cameraImage capture by NIS-Elements software and image analysis by machine learning (R-CNN)D. magnaHeart malformation detection
[15]High-speed CCD camera mounted to an inverted microscopeImageJ Time Series Analyzer plug-inD. magna, D. similis, and Moina sp.Heart rate, blood flow rate, stroke volume, ejection fraction, fractional shortening, cardiac output, and heartbeat regularity
[37]High-speed CCD camera mounted to an inverted microscopeImageJ Kymograph plug-inD. magnaHeart rate, stroke volume, ejection fraction, fraction shortening, cardiac output, and heartbeat regularity
[38]High-speed CCD camera mounted to an inverted microscopeOpenCVD. magnaHeart rate and heartbeat regularity
Table
Table 2. Comparison of prediction power of U-Net and Mask RCNN for cardiac size prediction in different water flea species.
Table 2. Comparison of prediction power of U-Net and Mask RCNN for cardiac size prediction in different water flea species.
Dice CoefficientIOUSensitivitySpecificityN
U-Net
D. magna0.930 ± 0.0420.872 ± 0.0700.946 ± 0.0840.987 ± 0.00960
D. pulex0.817 ± 0.1240.707 ± 0.1610.804 ± 0.1730.989 ± 0.006100
Moina sp.0.659 ± 0.2090.526 ± 0.2280.732 ± 0.2070.970 ± 0.029100
Mask RCNN
D. magna0.969 ± 0.0080.940 ± 0.0150.967 ± 0.0190.995 ± 0.00560
D. pulex0.958 ± 0.0110.919 ± 0.0200.945 ± 0.0250.998 ± 0.001100
Moina sp.0.930 ± 0.0540.874 ± 0.0830.961 ± 0.0320.994 ± 0.009100
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Saputra, F.; Farhan, A.; Suryanto, M.E.; Kurnia, K.A.; Chen, K.H.-C.; Vasquez, R.D.; Roldan, M.J.M.; Huang, J.-C.; Lin, Y.-K.; Hsiao, C.-D. Automated Cardiac Chamber Size and Cardiac Physiology Measurement in Water Fleas by U-Net and Mask RCNN Convolutional Networks. Animals 2022, 12, 1670. https://doi.org/10.3390/ani12131670

AMA Style

Saputra F, Farhan A, Suryanto ME, Kurnia KA, Chen KH-C, Vasquez RD, Roldan MJM, Huang J-C, Lin Y-K, Hsiao C-D. Automated Cardiac Chamber Size and Cardiac Physiology Measurement in Water Fleas by U-Net and Mask RCNN Convolutional Networks. Animals. 2022; 12(13):1670. https://doi.org/10.3390/ani12131670

Chicago/Turabian Style

Saputra, Ferry, Ali Farhan, Michael Edbert Suryanto, Kevin Adi Kurnia, Kelvin H.-C. Chen, Ross D. Vasquez, Marri Jmelou M. Roldan, Jong-Chin Huang, Yih-Kai Lin, and Chung-Der Hsiao. 2022. "Automated Cardiac Chamber Size and Cardiac Physiology Measurement in Water Fleas by U-Net and Mask RCNN Convolutional Networks" Animals 12, no. 13: 1670. https://doi.org/10.3390/ani12131670

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop