Next Article in Journal
Characteristic Binding Landscape of Estrogen Receptor-α36 Protein Enhances Promising Cancer Drug Design
Next Article in Special Issue
Exploring Omega-3′s Impact on the Expression of Bone-Related Genes in Meagre (Argyrosomus regius)
Previous Article in Journal
Bioassays for Identifying and Characterizing Plant Regulatory Peptides
Previous Article in Special Issue
Bone Formation in Zebrafish: The Significance of DAF-FM DA Staining for Nitric Oxide Detection
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Recent Advances in Bioimage Analysis Methods for Detecting Skeletal Deformities in Biomedical and Aquaculture Fish Species

by 1,*, 1, 1 and 2
1
Department of Computer Science and Electrical Engineering, Montefiore Institute, University of Liège, 4000 Liège, Belgium
2
Laboratory for Organogenesis and Regeneration (LOR), GIGA Institute, University of Liège, 4000 Liège, Belgium
*
Author to whom correspondence should be addressed.
Biomolecules 2023, 13(12), 1797; https://doi.org/10.3390/biom13121797
Submission received: 31 October 2023 / Revised: 5 December 2023 / Accepted: 9 December 2023 / Published: 14 December 2023

Abstract

:
Detecting skeletal or bone-related deformities in model and aquaculture fish is vital for numerous biomedical studies. In biomedical research, model fish with bone-related disorders are potential indicators of various chemically induced toxins in their environment or poor dietary conditions. In aquaculture, skeletal deformities are affecting fish health, and economic losses are incurred by fish farmers. This survey paper focuses on showcasing the cutting-edge image analysis tools and techniques based on artificial intelligence that are currently applied in the analysis of bone-related deformities in aquaculture and model fish. These methods and tools play a significant role in improving research by automating various aspects of the analysis. This paper also sheds light on some of the hurdles faced when dealing with high-content bioimages and explores potential solutions to overcome these challenges.

1. Introduction

In the realm of biomedical research, model fish species like zebrafish (Danio rerio), and medaka (Oryzias latipes) are highly regarded as valuable vertebrate models. They are extensively used in a variety of biomedical applications, encompassing drug testing, morphometric screening, genome editing, toxicology assessments, and behavior analysis in vertebrates [1,2,3,4,5,6,7,8]. These model fish exhibit significant genetic and metabolic pathway similarities to both fish and mammals, sharing over 70% of their genes with humans [9,10,11,12]. Notably, zebrafish and medaka models are particularly advantageous due to their ease of maintenance and reproduction. Together with other technical advantages such as small size, low maintenance cost, high fecundity, and amenability to genetic engineering tools, the reason these fish are so popular among scientists is their suitability for in vivo imaging [13,14]. The embryonic and larval stages of these animals are translucent, allowing for the application of advanced imaging technologies to observe biological processes in a living animal. This property bears great potential for biomedical research when combined with the availability of transgenic and mutant lines that allow modeling human skeletal diseases and tracking specific organs and cell types with fluorescent markers [15]. Such characteristics not only offer an incredible tool for fundamental research, but also greatly benefit drug discovery. According to the Business Research Insights website, the global zebrafish model services market size was USD 434.4 million in the year 2022 and is projected to reach USD 618.23 million by the year 2031, with a compound annual growth rate (CAGR) of 14.4% during the forecast period.
Fish is recognized as a valuable source of high-quality protein and essential nutrients that are integral to a healthy human diet. Within the aquaculture industry, fish holds a primary position as the predominant source of cultivated seafood for human consumption. According to the European Commission’s Ocean and Fisheries website, marine and freshwater fish constitute approximately 49% of the total aquaculture production. Commonly consumed food fish species include gilthead seabream (Sparus aurata), meagre (Argyrosomus regius), and salmon (Salmo salar), which are saltwater species, while rainbow trout (Oncorhynchus mykiss) is a freshwater counterpart. In their natural habitats, such as the sea or rivers, healthy fish thrive without external interventions in terms of food and care. However, in fish farms, fish are reared within controlled or artificial environments, such as ponds, tanks, or cages, which necessitate external care and provisioning of food. Given the escalating global demand for aquaculture products, the industry faces significant pressure to enhance its supply. To meet this demand, fish farmers adopt intensive production practices, which can result in challenges like deteriorating water quality, higher fish density per unit of water volume, and limited food availability for the fish. These factors may contribute to stressed fish, the development of physical abnormalities, and susceptibility to serious diseases [16]. Fish with disease or deformities are rejected by the potential retailers or customers, thereby representing a significant economic loss to the fish farmers [17]. Major economic losses are directly due to the development of skeletal disorders altering the external shape of reared fish, i.e., opercular and vertebral column deformities [18]. Moreover, tedious technical effort and time are required to manually cull out the deformed fish from the productive cycle, which should be carried out as early as possible in order to not waste resources on growing suboptimal fish.
To detect and classify the deformities in the reared or model fish, manual inspection or analysis is employed, which requires significant time and technical effort. Moreover, direct physical interaction with the fish can induce fear or stress that may reflect on its behavior. Due to abnormal behavior or stress, fish can not swim or take proper diet, which can lead to poor health of the fish [16]. To improve animal welfare both in aquaculture and biomedical research, scientists are looking for methods requiring minimal manual interaction with the animals, with more focus on their health and quality of life. Computer vision is one such area that is increasingly being adopted by fish farmers and biomedical researchers to monitor the health and/or behavioral changes of the animal/fish. It may be helpful in identifying the causes of fish stress or any health hazard with minimal interaction with the animal. Computer vision and image processing techniques can also be helpful to speed up other routine procedures such as animal feeding [19], animal sorting, and animal counting by automatizing these tasks. According to the website, https://this.fish/blog/ai-guide-tracking-ais-explosive-growth-in-aquaculture/ (accessed on 2 December 2023) “this.fish”, the top 10 artificial intelligence (AI) and software start-up companies for the aquaculture industry have raised USD 282 million in the past 5 years, illustrating the importance and prevalence of AI-based smart farming in aquaculture.
Nowadays, automatic or semi-automatic computer-vision-based image processing techniques are being used in aquaculture industries and biomedical research to speed up the detection and diagnosis of diseases in the fish under study. These computer-vision-based techniques employ artificial intelligence methods such as machine learning or deep learning, which not only speed up the diagnosis but are also helpful in improving the accuracy of the detection. Deep learning represents a cutting-edge AI approach that empowers computers to learn from data and perform tasks on par with human capabilities. It utilizes multi-layered neural networks to directly acquire task-specific features from the data and make informed decisions when confronted with unseen data after the learning process. This methodology avoids the use of manually engineered features (as used in older image processing approaches) and, instead, defines a learning process that autonomously extracts features from the data, reducing the need for human intervention. In scenarios involving image data, such as image analysis, specialized types of neural networks known as deep convolutional neural networks (CNNs) have been designed. Such architectures play a pivotal role in computer vision applications, including tasks like object classification, object recognition, segmentation, and object counting. For the analysis of biomedical images, deep-learning-based convolutional neural networks (CNNs) are widely employed and increasingly favored for their accuracy, which rivals human performance [20]. Although debatable, CNNs are considered an imitation of the working of neurons in the human brain for visual perception and understanding objects in the images. They use a set of consecutive convolutional blocks or layers in order to understand the useful patterns for recognizing the objects in the images. The fundamental component of a CNN is the convolutional layer, comprising a collection of filters (or kernels), whose parameters are trained during the learning process [21,22]. Convolutional layers excel in extracting features from images by addressing spatial redundancy through weight sharing, and the features become more distinct and informative while going deeper into the layers. The role of the activation layer is to fire or activate the particular neurons while processing the information, and spatial invariance is achieved using pooling layers. In the end, a condensed feature representation is generated, encapsulating the essential content of the image in fully connected layers [23]. A typical CNN architecture for object classification tasks in images is shown in Figure 1.
In this paper, we review various computer-vision-based automatic and semi-automatic image analysis methods and tools that are used in morphometric and phenotype studies of the aquaculture and biomedical model fish. Our literature survey was performed using searches in PubMed, Scopus, Google Scholar, Web of Science, Bioimage Informatics Index (https://biii.eu (accessed on 2 October 2023)), and Papers With Code (https://paperswithcode.com/ (accessed on 5 October 2023)) databases, and thanks to personal communications with researchers in the field, including members of the BioMedAqu project. A related, recent review appears in [24]. The latter, however, primarily focuses on the application of AI algorithms for behavioral analysis, genomics and neuroscience in zebrafish research, while our paper concentrates on the application of AI algorithms in the analysis of bone-related deformities in biomedical and aquaculture fish species.
Our paper is structured as follows. In Section 2, we first highlight the popular imaging techniques used in acquiring fish bioimages. In the following sections, we delve into the image analysis techniques employed in biomedical (Section 3) and aquaculture investigations (Section 4) to identify and categorize different types of bone-related deformities in both model and food fish species. Table A1 in Appendix A focuses on user-friendly, AI-based image analysis tools used in fish morphometric and phenotype research along with their specifications. Finally, we also highlight the challenges encountered during the image analysis tasks (Section 5) due to image acquisition settings, data scarcity, and algorithmic constraints and requirements.

2. Imaging Techniques Used in Fish Bioimages

One of the main advantages of using zebrafish as a model animal over other animals is its transparent body during early, external development life stages, especially from 0 to 10 days post-fertilization (dpf). The transparent body of the larva makes it easy for the biologists to see through its developing organs and bones during in vivo studies and also helps to produce bioimage datasets using various image acquisition equipment [25,26]. Given that image acquisition precedes image analysis, it is crucial to employ suitable imaging methods and protocols to ensure effective and accurate image analysis, particularly when conducting AI-based image analysis. Due to the small size of the zebrafish and medaka embryos and larvae, advanced optical microscopy imaging methods are employed to capture maximum information at the microscopic level [27]. Microscopic imaging methods necessitate a meticulous pipeline to be adhered to, ensuring the prevention of unwarranted variations in acquisition adjustments and parameters that might introduce artifacts capable of influencing the outcomes of image analysis algorithms [28]. Beyond fundamental considerations like luminosity and focus control, special attention to the fish’s positioning and the characteristics of the glass plates is also needed to mitigate potential issues related to light refraction. This precautionary approach aims to prevent problems like shadowed areas in the images that could disrupt the subsequent analysis [29]. Since most phenotype and morphometric studies in biomedical research require capturing the fine-grained information at the sub-cellular level, microscopy methods such as bright-field or fluorescence microscopy are prevalent compared to other imaging approaches [30,31]. More recently, confocal and light-sheet microscopy deliver three-dimensional images [32], while Raman spectroscopy, Fourier-transform infrared spectroscopy, or mass spectrometry imaging are able to reveal the spatial distribution of individual (bio)molecules or classes of molecules [33,34,35,36], resulting in ever more high-content and demanding analysis requirements.
Apart from microscopy methods, X-ray radiography techniques are also popular in biomedical and aquaculture research for analyzing the skeletal structures of the juvenile and adult fish, including microCT imaging [37,38,39]. While microscopy imaging methods are employed in the early life stages (embryonic and larval) of the model fish due to its body’s optical clarity and small size, radiography methods are employed in the later life stages to visualize hard tissues. The adult model fish serves as a distinct and valuable resource for studying pathogenic and therapeutic aspects of adult human bone diseases. This is attributed to the fact that certain functions such as bone turnover, repair, degeneration, and metabolic responses are not fully mature in embryos [40]. Similarly, in aquaculture research, radiography imaging methods are utilized for juvenile and adult fish for several types of phenotype and morphometric studies [41,42].

3. Bioimage Analysis in Biomedical Research

In recent years, considerable progress has been obtained in the field of AI development. In particular, deep-learning techniques are used for automatic image analysis in biomedical sciences and are becoming the predominant choice in various morphometric and phenotypic studies [43,44]. For single-cell phenotype assays that require gathering intricate data at the cellular or sub-cellular level to discern features linked to cellular shape, protein localization, and intracellular movement, classifying phenotypes poses a notable challenge. This challenge becomes more critical in scenarios involving high-content screening where many high-resolution images are acquired. In such situations, deep-learning methods have proven to be more effective than conventional approaches, as evidenced by the research of [45,46]. In morphometric and phenotype research, segmenting various parts of the fish is a key operation for analyzing development in the fish. For example, EmbryoNet (version 1.0.1) [47] is a deep-learning-based software method to identify the phenotype defects in the embryonic stage of the zebrafish. The aim of this approach is to bridge the gap between observed phenotypic traits in embryos and the underlying molecular signaling pathways responsible for those traits. The diverse set of observable traits or phenotypes that researchers monitor during embryonic development include morphological changes, cellular behaviors, or other features. The problem of phenotype classification in zebrafish in high-throughput screening using the end-to-end deep-learning method is described in [48]. In this study, the authors tackle the challenge of categorizing morphological alterations in zebrafish found in multi-fish wells, which often have fish overlapping with one another. Many morphometric studies involve analyzing the skeletal parts of the fish, especially the jaws, operculum, and vertebral column. In order to analyze these parts, researchers in the field of skeletal biology largely rely on advanced imaging techniques followed by morphometric analysis of skeletal structures in animal models and human patients [49]. This makes it possible to explore how specific molecular mechanisms translate into phenotypic changes in both physiological and disease contexts. The great advantage of such an approach is that it allows for translating qualitative biological information into mathematical language, amenable to statistical analysis. Analyzing malformations during early developmental stages offers other valuable insights into the potential toxicity of chemicals. In [50], an automated software tool is developed to quantify the morphometric defects during developmental toxicity screening in zebrafish embryos. In this approach, morphometric features are extracted and organized in a hierarchical manner using length and surface areas from contour information of different parts of the zebrafish candidate. In order to detect certain features, information about previously detected features should also be included. Finally, the detected features are the boundary coordinates of the contours of the objects such as eyes, head, and swim bladder. of embryos. One of the highly sensitive indicators of developmental toxicity is the absence or presence of abnormalities in the tail, spine, or other parts of the fish.
In the work of [29], a machine-learning-based phenotype classification method was developed in which the pipeline comprises a dual-step strategy, beginning with a three-class classification model. Initially, it separates images of “Dead” or “Chorion” from “Normal” embryos before addressing other anomalies such as “downward curved tail”, “upward curved tail” or “short tail” etc. When contrasted with manual classification, the automated classification using machine learning yields a consensus agreement with biological experts’ voting in nine out of eleven assessed defects in 3-day-old zebrafish larvae, ranging from 90% to 100%. Another supervised machine learning approach to automatically classify the absence or presence of malformation in the spine of the medaka fish embryo is discussed in [51]. In that work, a dataset of 2D high-resolution microscopic images of medaka fish is used. Feature extraction is performed, first by segmenting the embryo from the images. Since many malformations are characterized by abnormal spine curvature, features such as body length or curvature angle are then extracted. These features are then fed into a machine-learning-based classifier for training. Since feature characterization depends on the geometry of the body representation of the embryo, the authors admitted that their method is not applicable when the tail of the deformed embryo makes a hook shape, hence not universal to any type of malformation or failure with a high degree of severity in the deformed tail. Figure 2 shows different types of deformities in the tail part of the zebrafish embryo.
Many morphometric studies involve analyzing the skeletal parts of the fish, especially jaws, operculum and the vertebral column. In order to analyze these parts, researchers in the field of skeletal biology largely rely on advanced imaging techniques followed by morphometric analysis of skeletal structures in animal models and human patients [49]. This makes it possible to explore how specific molecular mechanisms translate into phenotypic changes in both physiological and disease contexts. The great advantage of such an approach is that it allows for translating qualitative biological information into mathematical language, amenable to statistical analysis. In the past few years, several zebrafish in vivo assays for evaluating the osteogenic and mineralogenic activity of natural compounds and synthetic drugs were developed [13]. Among these, the morphometric analysis of the opercular bone in the zebrafish larva allows for an easy, cost-effective and fast evaluation of the osteogenic bioactivity of molecules of interest [52]. As pointed out in [53], analyzing the areas of the head and operculum regions in zebrafish larvae, along with quantifying the operculum-to-head ratio, serves as a reliable indicator of altered bone formation and mineralization. This method is well established and validated for screening bioactive molecules with potential effects on bone health. As such, this assay has been successfully implemented to identify natural extracts with promising pharmaceutical potential. However, obtaining biological quantitative data from microscopic images is still largely performed manually. With the aid of image processing software, operators manually segment specific regions of interest (e.g., the eye, the head, or the trunk), and extract data such as pixel areas or fluorescence intensities [53]. This laborious and time-consuming process also suffers from poor reproducibility due to operator-introduced biases. A set of various tools for the popular open-source image analysis software such as ImageJ (v1.54g) were recently developed to increase the automation of image processing obtained from the zebrafish operculum and other zebrafish-based biological assays [54]. However, such tools still require substantial input and supervision from the operator [41,55,56].
In [57], a deep-learning-based image segmentation method is proposed for the segmentation of the head and operculum area of zebrafish larvae from red channel fluorescence microscopy images. This approach allows for the complete automation of operculum and head area segmentation, greatly reducing the variability related to operator-introduced biases and, importantly, the time needed for extracting biological data from microscopic images, therefore substantially increasing the throughput of such an assay. This method reports a “Dice score” of 95 % , which is calculated as the ratio of the overlapping area between the actual and predicted regions, excluding the background. The model (https://github.com/navdeepkaushish/ (accessed on 4 November 2023)) is integrated in an open-source, web based image analysis tool called Cytomine [58]. In Figure 3, deep-learning-based operculum and head segmentation is shown.
At later stages, skeletal development in teleosts progresses under the influence of various biotic and abiotic factors. Consequently, the skeletal characteristics of these organisms are constantly shaped by environmental elements that interact with their genetic makeup. Rearing density has been described as an environmental variable that influences skeletal development, especially vertebral column formation in both small model and larger aquaculture fish, and high densities have been reported as driving factors that induce skeletal anomalies. In the work of [56], the authors provide comprehensive insights into the postcranial components of the medaka skeleton, and they investigate the structural and cellular transformations associated with the emergence of skeletal anomalies. They give an assessment of how rearing density impacts specific meristic counts (e.g., number of vertebrae) and the variability in the occurrence and types of skeletal irregularities. A reliable skeletal reference is provided to establish optimal laboratory conditions that can be used for the assessment of future experimental setups. The work described in [59] implemented a deep-learning model to automatically locate six important landmark positions on the vertebral column of the medaka fish, thereby assisting the biomedical researchers in speeding up the image analysis. The landmarks on the vertebral column are able to detect either upwards/downwards shifts or deviations from the normal axis or elongations/restrictions of the regions which could indicate vertebral body fusions or extranumerary vertebral bodies. Figure 4 shows six landmark points on the microscopy image of a medaka juvenile (40 dpf). For deep-learning model training and evaluation, a total number of 430 Alizarin-red-stained microscopy images of 2560 × 1920 size were used. In terms of performance, the authors reported an average Euclidean distance of about 9 pixels between actual and predicted landmark locations, estimated on a set of images that were not seen during model training. The model is integrated into the Cytomine image analysis tool [58].
Osteoporosis is a metabolic bone condition resulting from an imbalance in the processes of bone formation and resorption. This disrupted equilibrium leads to weakened bones that are prone to fractures, even from minor incidents. With an increasingly aging population, the prevalence of osteoporosis is on the rise, and currently, there are limited pharmaceutical interventions available, particularly for enhancing the bone-building activity of osteoblasts [60]. Encouragingly, extensive genetic studies conducted on individuals with low (LBM) and high bone mass (HBM) and genome-wide association studies (GWAS) in the general population have revealed fresh insights into pathways containing key players in bone formation [61,62,63,64,65,66]. These discoveries hold the potential for the development of new drug targets. Such studies involve analyzing bone structures of the zebrafish model in its earlier development stages. For quantification of the size and shapes of bone structures, anatomical landmark points are assigned to them for the geometric morphometric analysis. Manually annotating these landmarks is a tedious and time-consuming process that requires technical expertise as well. The work described in [59] uses deep learning to automatically detect the landmark locations of various bone structures in zebrafish larvae, thereby making the task of morphometric analysis easier for biologists. In this work, 25 landmark locations for bone structures are targeted (see Figure 5 for a sample image and its annotations), and a total of 113 microscopy images with size 2576 × 1932 are used for the training and the evaluation of several CNN-based deep-learning models. The authors reported an average distance between predicted and actual landmark locations of about 11 pixels. The trained model is integrated into the Cytomine image analysis tool as a software package called “S_Deep-Fish-Landmark-Prediction” (v1.2.0).

4. Bioimage Analysis in Aquaculture

The aquaculture industry plays a crucial role in providing a substantial food source for human consumption [67]. However, it faces mounting pressure to meet the growing demand for food supply. A decade ago, farmers relied on manual methods for various tasks, including sorting fish by size, identifying diseased individuals, removing deformed or deceased fish from healthy stock, and counting fish numbers. In addition, traditional invasive methods involve physically removing fish from the water and having an expert inspect them to identify potential diseases [68]. In recent times, deep-learning-based image processing techniques have emerged as an approach in the aquaculture sector to address the limitations of manual and labor-intensive procedures [43,69]. These methods offer the advantage of being less time-consuming and requiring less technical expertise. Importantly, they are non-invasive by nature, contributing to improved fish health and the promotion of sustainable aquaculture practices.
Although traditional machine learning techniques have been employed in aquaculture research for over a decade, there is a recent surge in the popularity of computer vision-based deep-learning methods, primarily because of their high performance and easy-to-use traits [70]. While relatively new in the aquaculture field, deep-learning analysis methods are proving beneficial by expediting routine tasks for fish farmers in a non-intrusive manner [69]. Simultaneously, they assist technicians and researchers within the aquaculture industry in identifying and categorizing fish disorders and deformities.
Recently [71], a method based on deep learning has been introduced for the non-invasive measurement of meagre fish length and weight. The approach uses stereo images of fish as inputs of a deep-learning object detector. This detector identifies and outlines individual fish by creating bounding boxes around them. Subsequently, each bounding box is employed to extract an image of the individual fish, which is then processed through a pre-trained CNN to pinpoint two significant landmarks on the fish: the snout tip and the base of the middle caudal rays. A landmark detection algorithm calculates the fish’s length in pixels by measuring the distance between these two landmarks. Finally, the pixel-based length is converted to physical units by using calibration data.
The European aquaculture sector places increasing importance on the production of gilthead seabream (Sparus aurata) [17]. However, the occurrence of skeletal deformities in farmed gilthead seabream poses a significant challenge for the industry [72]. This issue results in economic losses, negatively affects consumers’ perception of aquaculture, and raises concerns about the welfare of the fish [73]. While previous efforts have predominantly concentrated on reducing the occurrence of skeletal anomalies during the hatchery phase, current research is directed toward addressing the subsequent pre-ongrowing phase, where more severe deformities impacting the fish’s external appearance often occur [41]. Considering that the pre-ongrowing phase primarily occurs in land-based facilities, it serves as the final opportunity for farmers to implement quality controls and remove deformed fish before transferring them to sea cages. Additionally, the pre-ongrowing phase offers a more effective means of sorting and quality assurance compared to culling during or at the end of the hatchery phase. While fish with severe cranial anomalies can be readily identified and removed at the end of larval rearing due to the early development of their skull bones, vertebral axis deformities are infrequent at this stage, and they often manifest in more advanced developmental stages, which makes them harder to detect [74]. The work in [41] aims to ascertain whether tank volume or stocking density is a primary factor influencing the development of skeletal anomalies during the pre-ongrowing phase in gilthead seabream. The authors found that higher stocking densities led to a higher incidence of specific cranial and axial deformities in the fish. This research provides valuable insights for the aquaculture industry and offers practical applications for fish farmers to enhance the skeletal and morphological quality of farmed gilthead seabream. The research is further exploited with the following observations in mind: (1) Bone elements that were incompletely fused were treated as separate elements; (2) Supernumerary bones with typical morphology were not classified as anomalies but were accounted for in variations in meristic counts. Conversely, supernumerary elements with abnormal shapes were categorized as anomalies; (3) Only variations in shape that were distinctly and unequivocally identifiable were recognized as skeletal anomalies. For instance, deviations in the vertebral axis that were linked to deformation of the involved vertebrae were considered anomalies. Such morphometric features are marked by selecting 19 landmark locations over the fish. Any deviations in these landmark points indicate a potential anomaly in the vertebral column and overall shape of the fish. A sample image from the dataset with annotated landmarks is shown in Figure 6.
The work of automatically identifying these landmark locations is described in [59], where a deep-learning methodology was devised to identify the landmark locations on the radiography images of gilthead seabream. The deep-learning-based CNN is trained on 748 images of varying sizes and is evaluated on 100 test images, yielding an average distance of about 5 pixels between true and predicted landmarks. The method is integrated into the Cytomine image analysis tool.

5. Challenges in Bioimage Analysis Tasks

While traditional and deep-learning methods are valuable for various tasks in biomedical and aquaculture research, they are often challenged by complexities arising from factors like image acquisition, noise, multi-modality, or domain changes. Many of the methods described earlier are tailored to specific issues or rely on particular fish species and their image characteristics. A universal model capable of simultaneously addressing multiple tasks does not currently exist. In this section, we underscore the challenges posed by these factors and their implications for image analysis tasks. We also explore potential solutions that can be employed to tackle these issues.

5.1. High-Content, High-Throughput Imaging

Modern imaging instruments, combined with robotic devices, lead to vast amounts of high-resolution images that allow more specific and detailed analysis [5,75,76]. However, they will also generate terabytes of data with gigapixel images. Moreover, different types of confocal microscopy, time-lapse (video), or microCT methods deliver three-dimensional images that further increase the data load [31,33,34,35]. Analyzing and handling high-content images and observing features at the cellular or sub-cellular level become a bottleneck. Even for the current state-of-the-art analysis methods, it is very challenging to find informative patterns for processing vast amounts of complex structured data. The challenges encountered while processing these high-content image data of zebrafish have previously been discussed in [27] with regard to image quality, annotation, and storage. With the huge size of high-content images, it becomes imperative for the current existing image processing methods to process them in more standardized and generic ways. The authors emphasized the general requirement for the design of a workflow, information on the imaging data (metadata information) being analyzed, and availability of the storage infrastructure such as data center support. Image processing methods must include some preprocessing for noise reduction, correction of inhomogeneous illumination, and correction of attenuation before applying analysis methods. Usually, these preprocessing steps are application independent and can be easily found in many standard image processing toolboxes. While deep-learning-based approaches are better performing than their traditional counterparts in image analysis tasks, they still suffer from performance issues while dealing with high-resolution biomedical imaging. If the objects are very small in size, have distorted shapes, or the annotations are not fully outlined, these models can not perform well. Proper and careful preprocessing might be needed before being fed into neural networks. Another issue while using deep-learning-based models is caused by the fact that images have to be resized (usually downsized and tiled) to make them suitable for use in CNNs. As microscopic or histological images are of very high resolution, downscaling or squeezing them will lead to extensive loss of pixel-level information that should be retained during the study of the micro-anatomy of cells, tissues, and organs. Strategies such as multiple instance learning and weakly-supervised learning are currently being developed to deal with very large images in digital histology [77] and might be similarly useful in high-resolution fish imaging.

5.2. Choice of Image Analysis Methods/Protocols

Traditional image analysis methods that use simple image processing functions are good when the information they process is small. With the growing dimensionality of the data, the use of traditional methods is time-consuming and requires a lot of human intervention with technical expertise to analyze the data. Artificial intelligence (AI)-based techniques, especially deep-learning methods, are helpful to speed up the analysis by making the process semi- or fully automatic. However, training deep-learning-based CNN for high-content image analysis is an uphill task, and applying them effectively in biomedical image analysis tasks is still challenging. In most biomedical images, preserving spatial information is very crucial; thus, if the architecture of the CNN does not preserve the spatial information during training, it may fail to produce the desired results. The choice of the neural network architecture is also dependent upon the imaging data being used for training. Another prominent challenge to deal with in biomedical image analyses is a class imbalance, occurring, for example, when segmenting small cells or tissues of a few pixels areas from large-sized images [57,78,79]. Proper care is needed to take this problem into account while designing an image processing pipeline. Overall, there is also a need for more standardized implementations of image processing workflows (including software dependencies, input and output data formats, and application programming interfaces) to enable reproducible benchmarking [80] and to ease the choice of the best-performing method on a given problem. In addition, tools to foster interdisciplinary collaboration between biologists and computer scientists, providing easy ways to annotate images, execute, and proofread algorithm predictions, are also required [81].

5.3. Lack of Annotated Bioimages

The scarcity of well-annotated bioimage data is one of the prime concerns while designing an image analysis tool or method for biomedical research that requires annotated datasets to be trained, tuned, and validated. Natural image data sets are available in abundance on different platforms with open-source access. In contrast, bioimage datasets are acquired with expensive instruments in a laboratory in controlled conditions and are not easily accessible to the common people. The majority of bioimage datasets originate from either patients or model animals, and they are often restricted to specific laboratories or individuals. Furthermore, these datasets typically consist of a relatively small number of images, ranging from a few hundred to a few thousand, in contrast to natural image datasets that often encompass millions or even billions of well-annotated images. A limited set of well-annotated images is frequently inadequate for training deep-learning models, leading to suboptimal performance. Models trained with insufficient image data often struggle, making it challenging for analysts to deploy them effectively for image analysis tasks. One solution to address the problem of data availability is to use deep transfer learning approaches where a pre-trained and well-optimized CNN is used for training on small bioimage datasets. The authors in [82] exploit the deep transfer learning approaches to address the challenges faced in bioimaging due to its complex structure and lack of data availability. Another solution to deal with small datasets is to use data augmentation [83] techniques in which the data set is increased on the fly during training, using some image processing functions such as random rotation, shift, crop, shear, etc., depending upon the requirement. There are ongoing initiatives to set large repositories of fish bioimages with different modalities (microscopy, radiography, etc.) [84], and the European Union encourages researchers and academicians to provide their annotated bioimages for open-access research. This initiative further inspires fish communities to build substantially large fish bioimage datasets that would be helpful in advancing the research in aquaculture.

5.4. Miscellaneous

Although computer-vision-based deep-learning models outperform other traditional computer vision image processing methods and are increasingly becoming a favorite choice among the biomedical research community, they still suffer from a lack of interpretability. Even in general computer vision tasks, deep learning is mostly considered as black box learning, and research is still ongoing on how to interpret its results and learning mechanisms [85]. Moreover, medical images are content-sensitive, i.e., pixel-level information is needed, and resizing or squeezing to the size required by the network architectures adversely affects the performance due to loss of information. In aquaculture, real-time deployment of AI models for actually sorting the fish will require thorough testing of the working conditions (underwater, while swimming, in the air) and may be challenging due to variations in lighting, pose, background, occlusion, and water turbidity [86], leading to a decrease in the performance of the AI models. In such a complex setting, a considerable number of sensors, cameras, and other equipment must be deployed to gather and analyze real-time information, resulting in a substantial upfront capital investment [69]. Deploying large AI models is also demanding in terms of memory requirements, and incorporating these models into small memory chips poses a significant challenge. These challenges can be addressed using smart automation systems that include the integration of artificial intelligence, robotics, Internet of Things (IoT), edge computing, and advanced distributed algorithms for real-time image analysis [87,88,89]. Real-time applications of AI models in aquaculture would include, among others, fish counting, fish monitoring, fish feeding, fish classification (normal or deformed), fish behavior analysis, or fish sorting. From a biomedical imaging point of view, deep-learning-based models should be designed in more generic ways so that they can be trained without losing much of the information from the high-resolution images. Moreover, most biomedical researchers are from a non-mathematical background, and they might be more interested in knowing the objective interpretation of why rather than the subjective interpretation of how these deep-learning-based neural networks are predicting the discriminating features that often defy the common logic of human interpretation.

6. Conclusions

This paper highlights the current state-of-the-art machine and deep-learning methods being used in biomedical and aquaculture research for morphometric studies related to bone development. In this context, we describe the application of various deep-learning methodologies that can be applied in several bioimage analysis tasks, such as fish body part segmentation or anatomical landmark identification. The methods are not only helpful to speed up the image analysis, but also efficient in terms of accuracy, objectivity, and ease of availability.

Funding

This work was supported by the EU MSCA-ITN project BioMedAqu (GA 766347) that has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 766347. N.K. was an MSCA Ph.D. fellow and was later funded by the Faculty of Applied Sciences of the University of Liège. M.M. is a “Maître de Recherche au F.N.R.S.”. R.M. is supported by the BigPicture EU Research and Innovation Action (Grant agreement number 945358). This work was also partially supported by Service Public de Wallonie Recherche under Grant No. 2010235—ARIAC by DIGITALWALLONIA4.AI.

Institutional Review Board Statement

All studies reported here adhered to the code of ethics for scientific research, compliant with European Directive 2010/63/EU. Details can be found in the cited literature.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this review article.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Appendix A

Table A1. Image Analysis Tools and their specifications.
Table A1. Image Analysis Tools and their specifications.
NameRef. PaperImage ModalityFish TypeApplicationOpen Source?Data AvailabilityResearch Area
EmbryoNet[47]
D. Capek et al.
Light microscopyZebrafish embryos (1–26 hpf  1 ),
Medaka embryos (0–48 hpf),
Stickleback larvae (0–140 hpf)
Phenotype classificationYesYesBiomedical
ZF-AutoML[90]
R. Sawaki et al.
Light microscopyZebrafish larvae (0–96 hpf)Phenotype classificationYesNoBiomedical
ZFTool[91]
M. J. Carreira et al.
Fluorescent Green Channel MicroscopyZebrafish larvae (0–72 hpf)Toxicity screeningYesYesBiomedical
FishInspector[50]
E. Teixido et al.
Light microscopyZebrafish embryos (0–96 hpf)Phenotype screeningYesYesBiomedical
ZebrafishMiner[92]
M. Reischl et al.
Fluorescent microscopyZebrafish embryos and larvae (32 hpf)Fluorescent quantification in body partsNoNoBiomedical
ZFBone[54]
M. Tarasco et al.
Fluorescent microscopyZebrafish larvae (6 dpf 2 )Morphometric analysisYesYesBiomedical
IMAFISH_ML[93]
A. Navarro et al.
Microscopy, RGB, radiographyAdult gilthead seabream, meagre, red porgyMorphometric analysisYesNoAquaculture
Cytomine[58]
R. Marée et al.
All typesAll typesGeneral image analysisYesYesBiomedical, Aquaculture
1 hours post-fertilization; 2 days post-fertilization.

References

  1. Selderslaghs, I.W.; Hooyberghs, J.; Blust, R.; Witters, H.E. Assessment of the developmental neurotoxicity of compounds by measuring locomotor activity in zebrafish embryos and larvae. Neurotoxicology Teratol. 2013, 37, 44–56. [Google Scholar] [CrossRef] [PubMed]
  2. Rihel, J.; Prober, D.A.; Arvanites, A.; Lam, K.; Zimmerman, S.; Jang, S.; Haggarty, S.J.; Kokel, D.; Rubin, L.L.; Peterson, R.T.; et al. Zebrafish behavioral profiling links drugs to biological targets and rest/wake regulation. Science 2010, 327, 348–351. [Google Scholar] [CrossRef] [PubMed]
  3. Bugel, S.M.; Tanguay, R.L. Multidimensional chemobehavior analysis of flavonoids and neuroactive compounds in zebrafish. Toxicol. Appl. Pharmacol. 2018, 344, 23–34. [Google Scholar] [CrossRef]
  4. MacRae, C.A.; Peterson, R.T. Zebrafish as tools for drug discovery. Nat. Rev. Drug Discov. 2015, 14, 721–731. [Google Scholar] [CrossRef]
  5. Love, D.R.; Pichler, F.B.; Dodd, A.; Copp, B.R.; Greenwood, D.R. Technology for high-throughput screens: The present and future using zebrafish. Curr. Opin. Biotechnol. 2004, 15, 564–571. [Google Scholar] [CrossRef]
  6. Scholz, S.; Fischer, S.; Gündel, U.; Küster, E.; Luckenbach, T.; Voelker, D. The zebrafish embryo model in environmental risk assessment—Applications beyond acute toxicity testing. Environ. Sci. Pollut. Res. 2008, 15, 394–404. [Google Scholar] [CrossRef] [PubMed]
  7. Peterson, R.T.; MacRae, C.A. Systematic approaches to toxicology in the zebrafish. Annu. Rev. Pharmacol. Toxicol. 2012, 52, 433–453. [Google Scholar] [CrossRef]
  8. Braunbeck, T.; Kais, B.; Lammer, E.; Otte, J.; Schneider, K.; Stengel, D.; Strecker, R. The fish embryo test (FET): Origin, applications, and future. Environ. Sci. Pollut. Res. 2015, 22, 16247–16261. [Google Scholar] [CrossRef]
  9. Hill, A.J.; Teraoka, H.; Heideman, W.; Peterson, R.E. Zebrafish as a model vertebrate for investigating chemical toxicity. Toxicol. Sci. 2005, 86, 6–19. [Google Scholar] [CrossRef]
  10. Ali, S.; Champagne, D.L.; Spaink, H.P.; Richardson, M.K. Zebrafish embryos and larvae: A new generation of disease models and drug screens. Birth Defects Res. Part Embryo Today Rev. 2011, 93, 115–133. [Google Scholar] [CrossRef]
  11. Strähle, U.; Scholz, S.; Geisler, R.; Greiner, P.; Hollert, H.; Rastegar, S.; Schumacher, A.; Selderslaghs, I.; Weiss, C.; Witters, H.; et al. Zebrafish embryos as an alternative to animal experiments—A commentary on the definition of the onset of protected life stages in animal welfare regulations. Reprod. Toxicol. 2012, 33, 128–132. [Google Scholar] [CrossRef]
  12. Pruvot, B.; Curé, Y.; Djiotsa, J.; Voncken, A.; Muller, M. Developmental defects in zebrafish for classification of EGF pathway inhibitors. Toxicol. Appl. Pharmacol. 2014, 274, 339–349. [Google Scholar] [CrossRef] [PubMed]
  13. Spoorendonk, K.M.; Hammond, C.L.; Huitema, L.F.; Vanoevelen, J.; Schulte-Merker, S. Zebrafish as a unique model system in bone research: The power of genetics and in vivo imaging. J. Appl. Ichthyol. 2010, 26, 219–224. [Google Scholar] [CrossRef]
  14. Zon, L.I.; Peterson, R.T. In vivo drug discovery in the zebrafish. Nat. Rev. Drug Discov. 2005, 4, 35–44. [Google Scholar] [CrossRef] [PubMed]
  15. Driever, W.; Solnica-Krezel, L.; Schier, A.; Neuhauss, S.; Malicki, J.; Stemple, D.; Stainier, D.; Zwartkruis, F.; Abdelilah, S.; Rangini, Z.; et al. A genetic screen for mutations affecting embryogenesis in zebrafish. Development 1996, 123, 37–46. [Google Scholar] [CrossRef]
  16. Martos-Sitcha, J.A.; Mancera, J.M.; Prunet, P.; Magnoni, L.J. Welfare and stressors in fish: Challenges facing aquaculture. Front. Physiol. 2020, 11, 162. [Google Scholar] [CrossRef] [PubMed]
  17. Llorente, I.; Fernández-Polanco, J.; Baraibar-Diez, E.; Odriozola, M.D.; Bjørndal, T.; Asche, F.; Guillen, J.; Avdelas, L.; Nielsen, R.; Cozzolino, M.; et al. Assessment of the economic performance of the seabream and seabass aquaculture industry in the European Union. Mar. Policy 2020, 117, 103876. [Google Scholar] [CrossRef]
  18. Verhaegen, Y.; Adriaens, D.; De Wolf, T.; Dhert, P.; Sorgeloos, P. Deformities in larval gilthead sea bream (Sparus aurata): A qualitative and quantitative analysis using geometric morphometrics. Aquaculture 2007, 268, 156–168. [Google Scholar] [CrossRef]
  19. Hu, W.C.; Chen, L.B.; Huang, B.K.; Lin, H.M. A computer vision-based intelligent fish feeding system using deep learning techniques for aquaculture. IEEE Sens. J. 2022, 22, 7185–7194. [Google Scholar] [CrossRef]
  20. Santosh, K.; Das, N.; Ghosh, S. Deep Learning Models for Medical Imaging; Academic Press: Cambridge, MA, USA, 2021. [Google Scholar]
  21. Mostafa, S.; Wu, F.X. Diagnosis of autism spectrum disorder with convolutional autoencoder and structural MRI images. In Neural Engineering Techniques for Autism Spectrum Disorder; Elsevier: Amsterdam, The Netherlands, 2021; pp. 23–38. [Google Scholar]
  22. Shen, D.; Wu, G.; Suk, H.I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 2017, 19, 221–248. [Google Scholar] [CrossRef]
  23. Guo, Y.; Liu, Y.; Oerlemans, A.; Lao, S.; Wu, S.; Lew, M.S. Deep learning for visual understanding: A review. Neurocomputing 2016, 187, 27–48. [Google Scholar] [CrossRef]
  24. Fan, Y.L.; Hsu, F.R.; Wang, Y.; Liao, L.D. Unlocking the potential of zebrafish research with artificial intelligence: Advancements in tracking, processing, and visualization. Med. Biol. Eng. Comput. 2023, 61, 2797–2814. [Google Scholar] [CrossRef] [PubMed]
  25. Hussain, S.; Aponte-Rivera, R.; Barghout, R.M.; Trapani, J.G.; Kindt, K.S. In Vivo Analysis of Hair Cell Sensory Organs in Zebrafish: From Morphology to Function. In Developmental, Physiological, and Functional Neurobiology of the Inner Ear; Humana: New York, NY, USA, 2022; pp. 175–220. [Google Scholar]
  26. Bauer, B.; Mally, A.; Liedtke, D. Zebrafish embryos and larvae as alternative animal models for toxicity testing. Int. J. Mol. Sci. 2021, 22, 13417. [Google Scholar] [CrossRef] [PubMed]
  27. Mikut, R.; Dickmeis, T.; Driever, W.; Geurts, P.; Hamprecht, F.A.; Kausler, B.X.; Ledesma-Carbayo, M.J.; Marée, R.; Mikula, K.; Pantazis, P.; et al. Automated processing of zebrafish imaging data: A survey. Zebrafish 2013, 10, 401–421. [Google Scholar] [CrossRef] [PubMed]
  28. Shamir, L.; Delaney, J.D.; Orlov, N.; Eckley, D.M.; Goldberg, I.G. Pattern recognition software and techniques for biological image analysis. PLoS Comput. Biol. 2010, 6, e1000974. [Google Scholar] [CrossRef] [PubMed]
  29. Jeanray, N.; Marée, R.; Pruvot, B.; Stern, O.; Geurts, P.; Wehenkel, L.; Muller, M. Phenotype classification of zebrafish embryos by supervised learning. PLoS ONE 2015, 10, e0116989. [Google Scholar] [CrossRef]
  30. Kuchmiy, A.; Efimov, G.; Nedospasov, S. Methods for in vivo molecular imaging. Biochemistry 2012, 77, 1339–1353. [Google Scholar] [CrossRef]
  31. Abu-Siniyeh, A.; Al-Zyoud, W. Highlights on selected microscopy techniques to study zebrafish developmental biology. Lab. Anim. Res. 2020, 36, 12. [Google Scholar] [CrossRef]
  32. Bruneel, B.; Witten, P.E. Power and challenges of using zebrafish as a model for skeletal tissue imaging. Connect. Tissue Res. 2015, 56, 161–173. [Google Scholar] [CrossRef]
  33. Høgset, H.; Horgan, C.C.; Armstrong, J.P.; Bergholt, M.S.; Torraca, V.; Chen, Q.; Keane, T.J.; Bugeon, L.; Dallman, M.J.; Mostowy, S.; et al. In vivo biomolecular imaging of zebrafish embryos using confocal Raman spectroscopy. Nat. Commun. 2020, 11, 6172. [Google Scholar] [CrossRef]
  34. Bennet, M.; Akiva, A.; Faivre, D.; Malkinson, G.; Yaniv, K.; Abdelilah-Seyfried, S.; Fratzl, P.; Masic, A. Simultaneous Raman microspectroscopy and fluorescence imaging of bone mineralization in living zebrafish larvae. Biophys. J. 2014, 106, L17–L19. [Google Scholar] [CrossRef]
  35. Fiedler, I.A.; Schmidt, F.N.; Wölfel, E.M.; Plumeyer, C.; Milovanovic, P.; Gioia, R.; Tonelli, F.; Bale, H.A.; Jähn, K.; Besio, R.; et al. Severely impaired bone material quality in chihuahua zebrafish resembles classical dominant human osteogenesis imperfecta. J. Bone Miner. Res. 2018, 33, 1489–1499. [Google Scholar] [CrossRef]
  36. da Silva, K.M.; Iturrospe, E.; Bars, C.; Knapen, D.; Van Cruchten, S.; Covaci, A.; van Nuijs, A.L. Mass spectrometry-based zebrafish toxicometabolomics: A review of analytical and data quality challenges. Metabolites 2021, 11, 635. [Google Scholar] [CrossRef]
  37. Ding, Y.; Vanselow, D.J.; Yakovlev, M.A.; Katz, S.R.; Lin, A.Y.; Clark, D.P.; Vargas, P.; Xin, X.; Copper, J.E.; Canfield, V.A.; et al. Computational 3D histological phenotyping of whole zebrafish by X-ray histotomography. Elife 2019, 8, e44898. [Google Scholar] [CrossRef] [PubMed]
  38. Merrifield, G.D.; Mullin, J.; Gallagher, L.; Tucker, C.; Jansen, M.A.; Denvir, M.; Holmes, W.M. Rapid and recoverable in vivo magnetic resonance imaging of the adult zebrafish at 7T. Magn. Reson. Imaging 2017, 37, 9–15. [Google Scholar] [CrossRef] [PubMed]
  39. Babaei, F.; Hong, T.L.C.; Yeung, K.; Cheng, S.H.; Lam, Y.W. Contrast-enhanced X-ray micro-computed tomography as a versatile method for anatomical studies of adult zebrafish. Zebrafish 2016, 13, 310–316. [Google Scholar] [CrossRef] [PubMed]
  40. Carnovali, M.; Banfi, G.; Mariotti, M. Zebrafish models of human skeletal disorders: Embryo and adult swimming together. BioMed Res. Int. 2019, 2019, 1253710. [Google Scholar] [CrossRef] [PubMed]
  41. Dellacqua, Z.; Di Biagio, C.; Costa, C.; Pousão-Ferreira, P.; Ribeiro, L.; Barata, M.; Gavaia, P.J.; Mattei, F.; Fabris, A.; Izquierdo, M.; et al. Distinguishing the Effects of Water Volumes versus Stocking Densities on the Skeletal Quality during the Pre-Ongrowing Phase of Gilthead Seabream (Sparus aurata). Animals 2023, 13, 557. [Google Scholar] [CrossRef] [PubMed]
  42. Beckmann, M.C.; Gilliam, J.F.; Langerhans, R.B. X-ray imaging as a time-saving, non-invasive technique for diet analysis. Fish. Res. 2015, 161, 1–7. [Google Scholar] [CrossRef]
  43. Suganyadevi, S.; Seethalakshmi, V.; Balasamy, K. A review on deep learning in medical image analysis. Int. J. Multimed. Inf. Retr. 2022, 11, 19–38. [Google Scholar] [CrossRef]
  44. Liu, X.; Gao, K.; Liu, B.; Pan, C.; Liang, K.; Yan, L.; Ma, J.; He, F.; Zhang, S.; Pan, S.; et al. Advances in deep learning-based medical image analysis. Health Data Sci. 2021, 2021, 8786793. [Google Scholar] [CrossRef]
  45. Dürr, O.; Sick, B. Single-cell phenotype classification using deep convolutional neural networks. J. Biomol. Screen. 2016, 21, 998–1003. [Google Scholar] [CrossRef]
  46. Dong, B.; Shao, L.; Da Costa, M.; Bandmann, O.; Frangi, A.F. Deep learning for automatic cell detection in wide-field microscopy zebrafish images. In Proceedings of the 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), Brooklyn, NY, USA, 16–19 April 2015; pp. 772–776. [Google Scholar]
  47. Čapek, D.; Safroshkin, M.; Morales-Navarrete, H.; Toulany, N.; Arutyunov, G.; Kurzbach, A.; Bihler, J.; Hagauer, J.; Kick, S.; Jones, F.; et al. EmbryoNet: Using deep learning to link embryonic phenotypes to signaling pathways. Nat. Methods 2023, 20, 815–823. [Google Scholar] [CrossRef] [PubMed]
  48. Ishaq, O.; Sadanandan, S.K.; Wählby, C. Deep fish: Deep learning–based classification of zebrafish deformation for high-throughput screening. SLAS Discov. Adv. Life Sci. R&D 2017, 22, 102–107. [Google Scholar]
  49. Allen, M.R.; Krohn, K. Skeletal imaging. In Basic and Applied Bone Biology; Elsevier: Amsterdam, The Netherlands, 2014; pp. 93–113. [Google Scholar]
  50. Teixidó, E.; Kießling, T.R.; Krupp, E.; Quevedo, C.; Muriana, A.; Scholz, S. Automated morphological feature assessment for zebrafish embryo developmental toxicity screens. Toxicol. Sci. 2019, 167, 438–449. [Google Scholar] [CrossRef] [PubMed]
  51. Genest, D.; Puybareau, É.; Léonard, M.; Cousty, J.; De Crozé, N.; Talbot, H. High throughput automated detection of axial malformations in Medaka embryo. Comput. Biol. Med. 2019, 105, 157–168. [Google Scholar] [CrossRef] [PubMed]
  52. Rosa, J.T.; Tarasco, M.; Gavaia, P.J.; Cancela, M.L.; Laizé, V. Screening of Mineralogenic and Osteogenic Compounds in Zebrafish—Tools to Improve Assay Throughput and Data Accuracy. Pharmaceuticals 2022, 15, 983. [Google Scholar] [CrossRef] [PubMed]
  53. Tarasco, M.; Laizé, V.; Cardeira, J.; Cancela, M.L.; Gavaia, P.J. The zebrafish operculum: A powerful system to assess osteogenic bioactivities of molecules with pharmacological and toxicological relevance. Comp. Biochem. Physiol. Part Toxicol. Pharmacol. 2017, 197, 45–52. [Google Scholar] [CrossRef] [PubMed]
  54. Tarasco, M.; Cordelieres, F.P.; Cancela, M.L.; Laizé, V. ZFBONE: An ImageJ toolset for semi-automatic analysis of zebrafish bone structures. Bone 2020, 138, 115480. [Google Scholar] [CrossRef] [PubMed]
  55. Westhoff, J.H.; Steenbergen, P.J.; Thomas, L.S.; Heigwer, J.; Bruckner, T.; Cooper, L.; Tönshoff, B.; Hoffmann, G.F.; Gehrig, J. In vivo high-content screening in zebrafish for developmental nephrotoxicity of approved drugs. Front. Cell Dev. Biol. 2020, 8, 583. [Google Scholar] [CrossRef]
  56. Di Biagio, C.; Dellacqua, Z.; Martini, A.; Huysseune, A.; Scardi, M.; Witten, P.E.; Boglione, C. A baseline for skeletal investigations in Medaka (Oryzias latipes): The effects of rearing density on the postcranial phenotype. Front. Endocrinol. 2022, 13, 893699. [Google Scholar] [CrossRef]
  57. Kumar, N.; Carletti, A.; Gavaia, P.J.; Muller, M.; Cancela, M.L.; Geurts, P.; Marée, R. Deep Learning Approaches for Head and Operculum Segmentation in Zebrafish Microscopy Images. In Proceedings of the Computer Analysis of Images and Patterns: 19th International Conference, CAIP 2021, Virtual Event, 28–30 September 2021; pp. 154–164. [Google Scholar]
  58. Marée, R.; Rollus, L.; Stévens, B.; Hoyoux, R.; Louppe, G.; Vandaele, R.; Begon, J.M.; Kainz, P.; Geurts, P.; Wehenkel, L. Collaborative analysis of multi-gigapixel imaging data using Cytomine. Bioinformatics 2016, 32, 1395–1401. [Google Scholar] [CrossRef]
  59. Kumar, N.; Biagio, C.D.; Dellacqua, Z.; Raman, R.; Martini, A.; Boglione, C.; Muller, M.; Geurts, P.; Marée, R. Empirical Evaluation of Deep Learning Approaches for Landmark Detection in Fish Bioimages. In Proceedings of the European Conference on Computer Vision; Springer: Cham, Switzerland, 2022; pp. 470–486. [Google Scholar]
  60. Bergen, D.J.; Kague, E.; Hammond, C.L. Zebrafish as an emerging model for osteoporosis: A primary testing platform for screening new osteo-active compounds. Front. Endocrinol. 2019, 10, 6. [Google Scholar] [CrossRef] [PubMed]
  61. Caetano da Silva, C.; Ostertag, A.; Raman, R.; Muller, M.; Cohen-Solal, M.; Collet, C. wnt11f2 Zebrafish, an Animal Model for Development and New Insights in Bone Formation. Zebrafish 2023, 20, 1–9. [Google Scholar] [CrossRef] [PubMed]
  62. Foessl, I.; Bassett, J.; Bjørnerem, Å.; Busse, B.; Calado, Â.; Chavassieux, P.; Christou, M.; Douni, E.; Fiedler, I.A.; Fonseca, J.E.; et al. Bone phenotyping approaches in human, mice and zebrafish–Expert overview of the EU cost action GEMSTONE (“GEnomics of MusculoSkeletal traits TranslatiOnal NEtwork”). Front. Endocrinol. 2021, 12, 1476. [Google Scholar] [CrossRef]
  63. Dietrich, K.; Fiedler, I.A.; Kurzyukova, A.; López-Delgado, A.C.; McGowan, L.M.; Geurtzen, K.; Hammond, C.L.; Busse, B.; Knopf, F. Skeletal biology and disease modeling in zebrafish. J. Bone Miner. Res. 2021, 36, 436–458. [Google Scholar] [CrossRef]
  64. Caetano da Silva, C.; Edouard, T.; Fradin, M.; Aubert-Mucca, M.; Ricquebourg, M.; Raman, R.; Salles, J.P.; Charon, V.; Guggenbuhl, P.; Muller, M.; et al. WNT11, a new gene associated with early onset osteoporosis, is required for osteoblastogenesis. Hum. Mol. Genet. 2022, 31, 1622–1634. [Google Scholar] [CrossRef] [PubMed]
  65. Cotti, S.; Huysseune, A.; Larionova, D.; Koppe, W.; Forlino, A.; Witten, P.E. Compression fractures and partial phenotype rescue with a low phosphorus diet in the Chihuahua zebrafish osteogenesis imperfecta model. Front. Endocrinol. 2022, 13, 851879. [Google Scholar] [CrossRef] [PubMed]
  66. Tonelli, F.; Bek, J.W.; Besio, R.; De Clercq, A.; Leoni, L.; Salmon, P.; Coucke, P.J.; Willaert, A.; Forlino, A. Zebrafish: A resourceful vertebrate model to investigate skeletal disorders. Front. Endocrinol. 2020, 11, 489. [Google Scholar] [CrossRef]
  67. Tacon, A.G.; Metian, M. Fish matters: Importance of aquatic foods in human nutrition and global food supply. Rev. Fish. Sci. 2013, 21, 22–38. [Google Scholar] [CrossRef]
  68. Toni, M.; Manciocco, A.; Angiulli, E.; Alleva, E.; Cioni, C.; Malavasi, S. Review: Assessing fish welfare in research and aquaculture, with a focus on European directives. Animal 2019, 13, 161–170. [Google Scholar] [CrossRef] [PubMed]
  69. Yang, X.; Zhang, S.; Liu, J.; Gao, Q.; Dong, S.; Zhou, C. Deep learning for smart fish farming: Applications, opportunities and challenges. Rev. Aquac. 2021, 13, 66–90. [Google Scholar] [CrossRef]
  70. Li, D.; Du, L. Recent advances of deep learning algorithms for aquacultural machine vision systems with emphasis on fish. Artif. Intell. Rev. 2022, 55, 4077–4116. [Google Scholar] [CrossRef]
  71. Tonachella, N.; Martini, A.; Martinoli, M.; Pulcini, D.; Romano, A.; Capoccioni, F. An affordable and easy-to-use tool for automatic fish length and weight estimation in mariculture. Sci. Rep. 2022, 12, 15642. [Google Scholar] [CrossRef] [PubMed]
  72. Hough, C. Manual of control of malformations in fish aquaculture. Science and Practice. In Federation of European Aquaculture Producers; RapidPRess: Luxembourg, 2009; Volume 150. [Google Scholar]
  73. Dara, M.; Carbonara, P.; La Corte, C.; Parrinello, D.; Cammarata, M.; Parisi, M.G. Fish Welfare in Aquaculture: Physiological and Immunological Activities for Diets, Social and Spatial Stress on Mediterranean Aqua Cultured Species. Fishes 2023, 8, 414. [Google Scholar] [CrossRef]
  74. Boglione, C.; Costa, C. Skeletal deformities and juvenile quality. In Sparidae; Wiley: Hoboken, NJ, USA, 2011; pp. 233–294. [Google Scholar]
  75. Lubin, A.; Otterstrom, J.; Hoade, Y.; Bjedov, I.; Stead, E.; Whelan, M.; Gestri, G.; Paran, Y.; Payne, E. A versatile, automated and high-throughput drug screening platform for zebrafish embryos. Biol. Open 2021, 10, bio058513. [Google Scholar] [CrossRef] [PubMed]
  76. Kithcart, A.; MacRae, C.A. Using zebrafish for high-throughput screening of novel cardiovascular drugs. Basic Transl. Sci. 2017, 2, 1–12. [Google Scholar] [CrossRef]
  77. Laleh, N.G.; Muti, H.S.; Loeffler, C.M.L.; Echle, A.; Saldanha, O.L.; Mahmood, F.; Lu, M.Y.; Trautwein, C.; Langer, R.; Dislich, B.; et al. Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology. Med. Image Anal. 2022, 79, 102474. [Google Scholar] [CrossRef]
  78. Mac Namee, B.; Cunningham, P.; Byrne, S.; Corrigan, O.I. The problem of bias in training data in regression problems in medical decision support. Artif. Intell. Med. 2002, 24, 51–70. [Google Scholar] [CrossRef]
  79. Havaei, M.; Davy, A.; Warde-Farley, D.; Biard, A.; Courville, A.; Bengio, Y.; Pal, C.; Jodoin, P.M.; Larochelle, H. Brain tumor segmentation with deep neural networks. Med. Image Anal. 2017, 35, 18–31. [Google Scholar] [CrossRef]
  80. Rubens, U.; Mormont, R.; Paavolainen, L.; Bäcker, V.; Pavie, B.; Scholz, L.A.; Michiels, G.; Maška, M.; Ünay, D.; Ball, G.; et al. BIAFLOWS: A collaborative framework to reproducibly deploy and benchmark bioimage analysis workflows. Patterns 2020, 1, 100040. [Google Scholar] [CrossRef] [PubMed]
  81. Paul-Gilloteaux, P.; Tosi, S.; Hériché, J.K.; Gaignard, A.; Ménager, H.; Marée, R.; Baecker, V.; Klemm, A.; Kalaš, M.; Zhang, C.; et al. Bioimage analysis workflows: Community resources to navigate through a complex ecosystem. F1000Research 2021, 10, 320. [Google Scholar] [CrossRef] [PubMed]
  82. Mormont, R.; Geurts, P.; Marée, R. Comparison of deep transfer learning strategies for digital pathology. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA, 18–22 June 2018; pp. 2262–2271. [Google Scholar]
  83. Shorten, C.; Khoshgoftaar, T.M. A survey on image data augmentation for deep learning. J. Big Data 2019, 6, 60. [Google Scholar] [CrossRef]
  84. Hartley, M.; Kleywegt, G.J.; Patwardhan, A.; Sarkans, U.; Swedlow, J.R.; Brazma, A. The BioImage archive–building a home for life-sciences microscopy data. J. Mol. Biol. 2022, 434, 167505. [Google Scholar] [CrossRef] [PubMed]
  85. Ras, G.; Xie, N.; van Gerven, M.; Doran, D. Explainable Deep Learning: A Field Guide for the Uninitiated. J. Artif. Int. Res. 2022, 73, 329–396. [Google Scholar] [CrossRef]
  86. Sun, M.; Yang, X.; Xie, Y. Deep learning in aquaculture: A review. J. Comput. 2020, 31, 294–319. [Google Scholar]
  87. Rastegari, H.; Nadi, F.; Lam, S.S.; Abdullah, M.I.; Kasan, N.A.; Rahmat, R.F.; Mahari, W.A.W. Internet of Things in aquaculture: A review of the challenges and potential solutions based on current and future trends. Smart Agric. Technol. 2023, 4, 100187. [Google Scholar] [CrossRef]
  88. Kelasidi, E.; Svendsen, E. Robotics for Sea-Based Fish Farming. In Encyclopedia of Smart Agriculture Technologies; Springer: Berlin/Heidelberg, Germany, 2023; pp. 1–20. [Google Scholar]
  89. Wang, C.; Li, Z.; Wang, T.; Xu, X.; Zhang, X.; Li, D. Intelligent fish farm—The future of aquaculture. Aquac. Int. 2021, 29, 2681–2711. [Google Scholar] [CrossRef]
  90. Sawaki, R.; Sato, D.; Nakayama, H.; Nakagawa, Y.; Shimada, Y. ZF-AutoML: An Easy Machine-Learning-Based Method to Detect Anomalies in Fluorescent-Labelled Zebrafish. Inventions 2019, 4, 72. [Google Scholar] [CrossRef]
  91. Carreira, M.J.; Vila-Blanco, N.; Cabezas-Sainz, P.; Sánchez, L. Zftool: A software for automatic quantification of cancer cell mass evolution in zebrafish. Appl. Sci. 2021, 11, 7721. [Google Scholar] [CrossRef]
  92. Reischl, M.; Bartschat, A.; Liebel, U.; Gehrig, J.; Müller, F.; Mikut, R. ZebrafishMiner: An open source software for interactive evaluation of domain-specific fluorescence in zebrafish. Curr. Dir. Biomed. Eng. 2017, 3, 199–202. [Google Scholar] [CrossRef]
  93. Navarro, A.; Lee-Montero, I.; Santana, D.; Henríquez, P.; Ferrer, M.A.; Morales, A.; Soula, M.; Badilla, R.; Negrín-Báez, D.; Zamorano, M.J.; et al. IMAFISH_ML: A fully-automated image analysis software for assessing fish morphometric traits on gilthead seabream (Sparus aurata L.), meagre (Argyrosomus regius) and red porgy (Pagrus pagrus). Comput. Electron. Agric. 2016, 121, 66–73. [Google Scholar] [CrossRef]
Figure 1. A typical CNN architecture for image classification tasks.
Figure 1. A typical CNN architecture for image classification tasks.
Biomolecules 13 01797 g001
Figure 2. Different phenotypes in zebrafish tail. Larvae were imaged live under a dissecting microscope under transmitted light illumination: (A) Downward curved tail; (B) Upward curved tail; (C) Short tail; (D) Normal phenotype [29].
Figure 2. Different phenotypes in zebrafish tail. Larvae were imaged live under a dissecting microscope under transmitted light illumination: (A) Downward curved tail; (B) Upward curved tail; (C) Short tail; (D) Normal phenotype [29].
Biomolecules 13 01797 g002
Figure 3. Segmentation results obtained with deep learning on a zebrafish image [57]. Zebrafish larvae were stained for mineralized tissues using Alizarin red and imaged using a fluorescence microscope: (Left) Input image; (Right) Analyzed image.
Figure 3. Segmentation results obtained with deep learning on a zebrafish image [57]. Zebrafish larvae were stained for mineralized tissues using Alizarin red and imaged using a fluorescence microscope: (Left) Input image; (Right) Analyzed image.
Biomolecules 13 01797 g003
Figure 4. Six landmark locations on a medaka juvenile stained with Alizarin red for calcified tissues from microscopy dataset [59]. These landmarks are as follows: 1: rostral tip of the premaxilla; 2: base of the neural arch of the 1st (anteriormost) abdominal vertebra bearing a rib; 3: base of the neural post-zygapophyses of the first hemal vertebra (viz., vertebra with hemal arch closed by a hemaspine); 4: base of the neural post-zygapophyses of the first preural vertebra; 5: base of the neural post-zygapophyses of the preural-2 vertebra; 6: posteriormost (caudad) ventral extremity of the hypural 1.
Figure 4. Six landmark locations on a medaka juvenile stained with Alizarin red for calcified tissues from microscopy dataset [59]. These landmarks are as follows: 1: rostral tip of the premaxilla; 2: base of the neural arch of the 1st (anteriormost) abdominal vertebra bearing a rib; 3: base of the neural post-zygapophyses of the first hemal vertebra (viz., vertebra with hemal arch closed by a hemaspine); 4: base of the neural post-zygapophyses of the first preural vertebra; 5: base of the neural post-zygapophyses of the preural-2 vertebra; 6: posteriormost (caudad) ventral extremity of the hypural 1.
Biomolecules 13 01797 g004
Figure 5. Sample image of zebrafish with 25 landmark locations. Alizarin red staining was performed on fixed larvae and imaged under a dissecting microscope. The landmark locations are annotated as follows: 1 and 24: Maxilla; 2 and 23: Branchiostegal ray 2; 3 and 11: Opercle; 4, 12, 13, and 14: Cleithrum; 5 and 19: Anguloarticular; 6 and 25: Ceratobranchial; 7 and 8: Hyomandibular; 9 and 20: Entopterygoid; 10: Notochord; 21, 15, and 18: Parasphenoid; 17 and 22: Dentary; 16: showing anterior end marking.
Figure 5. Sample image of zebrafish with 25 landmark locations. Alizarin red staining was performed on fixed larvae and imaged under a dissecting microscope. The landmark locations are annotated as follows: 1 and 24: Maxilla; 2 and 23: Branchiostegal ray 2; 3 and 11: Opercle; 4, 12, 13, and 14: Cleithrum; 5 and 19: Anguloarticular; 6 and 25: Ceratobranchial; 7 and 8: Hyomandibular; 9 and 20: Entopterygoid; 10: Notochord; 21, 15, and 18: Parasphenoid; 17 and 22: Dentary; 16: showing anterior end marking.
Biomolecules 13 01797 g005
Figure 6. Sample X-ray image of pre-ongrowing phase gilthead seabream with 19 landmarks [59]. The image was obtained in air, on euthanized fish. The landmark locations are as follows: A: frontal tip of premaxillary; B: rostral head point in line with the eye center; C: dorsal head point in line with the eye center; D: dorsal extremity of the 1st predorsal bone; E: edge between the dorsal 1st hard ray pterygophore and hard ray; F: edge between the dorsal 1st soft ray pterygophore and soft ray; G: edge between the dorsal last soft ray pterygophore and soft ray; H: dorsal concave inflexion point of caudal peduncle; I: middle point between the bases of hypurals 2 and 3 (fork); L: ventral concave inflexion point of caudal peduncle; M: edge between the anal last pterygophore and ray; N: edge between the anal 1st ray pterygophore and ray; O: insertion of the pelvic fin on the body profile; P: preopercle ventral insertion on body profile; Q: frontal tip of dentary; R: neural arch insertion on the 1st abdominal vertebral body; S: neural arch insertion on the 1st hemal vertebral body; T: neural arch insertion on the 6th hemal vertebral body; U: between the pre- and post-zygapophyses of the 1st and 2nd caudal vertebral bodies.
Figure 6. Sample X-ray image of pre-ongrowing phase gilthead seabream with 19 landmarks [59]. The image was obtained in air, on euthanized fish. The landmark locations are as follows: A: frontal tip of premaxillary; B: rostral head point in line with the eye center; C: dorsal head point in line with the eye center; D: dorsal extremity of the 1st predorsal bone; E: edge between the dorsal 1st hard ray pterygophore and hard ray; F: edge between the dorsal 1st soft ray pterygophore and soft ray; G: edge between the dorsal last soft ray pterygophore and soft ray; H: dorsal concave inflexion point of caudal peduncle; I: middle point between the bases of hypurals 2 and 3 (fork); L: ventral concave inflexion point of caudal peduncle; M: edge between the anal last pterygophore and ray; N: edge between the anal 1st ray pterygophore and ray; O: insertion of the pelvic fin on the body profile; P: preopercle ventral insertion on body profile; Q: frontal tip of dentary; R: neural arch insertion on the 1st abdominal vertebral body; S: neural arch insertion on the 1st hemal vertebral body; T: neural arch insertion on the 6th hemal vertebral body; U: between the pre- and post-zygapophyses of the 1st and 2nd caudal vertebral bodies.
Biomolecules 13 01797 g006
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kumar, N.; Marée, R.; Geurts, P.; Muller, M. Recent Advances in Bioimage Analysis Methods for Detecting Skeletal Deformities in Biomedical and Aquaculture Fish Species. Biomolecules 2023, 13, 1797. https://doi.org/10.3390/biom13121797

AMA Style

Kumar N, Marée R, Geurts P, Muller M. Recent Advances in Bioimage Analysis Methods for Detecting Skeletal Deformities in Biomedical and Aquaculture Fish Species. Biomolecules. 2023; 13(12):1797. https://doi.org/10.3390/biom13121797

Chicago/Turabian Style

Kumar, Navdeep, Raphaël Marée, Pierre Geurts, and Marc Muller. 2023. "Recent Advances in Bioimage Analysis Methods for Detecting Skeletal Deformities in Biomedical and Aquaculture Fish Species" Biomolecules 13, no. 12: 1797. https://doi.org/10.3390/biom13121797

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop