Combining Image-Based Phenotyping and Multivariate Analysis to Estimate Fruit Fresh Weight in Segregation Lines of Lowland Tomatoes

Q1

Introduction

In the introduction, it is necessary to add information about phenotyping techniques and include examples of current relevant works on high-precision phenotyping

A1

Thank you for your suggestion. We have improved it according to your suggestions

Revision

Lines 92-104: The level of accuracy in this technology is determined by the camera sensors used, such as RGB cameras, multispectral cameras, hyperspectral cameras, Lidar cameras, and MRI [30,31]. The higher precision of the camera sensor, the more comprehensive the data obtained, especially, when combined with automation and big data concepts such as high-throughput phenotyping [31–36]. In this context, RGB cameras represent an affordable tool which can be used to indirectly obtain precious information on plant and/or crop agronomic potential in a non-destructive way [30,37]. It can predict the potential of an object based on the expected goal with a low error rate [29,38–40], so that IBP characters based on RGB camera can be used to predict tomato fruit weight through modeling. This modeling will make the evaluation process more effective, both in the breeder selection and in the robotic harvest process

Q2

Materials and Methods

Cite and reference the software used in the assessments. Improve the quality of the presented figure.

A2

Thank you for your suggestion. We have improved it according to your suggestions

Revision

Line 196-197: Fiji® open-source software (Schindelin et al. 2012) was used to semi-automatically obtain phenotypic measurements from individual scans.

Line 206-209: To this end, the HSB histogram of the whole stack was computed, and the ideal threshold (THR) automatically identified as the point of maximum distance between the histogram and the line connecting its peak to the farthest end (Woolf et al. 2021).

Line 218-219: Meanwhile, the use of Fiji software in image processing on tomatoes was also reported by Ayenan et al. (2020), and Baraj et al. (2021).

Lines 274-276: Image-based phenotyping characters with a significant positive correlation are contin-ued in path and multiple regression analyses with agricolae package in Rstudio 3.6.3 [59].

Q3

Results

Enhance the quality of the presented figure and modify the editing of Figure 2 so that the parts with a manifestation equal to 0 are not blank in the figure and display the values.

A3

Thank you for your suggestion. We have improved it according to your suggestions

Revision

Q4

Discussion

In the discussion, we will further discuss high-precision phenotyping and its applicability in plant genetic improvement.

Citation of works regarding the use of images in data processing and the use of multispectral data.

A4

Thank you for your suggestions. We have added some information for further discussion of high-precision phenotyping

Revision

Lines 587-595: However, the development of this model still needs to be optimized by using more precise sensors or high-precision phenotypes. Developing concepts with high-precision phenotyping (lmultipectral camera, hyperspectral camera, etc) will produce increasingly complex data, so more comprehensive analysis is needed in predicting fruit weight and other characters such as biotic, fruit quality and abiotic stress [30,35,106–108]. Moreover, this concept is focused on robot-based harvesting. However, overall, these results provide a good basis for determining the potential fruit weight per row non-destructively in tomato breeding

Q5

I also suggest citing works where, in comparison with other predictive techniques, regression stands out and in which situations this occurs.

A5

Thank you for your suggestions. We have added some information for comparision with other  studies.

Revision

Lines 576-578: These regression results are also supported by Nyalala et al. [28] and [40], where the destructive regression of whole fruit area in this study had high R² and R²-adjusted and low RMSE like this study

Q1

It’s better to mention the information of observed characters in abstract and explain the relationships between these characters and the fruit weight.

A1

Thank you for your suggestions. We have added information regarding the observed characters in abstract

Revision

Lines 35-38: Several phenotypic traits were extracted from each image, including the slice and whole fruit area (FA), round (FR), width (FW), height (FH), and red (RI), green (GI) and blue index (BI). Such morpho-colorimetric were used as inputs of a genetic and multivariate analysis for non-destructively predicting the fresh weight of the fruit (FFW)

Q2

In introduction, more information about why these observed characters should be measured for weight evaluation should be given.

A2

Thank you for your suggestions. We have added information regarding the IBP characters to predict the furit weight in introduction

Revision

Lines 102-104: so that IBP characters can be used to predict tomato fruit weight through modeling. This modeling will make the evaluation process more effective, both in the breeder selection and in the robotic harvest process

Q3

  In nondestructive validation of the model, the plant was shoot before a white background. How to ensure the model performance when it applied in field?

A3

Thank you for your question. We used a white background because we wanted to highlight the scale of our photo ruler. Moreover, we know that the most identical IBP character in fruit weight prediction is full area, so scale is very important in the success of this data. In addition, this development is still in the early stages so the process of optimizing the model at scale is important to carry out. Based on these considerations, in this study we used a white background to non-destructively optimize the scale of fruit photos. Future research will optimize the use of labeling in looking at the potential of a strain as a whole, so that the level of complexity will be higher than at this early stage of development.

Revision

Lines 178-219: Whole and sliced fruits were detached from the tested tomato plants (five fruits per line) and imaged for phenotypic traits extraction. Each sample was placed onto a portable lightbox photography studio (50 cm x 50 cm x 50 cm) and illuminated with an 8-watt LED light against a white background to better distinguish the fruit shape. Taking photos is also arranged by placing the fruit in the best or widest position, so that the fruit has an optimal area to photograph. This is so that the photo can represent the true character. A Canon® EF 28-135mm f/3.5-5.6 commercial camera (Canon Inc., Tokyo, Japan) was fixed on the top-hole of the box ensuring a 90-degree vertical shooting. For each sample, 1 top-view RGB image was collected by setting the camera with 5.6 F-stop, 1/160 second exposure time, ISO 800 and no-flash mode. The scene also included a graduated ruler with markings every 1 mm as a reference scale to xy-metrically calibrate the image. Meanwhile, non-destructive shooting is done with a white background too. This aims to sharpen the ruler scale when taking photos of fruit directly in the field, so that the model could be optimize in prediction . In addition, non-destructive photography is also done by taking the best or widest angle of the target fruit in the field. This also aims to ensure that the model formed at the initial development stage can be optimized in predicting fruit weight as the main character

Q4

The reason of the calculation coefficient from the observed characters to the fruit weight should be addressed more according to the table 1 and figure 2

A4

Thank you for your advice. We have added several sentences in the discussion according to your suggestions. However, in particular, the selection of intact fruit fields is a combination of various approaches, namely heritability, correlation, path analysis, stepwise regression and gain advance. These results explain that whole fruit area is the best character in predicting fruit weight. Table 1 and Table 2 only roughly select characters that have the potential to be continued in a more in-depth analysis. Therefore, we explain this concept thoroughly and do not just focus on table 1 and figure 2.

Revision

Lines 452-455: In general, heritability is a genetic parameter that can predict the role of genetics on phenotype, so it is important to calculate, estimate and compare each character based on its heritability [60,61], especially toward the main characters like fruit weight [6].

Lines 479-481: Almost all of these characters were identified as having good heritability compared to fruit weight, except slice fruit area and slice fruit width (Table 1). This indicates that in general the character . All of these characters were analyzed in more depth using mul-tivariate analysis.

Lines 533-540: On the other hand, the whole fruit width character has a better direct influence and heritability value than the whole fruit area (Tables 1 and 2). However, this potential is still considered rough and not enough to make the whole fruit width a selection crite-rion for fruit weight based on the genetic validation approach. In addition, the character of whole fruit area has a very good correlation with whole fruit width, so that whole fruit area can still represent the potential influence of whole fruit width on fruit weight. Therefore, the intact fruit area character is emphasized as a secondary and main char-acter in forming a prediction model for tomato fruit weight.

Q5

The imaging angle affects the projection shape and size of fruits, which is not explained enough in current version.

A5

Thank you for your suggestion. We have added information in MM section. We explain more that how we get the pictures according to your suggestion

Revision

Lines 182-185: Taking photos in destructive concept is also arranged by placing the fruit in the best or widest position, so that the fruit has an optimal area to photograph. This is so that the photo can represent the true character

Lines 192-195: In addition, non-destructive photography is also done by taking the best or widest angle of the target fruit in the field. This also aims to ensure that the model formed at the initial development stage can be optimized in predicting fruit weight as the main character

Q6

  In real field or greenhouse, the image of the fruit would have complex background which lead to the error of fruit contour extraction. How to control the impact of this error in algorithm design?

A6

Thank you for your question. We agree with your statement that in the field or in the greenhouse, fruit images in the plant have complex background which leads to the error of fruit contour extraction. However, this research is an initial stage which is more focused on the concept of plant breeding, so we controlled it by using a white background and a ruler as the scale. In general, breeders are often overwhelmed in handling large tomato populations, especially in characterization or observing fruit weight which is very identical to water content. This is a solution for breeders in evaluating their lines both destructively and non-destructively, especially on tomato fruit characteristics. The next development will be more complex to assess the productivity performance in the line and its relationship to the robotic concept in the complex harvesting process. It is possible that a more complex algorithm in the fruit labeling process will be optimized because the fruit background is no longer white but is natural compared to the existing background. However, this result is enough to be a good solution in the initial stages of developing the concept.

Revision

-

Q7

The whole fruit area was the only one factor for the fruit weight model formulation. Is there any possible linear or nonlinear relationships with other factors?

A7

Thank you for your question. Based on the results of analysis on various populations of breeding generations, it shows that whole fruit area is a character that has a high correlation with fruit weight. Apart from that, if analyzed using general linear regression, the regression results show very good values, so for this first stage of the process we assume that the whole fruit area character is a strong character for predicting fruit weight. However, this stage is still in the initial development stage which focuses on independent fruit, so there is a possibility that the model will become more complex in the development stage, especially in the concept of robot-based tomato harvesting. This allows for other IBP characteristics to influence the model and the possibility that the regression in general is no longer linear as in this study.

Revision

-

Q1

Line 118: Why were five tomato samples taken from all plants in each generation except F5?

A1

Thank you very much for your suggestion. The number of tomatoes at the beginning of the generation was greater than at F5 for several reasons. First, the population in the first generation is relatively more diverse than the next generation. Apart from that, the F5 generation is also the material used for the final or non-destructive validation process, so the number of tomatoes for each genotype used is not as large as for model formation.

Revision

-

Q2

Line 127:  Information about the content of the AB mix fertilizer used should be included. There is already a relationship between plant nutrition and fruit size. Therefore, the fertilizer content should be written clearly. For example; AB Mix is NO3 = 9.9%, NH4 = 0.48%, P2O5 = 4.83%, K2O = 16.50%, MgO = 2.83%, CaO = 11.48%, SO3 = 3.81 etc.

A2

Thank you for your suggestion. We have adde this information according to your suggestion

Revision

Line 147-149: Goodplant Ab mix fertilizer (N Total = 20.7%, P2O5 = 5.1%, K2O = 24.80%, MgO = 5.1%, CaO = 14.5%, S= 8.9%, Fe = 0.10%, Mn= 0.05%, Cu= 0.05%, B = 0.03%, Zn = 0,02%, Mo = 0.001%)

Q3

Line 137:  Should the element rates in Mutaria NPK fertilizer be written? like 16 16 16

A3

Thank you for your suggestion. We have added in MS according to your suggestion

Revision

Line 159: Mutiara NPK (16:16:16) fertilizer

Q4

Line 147: The recommended dose of Antracol 70 WP in tomatoes is 3g/l. Why did you use 2 g/l? Pesticides are substances harmful to human health. It may be considered to use low doses for human health, but can sometimes using such substances in low doses increase the resistance of plant pathogens in the long term?

A4

Thank you very much for your questions. I think so for your think that Antracol 70 WP in tomatoes is 3 g/L. However, in this research we including just 2 g/l. This is because we see this only as part of prevention as problems related to fungal pathogens did not appear significant in this study. In addition, we are developing a strain so it would be best to use less pesticide to see the resistance potential of that tomato lines.

Revision

-

Q1

I suggest improvements in the texts that explain the methodological steps in Figure 1.

A1

Thank you very much for your suggestion. we have improved it according to your suggestions

Revision

Lines 178-219 : Whole and sliced fruits were detached from the tested tomato plants (five fruits per line) and imaged for phenotypic traits extraction. Each sample was placed onto a portable lightbox photography studio (50 cm x 50 cm x 50 cm) and illuminated with an 8-watt LED light against a white background to better distinguish the fruit shape. Taking photos in destructive concept is also arranged by placing the fruit in the best or widest position, so that the fruit has an optimal area to photograph. This is so that the photo can represent the true character. A Canon® EF 28-135mm f/3.5-5.6 commercial camera (Canon Inc., Tokyo, Japan) was fixed on the top-hole of the box ensuring a 90-degree vertical shooting. For each sample, 1 top-view RGB image was collected by setting the camera with 5.6 F-stop, 1/160 second exposure time, ISO 800 and no-flash mode. The scene also included a graduated ruler with markings every 1 mm as a reference scale to xy-metrically calibrate the image. Meanwhile, non-destructive shooting is done with a white background too. This aims to sharpen the ruler scale when taking photos of fruit directly in the field, so that the model could be optimize in prediction . In addition, non-destructive photography is also done by taking the best or widest angle of the target fruit in the field. This also aims to ensure that the model formed at the initial development stage can be optimized in predicting fruit weight as the main character.

Fiji® open-source software (Schindelin et al. 2012) was used to semi-automatically obtain phenotypic measurements from individual scans. Firstly, the ruler markings were manually selected and used to calculate a scaling factor for obtaining absolute morphometric values. Subsequently, foreground objects (i.e., fruit) were segmented from the background pixels by applying a triangle thresholding method. In particular, the original RGB colorspace was automatically converted to a Hue, Saturation, Brightness (HSB) stack to emphasize spectral dissimilarities between vegetal and non-vegetal features. In the HSB model, H (0-360°) differentiates pure colors while S (0-100%) and B (0-100%) characterizes the shade and the overall brightness of the color, respectively, offering manipulable indicators for intuitive selection of foreground and background color-fingerprints. To this end, the HSB histogram of the whole stack was computed, and the ideal threshold (THR) automatically identified as the point of maximum distance between the histogram and the line connecting its peak to the farthest end (Woolf et al. 2021). Thus, the pixels outside THR were labelled as ‘0’ (i.e., background), while the remaining points were coded as ‘1’ (i.e., fruit). The original RGB image was superimposed to the resulted binary mask to retrieve solely the region of interest (ROI). Then, the ROI was automatically analysed to measure the whole and slice fruit area (FA), round (FR), width (FW), height (FH), and red (RI), green (GI) and blue index (BI). Specifically, FA was obtained as the surface of all the selected pixel (cm2), FR was the length of the outside boundary of the ROI (cm), while FW and FH represented the maximum x- and y-axis length of a fitted ellipse (cm), respectively. Finally, RI, GI and BI were calculated as the average red-, blue- and green-channel value of all the ROI pixels.

Q1

The study predicted tomato fruit weight using fruit structural parameters through simple linear regression. In my opinion, the study has low novelty, inappropriate design lacking a sharp justified focus, and no practical implications. The manuscript is poorly written with grammar issues, weak literature review, and vague methodology description.

A1

Thank you for your constructive input. We will use this as an evaluation of our future research. However, this research actually has the novelty of helping tomato breeders identify or characterize fruit properties non-destructively in the field. Most tomato fruit observation patterns are carried out destructively and use relatively small samples originating from the same fruit lot or lines. The differences between these fruits may be relatively subtle or simply caused by environmental differences. This is different from this research which builds models from diverse breeder material, both within populations, varying between generations, and varying between growing environments. This makes the distribution of fruit observation samples very wide, so that the model formed is stronger. In fact, it is also hoped that this research can further explore the development of existing models. However, based on systematic analysis and genetic character validation, only the IBP whole fruit area character can predict and select the potential of a line for fruit weight, so the analysis process is relatively simple. However, we realize that this observation still needs to be developed more widely, especially in the use of high camera sensors. The camera used is still based on an RGB camera, so using a camera with high resolution has the opportunity to produce better novelties in the future. However, overall for this early development stage, the use of RGB cameras has been considered good in predicting fruit and helping breeders to select the best lines to plant in the next generation.

Revision

-

Q2

Fruit weight estimation is not a novel research topic by any means, even according to the claimed image-based phenotyping (IBP) method by the manuscript. With that being said, not a single study that explored fruit weight estimation was properly reviewed in introduction, let alone a comprehensive literature review. As a result, no actual justification was provided for the study, despite line 98-100 attempted to do so. No specific, technical knowledge gaps are identified in current literature.

A2

Thank you for your constructive input. Kami telah memperjelas pendahuluan kami terakit literature yang menjelaskan penggunaan IBP dalam memprediksi tomato fruit weight. Penelitian kami memiliki perbedaan yang jelas dibandingkan beberapa penelitian yang ada, dimana kami menggunakan dua pendekatan, yakni destructive dan non-destructive concepts. Selain itu, kami menggunakan beberapa galur generasi tomat yang memiliki tingkat kompleksitas lebih tinggi dibandingkan penelitian sebelumnya, apalagi galur tomat tersebut ditanam pada dataran rendah yang bukan merupakan habitat umumnya. Hal ini menjadikan data sangat beragam dan dapat dijadikan sebagai dasar pemodelan yang robust dibandingkan hanya berdasarkan tomat terseleksi dari galur yang homogen. 

Revision

Lines 90-107: Image-based phenotyping (IBP) is a digitalization approach in the 4.0 era. This technology utilizes images obtained from an object and analyzed precisely with high accuracy [26–29]. The level of accuracy in this technology is determined by the camera sensors used, such as RGB cameras, multispectral cameras, hyperspectral cameras, Lidar cameras, and MRI [30,31]. The higher precision of the camera sensor, the more comprehensive the data obtained, especially, when combined with automation and big data concepts such as high-throughput phenotyping [31–36]. In this context, RGB cameras represent an affordable tool which can be used to indirectly obtain precious information on plant and/or crop agronomic potential in a non-destructive way [30,37]. It can predict the potential of an object based on the expected goal with a low error rate [29,38–40], so that IBP characters based on RGB camera can be used to predict tomato fruit weight through modeling. This modeling will make the evaluation process more effective, both in the breeder selection and in the robotic harvest process. Several research reports have also utilized IBP technology in the analysis of both tomato fruits [28,40–42] and canopy [43,44]. Specifically, Nyalala et al. [28,40] stated clearly regarding the effectiveness of using IBP in predicting tomato fruit weight. 

Q3

While the study aimed to predict fruit weight, for some reason a heritability analysis was added to the study, which in my opinion arguably is irrelevant to the core of the study yet occupied the most length of the manuscript. No clear research objectives to explain why the genotype aspect of the tomato plants matters for a phenotype modeling study.

A3

Thank you for your suggestion. Maybe there is a missed perception. In plant breeding, heritability is a tool to determine the role of genetics on phenotype when we use populations with a high level of diversity. This is important for breeders when they want to select plant genotypes (Acquaah 2007, Acquaah 2012, Syukur et al. 2015). Moreover, the general aim and basis of this topic material is closely related to plant breeding, where the general aim of this research is focused on assisting breeders in selecting genotypes based on fruit characteristics. Meanwhile, the material used is material from 5 generations (Rose Elders, Karina Elders, F2 generation, backcross generation, and F5 generation), so to ensure whether the model we are developing is correct, it is necessary to pay attention to its heritability. Therefore, we discuss the concept of this research more towards breeding as our novelty. There are not many publications that use IBP for multigenerational research like this research.

Revision

-

Q4

Methodology of the study was poorly explained. How tomato images are processed is unclear. Precise definitions of fruit height, width, total fruit area, fruit round, red index, green index, and blue index and how they were measured are unclear. How fruit weights were measured is unclear. Fruit weight modeling technique, predictor and response variables to the model are unclear. It is unclear whether any information obtained from the heritability analysis was utilized in the fruit weight modeling.

A4

Thank your for your correction. We have revised it according to your suggestion, especially for the image analysis flow

Revision

Lines 178-219: Whole and sliced fruits were detached from the tested tomato plants (five fruits per line) and imaged for phenotypic traits extraction. Each sample was placed onto a portable lightbox photography studio (50 cm x 50 cm x 50 cm) and illuminated with an 8-watt LED light against a white background to better distinguish the fruit shape. Taking photos in destructive concept is also arranged by placing the fruit in the best or widest position, so that the fruit has an optimal area to photograph. This is so that the photo can represent the true character. A Canon® EF 28-135mm f/3.5-5.6 commercial camera (Canon Inc., Tokyo, Japan) was fixed on the top-hole of the box ensuring a 90-degree vertical shooting. For each sample, 1 top-view RGB image was collected by setting the camera with 5.6 F-stop, 1/160 second exposure time, ISO 800 and no-flash mode. The scene also included a graduated ruler with markings every 1 mm as a reference scale to xy-metrically calibrate the image. Meanwhile, non-destructive shooting is done with a white background too. This aims to sharpen the ruler scale when taking photos of fruit directly in the field, so that the model could be optimize in prediction . In addition, non-destructive photography is also done by taking the best or widest angle of the target fruit in the field. This also aims to ensure that the model formed at the initial development stage can be optimized in predicting fruit weight as the main character.

Fiji® open-source software (Schindelin et al. 2012) was used to semi-automatically obtain phenotypic measurements from individual scans. Firstly, the ruler markings were manually selected and used to calculate a scaling factor for obtaining absolute morphometric values. Subsequently, foreground objects (i.e., fruit) were segmented from the background pixels by applying a triangle thresholding method. In particular, the original RGB colorspace was automatically converted to a Hue, Saturation, Brightness (HSB) stack to emphasize spectral dissimilarities between vegetal and non-vegetal features. In the HSB model, H (0-360°) differentiates pure colors while S (0-100%) and B (0-100%) characterizes the shade and the overall brightness of the color, respectively, offering manipulable indicators for intuitive selection of foreground and background color-fingerprints. To this end, the HSB histogram of the whole stack was computed, and the ideal threshold (THR) automatically identified as the point of maximum distance between the histogram and the line connecting its peak to the farthest end (Woolf et al. 2021). Thus, the pixels outside THR were labelled as ‘0’ (i.e., background), while the remaining points were coded as ‘1’ (i.e., fruit). The original RGB image was superimposed to the resulted binary mask to retrieve solely the region of interest (ROI). Then, the ROI was automatically analysed to measure the whole and slice fruit area (FA), round (FR), width (FW), height (FH), and red (RI), green (GI) and blue index (BI). Specifically, FA was obtained as the surface of all the selected pixel (cm2), FR was the length of the outside boundary of the ROI (cm), while FW and FH represented the maximum x- and y-axis length of a fitted ellipse (cm), respectively. Finally, RI, GI and BI were calculated as the average red-, blue- and green-channel value of all the ROI pixels. Meanwhile, the use of Fiji software in image processing on tomatoes was also reported by Ayenan et al. (2020), and Baraj et al. (2021).

Q5

Grammar issues such as line 33, Table 4 “Karakter”.

A5

Thank your for your correction. We have revised according to your suggestion. Regarding to grammar, the MDPI will be facilated this service  

Revision

Table 4: Characters

Q6

Figure 3-5 do not have necessary units.

A6

Thank your for your correction. We have revised according to your suggestion.

Revision

Figure 3-5