vitisFlower®: Development and Testing of a Novel Android-Smartphone Application for Assessing the Number of Grapevine Flowers per Inflorescence Using Artificial Vision Techniques
Abstract
:1. Introduction
2. Experimental Section
2.1. Image Analysis for Flower Counting in Grapevine Inflorescences
- -
- Image pre-processing: this step basically consists of automatically segmenting the inflorescence from the background using color discrimination criteria (invariant to light conditions) for computing a region of interest (ROI).
- -
- Image analysis: in this step, the detection of flower candidates is achieved. Flowers are quasi-spherical in shape, so they produce a point of maximum light reflection. Therefore, flower candidates are identified in the area of the image delimited by the ROI as those connected components being regional maxima in the lightness channel of the Lab color space (concretely the Lab space used was CIE 1976 L*a*b* [24]).
- -
- Image post-processing: this final stage intends to remove those regional maxima not corresponding to real flowers in the image. It is carried out by sequentially applying these two statistical filters:
- Region size filter: removal of those candidates with a size larger than expected, taking into account the statistical size distribution of the candidates.
- Shape filter: due to the geometry of a flower, the area of maximum light reflection on its surface is expected to describe a quasi-circular shape; therefore, this filter eliminates those candidates describing elongated configurations.
2.2. Application’s Technical Overview
- The use of OpenCV4Android through Java code introduces limitations in image handling, which influence efficiency. In this respect, C/C++ provides higher versatility than Java, allowing image handling at lower levels.
- The power and versatility that is offered by C/C++ pointers allows one to reduce memory usage by reducing image and other variable copies. This advantage considerably helped to keep the application within the standard Android heap size.
- The implementation of the core using the standard C/C++ OpenCV library instead of its Android version outstandingly increases its portability, since there are OpenCV compilations for numerous operating systems, like iOS and Windows, among others.
- The use of the standard C/C++ OpenCV for implementing the image analysis algorithms facilitates their development and testing, since it can be carried out directly on a PC, avoiding the need for the use of smartphones or emulators to this effect.
2.3. Application’s Performance Description
- Implementation of a simple and friendly graphical user interface: the interface shows only the relevant information to the user, preferably by using symbols or illustrations instead of descriptive text.
- Linear execution: the application has a user-independent execution line in which the user is exempted from making any important decision. It allows the user to utilize the application without any knowledge about its internal performance.
- Home: the application shows the vitisFlower® logo along with basic information about the aims and authorship.
- Instructions for image capture: the application briefly informs the user of some basic notions for appropriately taking a photo.
- Image capture: the camera application available in the user’s Android device is invoked to make a capture. If the camera application is properly configured, it shows the captured image and allows one to discard it to take a new one in case the previous one was not properly acquired, for example because of the presence of leaves in the image, or due to the fact that the scene was not correctly focused, or it was overexposed.
- Image analysis: this state is transparent to the user. It is in charge of analyzing the image taken in the previous state for detecting and counting flowers. For reducing the computational workload, the image is scaled down to a resolution of 2 Mpx prior to its analysis.
- Results display: the results of the image analysis are presented to the user. On the one hand, the image with the detected flowers marked with red crosses is displayed. It easily allows one to graphically inspect the obtained results. On the other hand, the number of detected flowers is also shown. At this point, the user decides to save the results or to discard them if they are not satisfactory.
- Image storage: this stage is reached if the user decided to save the results in the previous step. The processed image is saved in a folder called “VitisFlowerImages” created by the application and located in the root folder of the device’s internal storage. The image is saved and named as follows [name]_[date]_[detected number of flowers].jpg, where:
- [name]: a dialog box allowing one to insert an image name. If it is omitted, this field takes the value “image”.
- [date]: the complete date of the image capture with the following format and information: day-month-year_hour.minutes.seconds.miliseconds.
- [detected number of flowers]: the number of flowers detected in the image.
2.4. Testing and Validation of the Application
2.4.1. Application’s Performance Evaluation
Feature | Price/Release Date | Sensor Model | Resolution | Lens Size | Aperture | ISO | |
---|---|---|---|---|---|---|---|
Device | |||||||
Sony Xperia Z2 | 549.0 €/2014 | Sony IMX220 | 20.7 Mpx | 1/2.3″ | f/2.0 | 50–800 | |
BQ Aquaris E5 | 209.90 €/2014 | Sony IMX214 | 13 Mpx | 1/3.2″ | f/2.2 | 100–1600 |
- TP: the number of flowers automatically detected corresponding to the actual flowers labelled in the gold standard.
- FP: the number of flowers automatically detected that do not correspond to actual flowers in the gold standard. Redundant TPs (a redundant true positive is when a flower is detected more than once) were also considered as FPs.
- FN: the number of actual flowers labelled in the gold standard that were not automatically found.
2.4.2. Application’s Computational Efficiency Study
Feature | Price/Release Date | Chipset | CPU | GPU | RAM Memory | Android Version | |
---|---|---|---|---|---|---|---|
Device | |||||||
Sony Xperia Z2 | 549.0 €/2014 | Qualcomm MSM8974AB Snapdragon 801 | Quad-core 2.3-GHz Krait 400 | Adreno 330 | 3 GB | 5.0.1 Jelly bean | |
Sony Xperia Z2 Tablet | 449.0 €/2014 | Qualcomm MSM8974AB Snapdragon 801 | Quad-core 2.3-GHz Krait 400 | Adreno 330 | 3 GB | 4.4.4 Kit kat | |
BQ Aquaris E5 | 209.90 €/2014 | MediaTek MT6582 | Quad-core 1.3-GHz ARM Cortex-A7 | Mali-400 MP2 | 1 GB | 4.4 Kit kat | |
Motorola Moto G | 172.0 €/2013 | Qualcomm MSM8226 Snapdragon 400 | Quad-core 1.2-GHz Cortex-A7 | Adreno 305 | 1 GB | 4.4.2 Kit kat |
- Closing all recent applications in the device.
- Selection of the flight mode.
- Re-starting the device.
- Waiting for 20 s for the operating system to completely load.
- Starting the benchmarking version of vitisFlower.
- Running the tests 5 times.
3. Results and Discussion
3.1. Results of Performance Evaluation
- Analyzing inflorescences facing the Sun. The opposite orientation leads to light reflection and refraction patterns that can negatively affect the results.
- Casting a shadow on the inflorescence to create a homogeneous scene. If the illumination is poor due to low natural-light conditions, the use of the camera flash is recommended.
Sony Xperia Z2 | BQ Aquaris E5 | ||||
---|---|---|---|---|---|
Variety | Variety | ||||
Airen | 0.8223 | 0.9787 | Merlot | 0.9173 | 0.9517 |
Cabernet Sauvignon | 0.8363 | 0.9615 | Cabernet Sauvignon | 0.8855 | 0.9531 |
Chardonnay | 0.8770 | 0.9339 | Chenin Blanc | 0.7987 | 0.9563 |
Grenache | 0.8045 | 0.9763 | Grenache | 0.8391 | 0.9685 |
Riesling | 0.8411 | 0.9458 | Riesling | 0.9035 | 0.9212 |
Syrah | 0.8889 | 0.9376 | Sauvignon Blanc | 0.8664 | 0.9557 |
Tempranillo | 0.8308 | 0.9851 | Semillon | 0.8826 | 0.9158 |
3.2. Results of the Study of Computational Efficiency
3.3. Significance of the vitisFlower® Application for the Wine Industry
4. Conclusions/Outlook
Acknowledgments
Author Contributions
Conflicts of Interest
References
- Nuske, S.; Achar, S.; Bates, T.; Narasimhan, S.; Singh, S. Yield Estimation in Vineyards by Visual Grape Detection. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA, 25–30 September 2011; pp. 2352–2358.
- Nuske, S.; Wilshusen, K.; Achar, S.; Yoder, L.; Narasimhan, S.; Singh, S. Automated Visual Yield Estimation in Vineyards. J. Field Robot. 2014, 31, 837–860. [Google Scholar] [CrossRef]
- Font, D.; Pallejà, T.; Tresanchez, M.; Teixidó, M.; Martinez, D.; Moreno, J.; Palacín, J. Counting Red Grapes in Vineyards by Detecting Specular Spherical Reflection Peaks in RGB Images obtained at Night with Artificial Illumination. Comput. Electron. Agric. 2014, 108, 105–111. [Google Scholar] [CrossRef]
- Diago, M.P.; Correa, C.; Millán, B.; Barreiro, P.; Valero, C.; Tardaguila, J. Grapevine Yield and Leaf Area Estimation using Supervised Classification Methodology on RGB Images Taken Under Field Conditions. Sensors 2012, 12, 16988–17006. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Roscher, R.; Herzog, K.; Kunkel, A.; Kicherer, A.; Töpfer, R.; Förstner, W. Automated Image Analysis Framework for High-Throughput Determination of Grapevine Berry Sizes using Conditional Random Fields. Comput. Electron. Agric. 2014, 100, 148–158. [Google Scholar] [CrossRef]
- Dunn, G.M.; Martin, S.R. Yield Prediction from Digital Image Analysis: A Technique with Potential for Vineyard Assessments Prior to Harvest. Aust. J. Grape Wine Res. 2004, 10, 196–198. [Google Scholar] [CrossRef]
- May, P. Flowering and Fruitset in Grapevines; Phylloxera and Grape Industry Board of South Australia: Adelaide, Australia, 2004. [Google Scholar]
- Martin, S.R.; Dunn, G.M.; Hoogenraad, T.; Krstic, M.P.; Clingeleffer, P.R.; Ashcroft, W.J. Crop Forecasting in Cool Climate Vineyards. In Proceedings for the 5th International Symposium on Cool Climate Viticulture and Enology, Melbourne, Australia, 16–20 January 2000.
- Clingeleffer, P.R. Plant Management Research: Status and what it can Offer to Address Challenges and Limitations. Aust. J. Grape Wine Res. 2010, 16, 25–32. [Google Scholar] [CrossRef]
- Matthews, M.; Nuzzo, V. Berry Size and Yield Paradigms on Grapes and Wines Quality. Acta Hortic. 2007, 754, 423–436. [Google Scholar]
- Dry, P.R.; Longbottom, M.L.; McLoughlin, S.; Johnson, T.E.; Collins, C. Classification of Reproductive Performance of Ten Winegrape Varieties. Aust. J. Grape Wine Res. 2010, 16, 47–55. [Google Scholar] [CrossRef]
- Carbonneau, A.; Deloire, A.; Jaillard, B. La Vigne: Physiologie, Terroir, Culture; Dunod: Paris, France, 2007. [Google Scholar]
- Coombe, B. The Effect of Removing Leaves, Flowers and Shoot Tips on Fruit-Set in Vitis Vinifera L. J. Hortic. Sci. 1962, 37, 1–15. [Google Scholar]
- Friend, A.P.; Trought, M.C.T. Delayed Winter Spur-Pruning in New Zealand can Alter Yield Components of Merlot Grapevines. Aust. J. Grape Wine Res. 2007, 13, 157–164. [Google Scholar] [CrossRef]
- Collins, C.; Dry, P.R. Response of Fruitset and Other Yield Components to Shoot Topping and 2-Chlorethyltrimethyl-Ammonium Chloride Application. Aust. J. Grape Wine Res. 2009, 15, 256–267. [Google Scholar] [CrossRef]
- Poni, S.; Casalini, L.; Bernizzoni, F.; Civardi, S.; Intrieri, C. Effects of Early Defoliation on Shoot Photosynthesis, Yield Components, and Grape Composition. Am. J. Enol. Vitic. 2006, 57, 397–407. [Google Scholar]
- Brown, K.; Jackson, D.I.; Steans, G.F. Effects of Chlormequat, Girdling, and Tipping on Berry Set in Vitis Vinifera L. Am. J. Enol. Vitic. 1988, 39, 91–94. [Google Scholar]
- Coombe, B. Fruit Set in Grape Vines: The Mechanism of the CCC Effect. J. Hortic. Sci. 1970, 45, 415–425. [Google Scholar]
- Longbottom, M.L.; Dry, P.R.; Sedgley, M. A Research Note on the Occurrence of “Star” Flowers in Grapevines: Observations during the 2003–2004 Growing Season. Aust. J. Grape Wine Res. 2004, 10, 199–202. [Google Scholar] [CrossRef]
- De Bei, R.; Hook, J.; Fuentes, S.; Gilliham, M.; Tyerman, S.; Collins, C. Linking Canopy Architecture to Grape Quality using the LAI Canopy App. In Proceedings of the 19th International Meeting of Viticulture GIESCO, Montpellier, France, 31 May 2015; pp. 585–588.
- Fuentes, S.; de Bei, R.; Pozo, C.; Tyerman, S. Development of a Smartphone Application to Characterise Temporal and Spatial Canopy Architecture and Leaf Area Index for Grapevines. Wine Vitic. J. 2012, 27, 56–60. [Google Scholar]
- Diago, M.P.; Sanz-Garcia, A.; Millan, B.; Blasco, J.; Tardaguila, J. Assessment of Flower Number Per Inflorescence in Grapevine by Image Analysis Under Field Conditions. J. Sci. Food Agric. 2014, 94, 1981–1987. [Google Scholar] [CrossRef] [PubMed]
- IDC: Smartphone OS Market Share 2015, 2014, 2013, and 2012. Available online: http://www.idc.com/prodserv/smartphone-os-market-share.jsp (accessed on 13 August 2015).
- Connolly, C.; Fliess, T. A study of efficiency and accuracy in the transformation from RGB to CIELAB color space. IEEE Trans. Image Process. 1997, 6, 1046–1047. [Google Scholar] [CrossRef] [PubMed]
- Lorenz, D.H.; Eichhorn, K.W.; Bleiholder, H.; Klose, R.; Meier, U.; Weber, E. Growth Stages of the Grapevine: Phenological Growth Stages of the Grapevine (Vitis Vinifera L. Ssp. Vinifera)? Codes and Descriptions According to the Extended BBCH Scale? Aust. J. Grape Wine Res. 1995, 1, 100–103. [Google Scholar] [CrossRef]
- Skene, K. A Comparison of Effects of Cycocel and Tipping on Fruit Set in Vitis Vinifera L. Aust. J. Biol. Sci. 1969, 22, 1305–1312. [Google Scholar]
- Christensen, P. Timing of Zinc Foliar Sprays. I. Effects of Application Intervals Preceding and during the Bloom and Fruit-Set Stages. II. Effects of Day VS. Night Application. Am. J. Enol. Vitic. 1980, 31, 53–59. [Google Scholar]
- Dunn, G.M.; Martin, S.R. The Current Status of Crop Forecasting in the Australian Wine Industry. In Proceedings of the ASVO Seminar Series: Grapegrowing at the Edge, Tanunda, Barossa Valley, South Australia, 10 July 2003; pp. 4–8.
- Mobile Penetration in Spain & Digital Users. Available online: http://www.neomobile-blog.com/spain-2015-mobile-market-digital-user-habits/ (accessed on 13 August 2015).
- Google Play. Available online: https://play.google.com/store/apps/details?id=com.ur.android.grapegenius (accessed on 13 August 2015).
© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Aquino, A.; Millan, B.; Gaston, D.; Diago, M.-P.; Tardaguila, J. vitisFlower®: Development and Testing of a Novel Android-Smartphone Application for Assessing the Number of Grapevine Flowers per Inflorescence Using Artificial Vision Techniques. Sensors 2015, 15, 21204-21218. https://doi.org/10.3390/s150921204
Aquino A, Millan B, Gaston D, Diago M-P, Tardaguila J. vitisFlower®: Development and Testing of a Novel Android-Smartphone Application for Assessing the Number of Grapevine Flowers per Inflorescence Using Artificial Vision Techniques. Sensors. 2015; 15(9):21204-21218. https://doi.org/10.3390/s150921204
Chicago/Turabian StyleAquino, Arturo, Borja Millan, Daniel Gaston, María-Paz Diago, and Javier Tardaguila. 2015. "vitisFlower®: Development and Testing of a Novel Android-Smartphone Application for Assessing the Number of Grapevine Flowers per Inflorescence Using Artificial Vision Techniques" Sensors 15, no. 9: 21204-21218. https://doi.org/10.3390/s150921204