Next Article in Journal
An Experimental Analysis of Drift Detection Methods on Multi-Class Imbalanced Data Streams
Previous Article in Journal
Artificial Intelligence and Robotics Led Technological Tremors: A Seismic Shift towards Digitizing the Legal Ecosystem
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Machine Learning-Based Classification of Mushrooms Using a Smartphone Application

1
Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA
2
Department of Plant and Botany Science, Purdue University, West Lafayette, IN 47907, USA
3
Bindley Bioscience Center, Purdue University, West Lafayette, IN 47907, USA
4
School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(22), 11685; https://doi.org/10.3390/app122211685
Submission received: 23 September 2022 / Revised: 26 October 2022 / Accepted: 4 November 2022 / Published: 17 November 2022

Abstract

:
Worldwide, a large number of cases of harmful mushroom exposure and consumption result in hallucinations, sickness, and death. One contributing factor is that certain poisonous mushrooms closely resemble their edible counterparts, making it difficult for general public collectors to distinguish one from the other. We propose a method to classify mushroom types from field-collection images using a smartphone application based on a convolutional neural network. The application helps people without proper mycology background or training to distinguish poisonous mushrooms from edible ones with which they may be confused. We showed three case studies to classify two-, three-, and five-class models by optimizing their training steps by cross-validation. An android app was developed by transferring the server-based trained model and allowing users to obtain probability scores for the correct genus classification. Our experiments showed that this method could provide sensitivity and specificity of two-, three-, and five-class mushroom models ranging from 89% to 100% using an image from the field with diverse backgrounds and objects.

1. Introduction

Mushrooms are a highly sought delicacy by both chefs and amateur mushroom cooks. However, morphological similarities between edible and poisonous species often result in the unintended consumption of harmful fungi with severe or even fatal consequences. For instance, Amanita is a genus of poisonous mushroom species. According to reports, a person consuming only five fried slices of Amanita can experience symptoms including central nervous system depression, ataxia, waxing, waning obtundation, religious hallucinations, and hyperkinetic behavior [1]. Other mushrooms can be also highly toxic. For example, in a reported case, a 53-year-old consumed Gyromitra and experienced vomiting and diarrhea, subsequently expressing hypotension, anuria, jaundice, hemiplegia, and coma, followed by death on the 3rd day [2]. In total, at least 133,700 cases (7428/year) of mushroom toxicity, with 52 fatalities, were reported in the U.S. from 1999 to 2016 [3].
Smartphones have been available at virtually every geographical location in both developed and developing countries. Recent versions of smartphones boast a multicore central processing unit (CPU), large random-access memory (RAM), cellular and WiFi connectivity, and high-resolution cameras utilizing complementary metal-oxide semiconductor (CMOS) sensors, which render them a hand-held research computer. Several groups have proposed transforming smartphones from conventional communication devices into portable instruments via various attachments that transduce a target analyte into an imaging format. Some applications include the colorimetric analysis [4,5,6,7], luminescence detection [8,9,10,11,12], spectrometry [13,14,15], and surface plasmon resonance analysis [16,17,18]. In addition, the computing power of the modern smartphone allows easier integration of machine learning algorithms to assist decision-making when visual observations cannot be used for accurate discrimination of the samples [16,17,18,19].
Here, we propose utilizing a conventional smartphone as a mushroom-image classifier that can provide reliable categorization of mushroom images and provide necessary safety information for both public and professional mushroom pickers. Multiple approaches to analyzing mushrooms using machine learning employing support vector machines (SVMs) [20,21], decision trees, naive Bayes, and ensemble algorithms based on mushroom data from the UCI Machine Learning Repository [22], which contains 22 morphological features, have been described. However, the result generated from this mushroom database assumed that morphological characteristics such as gill shape and cap shape had already been extracted objectively and that all the data were already in numerical formats as opposed to the raw images. If an application were developed utilizing the methods, users would still be required to manually extract the mushroom’s morphological attributes, which may be based on a subjective interpretation of the morphological traits. In addition, the ecosystem in which particular mushrooms are found is another critical feature mycologists utilize for identification. Regrettably, the morphological feature-based classifications might not incorporate this information into the classifiers. This paper proposes a method to classify mushrooms belonging to several easily confused genera. In order to achieve this task, we trained and tuned deep convolutional neural network models [23]. Subsequently, the models were integrated into an Android mobile application so that anyone could determine the edibility of mushrooms by photographing them in situ.

2. Materials and Methods

2.1. Mushroom Images

As the first crucial step in training a machine learning classifier, a vast number of photos were downloaded from the internet using the “down-load-all-images” Chrome extension [24]. Then, the photos with additional elements that could not be deleted, such as titles, text overlays, watermarks, or authorship annotations, were eliminated from the database. Images of mushrooms outside of a natural context were also removed from the database. Finally, an expert mycologist examined all the collected photographs to confirm their website-assigned labels and verify their genuine identities. The total number of validated photos for each model and class within each model is summarized in Table 1.

2.2. Machine Learning Algorithm

While recognizing and classifying all dangerous mushroom species is the objective, we began by constructing databases for three distinct scenarios. The first model was able to classify species of the morphologically similar genera of Gyromitra (poisonous species) and Morchella (highly sought edible species). The second model was able to classify genera of Clavulina, Inocybe, and Marasmius, and the third model was able to classify genera of Agaricus (some edible species), Amanita (includes deadly species), Cantharellus (prized edibles), Pleurotus (prized edibles), and Tricholoma (some toxic species). The first two models relied on the same neural network architecture (Figure 1), but the third model employed a different one. A convolutional neural network and a feed-forward neural network comprised the architecture of the first two models. For the convolutional neural network, input variables were 224 × 224-pixel images in three-channel RGB color and six convolutional layer blocks. Max-pooling was applied after 1st, 3rd, 4th, and 5th convolutional layers. A rectified linear unit (ReLU) activation function was used after the six convolutional layers [25]. The feed-forward network consisted of three fully connected layers; the number of features in the last layer was equivalent to the number of classes. After each fully connected layer, exponential linear unit (ELU) [26] activation was used with 70% dropout rates. This neural network architecture was unable to obtain satisfactory classification results for the 5-class model. Therefore, we decided to utilize the fine-tuned ResNet system [23] with 152 layers model and added an output layer with a number of neurons equivalent to the number of mushroom species classes. The hardware specification of the server was Intel® Xeon® CPU @ 2.30GHz, 34GB RAM, and Tesla P100-PCIE-16GB.

2.3. Classifier Training and Testing

RGB channels were normalized with means of 0.485, 0.456, 0.406, and standard deviations of 0.229, 0.224, and 0.225, respectively, as suggested in the literature [27]. Optimal hyperparameters such as the number of layers, the number of neurons, and the learning rate were calculated using this model. In addition, batch size was set to 32, and a cross-entropy loss [28] was used as a cost function for all models with an optimizer set to Adam [29] with a learning rate of 0.0001, and L2 penalty to 0.0001. Moreover, the AMSGrad variant was utilized as suggested by Reddi et al. [30] while the first beta parameters were modified from 0.9 to 0.7 with 20 epochs. Using our training data, stratified k-fold cross-validation (k = 10) was performed to determine how well our systems were prepared to classify designated mushroom types.
To test the performance of our models, 96, 384, and 220 images were used for each model. Based on correct and incorrect predictions, a cross-validation table was generated and expressed as true positive (TP), true negative (TN), false positive (FP), and false negative (FN) results; these values were used to report the statistical parameters and receiver operating curves (ROC) as follows.
Sensitivity = T P T P + F N
Specificity = T N T N + F P
Accuracy = T P + T N T P + F P + T N + F N

2.4. Mobile App Development and Testing

Android Studio version 4.0 with SDK version 28 was used for the app development platform. To use trained PyTorch models in Android, we used PyTorch Android version 1.5.0. Three of the server-based pre-trained models were ported into the smartphone app called Purdue University Mushroom App (PUMA); the flow chart of the app is shown in Figure 2. The output is shown with classification results and associated probability. For this classification task, a sigmoid function with α = 5 was used to compute the estimated probability of mushroom scores predicted by our trained model:
p x = α 1 + e x × 100 %
where x is a feature from the penultimate feed-forward layer.
Pre-trained models from servers were deployed to various Android devices, such as Samsung Galaxy S9 and Note 20, Google Nexus 5, and Pixel 2, with the same images used in the testing on our server-based model. Using these Android smartphones, test results were recorded as confusion matrices, and the performance results were calculated using Equations (1)–(3). To explore the effect of image size on the statistical output, we tested the classification results for intentionally scaled-down images. For this experiment, original images were scaled from 90% to 10% of the initial size, and the respective statistical accuracy was calculated.

3. Results

3.1. Model Training

For the two-class model, Figure S1A displays the server-based cross-validation training results for the genera Gyromitra and Morchella. Both cases showed around 98%, 98%, and 99% for sensitivity, specificity, and accuracy, respectively. Figure S1B shows the receiver operating curve (ROC) for the training results. For the three-class model, Figure S2A displays the server-based training result expressed as statistical outputs for the genera Clavulina, Inocybe, and Marasmius. All cases showed around 96%, 98%, and 97% for sensitivity, specificity, and accuracy, respectively. Figure S2B shows the ROC for the training results. For the five-class model, Figure 3A displays the server-based training result expressed as statistical outputs for the genera Agaricus, Amanita, Cantharellus, Pleurotus, and Tricholoma. Except for Tricholoma, all cases showed around 95%, 98%, and 97% for sensitivity, specificity, and accuracy, respectively. Tricholoma recorded 90% sensitivity, the lowest value among all the cases. Figure 3B shows the ROC for the training results. For the significance test, Table S1 shows the average accuracy and confidence interval, while Table S2 shows the significance test for two-, three-, and five-class models. Detailed statistical comparisons of the performance of trained models are provided in the Supplemental Materials (Tables S1 and S2, Figure S5).

3.2. Model Testing

After the training session had generated classification models, a batch of set-aside non-training images was used for testing. For the server-based model and server-based classification, and for the server-based model transferred to smartphones, the results were expressed in two distinct ways. Figure S3A,B shows the test results for the two-class model for the server and for a smartphone. Classification accuracy was more than 90% in both cases. Figure S4A,B shows the test results for the three-class model. In server-based testing, the specificity for Clavulina was the lowest among the three classes but still higher than 90%. In phone-based testing, the specificity for Marasmius was the lowest result among the three classes but higher than 90% as well. Both server-and phone-based models showed classification accuracy well above 90%. Figure 4A,B shows the test results for the five-class model. In server-based testing, Agaricus had the lowest sensitivity value (~82%) among all the classes, whereas Tricholoma recorded the lowest sensitivity (90%) in phone-based testing. All the cases except Agaricus from the server-based training showed above 90% classification accuracy.

3.3. Mobile App

Once launched, the app starts with the PUMA logo, as shown in Figure 5A. Users are then able to interact with the app using four buttons: Camera, Gallery, Database, and Mode. The Camera button allows the user to take a picture by launching a default camera application on the mobile device system. With this default camera app, the user can take as many pictures as desired and then return to the main menu (Figure 5B). The Gallery button enables the user to select a photo from a default gallery app to make a prediction. However, when PUMA is launched for the first time, the Gallery button is disabled and remains so before the users choose a type of classification model by clicking Mode. The app provides three classification models, as described in the Materials and Methods section, which can be selected via drop-down menus. After the user chooses the type of classification, the label and the accompanying class probability of the identified species are displayed. On the same screen, the user can put their own collection identification number (collection ID) on the result (Figure 5C). Lastly, the database allows the users to see the classification history. The users can check this history with a representative thumbnail of collected images, including a classification label from pre-trained models, a probability of being a classified mushroom species, a collection ID of each mushroom, and a timestamp when the user performs the prediction by clicking Gallery (Figure 5D). Moreover, we provide geographical information using Google Maps (Figure 5E). To accomplish this, the app collects the location’s latitude and longitude when photos are taken.

3.4. Inter-Platform Comparison

One of the questions that arise when using models trained on different platforms is the transferability of the result. Most wild mushroom collection occurs in the remote field, where cellular signal strength can be weak or nonexistent. Therefore, we compared the classification accuracy of server-based testing and smartphone-based testing results to determine the degree of consistency. Figure 6 shows the comparison of three reported performance measures (sensitivity, specificity, and accuracy). Variation was defined as a ratio of the server-based performance measure to the smartphone-based measures. Figure 6A shows a ±5% discrepancy between the two methods for the three-class model, while Figure 6B demonstrates a wider gap (maximum of +15% and a minimum of −10%) between the two platforms. For the three-class classification task, Clavulina showed the most stable testing results from the inter-platform comparison with the lowest variation rates of sensitivity, specificity, and accuracy, whereas Marasmius recorded the highest variation rates of the performance measures. Overall, inter-platform testing in the three-class classification showed stable variation results. The five-class classification showed less stability. Agaricus recorded the highest variation rates in sensitivity, about 118%. In addition, Tricholoma species recorded about 90% variation in specificity. Except for those two types, the other three mushrooms recorded stable variation rates. In particular, Pleurotus recorded the lowest variation rate among the five classes. Furthermore, while accuracy showed minimal variations, other performance measures revealed larger variations between the two platforms. For example, smartphone-based sensitivity results reported higher rates than server-based results and vice-versa for specificity.

3.5. Inter-Phone Comparison

In addition to the inter-platform comparison, an inter-phone comparison was conducted to determine whether there was a brand and/or operating system version dependency. While all of the tested smartphone brands utilized the Android operating system, some were more recent than others, including Google’s Nexus 5, Pixel 2, Samsung’s Galaxy S8, and Samsung’s Galaxy Note 20. Table 2, Table 3 and Table 4 show the results of Equations (1)–(3). While the two-class model showed above 90% accuracy for all brands, there were some performance variations between different smartphone brands. For example, sensitivity for Gyromitra ranged from 100% (Note 20) to 91.6% (S8 and Nexus 5), while specificity was in the range of 92.3%-91.8% on all four models. For the three-class model, sensitivity for Clavulina varied from 100% (S8 and Note 20) to 94.5% (Nexus 5), Inocybe resulted in 99.2% (Note 20) to 93.7% (Nexus 5), and Marasmius reported 97.6% (Note 20) to 89.9% (Nexus 5). For the five-class model, Agaricus showed a large variation in PPV, from 92.3% (Note 20) to 85% (Pixel 2). Amanita reported variations of specificity from 94.8% (Note 20) to 92.5% (the other three models). The rest of the three classes showed variations of about a 2% difference depending on the smartphone model.

3.6. Image-Scaling Effect

Another component of the inter-phone comparison was a test of the effect of image resolution on classification accuracy. Since many phones have their own CMOS sensors with different native pixel resolutions, all the training data were challenged against lower-resolution images from 100% (full scale) to 10% scale of input images. In addition, users can take many mushroom images at varying distances from the object, which results in different pixel resolutions for each mushroom image. Results were summarized in Figure 7A for two- and three-class models and Figure 7B for the five-class model. As the image was downsized from the original image, classification performance gradually deteriorated, which was reflected in the accuracy values (Equation (3)). Despite a 50–70 percent reduction in size, the accuracy of the reconstructed image was close to that of the original. However, the five-class model’s sensitivity yielded significantly lower results than specificity. For example, the sensitivity for Tricholoma genera was overall 10–20% lower than that of other counterparts.

4. Discussion

A large number of established machine learning algorithms were previously tested against the standard datasets from the UCI machine learning repository [22]. Among the available datasets exists one for mushrooms; this dataset could be used to train and test newly developed applications. One of the limitations of utilizing this database is that the image features used by machine learning have already been extracted. While this works fine in some experimental settings, it is not trivial to consistently extract those morphological features of mushrooms in the real-life mushroom foraging situation. Additionally, relying on the phenotypic mushroom characteristics alone would ignore informative descriptors indicative of the mushrooms’ growth environments, such as surrounding nature.
Additionally, various mushroom orientations (top view, front view, under the cap view), and different sizes (zooming) effects, into a single machine learning platform. Therefore, users could simply take images with their smartphones, and a pre-trained library (for remote locations with unreliable network access) or a server-based library could be used to provide accurate classification of the mushroom species.
Our training data accurately reflect the reality of mushroom foraging. The data includes various image perspectives (photographs from various angles, photographs under the cap, images displaying the front, top, and side, etc.), and the surrounding environment. In addition, as shown in Figure 2, the collection of mushroom images may involve taking a large number of consecutive photographs and sorting them by collection identification numbers. At the earlier stage of the app development, we added to the data input a collection identification feature so that all the images on the phone were classified with their unique collection numbers (Figure 5C). Since the offline classification was conducted on the spot, all the longitude and latitude information was also saved for easy tracking of the geographical location of each collected sample. We have previously discussed the advantages and disadvantages of server-based and mobile testing [4]. While the server-based method is more flexible and constantly refreshes the up-to-date models for better accuracy, smartphone-based testing conducts the classification without any network service present. Most wild-mushroom foraging happens in remote areas of the natural habitat lacking reliable network service. Therefore, offline, on-phone training was a critical feature to implement in this study.
However, training deep neural networks on a smartphone is so difficult that implementing this feature would be impractical. Consequently, transferring the pre-trained model from the server was the most logical course of action. The results were dependent on the classifier and the number of classes. While high levels of overall accuracy were relatively easy to reach, more granular metrics, such as sensitivity and specificity, showed some variation between the two platforms. Interestingly, the smartphone-based method demonstrated greater sensitivity, resulting in a lower false-negative rate. If our sample was a toxic mushroom, the smartphone would be less likely to misidentify it as an edible one.
Based on the results of inter-phone testing, our trained models generated consistent performance metrics regardless of the employed platform. We automatically resize captured images to the optimal image resolution for each of the devices to ensure that our mobile application and trained models produce the best results. The sheer diversity of mushroom genera and species is one limitation of our classification model. The current model provides proof-of-concept for classifying mushrooms at the genus level, whereas our future work will concentrate on refining classification models to discriminate among species within specific genera. The current supervised-learning method provides the pre-trained class with the closest matching label. While the probability calculation provided a measure of image similarity to the classes defined in database, the conventional discriminative classification is by definition unable to correctly process images from new and merging classes. Future work will involve the implementation of non-exhaustive learning techniques capable of automatically detecting the presence of new, never-before-seen classes in order to enable the detection and labeling of species not defined in the initial training database [31,32].

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/app122211685/s1, Figure S1: Comparison of sensitivity and specificity for the two classes of Gyromitra and Morchella.; Figure S2: Comparison of sensitivity and specificity for the three classes of Clavulina, Inocybe, and Marasmius; Figure S3: Comparison of sensitivity and specificity for the two classes of Gyromitra and Morchella.; Figure S4: Comparison of sensitivity and specificity for Clauvulina, Inocybe, and Marasmius classes.; Figure S5: Comparison of the class accuracies in three different classifiers computed using 10× cross-validation.; Table S1: Computed mean accuracies and confidence interval for classification performed using 2-, 3-, and 5-classes models.; Table S2: Significance test results.

Author Contributions

Data curation, J.J.L., B.R. and M.C.A.; Formal analysis, J.J.L.; Funding acquisition, E.B.; Investigation, J.J.L., M.C.A., B.R. and E.B.; Methodology, J.J.L. and E.B.; Project administration, E.B.; Supervision, M.C.A. and E.B.; Writing—original draft, J.J.L. and E.B.; Writing—review & editing, M.C.A., B.R. and E.B. All authors have read and agreed to the published version of the manuscript.

Funding

U.S. Department of Agriculture, Agricultural Research Service, under Agreement No. 59-8072-1-002.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

This research was supported by the Center for Food Safety Engineering at Purdue University, funded by the U.S. Department of Agriculture, Agricultural Research Service, under Agreement No. 59-8072-1-002. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the U.S. Department of Agriculture.

Conflicts of Interest

The author declare no conflict of interest.

References

  1. Satora, L.; Pach, D.; Ciszowski, K.; Winnik, L. Panther cap Amanita pantherina poisoning case report and review. Toxicon 2006, 47, 605–607. [Google Scholar] [CrossRef] [PubMed]
  2. Giusti, G.; Carnevale, A. A case of fatal poisoning by Gyromitra esculenta. Arch. Toxicol. 1974, 33, 49–54. [Google Scholar] [CrossRef] [PubMed]
  3. Brandenburg, W.E.; Ward, K.J. Mushroom poisoning epidemiology in the United States. Mycologia 2018, 110, 637–641. [Google Scholar] [CrossRef]
  4. Jung, Y.; Kim, J.; Awofeso, O.; Kim, H.; Regnier, F.; Bae, E. Smartphone-based colorimetric analysis for detection of saliva alcohol concentration. Appl. Opt. 2015, 54, 9183–9189. [Google Scholar] [CrossRef] [PubMed]
  5. Rateni, G.; Dario, P.; Cavallo, F. Smartphone-based food diagnostic technologies: A review. Sensors 2017, 17, 1453. [Google Scholar] [CrossRef] [PubMed]
  6. Silva, A.F.S.; Rocha, F.R. A novel approach to detect milk adulteration based on the determination of protein content by smartphone-based digital image colorimetry. Food Control 2020, 115, 107299. [Google Scholar] [CrossRef]
  7. Sumriddetchkajorn, S.; Chaitavon, K.; Intaravanne, Y. Mobile device-based self-referencing colorimeter for monitoring chlorine concentration in water. Sens. Actuators B Chem. 2013, 182, 592–597. [Google Scholar] [CrossRef]
  8. Anand, R.K.; Rahn, K.L.; Rhoades, T.D. Alternating Current Voltammetry at a Bipolar Electrode with Smartphone Luminescence Imaging for Point-of-Need Sensing. ChemElectroChem 2020, 7, 1172–1181. [Google Scholar]
  9. Jung, Y.; Coronel-Aguilera, C.; Doh, I.-J.; Min, H.J.; Lim, T.; Applegate, B.M.; Bae, E. Design and application of a portable luminometer for bioluminescence detection. Appl. Opt. 2020, 59, 801–810. [Google Scholar] [CrossRef]
  10. Kim, H.; Jung, Y.; Doh, I.-J.; Lozano-Mahecha, R.A.; Applegate, B.; Bae, E. Smartphone-based low light detection for bioluminescence application. Sci. Rep. 2017, 7, 40203. [Google Scholar] [CrossRef] [Green Version]
  11. Li, S.; Liu, J.; Chen, Z.; Lu, Y.; Low, S.S.; Zhu, L.; Cheng, C.; He, Y.; Chen, Q.; Su, B. Electrogenerated chemiluminescence on smartphone with graphene quantum dots nanocomposites for Escherichia Coli detection. Sens. Actuators B Chem. 2019, 297, 126811. [Google Scholar] [CrossRef]
  12. Zhu, Z. Smartphone-based apparatus for measuring upconversion luminescence lifetimes. Anal. Chim. Acta 2019, 1054, 122–127. [Google Scholar] [CrossRef] [PubMed]
  13. Das, A.J.; Wahi, A.; Kothari, I.; Raskar, R. Ultra-portable, wireless smartphone spectrometer for rapid, non-destructive testing of fruit ripeness. Sci. Rep. 2016, 6, 32504. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Wang, Y.; Liu, X.; Chen, P.; Tran, N.T.; Zhang, J.; Chia, W.S.; Boujday, S.; Liedberg, B. Smartphone spectrometer for colorimetric biosensing. Analyst 2016, 141, 3233–3238. [Google Scholar] [CrossRef]
  15. Zhang, C.; Cheng, G.; Edwards, P.; Zhou, M.-D.; Zheng, S.; Liu, Z. G-Fresnel smartphone spectrometer. Lab Chip 2016, 16, 246–250. [Google Scholar] [CrossRef]
  16. Fan, Z.; Geng, Z.; Fang, W.; Lv, X.; Su, Y.; Wang, S.; Chen, H. Smartphone Biosensor System with Multi-Testing Unit Based on Localized Surface Plasmon Resonance Integrated with Microfluidics Chip. Sensors 2020, 20, 446. [Google Scholar] [CrossRef] [Green Version]
  17. Lertvachirapaiboon, C.; Kiyokawa, I.; Baba, A.; Shinbo, K.; Kato, K. Colorimetric determination of hydrogen peroxide based on localized surface plasmon resonance of silver nanoprisms using a microchannel chip. Anal. Lett. 2019, 52, 1939–1950. [Google Scholar] [CrossRef]
  18. Masson, J.-F. Portable and field-deployed surface plasmon resonance and plasmonic sensors. Analyst 2020, 145, 3776–3800. [Google Scholar] [CrossRef]
  19. Udrea, A.; Mitra, G.; Costea, D.; Noels, E.; Wakkee, M.; Siegel, D.; de Carvalho, T.; Nijsten, T. Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms. J. Eur. Acad. Dermatol. Venereol. 2020, 34, 648–655. [Google Scholar] [CrossRef]
  20. Pinky, N.J.; Islam, S.M.; Rafia, S.A. Edibility Detection of Mushroom Using Ensemble Methods. Int. J. Image Graph. Signal Process. 2019, 11, 55. [Google Scholar] [CrossRef] [Green Version]
  21. Verma, S.; Dutta, M. Mushroom classification using ANN and ANFIS algorithm. IOSR J. Eng. 2018, 8, 94–100. [Google Scholar]
  22. UC Irvine Machine Learning Repository. Available online: http://archive.ics.uci.edu/ml (accessed on 11 January 2022).
  23. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
  24. Download Image. Available online: https://download-all-images.mobilefirst.me/ (accessed on 11 January 2022).
  25. Agarap, A.F. Deep learning using rectified linear units (relu). arXiv 2018, arXiv:1803.08375. [Google Scholar]
  26. Clevert, D.-A.; Unterthiner, T.; Hochreiter, S. Fast and accurate deep network learning by exponential linear units (elus). arXiv 2015, arXiv:1511.07289. [Google Scholar]
  27. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012. [Google Scholar]
  28. Cross Entropy Loss. Available online: https://pytorch.org/docs/master/generated/torch.nn.CrossEntropyLoss.html (accessed on 11 January 2022).
  29. Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
  30. Reddi, S.J.; Kale, S.; Kumar, S. On the convergence of adam and beyond. arXiv 2019, arXiv:1904.09237. [Google Scholar]
  31. Akova, F.; Dundar, M.; Davisson, V.J.; Hirleman, E.D.; Bhunia, A.K.; Robinson, J.P.; Rajwa, B. A machine-learning approach to detecting unknown bacterial serovars. Stat. Anal. Data Min. 2010, 3, 289–301. [Google Scholar] [CrossRef]
  32. Rajwa, B.; Dundar, M.M.; Akova, F.; Bettasso, A.; Patsekin, V.; Dan Hirleman, E.; Bhunia, A.K.; Robinson, J.P. Discovering the unknown: Detection of emerging pathogens using a label-free light-scattering system. Cytom. Part A 2010, 77A, 1103–1112. [Google Scholar] [CrossRef]
Figure 1. Conceptual diagram explaining the machine-learning-based mushroom image classification. (A) Shows the input images to output layers which are generated from a server. Once the training model is verified, it is transferred to a smartphone device. (B) Series of layers starting from 224 (H) × 224 (W) × 3 (channels) and subsequent convolution of 2 × 2. The image is flattened as a one-dimensional array of 4500 elements and fed to a feed-forward neural network. This network is subsequently reduced to 300, 300, and 3 neurons of fully connected layers corresponding to the number of classes.
Figure 1. Conceptual diagram explaining the machine-learning-based mushroom image classification. (A) Shows the input images to output layers which are generated from a server. Once the training model is verified, it is transferred to a smartphone device. (B) Series of layers starting from 224 (H) × 224 (W) × 3 (channels) and subsequent convolution of 2 × 2. The image is flattened as a one-dimensional array of 4500 elements and fed to a feed-forward neural network. This network is subsequently reduced to 300, 300, and 3 neurons of fully connected layers corresponding to the number of classes.
Applsci 12 11685 g001
Figure 2. Flow chart of the PUMA. At the start, users were asked to select the classification model. There are two different ways of importing images which are from the live camera shots and from the image gallery. Once images are selected, the classifier automatically compares them against the pre-trained model and reports the classification results into the in-phone database. Database entry includes accuracy, classified name, date, time, and geographical locations.
Figure 2. Flow chart of the PUMA. At the start, users were asked to select the classification model. There are two different ways of importing images which are from the live camera shots and from the image gallery. Once images are selected, the classifier automatically compares them against the pre-trained model and reports the classification results into the in-phone database. Database entry includes accuracy, classified name, date, time, and geographical locations.
Applsci 12 11685 g002
Figure 3. Comparison of sensitivity and specificity for the five classes of Agaricus, Amanita, Cantharellus, Pleutrotus, and Tricholoma. (A) classification rate from server-based training and (B) receiver operating curve.
Figure 3. Comparison of sensitivity and specificity for the five classes of Agaricus, Amanita, Cantharellus, Pleutrotus, and Tricholoma. (A) classification rate from server-based training and (B) receiver operating curve.
Applsci 12 11685 g003
Figure 4. Comparison of sensitivity and specificity for the five classes of Agaricus, Amanita, Cantharellus, Pleutrotus, and Tricholoma. (A) Classification rate from laptop training and testing. (B) Test results from importing the trained model to the smartphone handset.
Figure 4. Comparison of sensitivity and specificity for the five classes of Agaricus, Amanita, Cantharellus, Pleutrotus, and Tricholoma. (A) Classification rate from laptop training and testing. (B) Test results from importing the trained model to the smartphone handset.
Applsci 12 11685 g004
Figure 5. Screenshot of PUMA. (A) starting screen (B) initial menu where the user can select the app functions. (C) Results screen where the predicted class with respective probability is shown. Collection ID can be typed as well. (D) A snapshot of the in-phone database with thumbnail images and predicted classes. (E) Foraging map; when the user clicks on a result, the application automatically transmits the image data’s coordinates to mapping software so that the collection’s location can be displayed.
Figure 5. Screenshot of PUMA. (A) starting screen (B) initial menu where the user can select the app functions. (C) Results screen where the predicted class with respective probability is shown. Collection ID can be typed as well. (D) A snapshot of the in-phone database with thumbnail images and predicted classes. (E) Foraging map; when the user clicks on a result, the application automatically transmits the image data’s coordinates to mapping software so that the collection’s location can be displayed.
Applsci 12 11685 g005
Figure 6. Comparison of statistical outputs when two different platforms are used: Smartphone vs. server-based classification. (A) For 3-class model (B) 5-class model.
Figure 6. Comparison of statistical outputs when two different platforms are used: Smartphone vs. server-based classification. (A) For 3-class model (B) 5-class model.
Applsci 12 11685 g006
Figure 7. Comparison of relative accuracy when image sizes are varied. (A) For 3-class model (B) 5-class model. When image size falls below 20–25%, overall accuracy decreases to ~80% range.
Figure 7. Comparison of relative accuracy when image sizes are varied. (A) For 3-class model (B) 5-class model. When image size falls below 20–25%, overall accuracy decreases to ~80% range.
Applsci 12 11685 g007
Table 1. Total number of images used for machine learning training and testing.
Table 1. Total number of images used for machine learning training and testing.
TotalTrainingTesting
2-class modelGyromitra23819048
Morchella23819048
3-class modelClavulina497369128
Inocybe497369128
Marasmius497369128
5-class modelAgaricus25721542
Amanita32828840
Cantharellus45340944
Pleurotus39934950
Tricholoma45641244
Table 2. Inter-phone testing results—2 classes.
Table 2. Inter-phone testing results—2 classes.
Pixel 2Nexus 5Galaxy s8Galaxy Note 20
Gyromitrasensitivity0.93620.91670.91671
specificity0.91840.91670.91670.9231
accuracy0.92710.91670.91670.9583
Morchellasensitivity0.91840.91670.91670.9231
specificity0.93620.91670.91671
accuracy0.92710.91670.91670.9583
Table 3. Inter-phone testing results—3 classes.
Table 3. Inter-phone testing results—3 classes.
Pixel 2Nexus 5Galaxy s8Galaxy Note 20
Clavulinasensitivity0.99210.945311
specificity0.99220.99220.99220.9922
accuracy0.99220.97660.99480.9948
Inocybesensitivity0.94660.96090.95450.9618
specificity0.98430.93770.99210.9921
accuracy0.97140.94550.97920.9818
Marasmiussensitivity0.96060.89920.97620.9764
specificity0.97290.97270.98060.9844
accuracy0.96880.94810.97920.9818
Table 4. Inter-phone testing results—5 classes.
Table 4. Inter-phone testing results—5 classes.
Pixel 2Nexus 5Galaxy s8Galaxy Note 20
Agaricussensitivity0.91890.92310.92310.9231
specificity0.96690.97770.97770.9831
accuracy0.95870.96790.96790.9724
Amanitasensitivity0.9250.9250.9250.9487
specificity0.98310.98310.98310.9831
accuracy0.97250.97250.97250.977
Cantharellussensitivity0.8750.89360.89360.8936
specificity0.98820.98830.98830.9882
accuracy0.96330.96790.96790.9677
Pleurotussensitivity0.97920.97920.97920.9792
specificity0.98240.98240.98240.9822
accuracy0.98170.98170.98170.9816
Tricholomasensitivity0.84440.86360.86360.8636
specificity0.96530.96550.96550.9653
accuracy0.94040.9450.9450.9447
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Lee, J.J.; Aime, M.C.; Rajwa, B.; Bae, E. Machine Learning-Based Classification of Mushrooms Using a Smartphone Application. Appl. Sci. 2022, 12, 11685. https://doi.org/10.3390/app122211685

AMA Style

Lee JJ, Aime MC, Rajwa B, Bae E. Machine Learning-Based Classification of Mushrooms Using a Smartphone Application. Applied Sciences. 2022; 12(22):11685. https://doi.org/10.3390/app122211685

Chicago/Turabian Style

Lee, Jae Joong, M. Catherine Aime, Bartek Rajwa, and Euiwon Bae. 2022. "Machine Learning-Based Classification of Mushrooms Using a Smartphone Application" Applied Sciences 12, no. 22: 11685. https://doi.org/10.3390/app122211685

APA Style

Lee, J. J., Aime, M. C., Rajwa, B., & Bae, E. (2022). Machine Learning-Based Classification of Mushrooms Using a Smartphone Application. Applied Sciences, 12(22), 11685. https://doi.org/10.3390/app122211685

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop