Next Article in Journal
The Digital Fingerprinting Analysis Concerning Google Calendar under Ubiquitous Mobile Computing Era
Next Article in Special Issue
Provable Fair Document Exchange Protocol with Transaction Privacy for E-Commerce
Previous Article in Journal
Everywhere Equivalent 2-Component Links
Previous Article in Special Issue
Live Mobile Distance Learning System for Smart Devices
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Development of Hybrid Drug Information System Using Image Recognition

1
Department of Computer Software Engineering, Soonchunhyang University, Asan 336-745, South Korea
2
Department of Computer Science & Engineering, University of Colorado Denver, Denver, CO 80217, USA
*
Author to whom correspondence should be addressed.
Symmetry 2015, 7(2), 376-382; https://doi.org/10.3390/sym7020376
Submission received: 23 January 2015 / Revised: 31 March 2015 / Accepted: 13 April 2015 / Published: 16 April 2015
(This article belongs to the Special Issue Advanced Symmetry Modelling and Services in Future IT Environments)

Abstract

: In order to prevent drug abuse or misuse cases and avoid over-prescriptions, it is necessary for medicine taker to be provided with detailed information about the medicine. In this paper, we propose a drug information system and develop an application to provide information through drug image recognition using a smartphone. We designed a contents-based drug image search algorithm using the color, shape and imprint of drug. Our convenient application can provide users with detailed information about drugs and prevent drug misuse.

1. Introduction

Many people have wondered about their prescription drug, leftover drugs in their cabinet or some drugs found in their teenager’s room. According to the 2013 survey by Korea alliance of patients organization (KAPO), 91% of patients who receive a prescription want to get the detailed information for their doses of medicine [1]. Moreover, the survey by Korea Consumer Agency (KCA) showed that 38.7% of consumers experienced side effects of medicine in the 3 years from 2008 to 2010 [2].

While medicine takers may not aware of the details of a drug's brand name and manufacturer, quite often they are interested in basic facts about indications, dosage, interactions, and side effects of the medicine. In order to avoid over-prescriptions, to prevent medicinal misuse and side effects of medicine, it is necessary for medicine takers to be provided with the detailed information about the medicine.

All approved drugs in most countries are required by the Food and Drug Administration to have a unique imprint. The unique imprint of a drug product is determined by the drug manufacturer. The purpose of an imprint is to enable the identification of drug strength, ingredients, and manufacturer by patients, health care providers and law enforcement officials. Therefore, we designed a drug information system using this imprint.

In this paper, we propose and develop an application to provide the information about a drug through automatic image recognition after a user takes a picture of the drug with their smartphone. To provide drug information, we propose a drug image recognition technique using the color, shape and imprint of drug. We designed a smartphone and smart pad based application to provide a convenient drug search service for users.

This paper is organized as follows. Section 2 surveys the related works. We present drug image recognition in Section 3 and we propose our drug information system in Section 4. Experiment results are presented in Section 5. Finally, we conclude this paper and discuss future works in Section 6.

2. Related Works

Several automated drug information systems proposed in [3,4]. The Medicines Line (1300 MEDICINE) [3] is an automatic telephone information system to provide information about drugs to aged and visually impaired persons. The Chemical Named Entity Recognizer (CheNER) [4] is a tool for automated identification of IUPAC chemical mentions. However, these systems used speech recognition for drug or chemical names.

Several web based drug information systems have developed as shown Figure 1. Web based search systems can be performed in two ways: keyword search and content search based drug image. Current systems include: Drug Information [5], Drugs [6], Drug Guide [7], DrugInfo [8] and Naver Health [9]. However, usually, users must manually input the name, category, national drug, codes, imprint, discernment letter, dosage form, shape or color of the drug to search drug information. Especially with keyword search cases, there is a fatal problem that the user must know the drug name. These web based systems are time-consuming and inconvenient for a drug search because the drug search is user-defined and provides limited features.

3. Drug Image Recognition

We assume that drug is a finished dosage form, e.g., a tablet or capsule that contains a drug substance in association with one or more other ingredients. The drug is identified by shape, color and imprint. A drug imprint is defined as any single number, letter, mark, symbol, logo, or combination of numbers and letters, including national drug code and company names. The unique imprint permits the identification of the drug strength, drug product, active ingredients and the manufacturer of the product. The imprint is marked with an identification code by means of engraving, embossing, debossing, or printing with ink. Engraving is imprinted with a code that is cut into the dosage form surface after it has been completed. Embossing is imprinted with a mark raised above the dosage form surface. Debossing is imprinted with a mark below the dosage form surface.

To recognize the drug image, we use the method of comparing the image registered in the database and drug image that the drug user has taken using a smartphone as a pixel unit of the finely divided image. The programmer or user can decide how to split the image to pixels. We split the image in pixels and then compare the drug image per pixel bundle. Our image recognizers are designed differently according to certain kinds of drug shape and picture discernment. We assort drugs according to shape, hoof shape, color discernment, letters and picture discernment.

Table 1 shows the different settings for the size of pixels per bundle according to various imprint cases. The more blurry the imprint, the more detailed image comparison is needed. However, detailed comparison will cause the serious problem that the search processing time is longer. Thus, we need a compromise between the search precision and processing time. In drug image recognition, we also consider the interference of background.

Figure 2 shows an example that is well taken photo sharpening without background interference. In this case, it is possible to search exactly for the drug information in a short time because only the drug image part is used as a comparison.

Figure 3 shows a severe case: a photo with background interference. If there is interference of background in the drug photo, it affects the image comparison time.

4. Implementation

To provide drug information using drug image recognition automatically, we designed and implemented a drug information system and smartphone application “DrugWellness”. Figure 4 shows the proposed architecture of our drug information system. Our drug information system consists of web server, data server, and smartphone application.

In order to implement the drug information system and hybrid DrugWellness application, we used Open Computer Vision (OpenCV), Node.js and HTML5. We have constructed a database for drug information and image based on a drug information handbook [10]. Figure 5 shows the screenshots of our DrugWellness application based on an Android environment.

Figure 6 shows the snapshot of detailed drug information after a drug image search. The search results screen is configured as a dynamic web page. This page consists of drug name, drug image, efficacy of the drug, posology and method of administration, special warning and precautions for use, possible side effects, how to store and so on.

5. Experiment Results

We evaluate two aspects of our drug information system: (1) drug image search precision, and (2) the total processing time of end-to-end system. For experiments, we constructed drug image database that consists of 10,000 drug images. We have conducted experiments using 25 drugs with an ink print and 25 engraved drugs.

Figure 7 shows the precision of the drug image search according to the size of pixels per comparison bundle. The result shows that the imprint printed with ink has a higher precision than the engraved imprint. Moreover, the precision drops significantly as the size of pixels per comparison bundle grows. If the size of the pixels per comparison bundle is 100 or less, the precision is more than 90%.

Figure 8 shows the total processing time of end-to-end system according to the size of pixels per comparison bundle. The result shows that the imprint printed with ink has higher performance than the engraved imprint. The total processing time was greater as the size of pixels per comparison bundle was reduced. Also, the processing time difference between the imprint printed with ink and the engraved imprint increased as the size of pixels per comparison bundle was reduced.

4. Conclusions

For safety of drug use, it is essential to provide accurate information about the drug to the user. Previous drug information systems have supported a keyword search based on drug name or category. In this paper, we proposed a drug information system to provide the information about a drug through automatic image recognition. In experiment results, more comparison computation time was required for drug imprints that were engraved or debossed than those printed with ink. In future work, we will study the drug image recognition and search algorithm to improve the search speed.

Acknowledgments

This research was supported by the Ministry of Science, ICT & Future Planning (MSIP), Korea, under the Convergence Information Technology Research Center (C-ITRC) support program (NIPA-2014-H0401-14-1022) supervised by the National IT Industry Promotion Agency (NIPA).

Author Contributions

HwaMin Lee and Min-Hyung Choi conceived and designed this system. Doo-Soon Park developed the idea of experiments and helped to improve the organization of the paper. HwaMin Lee wrote this paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Patients “I would like to know whether the match doctor and pharmacist prescription pharmaceuticals”, Available online: http://www.dailypharm.com/News/174530 accessed on 22 January 2015.
  2. Gyeung-Rye, K. Research on the Actual Condition for Medicine Side-effects; Korea Consumer Agency: Seoul, Korea, 2008. [Google Scholar]
  3. Németh, G.; Olaszy, G.; Bartalis, M.; Kiss, G.; Zainkó, C.; Mihajlik, P.; Haraszti, C. Automated Drug Information System for Aged and Visually Impaired Persons. In Computers Helping People with Special Needs; Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A., Eds.; Springer Berlin: Heidelberg: Berlin, Germany, 2008; Volume 5105, pp. 238–241. [Google Scholar]
  4. Usié, A.; Alves, R.; Solsona, F.; Vázquez, M.; Valencia, A. CheNER: Chemical named entity recognizer. Bioinformatics 2014, 30, 1039–1040. [Google Scholar]
  5. Drug Information, Available online: http://druginfo.nlm.nih.gov/ accessed on 22 January 2015.
  6. Drugs, Available online: http://www.drugs.com/ accessed on 22 January 2015.
  7. Drug Guide, Available online: http://www.drugguide.com/ accessed on 22 January 2015.
  8. DrugInfo, Available online: http://www.druginfo.co.kr/ accessed on 22 January 2015.
  9. Naver Health, Available online: http://health.naver.com/ accessed on 22 January 2015.
  10. Lexicomp, Drug information handbook, 23rd ed; Lexicomp: Hudson, NY, USA, 2014.
Figure 1. Web based drug information systems. (a) Drugs.com’s Pill Identifier; (b) Naver Health’s Drug Search.
Figure 1. Web based drug information systems. (a) Drugs.com’s Pill Identifier; (b) Naver Health’s Drug Search.
Symmetry 07 00376f1 1024
Figure 2. Example of no interference of background.
Figure 2. Example of no interference of background.
Symmetry 07 00376f2 1024
Figure 3. Example of interference of background.
Figure 3. Example of interference of background.
Symmetry 07 00376f3 1024
Figure 4. The architecture of our drug information system.
Figure 4. The architecture of our drug information system.
Symmetry 07 00376f4 1024
Figure 5. The implement results of DrugWellness application.
Figure 5. The implement results of DrugWellness application.
Symmetry 07 00376f5 1024
Figure 6. A screenshot of drug information after drug image search.
Figure 6. A screenshot of drug information after drug image search.
Symmetry 07 00376f6 1024
Figure 7. Precision of drug image search according to the size of pixels per comparison bundle.
Figure 7. Precision of drug image search according to the size of pixels per comparison bundle.
Symmetry 07 00376f7 1024
Figure 8. Total processing time of end-to-end system according to the size of pixels per comparison bundle.
Figure 8. Total processing time of end-to-end system according to the size of pixels per comparison bundle.
Symmetry 07 00376f8 1024
Table 1. Setting value examples in each imprint case.
Table 1. Setting value examples in each imprint case.
Case ExampleSetting Values
Symmetry 07 00376f9If imprint is printing with ink, 100 pixels per image bundle
If imprint is engraving, 50 pixels per image bundle
If imprint is debossing or embossing, 10 pixels per image bundle

Share and Cite

MDPI and ACS Style

Lee, H.; Park, D.-S.; Choi, M.-H. A Development of Hybrid Drug Information System Using Image Recognition. Symmetry 2015, 7, 376-382. https://doi.org/10.3390/sym7020376

AMA Style

Lee H, Park D-S, Choi M-H. A Development of Hybrid Drug Information System Using Image Recognition. Symmetry. 2015; 7(2):376-382. https://doi.org/10.3390/sym7020376

Chicago/Turabian Style

Lee, HwaMin, Doo-Soon Park, and Min-Hyung Choi. 2015. "A Development of Hybrid Drug Information System Using Image Recognition" Symmetry 7, no. 2: 376-382. https://doi.org/10.3390/sym7020376

Article Metrics

Back to TopTop