Design of a Novel Remote Monitoring System for Smart Greenhouses Using the Internet of Things and Deep Convolutional Neural Networks

: To support farmers and improve the quality of crops production, designing of smart greenhouses is becoming indispensable. In this paper, a novel prototype for remote monitoring of a greenhouse is designed. The prototype allows creating an adequate artiﬁcial environment inside the greenhouse (e.g., water irrigation, ventilation, light intensity, and CO 2 concentration). Thanks to the Internet of things technique, the parameters controlled (air temperature, relative humidity, capacitive soil moisture, light intensity, and CO 2 concentration) were measured and uploaded to a designed webpage using appropriate sensors with a low-cost Wi-Fi module (NodeMCU V3). An Android mobile application was also developed using an A6 GSM module for notifying farmers (e.g., sending a warning message in case of any anomaly) regarding the state of the plants. A low-cost camera was used to collect and send images of the plants via the webpage for possible diseases identiﬁcation and classiﬁcation. In this context, a deep learning convolutional neural network was developed and implemented into a Raspberry Pi 4. To supply the prototype, a small-scale photovoltaic system was built. The experimental results showed the feasibility and demonstrated the ability of the prototype to monitor and control the greenhouse remotely, as well as to identify the state of the plants. The designed smart prototype can offer real-time remote measuring and sensing services to farmers.


Introduction
Currently, the problems of climate change and environmental damage (due to CO 2 emissions when burning fossil sources such as gas, oil, coal, etc.) have further serious problems of food and agricultural productivity [1]. The procedure of producing highquality crops is very important to meet the increasing demand of food products around the world. The industry of greenhouses is become a more fast-growing sector around the world. The greenhouse splits the crop from the environment, thus providing some sort of housing from the direct influence of external weather conditions. A greenhouse is mainly designed as a light transparent shelter to improve environmental conditions for plant production quality. Greenhouses are used to make a suitable atmosphere for planting and preventing plants from exposure to harsh environmental conditions, such as heavy rainfall or high solar radiation [2].
High or low temperatures, high humidity, CO 2 concentration, aeration, condensation of water, and water evaporation inside the greenhouse are among the major challenges faced by traditional greenhouses. Therefore, to achieve maximum returns from greenhouse cultivations, it is vital to maintain an environment that minimizes energy consumption [3].
Many parameters such as air temperature (Ta), relative humidity (RH), soil moisture (SM), light intensity (LI), and carbon dioxide (CO 2 ) concentration are involved and dependent on each other; this makes the greenhouse climate control a complicated procedure [4].
In the last decades, greenhouse climate monitoring and control problems have received considerable attention in agriculture engineering research [5]. Recently, researchers are more and more attracted to the application of the Internet of things (IoT) [6] to modernize greenhouses by designing smart monitoring systems [7] and creating artificial environments [8].
For example, Castañeda-Miranda and Castaño-Meneses [9] developed an automatic system for monitoring crops inside a greenhouse using solar energy and the IoT technique so that users could easily monitor the temperature, watering, and light through a mobile application. A wireless system enabling communication between the central control unit and four robots that worked in a model greenhouse was developed by Kumar et al. [10]; the results showed the potential of the system for application in real-life greenhouse operations. Chie et al. [11] designed an IoT-based system to monitor the environmental factors of an orchid greenhouse and the growth status of Phalaenopsis at the same time. As indicated by the authors, the system shows a great potential to provide quantitative information with high spatiotemporal resolution to floral farmers. Liao et al. [12] compared microclimate parameters inside two different tropical greenhouses using a custom-built wireless sensor for data fusion. A detailed review about the use of the IoT in the agricultural sector including greenhouses, various sensors which aid the IoT and agriculture, their applications, challenges, advantages, and disadvantages are reported in this paper [13].
Diesel is mainly used to supply greenhouses (e.g., for water pumping, irrigation, etc.) in remote areas, which is costly (e.g., in Saharan regions) [14]; however, currently, the utilization of solar energy (such as photovoltaic) for supplying greenhouses in remote areas is considered to be among the important applications of renewable energy sources [15]. The possibility to apply photovoltaic energy and the IoT to monitor greenhouses was shown by Aschilean et al. [16].
Another issue is the greenhouse crop diseases identification and early classification; recently, attempts have been made to design efficient approaches based on the IoT technique and image processing [17], for example, Mishra et al. [18] designed an IoT-based strawberry disease prediction system for smart farming; the capability of the model in disease prediction was shown. Kim et al. [19] used the IoT technique and a machine learning algorithm to classify plant diseases at an early stage. An IoT-based monitoring system for precision agricultural applications such as epidemic disease control was developed by Pavel et al. [20]; an expert system was also developed to make decisions regarding the diseases. A survey on the current techniques and prediction models based on image processing and the role of the IoT being applied for identification, detection as well as quantification of tomato plant diseases was shown by Khattab et al., Verma et al., and Diyan et al. [21][22][23].
The objective of this work was to design an effective smart monitoring system for modern greenhouse applications. To do this, the IoT technique, a clean source of energy, and deep convolutional neural networks (DCNN) were used. The proposed modern greenhouse will help farmers to: This paper is organized as follows: the next section provides the Materials and Methods including different steps to design the prototype (webpage, mobile application, and diseases identification and classification). The experimental results are reported in Section 3. The final Section 4 gives some concluding remarks and perspectives. Figure 1 shows a block diagram of the proposed monitoring system. It consists mainly of sensors (capacitive soil moisture, relative humidity, air temperature, light intensity, CO 2 , and ultrasonic), actuators (valve, water pump, fan, and servomotor), LCD (liquid crystal display; allows visualizing the measured data). A low-cost microcontroller (Arduino Mega 2560) was used to control and monitor different parameters inside the greenhouse. The parameters monitored were Ta, SM, RH, LI, CO 2 concentration, and water level (WL).

Monitoring and Displaying Greenhouse Parameters
• design of a low-cost monitoring prototype, • development of a webpage for monitoring parameters inside the greenhouse, • development of deep neural networks for diseases detection and classification, • development of an Android application for notifications about anomalies.
This paper is organized as follows: the next section provides the Materials and Methods including different steps to design the prototype (webpage, mobile application, and diseases identification and classification). The experimental results are reported in Section 3. The final Section 4 gives some concluding remarks and perspectives. Figure 1 shows a block diagram of the proposed monitoring system. It consists mainly of sensors (capacitive soil moisture, relative humidity, air temperature, light intensity, CO2, and ultrasonic), actuators (valve, water pump, fan, and servomotor), LCD (liquid crystal display; allows visualizing the measured data). A low-cost microcontroller (Arduino Mega 2560) was used to control and monitor different parameters inside the greenhouse. The parameters monitored were Ta, SM, RH, LI, CO2 concentration, and water level (WL). The control process is summarized in the following steps:  Step #1: Initialization, defining, loading reference parameters (CO2ref, Taref, SMref, RHref, WLref, and LIref) based on thr experimental thresholds.  Step #2: Measurements of the actual parameters (Ta, RH, SM, WL, CO2 and LI).  Step #3: Comparison of the parameters (measured versus references) for each sensor (e.g., if the measured Ta is outside of the Taref interval, the controller sends a signal to activate the corresponding relay and start the fan to refresh the environment inside the greenhouse). The control process is summarized in the following steps:
Step #3: Comparison of the parameters (measured versus references) for each sensor (e.g., if the measured Ta is outside of the Taref interval, the controller sends a signal to activate the corresponding relay and start the fan to refresh the environment inside the greenhouse).
Step #4: Sending a signal to the actuators by activating the corresponding relays: Most sensor networks require careful manual installation and configuration to assure that software components are properly associated with the physical instruments that they represent. The calibration method for each sensor is shown in Appendix A.

IoT and Webpage Development
To design the webpage, CCS, HTML, JavaScript languages and the Firebase platform were used. A Wi-Fi module named NodeMCU ESP8266 and a Wi-Fi camera module (ESP32) were used to collect and upload the measured data and images of the plants to the webpage for visualization and possible diseases classification. Figure 2 shows a block diagram for uploading the measured parameters and images to the designed webpage using the IoT technique.
Energies 2021, 14, x FOR PEER REVIEW 4 of 17 -LED: turn on the light. CO2ref, Taref, SMref, RHref, WLref, and LIref are reference parameters. Most sensor networks require careful manual installation and configuration to assure that software components are properly associated with the physical instruments that they represent. The calibration method for each sensor is shown in Appendix A.

IoT and Webpage Development
To design the webpage, CCS, HTML, JavaScript languages and the Firebase platform were used. A Wi-Fi module named NodeMCU ESP8266 and a Wi-Fi camera module (ESP32) were used to collect and upload the measured data and images of the plants to the webpage for visualization and possible diseases classification. Figure 2 shows a block diagram for uploading the measured parameters and images to the designed webpage using the IoT technique. The HTML language was mainly used to make the page structure, CCS-for designing a suitable environment, and JavaScript-for creating a dynamic environment. The Firebase designed by Google was used for hosting the webpage and database management.

Mobile Application and Notification
A mobile application was developed using Expo React-Native and a NodeMCU ESP8266 module. This application allows us to visualize the greenhouse parameters remotely (See Figure 3a).
To notify users about the state of the greenhouse (missed power supply, failure sensors, Internet failure, and diseases of the plants), a A6 GSM module with Arduino Uno were used. Figure 3b shows the workflow of this application. The HTML language was mainly used to make the page structure, CCS-for designing a suitable environment, and JavaScript-for creating a dynamic environment. The Firebase designed by Google was used for hosting the webpage and database management.

Mobile Application and Notification
A mobile application was developed using Expo React-Native and a NodeMCU ESP8266 module. This application allows us to visualize the greenhouse parameters remotely (See Figure 3a).
To notify users about the state of the greenhouse (missed power supply, failure sensors, Internet failure, and diseases of the plants), a A6 GSM module with Arduino Uno were used. Figure 3b shows the workflow of this application.

Database and Deep Learning CNNs for Diseases Classification
Classification and identification of plant diseases at early stages using image processing and analysis of environmental sensing data not only help farmers to get healthy plants but also maximize the production. In this work, we focused mainly on one type of vegetables (tomatoes).
The database used in this study comprises various images (e.g., tomato diseases) [24]. To detect and classify the tomato diseases, a deep convolutional neural network (DCNN)

Database and Deep Learning CNNs for Diseases Classification
Classification and identification of plant diseases at early stages using image processing and analysis of environmental sensing data not only help farmers to get healthy plants but also maximize the production. In this work, we focused mainly on one type of vegetables (tomatoes).
The database used in this study comprises various images (e.g., tomato diseases) [24]. To detect and classify the tomato diseases, a deep convolutional neural network (DCNN) was developed. Figure 4 shows the basic workflow of the DCNN for multiclass images classification. It consists of two parts, (1) the features extraction part, which contains some layers (convolutional and maximum pooling) and (2) the classification part, which performs nonlinear transformations of the extracted features (flattened and fully connected layers). The output could be the softmax function for predicting the class.

Database and Deep Learning CNNs for Diseases Classification
Classification and identification of plant diseases at early stages using image processing and analysis of environmental sensing data not only help farmers to get healthy plants but also maximize the production. In this work, we focused mainly on one type of vegetables (tomatoes).
The database used in this study comprises various images (e.g., tomato diseases) [24]. To detect and classify the tomato diseases, a deep convolutional neural network (DCNN) was developed. Figure 4 shows the basic workflow of the DCNN for multiclass images classification. It consists of two parts, (1) the features extraction part, which contains some layers (convolutional and maximum pooling) and (2) the classification part, which performs nonlinear transformations of the extracted features (flattened and fully connected layers). The output could be the softmax function for predicting the class. The DCNN code was implemented using the Python language in Raspberry Pi 4. The different steps for implementing the model into a Raspberry device are summarized below:

•
Step 1: Train the DCNN model • Step 2: Call the model on Raspberry with tf.lite.Interpreter() • Step 3: Program the ESP32 camera to save images every time we access its IP address • Step 4: Resize and change the type of the image to fit our model • Step 5: Predict this image with the model and get the result • Step 6: Use the Pyrebase library to make connections with the database • Step 7: Save the result to the Firebase database • Step 8: Read with NodeMCU and send an SMS if there is a problem

Standalone Photovoltaic Power System
A standalone PV system was used to supply the different components of the greenhouse, including electronic boards, water pumps, fans, lights, and the servomotor. It comprised three PV modules (360 W) connected in parallel, with one battery (12 V, 200 Ah), a charge regulator (24 V/30 A), a DC-DC buck converter, and a voltage regulator (in = 7-12 V, out = 5 or 3.3 V). Figure 5 shows the block diagram of the standalone PV power system utilized to supply the greenhouse.
Energies 2021, 14, 5045 6 of 15 A standalone PV system was used to supply the different components of the greenhouse, including electronic boards, water pumps, fans, lights, and the servomotor. It comprised three PV modules (360 W) connected in parallel, with one battery (12 V, 200 Ah), a charge regulator (24 V/30 A), a DC-DC buck converter, and a voltage regulator (in = 7-12 V, out = 5 or 3.3 V). Figure 5 shows the block diagram of the standalone PV power system utilized to supply the greenhouse.

Photovoltaic Power Supply System
To test the PV power system, we plotted in Figure 7 the data acquisition system, the output current, voltage, solar irradiance, and air temperature for a short period (25 May 2021, from 10 am to 12 am, with a time step of 5 s). To track the maximum power, the maximum power point algorithm was integrated, and more details can be found in our previous work [25,26].

Photovoltaic Power Supply System
To test the PV power system, we plotted in Figure 7 the data acquisition system, the output current, voltage, solar irradiance, and air temperature for a short period (25 May 2021, from 10 am to 12 am, with a time step of 5 s). To track the maximum power, the maximum power point algorithm was integrated, and more details can be found in our previous work [25,26]. From Figure 7b, it can be seen that the produced PV current and voltage were enough to supply the main components of the greenhouse.  (Table A1). The whole prototype cost around 150 dollars. From Figure 7b, it can be seen that the produced PV current and voltage were enough to supply the main components of the greenhouse.  (Table A1). The whole prototype cost around 150 dollars.  Figure 9a shows the main webpage designed for the smart greenhouse. The users can easily visualize the measured parameters online.

Monitoring and Data Visualization
To test the monitoring system, we illustrated in Figure 9b Figure 9a shows the main webpage designed for the smart greenhouse. The users can easily visualize the measured parameters online.

Monitoring and Data Visualization
To test the monitoring system, we illustrated in Figure 9b an example of the data posted to the webpage. The uploaded data were the real-time values of Ta, RH, LI, CO 2 concentration, air pollution, WL, and SM. The data were collected on 19 June 2021 for a period of 112 min. These results confirm and show clearly the correct functioning of the developed monitoring system and the webpage.

Warning SMS
A mobile application was developed in order to check the state of the greenhouse. Thus, users could be notified with a simple phone SMS in case of any problem in the system. The following Figure 10 displays a type of an SMS message received by a user (e.g., no Internet connection).

Warning SMS
A mobile application was developed in order to check the state of the greenhouse. Thus, users could be notified with a simple phone SMS in case of any problem in the system. The following Figure 10 displays a type of an SMS message received by a user (e.g., no Internet connection).

Mobile Application (Android)
The main Android screens of the developed mobile application are depicted in Figure  11. This application can help farmers to visualize the collected parameters and monitor the greenhouse remotely.

Plant Diseases Classification
The database used in this study consists of six categories (five disease classes and one health class). It contains a single leaf, multiple leaves, a single background, and a complex background. All the images were unified to 227 × 227 pixels. The diseases examined (in case of tomato plants) were bacterial spot, black leaf mold, gray leaf spot, late blight, and powdery mildew. Figure 12 shows an example of the images (diseases of tomato plants) available in the database.

Mobile Application (Android)
The main Android screens of the developed mobile application are depicted in Figure 11. This application can help farmers to visualize the collected parameters and monitor the greenhouse remotely.

Mobile Application (Android)
The main Android screens of the developed mobile application are depicted in Figure  11. This application can help farmers to visualize the collected parameters and monitor the greenhouse remotely.

Plant Diseases Classification
The database used in this study consists of six categories (five disease classes and one health class). It contains a single leaf, multiple leaves, a single background, and a complex background. All the images were unified to 227 × 227 pixels. The diseases examined (in case of tomato plants) were bacterial spot, black leaf mold, gray leaf spot, late blight, and powdery mildew. Figure 12 shows an example of the images (diseases of tomato plants) available in the database.

Plant Diseases Classification
The database used in this study consists of six categories (five disease classes and one health class). It contains a single leaf, multiple leaves, a single background, and a complex background. All the images were unified to 227 × 227 pixels. The diseases examined (in case of tomato plants) were bacterial spot, black leaf mold, gray leaf spot, late blight, and powdery mildew. Figure 12 shows an example of the images (diseases of tomato plants) available in the database. To develop the DCNN model, 4970 unified images were used; 3976 images (80%) were used for the training process, 994 images (20%)-for testing the model. The structure of the developed DCNN model is given below:  To develop the DCNN model, 4970 unified images were used; 3976 images (80%) were used for the training process, 994 images (20%)-for testing the model. The structure of the developed DCNN model is given below: The code was implemented and run under Google Colab, which is a free online cloud-based Jupyter Notebook environment. The DCNN model was trained under a GPU (Tesla K80 with 12 GB of GDDR5, Intel Xeon Processor with two cores @ 2.20 GHz and 13 GB RAM). Figure 13 shows the loss and the accuracy during the training process. The loss was close to 0.001 and the accuracy was about 0.99. These results confirm the good training of the model; thus, the DCNN model was ready to classify and identify the diseases of tomato plants. The code was implemented and run under Google Colab, which is a free online cloud-based Jupyter Notebook environment. The DCNN model was trained under a GPU (Tesla K80 with 12 GB of GDDR5, Intel Xeon Processor with two cores @ 2.20 GHz and 13 GB RAM). Figure 13 shows the loss and the accuracy during the training process. The loss was close to 0.001 and the accuracy was about 0.99. These results confirm the good training of the model; thus, the DCNN model was ready to classify and identify the diseases of tomato plants. To evaluate the performance of the developed DCNN model, we calculated the following error metrics: (1) where TP-number of true positive, TN-number of true negative, FP-number of false positives, FN-number of false negative. The results of the calculated error metrics are listed in Table 1. According to Table 1, good accuracy (88%) was obtained for diseases classification of tomato plants. However, the results could be improved by using a large database with high-quality images. To evaluate the performance of the developed DCNN model, we calculated the following error metrics: where TP-number of true positive, TN-number of true negative, FP-number of false positives, FN-number of false negative. The results of the calculated error metrics are listed in Table 1. According to Table 1, good accuracy (88%) was obtained for diseases classification of tomato plants. However, the results could be improved by using a large database with high-quality images.
Although the smart prototype designed has some advantages such as low-cost, free source of energy, good flexibility, capability to monitor diseases, good skills and knowledge, particularly in the implementation of the DCNN, to classify diseases are indispensable.

Conclusions and Outlook
A smart greenhouse was designed and experimentally verified at a laboratory scale. The greenhouse prototype was equipped with a smart monitoring system able to control and remotely monitor greenhouse parameters. The greenhouse was supplied by a free source of energy, a small-scale autonomous PV system, which was basically used to supply the main components of the greenhouse such as sensors, electronic boards, water pumps, LED lights, fans, servomotors, and relays. Thus, an artificial climate inside the greenhouse was created, with appropriate control of air temperature, relative humidity, soil moisture, CO 2 concentration, light intensity, by means of precise ventilation, lighting, and irrigation. Users could be also notified by an SMS about the state of the plants before the outbreak of the disease. The combination of the Internet of Things and deep learning have showed their capability to monitor the health of the plant investigated in this study. Detection and classification of plant diseases can help farmers to monitor plants growth efficiently for better crops production.
Future work will focus on the application of advanced predictive control algorithms for parameters control inside the greenhouse. Furthermore, the application of computer vision and other deep learning frameworks will be investigated. A large dataset with different cultivated plants (images) will be used.  Informed Consent Statement: Not applicable for this study.

Data Availability Statement:
The data used in the present study is publicly available (https://data. mendeley.com/datasets/ngdgg79rzb/1).