Deep Learning based Rapid Diagnosis System for Identifying Tomato Nutrition Disorders

Abstract

Nutritional disorders are one of the most common diseases of crops and they often result in significant loss of agricultural output. Moreover, the imbalance of nutrition element not only affects plant phenotype but also threaten to the health of consumers when the concentrations above the certain threshold. A number of disease identification systems have been proposed in recent years. Either the time consuming or accuracy is difficult to meet current production management requirements. Moreover, most of the systems are hard to be extended, only detect a few kinds of common diseases with great difference. In view of the limitation of current approaches, this paper studies the effects of different trace elements on crops and establishes identification system. Specifically, we analysis and acquire eleven types of tomato nutritional disorders images. After that, we explore training and prediction effects and significances of super resolution of identification model. Then, we use pre-trained enhanced deep super-resolution network (EDSR) model to pre-processing dataset. Finally, we design and implement of diagnosis system based on deep learning. And the final results show that the average accuracy is 81.11% and the predicted time less than 0.01 second. Compared to existing methods, our solution achieves a high accuracy with much less consuming time. At the same time, the diagnosis system has good performance in expansibility and portability.

1. Introduction

Tomato is one of the most important products containing various nutrients and consumed bylarge number of people every day. The disease of tomato crops is one of the most importantreasons for causing the serious loss of production, impacting the profits of tomato enterprise. The growth of crops is influenced by many factors. Soil is one of the important basic substances, and it is crucial to the development of agriculture [1]. Fertilization is one of the most common and popular means of agricultural production, which can effectively improvesoil properties and improve yield and quality of crops. The imbalance of trace elements willaffect the growth of crops, affect the quality of crops, reduce the total production, and havecertain harm to the crops and consumers [2]. There are many categories of tomato nutritional disorder diseases can lead to affect productivity and quality. It is challenge to implement adiagnosis system with high accuracy for the images among dataset with tiny difference insymptoms. Until now, most of these industry manager exploits manual inspection method of recognition tomato nutritional disorder [3]. Obviously, it is a time-consuming task in this way. Therefore, develop an easy-to-use diagnosis system of tomato nutritional disorder can be benefited to prevent the spread of diseases and keep economy of industry. The key motivation of this work is to achieve a system to identify tomato nutrition disorders by using commondigital camera.

We focus on identifying eleven types of very important tomato nutritional disorders thatresult from lack or excessive certain trace elements. These eleven trace elements include ninetypes of lack elements are boron (LB), iron (LFe), potassium (LK), manganese (LMn), nitrogen (LN), phosphorus (LP), sulfur (LS), and zinc (LZn). Three types of excessive elements are boron (ExB), manganese (ExMn), and nitrogen (ExN). The main subject of this research is to achieve an easy-to-use tomato nutrition disorders diagnosis system to identify the type of disease and provide appropriate solutions according to the results.

Generally speaking, a diagnosis system for discriminating crop disease can be summarized into three kinds. The first one is based on image-processing technology [4]. This method prevalently used in image segmentation of plant diseases for several decades. The method of image segmentation usually depends largely on great changes in texture, color or shape. Hence, it is hard to identifying the diseases by using image-processing technology for different categories of diseases images with small differences. The second kind is using machinelearning methods to classify features which are extracted by human. such as k-nearestneighbor [5], random forest [6],and support vector machine (SVM) [7]. This method depends on the classification features which are selected and adopted by human experience. Hence, such methods are hard to apply for farm enterprises for the drawbacks on flexibility and timeliness. The third one is based on deep learning [8]. Please note that the deep learning based approachs have been achieved remarkable success in many applications [9, 10, 11 ], especially in image classification. Generally, such method provides an end-to-end pipeline, by training related samples, achieve classification task automatically. So far, the methods based on convolutional neural network (CNN) have been considered the best of identification task[12,13].

In this paper, a novel automatic tomato nutrition disorders diagnosis is proposed based ondeep learning and enhanced deep super-resolution. Our approach with the following steps. Firstly, we build high quality dataset. We collect related images, label each image and enhancesmall differences between images by using pre-processing methods. Also, in order to train the model better, we exploit data augmentation methods. Secondly we build an identification model to achieve diagnosis system of tomato diseases. We build the identification architecture based on CNNs, training and validation until the model reach at a good performance. Finally, we establish diagnosis system. Based on identification model, we build an easy-to-usediagnosis system both on client and online. The client system can realize real-time diagnosis. And the online system can deal with images which not have good prediction results. With the diverse samples of images which will be uploaded by users, the system has expansibility and portability.

2. Related Work

Many identification systems have been developed based on images of plant disease. Forexample, SVM[7] represents as the classical method of machine learning algorithms in the literature. Dubey et al proposed a disease classification method for apple fruit which based onimage processing technology [14]. To segment the images of grape from background and leaves, Behroozi-Khazaei and Maleki presented an method based on genetic algorithm (GA) and artificial neural network (ANN) [15]. Ma et al presented an algorithm based on the information on color and region growing for identifying foliar diseases in greenhousevegetable [16].

CNN has substantially improved the state-of-the-art methods on object identification. Tosome extent, the identification systems of disease based on deep CNN have recently madeground breaking advances on agricultural.Therefore, it has been widely studied and used [17]. Given large-scale training data, Brahimi et al proposed a system can identify nine diseases of tomato based on deep learning [18]. To classify and detect diseases and pets of tomato plant, Fuentes et al designed an end-to-end architectures based on deep CNNs, they collected five thous and disease images of tomato plant, including nine types of disease and pets. Beforetraining the network, they use data augmentation methods to process the dataset. At finally experimental result shows that the network training on the dataset with augmentation methodshas better performance on identification and detection task [19]. In order to identify disease of apple foliage better, Liu et al designed CNN architectures based on Alexnet [20]. They collected more than thirteen thousand includes four categories diseased images of apple leavesto create the dataset. The proposed system plays real-time fruit detection, meanwhile it can be working at complex surrounding. According to the result, the method based on deep learning has significant performance on accuracy and timeliness without any extra image pre-processing.

Recently, many researchers have taken effort on studying the system to find some effective ways to promote the accuracy and rapidity of prediction results. Lu et. al proposed a diagnosis system to recognize wheat disease [21]. The proposed system designed based on a weakly supervised deep learning framework that can identify both category and location of wheat disease automatically. All the images of training data are collected in wild. However, oned rawback of this system is inconvenient to carry due to most predictors are computers orservices. In view of the limitation of the system, Mwebaze et. al achieves a diagnosis system to implement five diseases and severity measurements of disease application on smartphone [22]. The application implementation of such system based on LinearSVC [23], KNN [24] and Extra Trees [25] three classifiers. Only identify a fixed number of disease, without scalable is the main disadvantage of this approach.

3. Materials and Methods

3.1 System Overview

The aim of our research is to design a system on smartphone to identify tomato nutritiondisorders by using common digital camera. The system designed and developed based on deeplearning. The systems with ability to extend and transplant. A general overview of the systemis presented in Fig. 1. Mainly the system consists of four parts. The first part is dataset processing, we collect related images and divide the dataset into three sections. Then weexploit serval methods to enhance the details of the original image and augmented the datasetto reducing the probability of over-fitting. The second is the identification model part. In this part we design identification architecture based on CNNs, training and validation the modeluntil it reaches at a good performance. The third part is to developing a diagnosis system onclient, in this part we implement the system on smartphone. And the fourth part is todeveloping a diagnosis system online. In this part is to solve the disease images which are hard to be well recognized. By such online system can update the dataset and improve the overallscalability. Following we introduce each part of the system in detail.

 

Fig. 1. Overall architecture of diagnosis system.

3.2 Dataset Processing

We collect more than two thousand pieces of images, which consist in eleven categories of tomato nutritional disorders. Consideration based on small differences between different categories, all the images labeled by experts manually. The samples of the eleven diseases of tomato nutritional disorders show in Fig. 2.

 

Fig. 2. Some samples of eleven kinds nutritional disorders of tomatoes.

Commonly, the difference between categories is tiny and brings certain difficulties to the identification. If the images of dataset including more detail information, it will improve the training accuracy. Generally speaking, super resolution reconstruction is realized via a series of low-resolution images to improve its resolution. This scheme can enhance the details of a low-resolution image. At present, EDSR achieves a state-of-the-art performance in the field of super resolution [26]. The significant performance improvement on EDSR is to reducememory consumption compared with conventional residual networks of the training. Inconsideration of the training procedure numerically would be unstable when increasing the number of feature maps above a certain level. The baseline of EDSR models adopting theresidual scaling method [27] with factor 0.1 to solve this problem. And the architecture of EDSR shows in Fig. 3. Meanwhile, it takes relatively short time to generate a high-resolutionimage from a low-resolution one. Encouraged by the excellent performance of EDSR, we usepre-trained model of EDSR for image pre-processing. Some samples of the processing with ground truths are shown in Fig. 4. And the Image quality evaluation results show on Table 1.

 

Fig. 3. The architecture of EDSR [26].

From the result, we can find both visual perceptions and data results shows the processed image have more detail information. Alought deep learning based system has excellent performance when compared with computer vision algorithms. Because the overfitting is one of the main drawback [28]. There are several reasons of overfitting problem. One of the mostimportant reason is the number of images used for training [29]. Therefore, when the number of images is not enough for the training, data augmentation is an effective way to reduce the probability of overfitting [30]. Provide additional number of images for training, we exploitsome methods of geometrical transformations such as resizing and rotation. By this way togenerate six datasets which include: Rotation dataset(Rot), Resizing dataset(Res), Resizing & Rotation (Res & Rot), SR-Rotation(SR-Rot), SR-Resizing(SR-Res), SR - Resizing & Rotation (SR-Res Rot).

 

Fig. 4. Image enhanced by EDSR. The top is original image, the middle image is result of originalimage processing by bicubic method and the bottom image is the result of original image processing by EDSR

Table 1. Results of PSNR (dB), SSIM. 

 

3.3 Identification Model

CNN is one of a feed-forward neural network which has performed equally well for image processing [31-34]. Mainly, the convolution neural network is composed of the convolutionlayers and the pooling layers. The convolution layer is only to extract feature information from the adjacent layer.The convolutional layer usually contains a number of feature maps, in each feature map neurons are arranged in rectangular pattern, and neurons having the samefeature-map can share weights. Share weights is a most important contribution to CNN. By this way can reduce the computation of the whole network training and reduce the risk of over-fitting. Commonly, mean pooling and max pooling are two forms of the pooling layer. Sub-sampling layer can reduce the dimension of feature map. And by this way improve the accuracy and reduce overfitting to some extent. The model with redundant parameters affectstraining time of the network and increase the prediction time, therefore the network based onconvolution and sub-sampling layers can simplify the complexity and reduce the parameters of the model. Our identification architecture shows in Fig. 5.

 

Fig. 5. Nutrition nisorders classification architecture based on Alexnet.

3.4 Diagnosis System Client

The use of mobile terminals are increasing year by year, mainly benefit its portability. Our motivation is to turn the research into a more portable tool that allows agriculturalimplementers to diagnose diseases in field of greenhouse or in the wild, and providesuggestions to help them. On one hand, the agricultural environment is not covered by the Internet in many cases. So designing an application that relies too much on the web can beinconvenient. On the other hand, farm owners or managers not only need to know the information about crop defects, but also need measures to deal with these diseases. Based on these considerations, we design this user interaction program based on the convenience of operation, and implement a smartphone based service system that allows smartphones todiagnose leaf information by the image of leaves with defective.

3.5 Diagnosis System Online

Usually, the system using in the open environment such as big field, farm and greenhouse which a capture some images with low-confidence. We design and implement web-based diagnosis system to provide the farmers advice for the images with lower confidence values by pre-trained model.

Mainly, the system consists of three parts as follows: First we judge confidence value of captured image, if the confidence value less than the threshold, the image will upload to the server-side of the system by transmission control protocol (TCP) [35]. Otherwise, returns the result which predicted by pre-trained model to user directly. Then we upload the images and stored in the database on server-side. When experts login server-side, judge the image and send back the result to user by TCP. At the same time, we can acquire the new labelled images, when the images data up to a certain quantity, we copy the labelled images and update the dataset and improve the accuracy of recognition and expand the category of recognition by training and validation.

This online diagnosis system can solve the problem of incomplete dataset. By this way willimprove the accuracy and the category of the whole system prediction.

4. Experimental Results and Discussion

4.1 Tomato disease identification

For deep learning technology, parallelizing calculation is an important power, benefit from GPU's parallelizing power, we run our codes on a Windows10 64-bits PC equipped with an Intel(R) Core (TM) i5-7500 CPU @ 3.40GHz processor, 8 GB-RAM. a NVIDIA GTX 1060GPU, Our program is written in python language and the architecture are implemented with tensor flow.

In order to understanding which data augmentation method is more suitable for this study, we train the models on these datasets separately. Fig. 5 and Fig. 6 illustrate the accuracy and loss curves training on each dataset respectively.These curves are drawn at the learning rate of 0.001. From the curves we can find the accuracy of training slowly to convergence when the system training on ROT. Althougth training on RES&ROT and SR-RES&ROT have betterperformance of accuracy, they are prone to fluctuations and oscillations. Adopting RES, SR-ROT, SR-RES datasets, the accuracy can reach more than 99% after 10 epoches, the model will soon be convergence of high accuracy. Fig. 7 and Fig. 8 represent the accuracy and losscurves validation on each dataset respectively. From the curves we can find the accuracy of validation is hard to promote when it nearly reached at 85% when the system training on ROT. The response from loss curves are consistent with the information in accuracy curve.

We also test the prediction performance of six pre-trained models which are trained separately on ROT, RES, RES&ROT, SR-ROT, SR-RES, SR-RES&ROT. Our testing datasetcontains more than 200 untrained images. The prediction results are show in Table 2. Based on the above results, we choose the classification model which trained on SR-ROT.

 

Fig. 6. Iteration of training accuracies changes on six categories of datasets

 

Fig. 7. Iteration of loss changes during training process on six categories of datasets

 

Fig. 8. Iteration of validation accuracies changes on six categories of datasets

 

Fig. 9. Iteration of loss changes during validation process on six categories of datasets

Table 2. The accuracy of the model training under each dataset with different data augmentation methods.

 

4.2 Identification System Deployment

We implement an application by embedding the pre-trained identification model to the mobileclient. Our codes are written in objective-c language and implement the application on Apple Macintosh operating system, thus application can analysis diseases of captured image predict the results and give advice to users. We show some results on Fig. 10.

In detail, firstly, we store the treatments and suggestions for each disease in a lightweight database SQLite in the mobile phone. Then the application calculates the confidence value of the image which captured by user. After that, if the confidence value above the threshold (>75% in this paper) we use the structured query language to query the corresponding solution by the prediction results to find the corresponding solution and advice. Finally, providing results and suggestions directly back to user.

Fig. 10. Partial results of the application on mobile. User take a photograph by camera (left); the application returns top three results (middle); at the same time the application give some advice to user(right).

5. Conclusion and Future Works

In this paper we propose a novel diagnosis system for tomato nutrition disorders based on deeplearning, by such ways to training the model on tomato images of disease without need of extrapre-processing methods.

We exploit pre-trained EDSR model to enhance the images detail information. Then we generate six different datasets by using augmentation technologies of resizing and rotation. After we implement CNNs based architecture, we train the identification model on these six datasets separately. Furthermore, compared with different experimental results betweendifferent dataset indicates that SR-Rot is a superior choice. Training on such dataset can get better result of identification for disease identification than another five datasets. On one hand, we proposed diagnosis system client can provide an easy-to- use system for the managers of farm. On another hand, the diagnosis system client application can not provide assistanceaccurately in some cases, we also provide an online system to avoid this problem. By this way, we achieve the whole system with ability of extensibility and portability.

In the future, our objective is to optimize the CNN based identification architecture of improved identification accuracy and reduce model size. Besides, Fine grained image classification technology could be further research on.

6. Acknowledgments

This research was supported by project of Scientific Operating Expenses from Ministry of Education of China (2017PT19) and China Postdoctoral Science Foundation (2018M630222). All of the mentioned support is gratefully acknowledged.