Interpretable deep multimodal-based tomato disease diagnosis and severity estimation

Abstract

Abstract Plant diseases pose a significant threat to global food security, particularly in regions that rely heavily on crops that are vulnerable to disease, such as tomatoes. This research addresses the inefficiencies of traditional farming solutions by presenting a novel multimodal deep learning algorithm. The algorithm leverages EfficientNetB0 for image-based disease classification and utilizes Recurrent Neural Networks (RNN) to predict disease severity based on environmental data. By integrating visual and climatological inputs, our model addresses the limitations of unimodal systems, enhancing classification accuracy and interpretability. The model achieved a disease classification accuracy of 96.40% and a severity prediction accuracy of 99.20%. Additionally, the use of LIME and SHAP explainable AI techniques improves the understanding of disease severity classification outcomes. The contributions of this study align with precision agriculture practices and advance the resilience of local food systems, particularly in economies heavily dependent on tomato production. The proposed approach has the potential to mitigate the impacts of plant diseases and enhance food security by utilizing innovative technological solutions.