Detection of citrus diseases in complex backgrounds based on image-text multimodal fusion and knowledge assistance
文献类型: 外文期刊
作者: Qiu, Xia 1 ; Chen, Hongwen 1 ; Huang, Ping 1 ; Zhong, Dan 1 ; Guo, Tao 1 ; Pu, Changbin 1 ; Li, Zongnan 1 ; Liu, Yongling 1 ; Chen, Jin 3 ; Wang, Si 1 ;
作者机构: 1.Sichuan Acad Agr Sci, Inst Remote Sensing & Digital Agr, Chengdu, Peoples R China
2.Sichuan Acad Agr Sci, Sci & Technol Ctr Intelligent Agr, Chengdu, Peoples R China
3.Beijing Normal Univ, Inst Remote Sensing Sci & Engn, Fac Geog Sci, State Key Lab Remote Sensing Sci, Beijing 100875, Peoples R China
关键词: citrus disease; deep learning; multimodal fusion; background diversity; knowledge assistance
期刊名称:FRONTIERS IN PLANT SCIENCE ( 影响因子:5.6; 五年影响因子:6.8 )
ISSN: 1664-462X
年卷期: 2023 年 14 卷
页码:
收录情况: SCI
摘要: Diseases pose a significant threat to the citrus industry, and the accurate detection of these diseases represent key factors for their early diagnosis and precise control. Existing diagnostic methods primarily rely on image models trained on vast datasets and limited their applicability due to singular backgrounds. To devise a more accurate, robust, and versatile model for citrus disease classification, this study focused on data diversity, knowledge assistance, and modal fusion. Leaves from healthy plants and plants infected with 10 prevalent diseases (citrus greening, citrus canker, anthracnose, scab, greasy spot, melanose, sooty mold, nitrogen deficiency, magnesium deficiency, and iron deficiency) were used as materials. Initially, three datasets with white, natural, and mixed backgrounds were constructed to analyze their effects on the training accuracy, test generalization ability, and classification balance. This diversification of data significantly improved the model's adaptability to natural settings. Subsequently, by leveraging agricultural domain knowledge, a structured citrus disease features glossary was developed to enhance the efficiency of data preparation and the credibility of identification results. To address the underutilization of multimodal data in existing models, this study explored semantic embedding methods for disease images and structured descriptive texts. Convolutional networks with different depths (VGG16, ResNet50, MobileNetV2, and ShuffleNetV2) were used to extract the visual features of leaves. Concurrently, TextCNN and fastText were used to extract textual features and semantic relationships. By integrating the complementary nature of the image and text information, a joint learning model for citrus disease features was achieved. ShuffleNetV2 + TextCNN, the optimal multimodal model, achieved a classification accuracy of 98.33% on the mixed dataset, which represented improvements of 9.78% and 21.11% over the single-image and single-text models, respectively. This model also exhibited faster convergence, superior classification balance, and enhanced generalization capability, compared with the other methods. The image-text multimodal feature fusion network proposed in this study, which integrates text and image features with domain knowledge, can identify and classify citrus diseases in scenarios with limited samples and multiple background noise. The proposed model provides a more reliable decision-making basis for the precise application of biological and chemical control strategies for citrus production.
- 相关文献
作者其他论文 更多>>
-
Transcriptomics and metagenomics of common cutworm (Spodoptera litura) and fall armyworm (Spodoptera frugiperda) demonstrate differences in detoxification and development
作者:Tang, Ruixiang;Liu, Fangyuan;Wang, Jiao;Wang, Lei;Li, Jing;Fan, Zhenxin;Yue, Bisong;Lan, Yue;Liu, Xu;Guo, Tao
关键词:Spodoptera litura; Spodoptera frugiperda; Developmental stages; Transcriptome; Metagenomics
-
Attention based spatiotemporal graph attention networks for traffic flow forecasting
作者:Wang, Yi;Jing, Changfeng;Xu, Shishuo;Guo, Tao
关键词:Traffic flow forecasting; Spatiotemporal graph neural network; Network deepening; Network degradation; Dynamic spatiotemporal correlation; Intelligent transportation systems
-
Measurement of Urban-Rural Integration Level in Suburbs and Exurbs of Big Cities Based on Land-Use Change in Inland China: Chengdu
作者:Wang, Meimei;Yang, Yongchun;Guo, Tao
关键词:built-up land; urban-rural integration (URI); URI level; land-use change; path; Chengdu
-
Spatiotemporal modeling of land subsidence using a geographically weighted deep learning method based on PS-InSAR
作者:Li, Huijun;Zhu, Lin;Gong, Huili;Dai, Zhenxue;Guo, Tao;Guo, Gaoxuan;Wang, Jingbo;Teatini, Pietro;Teatini, Pietro
关键词:Land subsidence; Spatiotemporal modeling; GW-LSTM; PS-InSAR; Uncertainty analysis
-
Water Quality Index (WQI) as a Potential Proxy for Remote Sensing Evaluation of Water Quality in Arid Areas
作者:Zhang, Fei;Liu, Changjiang;Wang, Xiaoping;Wang, Weiwei;Cao, Naixin;Zhang, Fei;Liu, Changjiang;Wang, Xiaoping;Wang, Weiwei;Cao, Naixin;Chan, Ngai Weng;Shi, Jingchao;Kung, Hsiang-Te;Li, Xinguo;Guo, Tao
关键词:Water Quality Index (WQI); Ebinur Lake; remote sensing
-
Soil potassium regulation by changes in potassium balance and iron and aluminum oxides in paddy soils subjected to long-term fertilization regimes
作者:Han, Tianfu;Huang, Jing;Liu, Kailou;Liu, Shujun;Zhang, Lu;Zhang, Huimin;Han, Tianfu;Feng, Gu;Han, Tianfu;Feng, Gu;Liu, Kailou;Fan, Hongzhu;Shi, Xiaojun;Jiang, Xianjun;Chen, Jin;Liu, Guangrong;Xu, Yongmei;Zhang, Huimin
关键词:Potassium balance; Iron and aluminum oxides; Long-term fertilization; Paddy soil
-
Variability in CitXET expression and XET activity in Citrus cultivar Huangguogan seedlings with differed degrees of etiolation
作者:Qiu, Xia;Dong, Zhixiang;Ye, Shuang;Huang, Shengjia;Liu, Xinya;Xi, Lijuan;Wang, Zhihui;Gu, Xianjie;Sun, Guochao;Wang, Zhihui
关键词: