Partially occluded weed classification using vision transformers and convolutional neural networks for precision agriculture

Abstract

Abstract Automated weed detection is essential for site-specific herbicide application, that can result into the reduced environmental footprint of conventional agriculture. However, for field deployment of automated weeding devices, occlusion remains a critical challenge that can weaken the precision of weed identification. Here, we compare the performance of Vision Transformers (ViT-B16 & PvTv2) and Convolutional Neural Networks (EfficientNet-B0 & ResNet-50) in accurate weed detection, using controlled synthetic occlusion levels (0%, 25%, and 50%). We found that ViT-B16 has superior occlusion resilience, with image testing accuracy increasing from 80% to 86% under 50% occlusion. In contrast, the testing accuracy of PvTv2, EfficientNet-B0 and ResNet-50 dropped from 45 to 76% under similar conditions. Multivariable regression confirmed architecture type as the dominant testing accuracy driver ( p  ≤ 0.001), with ViTs outperforming CNNs by an average of 14.56% points. These results suggest that occlusion resilience is not uniform across architectural variants but depends critically on attention-based design. Consequently, for real time deployable automatic weed detection systems, hybrid architectures that balance ViT global context with CNN computational efficiency represent a critical future direction. Such approaches can support precise herbicide application, reduce chemical inputs, and enable more sustainable crop protection through reliable AI-driven automation.