1. The Problem of Vineyards Yield Prediction
The smartification of the agri-food sector is a key factor in improving both productivity and environmental impact. Real-time grape detection represents an important step toward supporting vineyard operators in the field by reducing labor-intensive and repetitive tasks. It also improves yield estimation by analyzing a larger number of vineyard rows, resulting in better sampling. It is therefore important to develop accurate, low-energy solutions suited to a wide range of possible conditions in vineyards. To this end, we propose object detection models designed to meet practical requirements – efficiency, speed, and accuracy. The models have been trained under real environmental conditions, including variations in grape size, weather, and lighting. Our AI solutions are designed to be scalable and adaptable across different technologies, making them suitable for a wide range of vineyard applications.
1. NeuroCorgi Circuit
Deploying artificial intelligence models on edge devices presents considerable challenges due to their stringent computational and architectural performance requirements. Ensuring real-time inference further necessitates low-latency hardware platforms capable of maintaining the desired frame rate. Moreover, energy efficiency remains a critical design constraint, as edge devices are frequently powered by limited battery resources. Thus, in order to deploy these models in real vineyards, it remains key to developing powerful, ultra-low-power and low-latency hardware computing platforms able to exploit highly resolute data generated by environmental sensors.
CEA has developed an edge device targeting AI inference with ultra-low-power and low latency. Remarkably, this solution enables the processing of up to HD images (1280×720) at 30 frames per second, consuming only a fraction of the energy required by the image sensor. It leverages two key principles: a feature extraction backbone with hard-wired weights and quantization, coupled with a transfer learning technique. Specifically, this Feature Extractor Accelerator consumes 23.2 mW, with a latency of 6.9 ms, while achieving 70.42% accuracy on the ImageNet dataset.
The EdgeAI Value Chain Demonstrator 3, aims at coupling NeuroCorgi circuit with the STM32MP2 computing platform, to deploy efficient grape detection in vineyards algorithms (NeuroCorgi-LeYolo demonstrator). NeuroCorgi circuit performs feature extraction of the images while STM32MP2 computes the LeYolo detection head. The combined system will be battery powered and embedded on vineyards robot (see Figure 1).
2. Grapes Detection on the Embrapa Dataset
The Embrapa Wine Grape Instance Segmentation Dataset is the most popular benchmark for grapes detection in RGB images. It contains 300 images of 5 different black and white varieties during the veraison (coloring) stage. Each grape cluster was labeled with bounding boxes for object detection.
For our experiments we have adapted our LeYolo detector, previously published in the EdgeAI project, with the NeuroCorgi backbone. Despite its frozen backbone, our NeuroCorgi-LeYolo model is close to the YOLO model with a similar mAP of 83% and precision of 81.6%. However, the NeuroCorgi backbone has the advantage of running on ASIC, which makes it fast and energy efficient.
The optimal use of vision transformers has also been the subject of research. Real-time detection transformers (RT-DETR) have been designed and optimized to operate in real time on the MP2 board. The custom RT-DETR model has only 1.86 million parameters (10.9 GFLOPs) and has been quantized with TensorFlow Lite asymmetric and per-tensor 8-bit integer precision. The inference time of the compiled .nbg model takes 0.2 seconds on MP2; most operations (73.3%) are compiled to run on the NPU, while the remaining 26.7% are executed by the GPU.
Our model performance metrics are presented in the table below, including results obtained with the best current version of YOLO (variant 26 nano), for comparison.
Models | %Recall | %Precision | %mAP-50 | %mAP-50-95 |
NeuroCorgi | 74.5 | 81.6 | 83 | 44.1 |
YOLO26 nano | 78.3 | 81.7 | 83 | 51.2 |
RT-DETR custom | 73.8 | 83.1 | 79.6 | 46.5 |
Figure 2‑- Grapes detection on a validation image (NeuroCorgi-LeYOLO)
Figure 3- Grapes detection on video frames (RT-DETR custom model)
4. Grape Disease Detection on Tardif dataset
The Tardif dataset is a useful, open-access collection of images annotated by experts and representing symptoms in different parts of the grapevine – leaves, shoots and bunches. These images cover the most common diseases – “FD symptomatic leaves”, “ESCA symptomatic leaves” and “Confounding leaves” and their confounding factor which can be easily confused with other categories and presents a challenging case for object detection algorithms. Our RT-DETR object detection model has been optimized for edge inference; performance benchmark results obtained on the STM32MP2 board show an inference time of 0.11 seconds; the optimized .nbg model runs 65% on the NPU and 35% on the GPU.
Figure 4 presents the various diseases that the model was able to detect in a randomly selected validation image. The detector shows 22 FD leaves (mAP@50 = 60.9%), 47 confounding leaves (mAP@50 = 47.5%), and 3 symptomatic shoots.
Figure 4- The disease detection with custom RT-DETR model
Blog signed by: URCA, INPG, CEA, ST, VPHV/CV/VPP teams