Reactive planning for a robot (Figure 1) involves real-time decision-making where the robot dynamically adjusts its actions based on immediate sensory inputs from its environment, without relying on a precomputed plan. Unlike deliberate planning, which involves creating a detailed sequence of actions in advance, reactive planning enables robots to respond swiftly to unexpected changes, such as obstacles or moving targets, by continuously processing sensor data and selecting appropriate behaviors. For example, a robot navigating a cluttered space might use reactive planning to avoid collisions by instantly altering its path based on proximity sensor readings. This approach is particularly effective in unpredictable or dynamic environments but may lack long-term optimization compared to deliberate strategies.
Figure 1. Reactive planning in a robot for finding its path.
Drones as unmanned aerial vehicles (UAVs) hold significant potential in enabling reactive planning for robots or Unmanned Ground Vehicles (UGVs) by providing a crucial aerial perspective and dynamic data streams that a ground-based system alone cannot achieve (Figure 2).
Figure 2. Guiding a UGV by a UAV.
Firstly, drones can act as mobile, elevated sensor platforms. By flying ahead or alongside a UGV, they can provide real-time, high-resolution environmental mapping and obstacle detection from an overhead view. This allows for the identification of obstacles (e.g., ditches, uneven terrain, or unexpected barriers) that might be obscured from the UGV’s ground-level sensors until it’s too late for a smooth, reactive maneuver. This early warning system allows the UGV’s reactive planning system to anticipate and prepare for changes in its immediate path, even before the obstacles enter its own sensor range.
Secondly, drones can rapidly assess and update the local environment, providing dynamic situational awareness. In fast-changing scenarios, such as search and rescue operations or navigating chaotic environments, a drone can quickly scout ahead, identifying new hazards or clear paths and relaying this information instantly. This continuous, updated environmental picture directly feeds into the UGV’s reactive planning modules, allowing it to dynamically adjust its trajectory, speed, and behavior in real-time, making it significantly more adaptable and efficient than if it were relying solely on its onboard sensors.
In essence, drones augment the reactive capabilities of ground robots by extending their sensory horizon, providing critical foresight, and enabling quicker, more informed decision-making in complex and unpredictable environments. Guiding UGV by UAV has abundant applications including:
- Obstacle Avoidance and Navigation
- Search and Rescue Operations
- Environmental Monitoring
- Military and Defense
- Agricultural Automation
- Disaster Response and Mapping
- Exploration and Research
- Logistics and Delivery
However, the information provided by existing UAV systems lack insight into the depth variations of the ground surface ahead of the UGV. Specifically, the UGV telemetry data reports the UGV height based on the above-take-off (ATO) point, not based on the above-ground-level (AGL), as seen in Figure 3. This makes the UAV reported height to be negative in points below ATO. On the other hand, the UAV perception is usually based on camera (and lidar) which fail to function under harsh weather (e.g., fog) and poor light conditions. Furthermore, they become blind to occlusions such as vegetation which is common in environmental monitoring applications.
Figure 3. A drone telemetry data is given with ATO as reference.
To overcome the limitations of conventional UAV perception systems, IMEC has developed an advanced solution under the EdgeAI project that fuses radar and camera data. Radar sensors, known for their resilience to poor lighting and adverse weather, also offer the ability to detect objects through occlusions. By integrating radar with camera input, the system delivers high-resolution, occlusion-robust visibility, enabling UAVs to support reliable reactive planning for UGVs.
Figure 4. The IMEC UNSAND technology for reactive planning by UAVs.
The sensor technology developed at IMEC provides accurate UAV altitude measurements relative to the above ground level (AGL) and identifies the type of terrain ahead of the UGV. The current system achieves a UAV height estimation accuracy of 40 cm. Additionally, it classifies ground surfaces into five categories—packed dirt, asphalt, grass field, water, and sand—with a classification accuracy of 96%. The developed UAV nadir surface analysis based on deep-learning (UNSAND) is shown in Figure 4.
Blog signed by: Hamed Javadi (IMEC team)