AI Smart Drone Controller based on ESP32-S3

Client

A consumer drone manufacturer adding AI-powered obstacle avoidance and subject tracking to their existing flight controller platform.

Challenge

Adding real-time object detection and tracking to a drone that has limited payload (under 250g total camera+compute) and must operate without internet connectivity.

Product Concept & Architecture

An ESP32-S3 acts as a vision co-processor, communicating with the existing STM32F7 flight controller via UART (MAVLink protocol). An OV5640 5MP camera captures 1080p video.

Engineering Solution

The ESP32-S3 runs ESP-DL with an INT8-quantized YOLO-Nano model (320KB) for person/obstacle detection at 30fps. Bounding box coordinates are converted to GPS-relative offsets and sent to the flight controller as MAVLink position targets.

Result

30fps 1080p object detection. 20ms inference latency. <40g total weight.