Real-Time Multimodal Object Detection for Robotic Perception: Integrating YOLOv8n with an Intel RealSense D455 Depth Camera

Abstract

This abstract discusses the development of an object detection system to support visual perception in delivery robots operating in environments with varying conditions. Reliable object detection is necessary for the robot to safely navigate and interact with the surrounding environment. The proposed system uses an Intel RealSense D455 depth camera integrated with the YOLOv8n model to process RGB imagery and depth colormap. The dataset consists of RGB Indoor, RGB Outdoor, and Depth Colormap Outdoor scenarios with variations in distance and object orientation. All data is annotated using Roboflow. The implementation was carried out on Ubuntu 22.04 with Python 3.10 and achieved real-time inference performance of at least 30 FPS on the HP Victus 15 FA0116TX. Integration with Humble's ROS 2 enables efficient communication between perception and navigation modules, thereby improving the reliability of object detection under diverse lighting conditions.