Obstacle Avoidance Using Hybrid Reinforcement Learning in Dynamic Environments

Abstract

This thesis presents a hybrid reinforcement learning approach for obstacle avoidance with the Festo Robotino 4 omnidirectional mobile robot in dynamic environments. The proposed method combines a classical Pure Pursuit controller with a Soft Actor-Critic (SAC) agent that uses a custom 1D CNN (LidarCNN1D) to extract spatial features from stacked LiDAR observations. The system is implemented in ROS 2 Humble and Gazebo, and trained with a three-stage curriculum that scales from a 10×10 m room with static obstacles up to a 6×6 m room with six dynamic pedestrians. Four methods — Hybrid SAC, Optimized SAC, Pure SAC, and Pure PPO — are compared to isolate the contribution of each architectural choice. The Hybrid SAC achieves the best results, reaching a 99% success rate in the hardest training stage and 84% in the most difficult unseen evaluation environment, while maintaining the lowest collision rate (16%). The findings show that CNN-based feature extraction, frame stacking and Pure Pursuit guidance combine to deliver robust real-time navigation suitable for indoor service robots.