Controlling temperature of complex systems with Reinforcement Learning

Abstract

This thesis investigates thermal control in a simplified CubeSat system using both classical and reinforcement learning approaches. A reduced-order thermal model is developed to capture the main heat transfer mechanisms, including conduction, radiation, and orbital heat input. A thermostat-based controller is first implemented as a baseline and shows limitations such as oscillatory behavior and lack of anticipation. To improve performance, a reinforcement learning controller based on the PPO algorithm is introduced and trained in a Unity ML-Agents environment. The results demonstrate that the RL agent achieves smoother control and maintains subsystem temperatures closer to desired ranges compared to the baseline. Overall, the study shows that reinforcement learning is a promising method for thermal control under time-varying and constrained conditions.