This study focuses on developing and designing AI agents for FPS games using the Unity engine and the ML-Agents toolkit. The Proximal Policy Optimization (PPO) algorithm combined with a staged curriculum learning strategy was used to train the agent. Experimental results show that curriculum learning significantly improved the agent’s navigation, combat abilities, and training efficiency. Through parameter tuning and reward optimization, the agent achieved higher cumulative rewards and better training stability. The findings demonstrate the feasibility of applying deep reinforcement learning to FPS games and provide a foundation for future research in complex environment modeling and adaptive game AI.