The management of extensive agent swarms presents significant challenges in dynamic, real-time environments, particularly within the context of game artificial intelligence, such as real-time strategy games. Traditional Particle Swarm Optimization (PSO) techniques demonstrate effectiveness in optimization tasks; however, they frequently exhibit suboptimal convergence and insufficient flexibility in complex and challenging scenarios. This study presents a hybrid methodology that combines Reinforcement Learning (RL) and Particle Swarm Optimization (PSO) to develop an adaptive swarm clustering system. This method utilizes a Deep Deterministic Policy Gradient (DDPG) agent operating externally through an API to dynamically adjust Particle Swarm Optimization (PSO) parameters, thereby maintaining a separation between adaptive intelligence and the simulation engine. This allows the swarm to effectively navigate and group within a procedurally generated 2D simulation environment with physical obstacles, unlike previous studies that rely on static mathematical benchmarks. A quantitative analysis employing Mixed Linear Model Regression (MLMR) indicates that this hybrid method significantly outperforms traditional, manually tuned PSO in terms of convergence time and diversity value. The RLGPSO model showed an 11.46% decrease in convergence time on highly complex maps. This result was statistically significant, with a p-value of 0.002 from the MLMR analysis. This research presents a framework for the deployment of intelligent, self-organizing agent swarms, enhancing the realism and efficacy of contemporary game artificial intelligence.