Bio-Swarm-Evolution Strategy Optimizer: A Hybrid Approach for Hyperparameter Optimization in Deep Learning

Abstract

We introduce the Bio-Swarm-Evolution Strategy Optimizer (BSES), a hybrid automated hyperparameter optimization method that fuses particle swarm optimization, genetic algorithm operators, and Hyperband’s successive halving to balance exploration, exploitation, and computational efficiency. Unlike grid or random search, BSES is designed for large, non-linear search spaces common in deep neural networks. Experiments on MNIST and CIFAR-10 show that BSES consistently outperforms Random Search, GA, and PSO, achieving higher validation accuracies and faster convergence with lower inter-run variability. Notably, BSES reaches 0.974 accuracy on MNIST and 0.827 on CIFAR-10, while baseline methods stagnate earlier. Additional CIFAR-10 trials confirm its robustness on more complex tasks. Analysis on the PathMNIST dataset further reveals optimal agent and iteration thresholds, highlighting BSES as an efficient and scalable optimizer for deep learning workflows.