Bulletin of Electrical Engineering and Informatics
Vol 15, No 5: October 2026

Optimizing and evaluating proximal policy optimization and soft actor-critic agents in Unity ML-Agents-based games

Muhammad Fakhri Loebis (Bina Nusantara University)
Andry Chowanda (Bina Nusantara University)



Article Info

Publish Date
01 Oct 2026

Abstract

This study presents a comparative analysis of proximal policy optimization (PPO) and soft actor-critic (SAC) for training autonomous delivery agents in high-fidelity 3D environments using Unity ML-Agents. Both algorithms were evaluated with identical hyperparameters and reward functions across five independent runs to ensure statistical robustness. Results show that PPO achieves 31.4% higher final reward (6658.82 versus 5067.27) and superior policy improvement consistency (ratio 2.00 versus 0.31). However, PPO exhibits greater reward variability (coefficient of variation (CV) 0.533 versus 0.107) and slower convergence (33 versus 6 steps). SAC demonstrates faster initial convergence and superior stability with lower performance variance. These findings indicate that PPO is preferable for applications prioritizing maximum final performance in stable environments, while SAC is more suitable for tasks requiring adaptability and consistent performance under dynamic conditions. This study provides practical guidance for researchers implementing reinforcement learning (RL) in Unity-based simulations for autonomous systems.

Copyrights © 2026






Journal Info

Abbrev

EEI

Publisher

Subject

Electrical & Electronics Engineering

Description

Bulletin of Electrical Engineering and Informatics (Buletin Teknik Elektro dan Informatika) ISSN: 2089-3191, e-ISSN: 2302-9285 is open to submission from scholars and experts in the wide areas of electrical, electronics, instrumentation, control, telecommunication and computer engineering from the ...