Software testing is a crucial stage in the software development cycle, and it can be divided into two main categories: manual testing and automated software testing. Automated testing can be further divided into Uni, Integration, and UI testing. This paper analyzes the scenario testing process and evaluation results using the Katalon Studio automation testing tool. The data taken from the manual test is a test case from the user acceptance test (UAT) document, while the automatic test data uses the Katalon studio tool. Then, the results of manual testing and automated testing will be compared. As a result of testing, automatic testing of the Harmonie application was successfully carried out. The data shows that automated testing with Katalon tools is 31.90% faster than manual testing. This study highlights the benefits of using automated testing for software development.