Application of A/B Testing Experimentation on Government Digital Products to Enhance Teachers' Skills and Capabilities in Indonesia - Widiarso, Bagoes Rahmat; Muthohar, Bhaskoro Abdillah; Tombe, Septi Rito; Novianti, Putri Wikie (Universitas Negeri Surabaya, 2023)
The Ministry of Education, Culture and Research and Technology (MoECRT) in Indonesia has developed a digital product named “Microlearning” within the Merdeka Mengajar digital Platform. “Microlearning” enables teachers to enhance their skills and capabilities by doing self-determined learning that is followed-up by the submission of Aksi-Nyata reports. We improved users’ journey flow and provided an alternative of user-interface designs in the platform, in which we hypothesized that these improvements could boost the number of teachers participating in the Aksi-Nyata report submissions - the main expected outcome of the “Microlearning” digital product. We conducted a randomized-control-trials experimentation in the digital platform to scientifically quantify its effect on the product, by including 1.6 million users who logged-in to the platform during a two weeks period. This experimentation (known as an A/B Testing) allowed us to simultaneously compare two groups within a specific period and within a controlled-environment. The bayesian-based experimentation revealed that the new developments increased the number of users who started to learn a topic, as well as increased the Aksi-Nyata report submission in the “Microlearning” product, as compared to the legacy-designs. A follow-up study showed even more convincing results, which shows the number of submitted Aksi-Nyata reports increased by 590% up to two months after implementing the analytics-based initiatives for all logged-in users in the platform.