Evaluating Statistical Power in t-Test and Welch’s Test Using Monte Carlo Simulation Approach - Tengku Irfan Wira Buana; Arisman Adnan (Jurusan Matematika FMIPA UNM, 2025)
Statistical hypothesis testing is a key method in inferential statistics for assessing whether group differences are simply due to chance or amount to actual effect. One of the central concepts in hypothesis testing is statistical power. Statistical power is the probability of correctly rejecting the null hypothesis when the alternative hypothesis is true. Low statistical power increases the risk of Type II errors, leading to misleading conclusions. This study explores the key factors influencing statistical power, including sample size, effect size, variance, and significance level. Monte Carlo simulation method was utilized in this study to examine the statistical power associated with the two-sample t-test across various combinations of sample size, effect size (mean difference), and population variance. Simulations were conducted by generating random samples, performing variance tests, and applying either the Student’s t-test or Welch’s t-test based on variance equality. The results confirmed that statistical power increases with larger sample sizes and greater effect sizes, while higher variance and stricter significance levels reduce power. Welch’s t-test was found to be more reliable than the standard t-test in cases of unequal variances, reinforcing its importance in real-world data analysis. These findings show the importance of careful study design in hypothesis testing. Researchers must consider and plan the study so that there is enough power to detect meaningful effects. Future studies should examine different statistical methods of power, and potentially extend the simulation to different non-normal distributions for hypothesis testing.