With the existence of 5G technology, it is also possible for the evolution of new data transmission techniques. OFDM is the first multicarrier modulation scheme that has been used in digital communication systems. The UFMC system is one of the recommendations as a new multicarrier modulation scheme for 5G communication systems that applies filtering per sub-band instead of using cyclic prefixes. For this reason, this study will analyze the performance of OFDM and UFMC using 64-QAM and 256-QAM modulation to find out the best performance on a 5G communication system with general parameters according to the Key Performance Indicators (KPI) for the 5G physical layer (PHY) is BER to SNR with target 10-4. The results show that the communication system with OFDM is better than UFMC using 64-QAM modulation with an SNR value of 22.14 dB.