Physical fitness was a necessity that requires someone to do physical activities. The purpose of this study ware to analyze the relationship between duration of work, physical and mental workload on physical fitness of construction worker PT. X Semarang. This study uses an observational analytic method with and cross sectional approach. The sample were 50 construction workers of PT. X Semarang. This sample was taken by simple random sampling technique. The variables of this study are age; duration of work; physical workload; mental workload. Including sports habit, smoking habits, and physical fitness. The data collection method uses a questionnaire and method Harvard step test 3-minute. Based on univariate analysis, the highest age of workers at a young age is (≤40 years) (66%). The work duration of poor worker (48%). Low physical workload (50%). Moderate mental workload (64%). Less exercise habits (49%). Smoking workers are classified as light smokers (52%), moderate physical fitness (44%). The test of statistical relation uses rank spearman. There was no relation between age (p value = 1,000), mental workload (p value = 0.753), exercise habits (p value = 0.184) with physical fitness. There is a relation between duration of work (p value = 0.014), physical workload (p value = 0.040), smoking habits (pvalue = 0.034) with physical fitness. PT. X Semarang was a company engaged in contstruction and workers who have moderate fitness. The workers need to exercise, 3 – 5 times a week to keep good physical fitness.