The agricultural sector as a buffer for food supply also contributes the largest workforce in Indonesia. However, low income and high poverty in farming households have raised questions regarding the condition of food security in areas where most of majority of the population are farmers. To examine this, the research uses the variable percentage of farmers per total worker from BPS and the Food Security Index (IKP) score from the National Food Agency in 416 districts in Indonesia from 2020 to 2022. Through Generalized Least Square (GLS) estimation, a significant negative correlation was found between the percentage of farmers and the IKP score. Every percentage increase in the number of farmers by 1% will be accompanied by a decrease in the IKP score of 0.107 points under ceteris paribus conditions. In short, the research results show that districts with a greater percentage of farmers and poor people from the agricultural sector, higher wages for agricultural labor and less rice production, will have lower level of food security. On the other hand, districts with high GRDP, large populations, higher rice consumption costs, lower meat consumption costs, and many poor people targeted by the Non-Cash Food Assistance program will have a higher level of food security. In relation to this matter, poverty reduction measures in districts where most of the workforce comes from the agricultural sector need to be accelerated to increase food security.