This study investigates the impact of digital trust (DT) and perceived security (PS) on online banking usage behavior in Vietnam, incorporating perceived risk (PR) as a mediator and digital financial literacy (DFL) as a moderator. Drawing on an extended Technology Acceptance Model (TAM) integrated with Commitment–Trust Theory, Perceived Risk Theory, and Protection Motivation Theory, a research model with eight hypotheses was developed and empirically tested. Data were collected from 385 online banking users in Ho Chi Minh City and surrounding provinces via a structured questionnaire. Partial least squares structural equation modeling (PLS-SEM) using SmartPLS 4 was employed for data analysis. The measurement model demonstrated satisfactory reliability and validity, with all constructs exceeding established thresholds for Cronbach’s alpha (≥ 0.70), composite reliability (≥ 0.70), and average variance extracted (≥ 0.50). The structural model results revealed that both DT (β = 0.278, p < 0.001) and PS (β = 0.231, p < 0.001) significantly and positively influenced intention to use (IU), while both constructs significantly reduced PR. PR exhibited a significant negative effect on IU (β = −0.189, p < 0.001), and mediation analysis confirmed that PR partially mediated both the DT–IU and PS–IU relationships. Furthermore, DFL significantly moderated the PR–IU link (β = 0.118, p = 0.013), weakening the negative effect of PR on IU for users with higher DFL. IU strongly predicted actual usage behavior (β = 0.637, p < 0.001). The model explained 51.8% of the variance in IU and 40.6% in actual behavior. These findings contribute to the digital banking literature by providing an integrated trust–security–risk framework with conditional boundary effects, and offer practical implications for Vietnamese commercial banks seeking to enhance digital service adoption through trust-building strategies and financial literacy programs.