AI-mediated Informal Digital Learning of English (AI-IDLE) is increasingly relevant for ESL learners who need low-pressure opportunities to practise speaking beyond formal classrooms. This qualitative descriptive study explored how Pakistani adult ESL learners perceived the role of AI-based applications in managing speaking anxiety and strengthening speaking self-efficacy. The study involved ten undergraduate ESL learners at the University of Swabi, Khyber Pakhtunkhwa, Pakistan, who used ChatGPT, Copilot, and Grok for informal English practice. Data were collected through participant-generated artifacts, including 44 AI chat screenshots and 78 minutes of voice-chat recordings, as well as semi-structured interviews totaling 238 minutes and 27 seconds. The data were analyzed using systematic thematic analysis. The findings revealed that learners perceived AI-IDLE as a safe and supportive rehearsal space where they could practise speaking, make mistakes, receive corrective feedback, and prepare for classroom tasks without fear of ridicule or negative judgment. Repeated AI-guided practice through role-play, vocabulary rehearsal, pronunciation work, and pre-presentation preparation helped learners develop perceived communicative competence and stronger speaking self-efficacy. These gains were associated with reduced pre-task anxiety before presentations, vivas, tests, and academic interviews. However, AI rehearsal did not fully remove anxiety rooted in classroom social hierarchies, peer judgment, high-stakes assessment, and unequal speaking opportunities. The study suggests that AI can serve as a meaningful speaking partner in ESL learning when used as an informal rehearsal tool, supported by teacher feedback, ethical guidance, and learner-driven practice routines.