Automated news monitoring and sentiment analysis system using web scraping and large language models - Zerusealtin David Naibaho; Hondor Saragih (Jurnal Mandiri IT, 2026)
Organizations increasingly require efficient systems to monitor and analyze vast online news data for timely and informed decision making. Manual monitoring is inadequate due to information overload and the time sensitive nature of digital content. This study presents the design, development, and evaluation of an automated web based news monitoring and sentiment analysis system integrating web scraping and artificial intelligence. The system was implemented using the Django web framework with a PostgreSQL database, Playwright browser automation for dynamic content extraction, and Google’s Gemini API for contextual sentiment classification. Three main functions were developed: automated data collection based on keywords and date ranges, AI driven sentiment analysis producing positive, negative, or neutral labels with contextual understanding, and automated reporting with interactive visualizations exportable to XLSX and CSV formats. Functional black box testing confirmed 100% success across 28 test cases, verifying reliability in authentication, data acquisition, sentiment analysis, and visualization. Performance evaluation showed that the system could collect 50–200 articles within 2–4 minutes and process sentiment analysis at 1–2 seconds per article. The proposed system effectively transforms manual workflows into fully automated operations, enabling systematic media monitoring, sentiment tracking, and data driven decision support.