The development of a data lakehouse system for the integration and management of cyber threat intelligence data in XYZ unit - Chan, Ricky; Dhaifullah, Rendi Hanif; Saragih, Hondor; Lediwara, Nadiza; Adha, Rochedi Idul (Institute of Computer Science (IOCS), 2025)
Cybersecurity systems are evolving to deal with increasingly complex digital threats. One of the main challenges in this field is integrating and managing Cyber Threat Intelligence (CTI) efficiently. This research aims to design and implement Data Lakehouse as a solution to manage CTI data in XYZ Unit. The system was built using Apache Spark, MinIO, Dremio, Nessie, and Apache Iceberg with a containerization approach using Docker to ensure flexibility and ease of implementation. The implementation results show that the system successfully integrates various CTI data sources and improves efficiency in data storage, processing, and analysis. MinIO is used as the primary storage, Apache Spark processes data at scale, Dremio enables real-time data analysis, and Nessie manages data version control to maintain its integrity. Blackbox testing proves that the system can work optimally, with results showing improved data integration and efficiency in managing cyber threat information. Thus, the developed Data Lakehouse can be an effective solution in supporting threat detection and strategic decision-making in XYZ Unit.