Multimodal deep learning framework for detection and attribution of adversarial information operations on social media platforms

Nick Holson M. Silalahi (Universitas Pertahanan Republik Indonesia, Bogor, Indonesia)
Jonson Manurung (Universitas Pertahanan Republik Indonesia, Bogor, Indonesia)
Bagus Hendra Saputra (Universitas Pertahanan Republik Indonesia, Bogor, Indonesia)

Journal of Defense Technology and Engineering
Vol. 1 No. 2 (2026): January, Journal of Defense Technology and Engineering

Adversarial information operations on social media platforms pose critical threats to national security, with state-sponsored actors exploiting multimodal content manipulation to conduct sophisticated disinformation campaigns. Existing detection approaches focus on single-modality analysis, lacking comprehensive frameworks for simultaneous detection, attribution, and coordination identification. This research develops an integrated multimodal deep learning framework combining RoBERTa-large transformer, Vision Transformer, Graph Convolutional Networks, and bidirectional LSTM, unified through cross-modal attention fusion with multi-task learning optimization. Experimental validation utilizes eight datasets including Russian IRA tweets (3.8M posts), Fakeddit (1M submissions), TweepFake (25K accounts), FakeNewsNet (23K articles), MM-COVID (6.7K posts), CREDBANK (60M tweets), and MEMES (12K items). Results demonstrate 93.24% detection accuracy, 79.34% attribution accuracy across 15 threat actor groups, 91.67% coordination F1-score, 88.62% narrative classification accuracy, and 448ms inference latency suitable for real-time deployment. Ablation studies reveal graph neural networks provide largest performance contribution (5.82% improvement), highlighting social network analysis importance for detecting coordinated behavior. Future directions include large-scale pre-training, adversarial training, continual learning, human-AI collaboration, multilingual expansion, federated learning, and causal inference methods.