Multimodal deep learning framework for detection and attribution of adversarial information operations on social media platforms - Nick Holson M. Silalahi; Jonson Manurung; Bagus Hendra Saputra (Journal of Defense Technology and Engineering, 2026)
Adversarial information operations on social media platforms pose critical threats to national security, with state-sponsored actors exploiting multimodal content manipulation to conduct sophisticated disinformation campaigns. Existing detection approaches focus on single-modality analysis, lacking comprehensive frameworks for simultaneous detection, attribution, and coordination identification. This research develops an integrated multimodal deep learning framework combining RoBERTa-large transformer, Vision Transformer, Graph Convolutional Networks, and bidirectional LSTM, unified through cross-modal attention fusion with multi-task learning optimization. Experimental validation utilizes eight datasets including Russian IRA tweets (3.8M posts), Fakeddit (1M submissions), TweepFake (25K accounts), FakeNewsNet (23K articles), MM-COVID (6.7K posts), CREDBANK (60M tweets), and MEMES (12K items). Results demonstrate 93.24% detection accuracy, 79.34% attribution accuracy across 15 threat actor groups, 91.67% coordination F1-score, 88.62% narrative classification accuracy, and 448ms inference latency suitable for real-time deployment. Ablation studies reveal graph neural networks provide largest performance contribution (5.82% improvement), highlighting social network analysis importance for detecting coordinated behavior. Future directions include large-scale pre-training, adversarial training, continual learning, human-AI collaboration, multilingual expansion, federated learning, and causal inference methods.