Evaluation and Comparison of the Use of Reinforcement Learning Algorithms on SSH Honeypot

Marco Ariano Kristyanto (Informatics Department, Universitas Surabaya, Surabaya, East Java)
Maya Hilda Lestari Louk (Informatics Department, Universitas Surabaya, Surabaya, East Java)

Teknika
Vol 13 No 1 (2024): Maret 2024

A honeypot is a tool or system used to record, redirect, and even lure hackers into penetrating and exploiting a system. The increasing development of technology causes cyber hackers to realize the existence of honeypots using various other software and tools. So, honeypots need a way to learn how hackers behave. The idea proposed is to combine honeypots with reinforcement learning algorithms so that honeypots become adaptive honeypots. This study suggests the concept by comparing the two Q learning-based RL algorithms, namely DQN and DDQN, to reach which algorithm is more optimal. The study results showed that the DDQN algorithm is more optimal in determining actions when compared to the DQN algorithm because using a double Q-value can help determine the action more accurately. Based on the result, the DDQN algorithm consumed less memory than the DQN Honeypot. The learning rate curve and the processing of DDQN algorithm commands can be used as an alternative algorithm that can be combined with honeypots because of the learning rate, which can make honeypots faster in the dynamic environment.