Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal JuTISI (Jurnal Teknik Informatika dan Sistem Informasi)
Ni’ma Kholila
Unknown Affiliation
Computer Science & IT

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Perancangan System Crawler dengan MenerapkanArsitektur Distributed Task Heri Santoso; Indyah Hartami Santi; Ni’ma Kholila
Jurnal Teknik Informatika dan Sistem Informasi Vol 8 No 1 (2022): JuTISI
Publisher : Maranatha University Press

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.28932/jutisi.v8i1.4205

Abstract

PDC Media Group is a company engaged in online trading. The need for data insight in the online marketplace is very important. Likewise, how to get quite a lot of data, of course, requires automation such as crawling data on the marketplace website. Due to the large amount of data, crawler systems are often not optimal in crawling data. The application of distributed tasks on the crawler system provides convenience in scaling the server both vertically and horizontally. Therefore, the large and growing data can be handled by the crawler system. The application was developed with the Python language, with the  application server using Google Cloud Computing. In a distributed task architecture requires a component in the form of a message broker. The message broker used in designing this system is RabbitMQ. Testing the crawler system uses 3 scenarios, namely with 1 worker, 2worker, and 3 worker. The results for the 1 worker scenario are 19.3 requests per second and 332 ms for response time. The results for the 2 worker scenario are 41.4 requests per second and 328 ms for response time. While the results for the 3 worker scenario are 60 requests per second and 331 ms for response time.
Co-Authors Heri Santoso Indyah Hartami Santi