Jurnal ELTEK
Vol. 24 No. 2 (2026): October 2026

Using edge computing and Large Language Models (LLM) for constructing a local agent

Setya Widyawan Prakosa (Universitas Jember)
Naufal Ainur Rizal (Universitas Negeri Jember)
Tias Kurniati (Chung Yuan Christian University)



Article Info

Publish Date
30 Oct 2026

Abstract

The growing demand for artificial intelligence systems that are responsive, privacy-preserving, and capable of operating autonomously is driving the adoption of Large Language Models (LLMs) on edge devices. However, deploying LLMs as the core of agents on devices with limited resources still faces computational and memory constraints. This research develops an LLM-based local agent architecture that runs locally on a Raspberry Pi without relying on cloud services. The system integrates local LLM inference with agentic functions, including intent processing, reasoning, tool invocation, and interaction with hardware via GPIO. Several quantized Small Language Models (SLMs) were evaluated on a Raspberry Pi 5 with 16 GB of RAM using CPU-based inference. The evaluation included total latency, Time to First Token (TTFT), token generation rate, peak RAM usage, peak CPU utilization, and peak temperature. The results showed significant performance differences among the models. Qwen 2.5 0.5B Q3 produced the lowest total latency of 23.9 seconds, the lowest TTFT of 269.0 ms, and the highest generation rate of 12.1 tokens/s. However, the results indicate that inference speed alone is not sufficient to determine the most suitable model as the core of a local agent. Most models reached peak CPU utilization above 95%, while RAM usage and thermal characteristics varied across configurations. Based on a combination of inference performance and resource usage, models in the 1B–1.7B range specifically LLaMA 3.2 1B Q4, Qwen 2.5 1.5B Q4, and SmolLM 1.7B Q4 show potential as a balanced set of candidates between responsiveness and model capacity. The research results demonstrate the feasibility of deploying LLMs as local agents on low-cost edge devices and provide a basis for selecting models based on a balance between inference efficiency, resource constraints, and agentic capabilities.

Copyrights © 2026






Journal Info

Abbrev

eltek

Publisher

Subject

Computer Science & IT Control & Systems Engineering Electrical & Electronics Engineering Engineering

Description

Jurnal ELTEK merupakan media yang diterbitkan oleh Politeknik Negeri Malang sebagai sarana diseminasi dan publikasi artikel hasil penelitian dan artikel konseptual yang dilakukan oleh para peneliti, akademisi, praktisi dan industri di bidang Teknik Elektro. Artikel yang diajukan untuk diterbitkan ...