International Journal of Engineering and Emerging Technology
Vol 3 No 2 (2018): July - December

Middleware ETL with CDC based on Event Driven Programming

I Gede Adnyana (Unknown)
Made Sudarma (3Department of Electrical and Computer Engineering, Udayana University)
Wayan Gede Ariastina (Department of Electrical and Computer Engineering, Udayana University)



Article Info

Publish Date
15 Jan 2019

Abstract

To achieve the real time data warehouse is strongly influenced by the process in the data warehouse known as Extract, Tranform, Loading (ETL). One way to optimize the ETL process is processing only data that undergoes changes on the On Line Transaction Processing (OLTP) system. This technique is known as Change Data Capture (CDC) which is designed to maximize the efficiency of the ETL process. In this research middleware was built as a place where the ETL process will be carried out, transaction data from the OLTP system will be captured and sent directly to the middleware for further processing. The method used to capture changes in OLTP systems is Change Data Capture (CDC) based on Event Driven Programming, where this technique relies on events that occur in OLTP in capturing data changes. Functional testing is done by making a simulation of the insert and update processes in test applications namely OLTP CRM system.The results of the research obtained are (1) Change Data Capture (CDC) based on Event Driven Programming can capture changes in data that occur in OLTP CRM database;(2) ETL process to load data from Normalized Data Store (NDS ) to data warehouse with Timestamp technique can load data that only undergoes changes that are processed to be loaded in Data Warehouse;(3) An increase in the amount of data that is processed has an effect on increasing processing time. Other factors that affect the value of process time are execution plan and cache memory

Copyrights © 2018






Journal Info

Abbrev

ijeet

Publisher

Subject

Civil Engineering, Building, Construction & Architecture Computer Science & IT Electrical & Electronics Engineering Mechanical Engineering

Description

International Journal of Engineering and Emerging Technology is the biannual official publication of the Doctorate Program of Engineering Science, Faculty of Engineering, Udayana University. The journal is open to submission from scholars and experts in the wide areas of engineering, such as civil ...