Indonesian Journal of Electrical Engineering and Computer Science
Vol 27, No 2: August 2022

Dual embedding with input embedding and output embedding for better word representation

Yoonjoo Ahn (Sangmyung University)
Eugene Rhee (Sangmyung University)
Jihoon Lee (Sangmyung University)



Article Info

Publish Date
01 Aug 2022

Abstract

Recent studies in distributed vector representations for words have variety of ways to represent words. We propose a various ways using input embedding and output embedding to better represent words than single model. We compared the performance in terms of word analogy and word similarity with each input and output embeddings and various dual embeddings which are the combination of those two embeddings. Performance evaluation results show that the proposed dual embeddings outperform each single embedding, especially with the way of simply adding input and output embeddings. We figured out two things in this paper, i) not only input embedding but also output embedding has such meaning to represent the words and ii) combining input embedding and output embedding as dual embedding outperforms the single embedding when we use input embedding and output embedding individually.

Copyrights © 2022