Sentiment and Toxicity Score Evaluation of DJI Avata Product Reviews Using Cross-Industry Standard Process for Data Mining

Singgalen, Yerik Afrianto

KLIK: Kajian Ilmiah Informatika dan Komputer
Vol. 4 No. 6 (2024): Juni 2024

This research employs the CRISP-DM framework to analyze consumer sentiment and preferences regarding DJI Avata drone products, aiming to provide data-driven strategic recommendations for marketing and product development. By systematically exploring business objectives, preparing and cleaning data, and modeling sentiment, the study reveals high consumer engagement and predominantly positive sentiment (51.91% positive, 31.16% neutral, 16.93% negative) towards the DJI Avata. The Support Vector Machine (SVM) algorithm demonstrated superior performance in sentiment classification, achieving an accuracy of 74.69%, with an AUC of 0.839, precision of 77.57%, recall of 69.68%, and F-measure of 73.23%. A comparative analysis between the VADER and TextBlob models, showing a moderate agreement (Cohen’s kappa statistic = 0.413) on 64.84% of the posts, highlighted the value of using multiple sentiment analysis tools. Furthermore, toxicity scores calculated via the Perspective API identified critical areas for improvement in user engagement. Subsequently, the toxicity results reveal the following scores: Toxicity with an average of 0.09461 and a peak of 0.90451, Severe Toxicity with an average of 0.00817 and a peak of 0.45895, Identity Attack with an average of 0.01139 and a peak of 0.58743, Insult with an average of 0.04543 and a peak of 0.70658, Profanity with an average of 0.06133 and a peak of 0.89080, and Threat with an average of 0.02063 and a peak of 0.69437. These detailed metrics provide a comprehensive understanding of the dataset's different dimensions and intensities of negative sentiments. The significant variation between average and peak values indicates the presence of highly negative interactions, which necessitates targeted intervention. Consequently, these findings inform the development of specific strategies to mitigate toxicity and enhance the overall user experience in digital communities. These insights informed strategic recommendations to enhance digital marketing efforts and product features, underscoring the CRISP-DM framework's efficacy in guiding comprehensive consumer sentiment analysis and fostering informed decision-making in the aerial photography and videography market.