ANALISIS DAMPAK EMOJI PADA KLASIFIKASI EMAIL BERBASIS TF-IDF MENGGUNAKAN SUPPORT VECTOR MACHINE (SVM)

Abstract

Email spam remains a major challenge in digital communication, reducing the effectiveness of email services. This study analyzes the effect of emoji features on the performance of a Support Vector Machine (LinearSVC) using Term Frequency- Inverse Document Frequency (TF-IDF) for spam and ham email classification. The dataset combines the TREC 2007 and Enron-Spam corpora, totaling 81,730 emails. Since most emails do not contain emoji, synthetic emoji injection was applied under three conditions: baseline (no emoji), random emoji, and density- based emoji, each evaluated using unigram and bigram representations, resulting in six experimental scenarios. All experiments employed the same preprocessing pipeline, Stratified 5-Fold Cross-Validation, and class_weight='balanced'. All scenarios achieved F1-scores above 98%, with the best performance obtained by the Density + Unigram scenario (98.97% accuracy, 98.71% precision, 99.37% recall, and 99.04% F1-score). However, paired t-tests showed that emoji features did not produce statistically significant performance improvements, whereas n- gram configuration had a statistically significant effect. These findings indicate that n-gram selection has a greater impact on spam classification performance than the inclusion of emoji features.