User Review Analysis: Psychological Impact of Duolingo Gamification Using Machine Learning

Gamification in Duolingo aims to enhance user motivation, yet its internal elements trigger diverse psychological responses. This study aims to analyze these responses through app reviews by comparing the effectiveness of four machine learning models, namely Naïve Bayes, Random Forest, Support Vector Machine (SVM), and Logistic Regression. The research design employs a quantitative comparative approach with review data classified into three labels, including Motivated, Frustrated, and Neutral. The methodology involves text preprocessing, label encoding, and addressing data imbalance using SMOTE and Class Weight techniques. The results indicate that all models achieved high accuracy, ranging from 92% to 93%. However, based on the Macro Average F1-Score metric, which measures the model’s performance balance across all classes including minority ones, the SVM and Logistic Regression models delivered the most optimal performance with a score of 0.44, outperforming Naïve Bayes and Random Forest, which scored 0.40. Feature analysis through Feature Importance reveals that the Penalty feature is the dominant factor triggering frustrated conditions, whereas the Daily Challenge feature serve