Identification of Inconsistent Reviews and Ratings on Apps Using Sentiment Analysis: Case Study on Indonesian Digital Media Platform

Ahmad Aulia Zakiyal Fikri; Hafidz Ridho

doi:10.25170/metris.v26i01.6779

Authors

Ahmad Aulia Zakiyal Fikri Department of Systems and Industrial Engineering, Faculty of Industrial Technology and Systems Engineering, Institut Teknologi Sepuluh Nopember
Hafidz Ridho Department of Systems and Industrial Engineering, Faculty of Industrial Technology and Systems Engineering, Institut Teknologi Sepuluh Nopember

DOI:

https://doi.org/10.25170/metris.v26i01.6779

Keywords:

Sentiment Analysis, digital media, Naive Bayes Classifier, word cloud

Abstract

Reviews and ratings on apps store show how users perceive an app. This is an important aspect that companies must pay attention to in order to plan improvement strategies. However, there is an inconsistency between reviews and ratings, which makes it difficult to take corrective action. Detikcom, one of the main players in the digital media industry in Indonesia, also faces this similar problem. On Google Play Store platform, it is known that Detikcom's 1-star rating (13%) is one of the highest compared to its competitors. However, the inconsistency between ratings and reviews can be found frequently in the review section. Reflecting on this case, this study focuses on building a model that can identify sentiment using the Naïve Bayes Classifier method and identifying the main driving factors of each sentiment category using K-means Clustering and word cloud. Based on the results of the developed model, the tendency of Detikcom user sentiment is generally positive (67.16%) with a test accuracy of 87.84%. The presence of positive sentiment is based on keywords such as “accurate”, “trusty”, and “up to date”, while negative sentiment is based on keyword “ads”.

References

Aljrees, T., Umer, M., Saidani, O., Almuqren, L., Ishaq, A., Alsubai, S., . . . Ashraf, I. (2024). Contradiction in text review and apps rating: prediction using textual features and transfer learning. Peer Computer Science, 10, e1722.

Annur, C. M. (2023, May 30). Media Sosial, Sumber Informasi Utama Masyarakat Indonesia. Retrieved from Databoks: https://databoks.katadata.co.id/datapublish/2023/05/30/media-sosial-sumber-informasi-utama-masyarakat-indonesia. Accessed March 29, 2024.

Berrar, D. (2018). Bayes’ Theorem and Naive Bayes Classifier. In Encyclopedia of Bioinfor-matics and Computational Biology (pp. 403-412). Elsevier.

Chandani, V., Satria, R., & Purwanto. (2015). Komparasi Algoritma Klasifikasi Machine Learning dan Feature Selection pada Analisis Sentimen Review Film. Journal of Intelligent System, 1(1), 56-60.

Eser, G., & Sahin, C. (2024). Sentiment Analysis and Rating Prediction for App Reviews Using Transformer-based Models. International Journal of Advanced Natural Sciences and Engineering Researches, 8, 372-379.

Handayani, F., & Pribadi, F. S. (2015). Implementasi Algoritma Naïve Bayes Classifier dalam Pengklasifikasian Teks Otomatis Pengaduan dan Pelaposan Masyarakat melalui Layanan Call Center 110. Jurnal Teknik Elektro, VII(1), 19-24.

Imam, F. R., Pramono, S. H., & Dahlan, E. A. (2012). Implementasi Opinion Mining (Analisis Sentimen) untuk Ekstraksi Data Opini Publik pada Perguruan Tinggi. Jurnal EECCIS (Electrics, Electronics, Communications, Controls, Informatics, Systems), 6(1), 37-43.

Ismaturrahmi, Khadafi, M., & Amirullah. (2025). Implementation of Web-Based Sentiment Analysis Application on Movie Reviews Using the Naive Bayes Algorithm. Journal of Informatics Engineering and Software Applications, 1(1), 129-138.

Khan, M., Khan, M. Q., & Malik, F. (2024). Optimized Sentiment Classification of Google Play Store App Ratings Using Advanced Machine Learning Models. VFAST Transactions on Software Engineering, 12(4), 252-266.

Kominfo Indonesia, K. (2008). Undang-undang (UU) Nomor 14 Tahun 2008 tentang Keterbukaan Informasi Publik. Jakarta: Kementerian Kominfo.

Muzaki, A., Febriana, V., & Cholifah, W. N. (2024). Analissi Sentimen pada Ulasan Produk di E-Commerce dengan Metode Naive Bayes. Jurnal Riset dan Aplikasi Mahasiswa Informatika (JRAMI), 5(4), 758-765.

Newman, N. (2023). Reuters Institute Digital News Report 2023. London: Reuters Institute for the Study of Journalism.

Pahlevi, R. (2022, June 16). Ini Media Online Paling Banyak Dikonsumsi Warga Indonesia. Retrieved from Databoks: https://databoks.katadata.co.id/teknologi-telekomunikasi/statistik/ea656802a0af1bf/ini-media-online-paling-banyak-dikonsumsi-warga-indonesia Accessed March 29, 2024.

Razaq, M. T., Nurhanah, D., & Nurrahmi, H. (2023). Analisis Sentimen Review Film Menggunakan Naive Bayes Classifier dengan Fitur TF-IDF. e-Proceeding of Engineering, 10(2), 1698-1712.

Sfenrianto, Purnamasari, I., & Bahaweres, R. B. (2016). Naïve Bayes Classifier Algorithm and Particle Swarm Optimization for Classification of Cross Selling (Case Study: PT TELKOM Jakarta). 2016 4th International Conference on Cyber and IT Service Management (pp. 1-4). Bandung: IEEE.

Tamara, D., Kisata, P., & Rosenanda. (2020). Consumer Behavior Towards DIgital and Printed Media in Indonesia. Psychology and Education, LVII(9), 241-245.

Tan, P.-N., Steinbach, M., & Kumar, V. (2005). Introduction to Data Mining. Pearson.