Ades Tikaningsih, Imam Tahyudin, Berlilana

Two-Stage Framework Using IndoBERT for Sentiment Analysis of Tokopedia Reviews under Extreme Class Imbalance

Introduction

Two-stage framework using indobert for sentiment analysis of tokopedia reviews under extreme class imbalance. Tackle extreme class imbalance in Tokopedia review sentiment analysis using a two-stage IndoBERT framework. Achieves 16.2% F1-score improvement, boosting minority class sensitivity.

Abstract

The rapid growth of the Indonesian e-commerce industry has generated a large volume of customer reviews for sentiment analysis, but the data distribution often suffers from extreme class imbalance. The review dataset exhibits a 97.6% dominance of the positive class, causing the single-stage transformer model to produce high accuracy that does not fully represent classification capability. The baseline model achieves a macro-averaged F1-score of 0.599, with a neutral-class recall of 26.3%. Approaches based on loss function adjustment, such as class-balanced loss, focal loss, weighted cross-entropy, and decision-threshold adjustment, are unable to fundamentally address this issue, yielding only limited performance improvements. This study proposes a two-stage classification approach that decomposes the multi-class classification task into two sequential binary classification stages using a BERT-based Indonesian-language transformer model (IndoBERT). The first stage separates the positive class from the non-positive class, while the second stage distinguishes between the neutral and negative classes in a more balanced decision space. The proposed approach achieves a macro-averaged F1-score of 0.761, representing a 16.2% improvement over the baseline and outperforming all loss-function-based methods. These findings suggest that, under conditions of extreme class imbalance, simplifying the decision space through gradual task decomposition is more effective than intervention at the loss-function level. Furthermore, error propagation analysis and qualitative evaluations demonstrate that this approach improves sensitivity to minority classes, although challenges remain in cases involving ambiguous expressions.

Review

This study addresses a pervasive and challenging issue in real-world sentiment analysis: extreme class imbalance, specifically within Indonesian e-commerce (Tokopedia) customer reviews. The authors succinctly highlight how a stark dominance of positive reviews (97.6%) leads to misleading high accuracy in single-stage transformer models and renders traditional loss-function-based adjustments largely ineffective for improving minority class performance. To counteract this, the paper proposes an innovative two-stage classification framework utilizing IndoBERT, an Indonesian-language transformer model. This hierarchical approach sequentially decomposes the multi-class problem into two more manageable binary classification tasks, ultimately demonstrating a substantial 16.2% improvement in macro-averaged F1-score compared to baseline models. The core strength of this research lies in its effective methodological departure from conventional imbalance handling techniques. By first separating the dominant positive class from non-positive reviews, and then subsequently distinguishing between neutral and negative sentiments in a more balanced decision space, the proposed framework fundamentally alters the classification problem. This strategic decomposition not only significantly boosts performance metrics for minority classes but also provides a crucial insight: for extreme class imbalance, restructuring the decision-making process can be far more impactful than mere parameter adjustments at the loss-function level. The use of a domain-specific model like IndoBERT further bolsters the relevance and applicability of their findings within the Indonesian context. While the proposed two-stage framework undeniably represents a significant advancement in tackling extreme class imbalance, the study itself acknowledges that challenges persist, particularly concerning ambiguous expressions. Future research could explore advanced natural language understanding techniques or introduce a third stage specifically designed to re-evaluate or classify reviews deemed ambiguous by the initial stages. Furthermore, investigating the transferability of this two-stage decomposition strategy to other languages or different domains with varying imbalance patterns would offer valuable insights into its broader applicability. Nevertheless, this work offers a highly valuable contribution to the field of sentiment analysis, providing a robust and practically effective solution for a common and difficult data distribution problem.

Full Text

Comments

You need to be logged in to post a comment.