Abstract

Toxic comments on social media discourage user engagement and have serious consequences for mental health and social well-being. Such negativity heightens feelings of anxiety, depression, and social isolation among users, ultimately diminishing their experience on these platforms. For businesses, these toxic interactions are detrimental as they lead to reduced user engagement, subsequently affecting advertising revenue and market share.
Creating a safe and inclusive online environment is essential for business success and social responsibility. This requires real-time detection of toxic behavior through automated methods. However, many existing toxicity detectors focus mainly on accuracy, often neglecting important factors including throughput, computational costs, and the impact of false positives and negatives on user engagement. Additionally, these methods are evaluated in controlled experimental settings (offline tests), which do not reflect the complexities of large-scale social media environments. This limitation hinders their practical applicability in real-world scenarios.
This thesis addresses these limitations by introducing a Profit-driven Simulation (PDS) framework for evaluating the real-time performance of deep learning classifiers in complex social media settings. The PDS framework integrates performance, computational efficiency, and user engagement, revealing that optimal classifier selection depends on the toxicity level of the environment. High-throughput classifiers are most effective in low- and high-toxicity scenarios, while classifiers offering moderate accuracy and throughput excel in medium-toxicity contexts.
Additionally, the thesis tackles the challenge of imbalanced datasets by introducing a novel method for augmenting toxic text data. By applying Reinforcement Learning with Human Feedback (RLHF) and Proximal Policy Optimization (PPO), this method fine-tunes Large Language Models (LLMs) to generate diverse, semantically consistent toxic data. This approach enhances classifier robustness, particularly in detecting minority class instances.
The thesis also proposes a Proximal Policy Optimization-based Cascaded Inference System (PPO-CIS), which dynamically assigns classifiers based on performance and computational costs. This system improves efficiency by using high-throughput classifiers for initial filtering and more accurate classifiers for final decisions, reducing the workload on human moderators.
Extensive evaluations on datasets such as Kaggle-Jigsaw and ToxiGen demonstrate significant improvements in processing time, detection accuracy, and overall user satisfaction, contributing to the development of scalable, cost-effective toxicity detection systems for social media platforms.