A Context-Aware Representation-Learning-Based Model for Detecting Human-Written and AI-Generated Cryptocurrency Tweets Across Large Language Models

The extensive use of large language models (LLMs), particularly in the finance sector, raises concerns about the authenticity and reliability of generated text. Developing a robust method for distinguishing between human-written and AI-generated financial content is therefore essential. This study addressed this challenge by constructing a dataset based on financial tweets, where original financial tweet texts were regenerated using six LLMs, resulting in seven distinct classes: human-authored text, LLaMA3.2, Phi3.5, Gemma2, Qwen2.5, Mistral, and LLaVA. A context-aware representation-learning-based model, namely DeBERTa, was extensively fine-tuned for this task. Its performance was compared to that of other transformer variants (DistilBERT, BERT Base Uncased, ELECTRA, and ALBERT Base V1) as well as traditional machine learning models (logistic regression, naive Bayes, random forest, decision trees, XGBoost, AdaBoost, and voting (AdaBoost, GradientBoosting, XGBoost)) using Word2Vec embeddings. The proposed DeBERTa-based model achieved an impressive test accuracy, precision, recall, and F1-score, all reaching 94%. In contrast, competing transformer models achieved test accuracies ranging from 0.78 to 0.80, while traditional machine learning models yielded a significantly lower performance (0.39–0.80). These results highlight the effectiveness of context-aware representation learning in distinguishing between human-written and AI-generated financial text, with significant implications for text authentication, authorship verification, and financial information security.