NLP-Based Investigation of Textual and Semantic Cues in Fake News Identification

Abstract

This paper presents an empirical investigation of textual and semantic cues for fake news detection using FAKES-XL, a multi-domain, multi-language benchmark with leak-proof splits. Current reports often conflate gains with source/topic leakage and rarely assess probability calibration, limiting deployability across sources and languages. The present study trained text-only, semantic-only, and fused models on five bundles spanning English, Spanish, German, Hindi, and Italian, with temporal/source-grouped, topic-disjoint, cross-lingual zero-shot, and entity-disjoint evaluations. The methodology incorporated precommitted textual features (n-grams, stylometry, readability) and semantic signals (contextual embeddings, discourse, knowledge and retrieval-based evidence), applied post-hoc calibration, and quantified uncertainty via stratified bootstrap. Outcomes included Macro F1, Area Under the Receiver Operating Characteristic (AUROC), Area Under the Precision-Recall Curve (AUPRC), and Expected Calibration Error (ECE), with per-source and per-language scorecards and latency profiling under deployment constraints ($<=50 ~\text{ms}$on GPU;$<=120 ~\text{ms}$on CPU). While numeric results are not reported here, the analysis quantified the marginal value of each cue family, ablated discourse/knowledge/retrieval components, and produced calibrated thresholds tuned on validation and frozen on test. The contributions are a controlled comparison under strict leakage guards and a calibration-first evaluation that informs threshold selection. These findings support practical moderation workflows by offering reproducible scorecards and deployment-ready operating points.