Stylistic analysis of translated languages: A perturbation-based XAI deep learning framework

Abstract

Text classification using traditional machine learning techniques has been used in natural language processing (NLP) tasks to distinguish translated from non-translated languages, with high accuracy scores indicating the distinctive style of translated languages. While deep learning (DL) has demonstrated impressive performance in terms of representation learning and capturing nuanced patterns in natural language data, DL models act as black boxes, making their results difficult to interpret. This study addresses this issue by demonstrating an explainable AI (XAI) DL framework in a case study of United Nations (UN) meetings. The framework consists of three stages: i) train a variational autoencoder (VAE) combined with BERT embeddings converted from translated and non-translated texts; ii) utilize the majority vote from three classifiers selected from a stacked ensemble to classify the VAE’s latent representations; iii) implement a perturbation-based XAI method to interpret the DL model’s decisions. The results indicate that the VAE-based model effectively distinguishes the two text types, with accuracy scores above 0.8. The XAI analysis reveals that interpreting the VAE-based model’s decision uncovers stylistic differences between the two text types beyond superficial lexical and syntactic features. This proof-of-concept study demonstrates the potential of the XAI DL framework in other NLP studies that aim to analyze style.