Deploying large language models for discourse studies: An exploration of automated analysis of media attitudes

Abstract

This study aims to provide an LLM (Large Language Model)-based method for the discourse analysis of media attitudes, and thereby investigate media attitudes towards China in a Hong Kong-based newspaper. Analysis of attitudes in large amounts of media data is crucial for understanding public opinions, market trends, social dynamics, etc. However, corpus-based approaches have traditionally focused on explicit linguistic expressions of attitudes, leaving implicit expressions unconsidered. To address this gap, the present study explored the possibility of using LLMs for the automated identification and classification of both explicit and implicit attitudes and evaluated the feasibility of implementing this approach on personal computers. The analysis was based on the framework proposed by Martin and White, which provides a structured approach for describing different aspects of media attitudes [1]. Meta’s open-source Llama2 (13b) was used for automated attitude analysis and was quantised for deployment on personal computers. The quantised LLM was used to analyse 40,000 expressions about China in a corpus of news reports from Oriental Daily News, a top-selling newspaper in Hong Kong. The results demonstrated that the quantised LLM can accurately capture both explicit and implicit attitudes, with a success rate of approximately 80%, comparable to that of proficient human coders. Challenges encountered during the implementation process and potential coping strategies were also discussed.