During this PhD project, Amalie Pauli studied how to measure and evaluate style-related aspects of text, particularly in text generated by artificial intelligence. With the rapid development and increasing use of large language models, such as chatbots, it has become important to assess not only the information these systems generate, but also how it is expressed—for example, whether the text contains persuasive language. However, measuring such language use is inherently difficult, as style is subjective, nuanced, and dependent on context.

Amalie Pauli focused on measuring large language models’ capabilities to generate persuasive language and examined how different conditions affect both the type and degree of persuasive language produced. The research shows, for example, how generated messages can reflect gender-stereotypical language patterns in persuasive language. In addition, the project critically evaluated existing assessment methods for a style-related task, demonstrating that commonly used approaches do not always measure what they are intended to measure. The work highlights the need for evaluation methods that are reliable (consistent), valid (accurately capturing the intended concept), and efficient to apply at scale.

The findings contribute to a better understanding of how to evaluate nuanced and subjective aspects of AI-generated text. This is important for developing trustworthy AI systems and for identifying potential risks related to persuasion, bias, and misinformation in automated communication.

The PhD study was completed at the Department of Computer Science, Faculty of Natural Sciences, Aarhus University.

This summary was prepared by the PhD student.

Time: Friday, 29 May 2026 at 10.00
Place: Building 5342, room 333 , Department of Computer Science, Aarhus University, Åbogade 34, 8200 Aarhus N
Title of PhD thesis: Persuasive Language and Style in Text: Reliability, Validity, and Efficiency
Contact information: Amalie Brogaard Pauli, e-mail: ampa@cs.au.dk 
Members of the assessment committee
Professor Anne Lauscher, Department of Data Science, University of Hamburg Business School, Germany
Professor Steffen Eger, Department Computer Science and AI, University of Technology Nuremberg, Germany
Associate Professor Stefanie Zollmann, (chair),  Department of Computer Science, Aarhus University, Denmark.
Main supervisor: Professor Ira Assent, Department of Computer Science, Aarhus University
Co-supervisor: Professor Isabelle Augenstein, Department of Computer Science, University of Copenhagen, Denmark. 
Language: The PhD dissertation will be defended in English

The defence is public.
The PhD thesis is available for reading at the Graduate School of Natural Sciences/GSNS, Ny Munkegade 120, building 1521, 8000 Aarhus C