Improving quality of LLM driven annotation of political data

Large language models are increasingly used to annotate political text, as they can offer gains in scalability compared to traditional human annotators. However, use of these models has produced concerns about annotation quality, reproducibility of studies and the introduction of systematic biases into the complex and often ambiguous task of labelling political concepts. This thesis surveys methods for improving political data annotation, focusing on prompt design, annotation workflows, evaluation metrics, and their respective risks. The reviewed studies suggest that no single model or prompting strategy can ensure annotation quality, with scientifically effective use of LLMs requiring methodological transparency, with a focus on constraining the degrees of freedom available when working with Large Language Models to ensure reproducibility.