MarxistLLM: Fine-tuning a language model with a Marxist worldview

Recently, there has been increasing interest in using language models to examine the society. As large language models embody societal values and perspectives, practitioners and scholars might not account for the biases that these models cause in use. Instead of seeing biases as a source of error with big data, we could think of them as theoretical perspectives in the sense of a worldview ( Weltanschauung ), such as rational choice theory, Marxist theory, and feminist theory. To highlight potential biases in large language models and their implications for society—and explore the opportunity to use language models for social science theory—we fine-tune large language models with a specific Weltanschauung . Specifically, we incorporate the writings of Marx and Engels to fine-tune these models, aiming to infuse them with a stronger Marxist vocabulary and worldview. We show that these fine-tuned models depict a society where affluence loses its importance, but at the same time the models are more sensitive to capitalism and economy than a standard baseline model. This investigation underscores the need to examine the values embedded in large language models before they are used to analyze society, and highlights the opportunity to incorporate a particular theoretic stance into them.