Extracting Non-Linear Representations for Concept Control

Large language models have received much attention in recent years, but despite their ability to generate coherent responses for prompts regarding a wide range of disciplines and applications, they remain prone to generating biased, offensive or hallucinatory text. This may lead to widespread consequences such as the spread of misinformation or discrimination, users following harmful advice and reduced trust in artificial intelligence. Hence, safety is one of the most important open problems with generative models. Representation engineering is a recent approach that has shown promising results with alleviating this issue. Existing representation engineering methods extract vectors representing the direction of concepts such as truth or toxicity from a language model using as few as two prompts. By adding increasing multiples of a ‘truth’ vector to the activations, the truthfulness of a model can be continuously increased, providing a simple and effective means of controlling the behaviour of a model, without the need for external datasets or the precomputation associated with fine-tuning. However, it is unlikely that these simple, linear representations are able to approximate a complex model’s ground-truth representation of such high-level, vague and nuanced concepts to a high degree of accuracy. This may explain why the quality of responses degrades significantly as the behaviour of the model is increasingly controlled by a linear representation. This thesis investigates the potential of non-linear representations to form better local approximations of concepts through the proposal of three non-linear representation methods: Namely, TaylorRep, CurveRep and SurfaceRep. TaylorRep generalises existing linear methods by representing concepts with Taylor approximations. Whilst this representation controls activations directly in the model’s latent space, the two other methods represent concepts in an intermediate, low-dimensional space using a polynomial and radial basis function network, respectively. Two important components of these methods are an algorithm to collect model activations required to fit non-linear representations and a method to control the model by mapping from low-dimensional representations to high-dimensional activations. Additionally, this thesis is the first to adapt representation engineering methods to image generation model control. We bridge this gap through a comparison to the implementation of existing image generation control methods. In addition to qualitative analysis of text and images generated by models controlled by these representations, we evaluate the methods on the BBQ, RealToxicityPrompts and TruthfulQA benchmarks. Although linear methods maintain the advantages of speed and simplicity, TaylorRep outperforms existing methods on both reduction and control of toxicity. Furthermore, the projection-based CurveRep and SurfaceRep methods have the advantage that their low-dimensional representations are readily visualisable.