Comparing negation processing in humans and text-to-image models: Is the negated state of affairs represented with binary and non-binary negations?

This study investigates the negation effect in multimodal models. It focuses on how accurately Stable Diffusion XL processes affirmative and negative statements describing states that differ in prominence of alternatives. We define prominence as whether a noun–predicate combination evokes a particularly salient or conventional alternative when negated. Prominent alternatives are cases where negation strongly suggests a specific, highly salient alternative (e.g., “The traffic light is not red” → green). Non-prominent alternatives are cases where negation does not evoke a single dominant alternative, but rather a set of less salient possibilities (e.g., “The box is not wooden” → metal, plastic, cardboard, etc.). To select the items used in the experiment, we draw on the 18 items from Capuano et al. (2023) with the highest cloze probability as prominent items, and the 18 items with the lowest cloze probability as non-prominent items.