Investigating learning and behavioral biases in OCD in a volatile and probabilistic environment: An EEG study.

In this study we will examine the electrophysiological markers of learning and decision-making in a stochastic and volatile environment, and their relation to symptoms of obsessive-compulsive disorder (OCD). The theoretical foundations of our research lay in the goal-directed model of behavior developed by Moors et al. (2017). The goal-directed cycle starts with the detection of a discrepancy or mismatch between the representation of a current or anticipated stimulus or state ([S]) and a goal (understood as the representation of a valued outcome; [Ov]). This discrepancy can be reduced via the broad strategies of acting (i.e., assimilation), replacing the goal by a different goal (i.e., accommodation), or biasing the interpretation of the stimulus (i.e., immunization). In the case of assimilation, people select the action option with the highest expected utility (i.e., the highest expectancy of reducing the discrepancy and reaching the valued outcome given the current state) from their action repertoire. This, in turn, activates the corresponding action tendency or intention, which can translate into overt behavior and produces an actual outcome. This outcome is fed back as the stimulus input to the next run of the cycle. The cycle is repeated until the discrepancy is resolved or until another more important goal takes over (for details, see Moors, 2022, 2025). Expectancies of behavior options in the repertoire are learned and updated according to reinforcement learning principles. When an action tendency is activated during the first run of the cycle, it not only leads to overt behavior and an actual outcome but also generates an expectation that the valued outcome will indeed occur. As a result, in the next run of the cycle, the initial goal becomes the representation of an outcome that is not only valued but also expected. When the actual outcome from the previous cycle is then fed back into the comparator of the new cycle, any discrepancy between expected and actual outcomes constitutes a reward prediction error. This error can be used to update the expectancy associated with the previously selected behavior in the action repertoire. Biases can arise in each of these stages. Examples are misestimations of discrepancies in the discrepancy detection stage, a lack of suitable action options in the agent’s action repertoire, and misestimations of action-outcome expectancies (or more broadly of costs and benefits) of action options in the action selection stage. An important factor that may contribute to the occurrence of misestimations is uncertainty. Two important properties of environments that breed uncertainty are stochasticity and volatility. Stochasticity concerns the probability that actions are followed by certain outcomes given certain stimuli (i.e., S:R-O contingencies). In deterministic environments, probabilities are 1 or zero (e.g., actions are always or never rewarded), whereas in probabilistic environments, they are smaller than 1 and larger than zero (e.g., actions are rewarded in 80% of cases). In deterministic environments, uncertainty is purely epistemic – stemming from a lack of information. In probabilistic environments, uncertainty has both epistemic and aleatoric components. The aleatoric component stems from the unpredictable outcome variability (i.e., randomness), also referred to as “risk” (Botelho et al., 2023), and is therefore irreducible. For instance, when an action yields a reward 80% of the time, and a punishment on 20% of the trials, agents can infer these probabilities, but they cannot predict the exact pattern of rewards (i.e., when a reward or punishment will be given). Volatility, on the other hand, concerns whether and how the S:R-O contingencies in the environment change over time. In stable environments, S:R-O contingencies remain constant, thereby not introducing any additional source of uncertainty, while in volatile environments, they change slightly (i.e., contingency degradation) or drastically (i.e., contingency reversal) at events called change-points. The uncertainty in volatile environments pertains to whether or not a change-point has occurred. This, in turn, generates uncertainty about the S:R-O contingencies that currently hold. The latter type of uncertainty is again often epistemic and therefore reducible. As stated earlier, agents learn about the correct S:R-O contingencies based on the actual outcomes of selected and executed actions, which are fed back to the comparator in the next run of the cycle. There, the agent evaluates whether the outcome or feedback signal is still discrepant with the desired – and now also expected – outcome (i.e., a reward prediction error - RPE). Thus, feedback signals not only inform the agent about goal progress, but also about whether the [S:R-O] expectancies held so far were correct. As such, they can help the agent update their represented S:R-O expectancies, and ultimately promote optimal action selection (i.e., action with the highest probability of producing a goal-congruent outcome). Crucially, however, in probabilistic and volatile environments, feedback signals (and thus RPEs) vary in their diagnostic value, that is, their ability to reduce epistemic uncertainty. Some outcomes are highly diagnostic or “valid” in the sense that they are in line with expectancies that promote optimal responding (i.e., reward/punishment following an optimal/non-optimal response). Other outcomes are low in diagnosticity or “noisy” in the sense that they deviate from expectancies that promote optimal responding (i.e., punishment/reward following an optimal/non-optimal response). In a deterministic context, punishments should typically decrease the probability of repeating the same action, while rewards should increase the probability of repeating. However, due to the context being probabilistic in nature, participants also had to consider whether feedback is valid or noisy in adjusting their action updates (either switching to an alternative option or staying with the same option). In such an environment, it is optimal to switch to an alternative action option after encountering valid errors and noisy rewards and to stay with the same action option after encountering noisy errors and valid rewards. A contingency reversal should harm the optimal switching patterns, because the validity of the feedback changes, requiring an adaptation. While the uncertainty and noisy outcomes in stochastic and volatile environments invite biases that can hinder optimal responding in the general population, we propose that some of these biases are more pronounced in people with OCD. Building on prior research, we hypothesize (H1) that people with OCD suffer from an increased signaling of the presence of a stimulus-goal discrepancy and a decreased signaling of its absence. In the context of reward-learning tasks, the former can also be called punishment hypersensitivity, while the latter can be called reward hyposensitivity. Punishment hypersensitivity is understood as an inflation of the discrepancy or incongruency signal following a punishment; reward hyposensitivity refers to a deflation of the congruency signal following a reward. Such aberrant signaling generates excessive performance-monitoring tendencies, effectively “telling” the agent that its current expectancies (i.e., mental model) and behavior need to be adapted. This leads to diminished ability to discriminate between valid from noisy feedback (i.e., to discern their information value). The disability is manifested as inflated estimates of the environmental volatility driven by excessive expectancy decreases after noisy errors and diminished expectancy increases after valid rewards. We will test these hypotheses in a sample from the general population divided into two groups: one that exhibits low levels of OCD symptoms and one that exhibits high levels of OCD symptoms. OCD symptoms will be measured via self-report. Learning and behavior will be assessed by employing a probabilistic contingency reversal task – an adaptation of the “Diamond Hunt” created by Buabang et al. (2023) – in which we independently manipulate the level of volatility (i.e., frequency of rule change) and stochasticity (i.e., the amount of noisy outcomes). This results in two dimensions of outcome feedback: valence (rewards vs. punishment) and informativeness (valid vs. noise). During the pre-reversal phase of the task, participants have to learn two probabilistic “stimulus: response-outcome” (S:R-O) contingencies across a series of (400) trials in order to maximize their monetary gain. Specifically, they have to learn for each of two stimuli (i.e., pairs of doors in two different colors; S1-S2) presented to them, which of two responses to select (i.e., opening either the right or left door; R1, R2) to increase their chance of receiving a valued outcome or reward (i.e., a diamond behind the door, signaling money gain; O1) rather than a disvalued outcome or punishment (i.e., a rock behind the door, signaling money loss; O2). For each stimulus (i.e., colored pair of doors), one response is optimal in that it has a high probability (i.e., 0.80) of resulting in the valued outcome, while the other response is suboptimal in that it has the inverse, low probability (i.e., 0.20). For instance, on trials where a red pair of doors appears (S1), choosing the right door (R1) is optimal, whereas on trials where a blue pair of doors appears (S2), choosing the left door (R2) is optimal. In each trial, the participants must make their choice within 3 seconds, or they are taken to the next trial without receiving either a rock or a diamond. After the pre-reversal phase, the contingencies are reversed so that the stimuli now have the opposite optimal response. At this point, participants enter the post-reversal phase. The reversal is done without notifying the participants. To perform optimally, they must adapt to the