Belief Updating Questionnaire Development

In everyday life, people continually revise their beliefs to keep up with a changing world, whether that means updating their opinion of a restaurant after tasting a new dish or reconsidering their views on complex social and political issues. When beliefs are updated accurately, they help guide effective behavior; for instance, believing a restaurant is good makes someone more likely to eat there. But when this updating process breaks down, it can lead to problems ranging from confirmation bias to delusions, where beliefs become increasingly disconnected from reality. Given this topic's utmost importance, well-controlled experimental tasks have been used to study belief updating behavior. However, these tasks trade off real-world applicability for repeated measurement and experimental control. It is currently unknown to what extent individual differences measured in such laboratory-style tasks relate to individual differences in real-world behavior, or whether laboratory or real-world belief updating is more closely related to mental health profiles. Here we preregister a set of studies aiming to answer these questions in three stages. Stage 1: We will develop and validate the Belief Updating Questionnaire (BUQ). The BUQ aims to measure stable, trait-like dispositions that capture how people update their beliefs in the real world through items pertaining to different constructs that are linked to varied patterns and nuances in real-world belief updating. We will focus on 3-4 potential subconstructs, tentatively named active information seeking (“I entertain alternative explanations when changing my mind.”), openness to belief revision (“My longstanding beliefs are open to revision.”), positive vs. negative bias (e.g., “I'm more likely to change my opinion when I hear bad news than when I hear good news.”), and propensity to treat unexpected events as reflective of changes in the current model (e.g., “When someone I know changes how they act around me, I expect it will not last.”). Previous pilot datasets have been in line with either the 4 constructs above or 3 constructs, by subsuming active information seeking and openness to belief revision under one broader construct of epistemic engagement. We will thus run exploratory factor analysis (EFA) to evaluate the assumed structure and calculate Cronbach's alpha for the full questionnaire as well as individual subconstructs to evaluate internal consistency. For an additional measure of internal consistency beyond Cronbach's alpha, we will also calculate split-half reliability. This involves splitting the items within each construct into two halves through a random split and calculating the correlation between the total scores of those two halves. These analyses will serve for further refinement of the BUQ items used in subsequent stages. Stage 2: We will use an independent sample to conduct a Confirmatory Factor Analysis (CFA) and assess convergent and divergent validity. The CFA will formally test the goodness-of-fit of the hypothesized three- or four- factor structure that was either confirmed or refined by the initial EFA. Key model fit indices (see below) will be evaluated to confirm that the observed data aligns with the factor structure obtained from our EFA, providing evidence for the construct validity of the BUQ. To assess the convergent validity of the BUQ, we will use three metrics: 1) a newly developed situation-based version of the BUQ, 2) established questionnaires on related constructs, and 3) established laboratory-based belief-updating paradigms. The situation-based belief-updating assessment will tap into two key constructs of the BUQ by presenting participants with hypothetical real-world situations and tracking their responses. This assessment of real-world updating measures the following constructs of a) propensity to treat unexpected events as reflective of changes in the current model (e.g. “You have been following user reviews for a new restaurant you are curious about. The average rating of the last five days across 1000 customers were: 2.5, 3.0, 2.8, 3.3, and 4.5. - What will the next average rating be?”) and b) positive vs. negative bias in belief updating, by including analogous hypotheticals of opposite valence (“You are looking at reviews for a hotel you are considering booking. The average ratings the last five days across 500 guests were: 3.6, 3.8, 3.7, 3.9, and 1.8 - What will the next average rating be?”). Other existing convergent validity scales will include the Davos Assessment of Cognitive Biases Scale (DACOBS), Causal Uncertainty Scale (CUS), Intolerance of Uncertainty Scale (IUS), Dogmatism Scale (DOG), and Peters Delusion Inventory (PDI). Laboratory-style tasks will consist of the predictive inference task (Nassar et al., 2016) and a multi-armed bandit task. Briefly, in the predictive inference task, participants predict the next outcome by positioning a bucket at the bottom of the screen, with the cover story that they are catching money dropped from a helicopter hidden behind clouds. The task requires estimation of spatial outcomes under uncertainty, capturing general belief-updating mechanisms. The particular task administered will include two conditions, one in which outlying data points are due to changes in the helicopter location (changepoints) and another where outlying data points do not persist in time (oddballs). Importantly, in this version of the task participants will not be instructed about the two conditions prior to performing the task. In the multi-armed bandits task, participants' will be required to learn from positive and negative feedback in order to make successful choices. Divergent validity measures will consist of self-report measures of unrelated constructs, such as the Patient Health Questionnaire - 9 (PHQ9), Marlowe-Crowne Social Desirability Scale (MC-SDS), the Apathy-Motivation Index (AMI), as well as well-established behavioral paradigms such as visual working memory task and digit-symbol matching task to evaluate the divergent validity of the BUQ. Stage 3: We will evaluate the temporal stability of the BUQ by conducting a test-retest reliability check. This will be performed by readministering the final and possibly refined version of the BUQ to a subset of the original participant pool after two-four weeks. The correlation between the initial and follow-up total scores and sub scores will be calculated to assess the consistency and stability of the measured trait-like disposition over time.