Constructed Response Paragraph Jigsaw Puzzle Test for Measuring Structure Building Ability

ABSTRACT This study examined whether paragraph jigsaw puzzle (PJP) test can be used to measure structure building ability, the ability to construct cohesive mental representations. We developed a constructed‐response version of the PJP test and scored it with a string‐similarity algorithm that allows for partial credit. The present study explored whether this constructed‐response PJP test could serve as a valid and reliable tool for assessing structure building ability. Studies 1 and 2 examined validity, and Studies 3 and 4 examined reliability. Study 1 showed an acceptable correlation ( r = 0.65) between the scores from constructed‐response PJP and the scores from multi‐media comprehension battery. Study 2 used think‐aloud protocols and revealed that participants employed more structure‐focused strategies with the constructed‐response than with the multiple‐choice format. Study 3 demonstrated higher alternative‐form reliability for the constructed‐response format, and Study 4 replicated this finding. Taken together, our results suggest that the constructed‐response PJP test could serve as a new tool for measuring structure building ability.