Instructional Mechanisms for Professional Writing: A Comparison of Scaffolded Annotation and ChatGPT

Professional writing skills are essential for crafting job application materials where applicants showcase their qualifications to recruiters and employers. Lettersmith is a digital tool that supports writing through scaffolded annotation, an instructional approach combining an expert-informed checklist, annotated examples, and self-tagging. We evaluated the efficacy of the instructional mechanisms that make up scaffolded annotation, as well as the use of ChatGPT, in facilitating writing cognitive processes and writing quality. Through a lab experiment with 146 first-year college students writing and revising a cover letter, we found that the combined mechanisms of scaffolded annotation within Lettersmith promoted a stronger understanding of the writing genre. Specifically, the use of a checklist combined with another writing support, like an example or self-tagging, was particularly effective for improving writing quality. Unstructured use of ChatGPT did not improve writing cognitive processes or writing quality more than Lettersmith.