Understanding and supporting AI alignment in graphic design

Graphic designs, compositions of visual elements created to convey a message, are ubiquitous in our world. They encompass a range of artifacts, including printed and digital media, brand styles, and graphical user interfaces for both professional and personal use. As a visual form of communication, graphic design is a widespread practice, and both professional designers and novices increasingly utilize artificial intelligence (AI) to support it. However, AI alignment remains challenging in graphic design because user intent is often unclear at the outset. Misaligned AI can lead to fixation on suboptimal ideas, reduce critical evaluation, and hinder the development of creative intent. In addition, design decisions in graphic design, such as color, typography, imagery, and the message, are interdependent. The challenge lies in understanding the structures in which these elements relate in practice, how this affects the evolution of user intent, and how to ensure AI realigns with that intent throughout the process. This thesis contributes to the understanding and support of AI alignment needs in graphic design. By analyzing empirical data on the design intent of novices and on professionals’ practices related to a representative and important design choice, color, the thesis proposes frameworks of graphic design decisions, their relationships, and the processes involved in making them. A key takeaway from these frameworks is the iterative co-evolution of purpose, content, and style choices. Building on these insights, the thesis introduces computational methods that model design processes and user intent. Based on these models, I propose multiple interaction techniques to support the alignment of open ended and evolving design intent by (1) aligning AI support with the structure of the design process, (2) making AI’s understanding of design intent interpretable, (3) allowing users to directly edit that understanding, and (4) emphasizing the connections and relationships among design decisions within the design space. User studies evaluated these interaction techniques to measure their impact on the design process.The results show that alignment with the design process increased usefulness and the ease of exploring options. Furthermore, the transparent and editable AI interpretation of intent helped users determine what they wanted from the design and how to articulate it.