Single-LLM Synthetic Market Evaluation for Banner Design
Abstract
We present a reproducible protocol for early-stage semantic validation of banner design strategies using a single language model (LLM) as a fixed evaluator. Instead of pixel-level renderings, stimuli are represented as feature-level design specifications that encode intentrelevant structure (e.g., information hierarchy, whitespace, density, contrast, CTA emphasis, and background noise). A deterministic calibration layer computes rule-based proxies for Readability, Constraint Compliance Score (CCS), and Editability, anchoring engineering constraints while keeping the pipeline fully synthetic and free of personal data. To assess intra-model reliability, we measure self-consistency via K repeated judgments under paraphrased prompts and report agreement rates alongside confidence. In a pilot with sanity-check pairs and intent trade-off pairs, we observe (i) high agreement and confidence for clear legibility manipulations, (ii) lower stability for contested intent trade-offs, and (iii) persona-conditioned preference reversals that are naturally represented as trade-off maps rather than a single scalar score. We position the approach as a lightweight triage step that reduces exploration cost before high-fidelity rendering or human-grounded validation.
Fields of Study & Subjects
Usability and User Interface Design
Digital Media and Visual Art
Recommender Systems and Techniques
Publication details
- Publication date
- 4 February 2026
- Document type
- Preprint