Evaluating story generation in small language models: Fine-Tuning and Metric Analysis

With the rapid advancement of Large Language Models (LLMs), Generative AI has achieved remarkable progress in producing human-like content. Of particular interest is the emerging trend of personalized LLMs, which tailor outputs to specific user contexts. This study focuses on the domain of storytelling and investigates the potential of Small Language Models (SLMs) – a rising alternative to LLMs due to their lower computational and storage requirements. We explore the capabilities of SLMs when fine-tuned for personalized storytelling using the QLoRA methodology. Our analysis covers two models: DistilGPT-2 and GPT-2 Large, from Hugging Face, and evaluates their performances using both referenced and unreferenced metrics. This research aims to highlight the practical viability of SLMs in personalized narrative generation and provide a comprehensive evaluation of their efficiency and effectiveness in the storytelling domain.