Text To Image Generation using CLIP & Stable Diffusion with Reduced Computational Cost

Advances in AI technology over the past few years have achieved tremendous progress in text-to-image generationthrough deep learning techniques that effectively bridge natural language and visual representation. Accordingly, a newtext-to-image transfer framework is being presented, enabled by CLIP and Stable Diffusion for improving coherent imagegeneration of high quality. Based on the proposed model, text prompts are first translated into semantic embeddings with the helpof CLIP. These embeddings are afterwards transformed into high-fidelity images by Stable Diffusion through an iterativerefinement loop. The latter will be qualitatively and quantitatively validated by cosine similarity measures and other metrics thatcheck the correspondence between the provided text and the resulting picture.The technology is cost effective and can be used in various fields, including, but not limited to, medical imaging, educationalmaterial development, and content creation. Additionally, this work discusses the problem of generative AI responsibility-forexample, their biases, resource efficiency, and interpretability-on top of allocating the risks of such technologies. To gain betterunderstanding of the context, further work is geared toward improving model performance while reducing the requiredcomputing resources and studying domain-specific fine-tuning methods.