Love, Lies, and Language Models: Investigating AI's Role in Romance-Baiting Scams

Artifact Description and Open-Science Mapping Note: To reduce risk of misuse and protect participants and institutions, we will not publicly distribute the code used to: Create the agent from our experimental study Evaluation of our system prompts for AI self-disclosure. This contains our complete dialogue system prompt which can be misused by dangerous actors. We publicly release the artifacts used in Section 5.2 Content Filters. This section evaluates whether modern content filters can detect romance-baiting conversations generated by LLMs. We focused on the moderation tools that vendors provide to developers for building agents. The artifact can be used to: generate synthetic datasets and evaluate them with public tools. It also contains the exact synthetic dataset used in the paper for reproducibility. dataset generation code (dataset_generation/). Scripts for generating the synthetic scam and non-scam conversations, including e-commerce scams, tax scams, neutral conversations, and multi-day romance-baiting scenarios used in our paper. It also includes the prompt templates and scenario configuration files used during generation. synthetic datasets (data/). The exact dataset used in the paper. We provide JSON conversations across four categories—romance-baiting, tax scams, e-commerce scams, and neutral conversations—totaling 1,011 conversations. evaluation code for content moderation systems (evaluation/). Orchestration script and evaluators for multiple commercial and open-source content filters (OpenAI Moderation API, Google Perspective API, and LLaMA Guard 3), as well as an LLM-based judge for false-positive analysis.