Artifacts of SoK: PHILTER: Uncovering Security and Functional Gaps in AI-based Phishing Website Detection Literature via an LLM-based Reasoning Framework

To support transparency and reproducibility of our evaluation methodology, we provide an artifact bundle (PHILTER.zip) containing the materials used to conduct and analyze our assessments of 55 phishing website detection methods. The artifact includes: Codebook (codebook/) This directory contains seven files, each corresponding to one of the seven PHILTER metrics described in Section 4 of our paper. Each file provides the metric's definition and evaluation criteria used in our assessments. LLM-assisted Assessment Pipeline (llm_assessment_pipeline.py)This script implements the LLM-assisted preliminary assessment pipeline described in Section 3.1 of the paper. It coordinates evaluator and arbitrator LLMs to extract metric-relevant evidence from papers and generate preliminary assessments with rationales. Assessment Results (assessments.json)This file contains both the LLM-generated preliminary assessments and the expert-validated final assessments for all 55 analyzed papers. The tables and aggregate statistics reported in the paper are computed from this file, supporting reproducibility of the reported results. Analysis Scripts (scripts/)This directory contains the scripts used to create all tables and aggregate statistics reported in the paper. Documentation (README.md)The README provides step-by-step instructions for setting up the environment and running the assessment pipeline and analysis scripts.