pqhQAQ/Pulsar-VLM: Pulsar-VLM: A Visual Reasoning Model for Pulsar Candidate Identification

1. Introduction We propose Pulsar-VLM, a visual reasoning model for pulsar candidate identification, built on pretrained multimodal large language models (MLLMs). The input to Pulsar-VLM includes diagnostic subplots (i.e., frequency-phase plots and DM curves) along with task instructions. By integrating the visual features of the diagnostic subplots with semantic textual descriptions, Pulsar-VLM is able to perform geometric morphological reasoning on the diagnostic plots in a manner analogous to how astronomers interpret them. Pulsar-VLM maintains high performance across diagnostic plots from commonly used radio telescopes, and thus provides substantial benefits to the pulsar research community. 2. Model Information Training Data The training dataset consists of both visual and textual modalities. The visual data includes DM curves and frequency–phase subplots extracted from diagnostic plots, while the textual data comprises geometric morphological reasoning chains. The visual training data is derived from diagnostic plots generated using PRESTO, based on observational data from FAST (Five-hundred-meter Aperture Spherical Radio Telescope) and Parkes telescopes. The details of the sources are as follows: FAST data source: This dataset includes training data from Wang et al. (2019) and the publicly available observational data from the Commensal Radio Astronomy FAST Survey. In total, it consists of 837 pulsar diagnostic plots and 21,186 RFI diagnostic plots. Parkes data source: This dataset is based on data published by Balakrishnan et al. (2021) and Morello et al. (2014). It includes 1,196 pulsar diagnostic plots and 1,425 RFI diagnostic plots. Training Detail We perform full fine-tuning on our model using the AdamW optimizer. The initial learning rate is set to 1e-5 and is decayed to 0 via a cosine schedule. To prevent overfitting, we apply a weight decay of 0.1. Training is conducted on four NVIDIA V100 GPUs (80GB each), with a batch size of 4 for 3 epochs. Mixed-precision training is employed to enhance computational efficiency and maximize resource utilization. Model Downloads The model is available for download from Hugging Face. Model Params Base Model Hugging Face Pulsar-VLM 7B deepseek-vl-7b-chat 🤗 Hugging Face 3. Evaluation Dataset To evaluate the model’s capability to identify pulsar candidates across data from different telescopes, we constructed an evaluation benchmark. This benchmark consists of diagnostic plots of confirmed pulsar signals, including our own CRAFTS dataset (which we collected and made publicly available), as well as other publicly available datasets from various radio telescope surveys. Dataset Telescope Pulsar Candidate Count Source CRAFTS dataset FAST 245 🤗 Hugging Face GPPS dataset FAST 1405 🔗 Official Website GC FANS dataset FAST 91 🔗 Official Website FAST dataset FAST 326 🔗 GitHub GBT-350 Dataset GBT 100 🔗 Official Website GBNCC Dataset GBT 173 🔗 Official Website SGAN Dataset Parkes 10586 🔗 GitHub GHRSS dataset GMRT 11 🔗 Official Website LOTAAS Dataset LOFAR 87 🔗 Official Website 4. Quickstart Installation To get started, ensure you are using a Python environment with version 3.8 or higher. Then, install the required dependencies by running the following commands: git clone https://github.com/pqhQAQ/Pulsar-VLM.git cd Pulsar-VLM pip install -e . Inference Usage We provide two scripts for performing inference: run_infer_single.py and run_infer_multi.py. These scripts allow you to classify pulsar candidates using Pulsar-VLM, either for a single candidate or for multiple candidates. 1. Single Candidate Inference Use run_infer_single.py to perform inference on a single pulsar candidate. The input consists of the Frequency-phase plot and the DM curve of the candidate. The script utilizes Pulsar-VLM to determine whether the candidate is a pulsar signal or radio frequency interference (RFI). Usage Open run_infer_single.py and modify the image_paths in the code to point to your input images. Specifically, set the paths for: frequency-phase_plot.png: Path to the frequency-phase plot image. DM_curve.png: Path to the DM curve image. After modifying the paths, run the script as follows: python3 run_infer_single.py The output will indicate whether the candidate is classified as a pulsar or RFI. 2. Multi Candidates Inference and Evaluation Use run_infer_multi.py to perform inference on multiple pulsar candidates and evaluate the overall performance. The input is a JSON file containing the paths to the frequency-phase plot and DM curve for each candidate, along with their corresponding ground truth answers. Usage Open run_infer_multi.py and modify the path to the input JSON file directly in the code. The JSON file should follow the format below: [ { "conversations": [ { "from": "user", "value": " /path_to/frequency-phase_plot.png /path_to/DM_curve.png Please judge if the signal is a pulsar signal or radio frequency interference. " }, { "from": "assistant", "value": " The signal is a pulsar signal. " } ] }, ... ] After editing the script, run the script as follows: python3 run_infer_multi.py The script will output the inference results for each candidate and compute the overall evaluation metrics, including: True Positives (TP) True Negatives (TN) False Positives (FP) False Negatives (FN) Recall Specificity 5. License This code repository and the Pulsar-VLM model are licensed under the Apache License 2.0, copyright 2026 Zhejiang Lab. Please note that: The LLM backbone of Pulsar-VLM is based on the pre-trained deepseek-vl-7b-chat. Your use of Pulsar-VLM shall therefore comply with the terms and conditions set forth in the DEEPSEEK LICENSE AGREEMENT. 6. Contact For any questions, please contact:Pan QiuhongEmail: panqh@zhejianglab.org