FocFormer-UNet: UNet With Focal Modulation and Transformers for Ultrasound Needle Tracking Using Photoacoustic Ground Truth

Ultrasound (US)-guided needle tracking is a critical procedure for various clinical diagnoses and treatment planning, highlighting the need for improved visualization methods to enhance accuracy. While deep learning (DL) techniques have been employed to boost needle visibility in US images, they often rely heavily on manual annotations or simulated datasets, which can introduce biases and limit real-world applicability. Photoacoustic (PA) imaging, known for its high contrast capabilities, offers a promising solution by providing superior needle visualization compared to conventional US images. In this work, we present FocFormer-UNet, a DL network that leverages PA images of the needle as ground truth for training, eliminating the need for manual annotations. This approach significantly improves needle localization accuracy in US images, reducing the reliance on time-consuming manual labeling. FocFormer-UNet achieves excellent needle localization accuracy, demonstrated by a modified Hausdorff distance of 1.43 1.23 and a targeting error of 1.22 1.14 on human clinical dataset, indicating minimal deviation from actual needle positions. Our method offers robust needle tracking across diverse US systems, improving the precision and reliability of US-guided needle insertion procedures. It holds great promise for advancing AI-driven clinical support tools in medical imaging. The following is the source code: https://github.com/DeeplearningBILAB/FocFormer-UNet. Open Science Framework (OSF) provides datasets and checkpoints at: https://osf.io/yxt9v/.