Deep ANN feature fusion with multi-stage data processing and visualization for enhanced breast cancer detection in ultrasound images

A major obstacle to breast cancer detection in ultrasound images is that they can be difficult to evaluate due to speckle noise and low contrast as well as very subtle differences between the appearance of benign vs. malignant lesions. With the advent of large amounts of digital medical imaging, particularly ultrasound, there exists a huge volume of heterogeneous ultrasound data; thus, it is imperative to develop both robust big data analytical frameworks as well as artificial neural network (ANN) architectures that will allow the identification of clinically relevant characteristics from these high dimensional noisy data streams. This paper proposes a new dynamic feature fusion technique called adaptive gated cross attention fusion (AGCAF). AGCAF utilizes a learnable gating function and two-way cross attention to dynamically weight each contribution of features from the DeIT and ViT backbones. The AGCAF framework includes a rigorous multiple stage image processing pipeline that uses CLAHE, anisotropic diffusion based speckle removal, and bilateral edge preserving filtering to make the boundaries of the lesions visible before the ANNs extract features. Tools used to visualize and validate the behavior of the model include T-SNE embedding, GRAD-CAM saliency maps, and roc curves. Extensive experiments using the BUSI, BUS-BRA, BrEaST, and BUSI_WHU databases show that the AGCAF architecture provides accuracy rates of 95.18 % and AUC rates of 96.02 % for 3 class classifications, and accuracy and AUC rates of 97.45 % and 97.43 % respectively for binary classifications. Results also show that AGCAF performed better than static concatenation ensemble and all baseline models individually. Additionally, cross database results and ablation studies were conducted to verify the robustness and clinical validity of the proposed method.