Fusion-Based CNN-ViT Model for Breast Cancer Classification and Detection Using Digital Mammograms

Abstract

Although it can affect men as well, breast cancer is currently the most common type of cancer that mostly affects women. It manifests when aberrant cells in breast tissue proliferate quickly and develop into tumors. Physicians use the mammogram procedure to examine the breast in order to diagnose early cancer. These mammograms fall into one of two cat- egories: benign or malignant.By merging Convolution Neural Networks (CNNs) with Vision Transformers (ViTs), this study attempts to improve classification accuracy and solve the variability and potential oversight in radiologists’ manual mammography interpretations.. ViTs capture the global context but need a lot of data and processing power, while CNN is an effective image classification system that leverages hierarchical feature extraction. In this study, we trained a CNN+ViT model using CLAHE- enhanced mammography pictures from Kaggle. For comparative study, we have also employed a few pre-trained models, including DenseNet, In- ception, SE Resnet, and XceptionNet. With an accuracy of 90.1%, the CNN+ViT model demonstrated strong performance. XceptionNet’s perfect accuracy could be a sign of overfitting