Abstract

According to the American Cancer Society, breast cancer is the world's most commonly diagnosed and deadliest form of cancer in women. A major determinant of the survival rate in breast cancer patients are the accuracy and speed of the malignancy grade determination. This thesis considers the classification problem related to determining the grade of a malignant tumor accurately and efficiently. A Fine Needle Aspiration (FNA) biopsy is a key mechanism for breast cancer diagnosis as well as for assigning grades to malignant cases. Carrying out a manual examination of FNA demands substantial work from the pathologist which may result in delays, human errors, and consequently lead to misclassified grades. In this context, the most common grading system for microscopic imaging for breast cancer is the Bloom and Richardson (BR) histological grading system which is based on the evaluation of tissues and cells. BR is not directly applicable to FNA biopsy slides due to distortion of tissue and even cell structures on the cytological slides. Therefore, in this thesis, to grade FNA images of breast cancer, instead of the BR grading scheme, six known cytological grading schemes, three newly proposed cytological grading schemes, and five grading systems based on convolutional neural networks were proposed to automatically determine the malignancy grade of breast cancer.

First, considering traditional Machine Learning methods, six cytological grading systems (CA-CGSs) based on six cytological schemes used by pathologists for FNA biopsies of breast cancer were proposed to grade tumors. Each system was built using the cytological criteria as proposed in the original CGSs. The six considered cytological grading schemes in this thesis were Fisher's modification of Black's nuclear grading, Mouriquand's grading, Robinson's grading, Taniguchi et al's, Khan et al's and Howell's modification in mitosis count criteria. To fulfill this task, different sets of handcrafted features using customized image processing algorithms were extracted for classification purpose. The proposed systems were able efficiently to classify FNA slides into G2 (moderately malignant) or G3 (highly malignant) cases using traditional machine learning algorithms. Additionally, three new cytological grading systems were proposed by augmenting three of the original CGSs by adding the low magnification features. However, the systems were not sensitive enough with regards to G3 cases due to the low number of available data samples. Therefore, a data balancing was performed to improve the sensitivity for G3 cases. Consequently, in the second objective of this work, data sampling and RUSBoost methods were applied to the datasets to adjust the class distribution and boost the sensitivity performance of the proposed systems. This enabled a sensitivity improvement of up to 30% which highlights the significance of class balancing in the task of malignancy grading of breast cancer.

Additionally, due to the considerable time and efforts required for handcrafted features-based cytological grading systems in order to achieve efficient feature engineering results, a deep learning (DL) approach was proposed to avoid the aforementioned challenges without compromising the grading accuracy. Thus, in this thesis, five different pre-trained convolutional neural network (CNN) models, namely GoogleNet Inception-v3, AlexNet, ResNet18, ResNet50, and ResNet101, combined with different techniques to deal with unbalanced data, were used to develop automated computer-aided cytological malignancy grading systems (CNN-CMGSs). According to the obtained results, the proposed CNN-CMGS based on GoogleNet Inception-v3 combined with the oversampling method provides the best accuracy performance for the problem at hand.

The results demonstrated that the proposed CGSs are highly correlated since they share some of the cytological criteria. Further, the overall accuracy of the CGSs is roughly the same and overall, the handcrafted features-based CGSs performed best even in the absence of class distribution rebalancing. Overall, for case classification, the best results were obtained for computer-aided CGSs based on the modified Khan et al.’s and Robinson’s schemes with accuracies of 97.77% and 97.28%, respectively. Meanwhile, for patient classification, the overall best results were obtained for computer-aided CGSs based on the modified Khan et al.’s and modified Fisher's schemes with accuracies of 96.50% and 95.71%, respectively. These results surpass previously reported results in the literature for computer-aided CGS based on BR histologic grading. Moreover, in clinical practice, Robinson’s typically has the best diagnostic accuracy with the highest reported experimental accuracy rate of 90%. Thus, the obtained results demonstrate that computer-aided breast cancer cytological grading systems using FNA can potentially achieve accuracy rates comparable to the more invasive histopathological BR-method.