Exploring Machine Learning Classifiers for Breast Cancer Classification

Abstract

Breast cancer is a major health concern affecting women and men globally. Early detection and accurate classification of breast cancer are vital for effective treatment and survival of patients. This study addresses the challenge of accurately classifying breast tumors using machine learning classifiers such as MLP, AdaBoostM1, logit Boost, Bayes Net, and the J48 decision tree. The research uses a dataset available publicly on GitHub to assess the classifiers' performance and differentiate between the occurrence and non-occurrence of breast cancer. The study compares the 10-fold and 5-fold cross-validation effectiveness, showing that 10-fold cross-validation provides superior results. Also, it examines the impact of varying split percentages, with a 66% split yielding the best performance. This shows the importance of selecting appropriate validation techniques for machine learning-based breast tumor classification. The results also indicate that the J48 decision tree method is the most accurate classifier, providing valuable insights for developing predictive models for cancer diagnosis and advancing computational medical research.

1. Introduction

Despite years of research, more women are being diagnosed with breast cancer. Validated risk assessment models can use mammographic density and polygenic risk to predict a woman's risk of breast cancer more accurately [1]. Breast cancer remains the dominant type affecting women, encompassing various pathological presentations, clinical characteristics, and outcomes. In the United States, it ranks as the second highest cause of cancer-related deaths [2]. Fig. 1 depicts the breast cancer illustration.

Fig. 1. Breast Cancer illustration: (a) breast cancer cells relation with the vascular system. (b) Major components present in blood [18].

Multiple observational studies have shown that regular mammography screening significantly decreases mortality rates associated with breast cancer [3]. Early diagnosis of breast cancer tumors can increase the chances of survival. In the domain of Machine Learning (ML) and Deep Learning (DL), Convolutional neural networks (CNNs) have emerged as effective tools for classifying breast cancer tumors in medical images. Ensemble learning methods such as Random Forest (RF) and gradient boosting can help feature engineering to improve accuracy. Radiomics is a method that extracts detailed features from medical images to help classify breast tumors more effectively. Classifying breast cancer involves examining genes, tissues, and images from scans like MRI and ultrasound. Combining data from various sources and using explainable AI and transfer learning can improve classification models. Accuracy can also be increased using some strategies, i.e., synthetic data generation, quantitative image marker identification, and data augmentation [4-7].

Some challenges observed in breast cancer classification techniques are unbalanced data, interoperability problems, scarcity of knowledge in the health domain, and confusion in annotations. Similarly, challenges may occur with robust generalization, cost considerations, computational requirements, the dynamic nature of breast cancer, and adaptation to different patient populations [5, 8, 9]. A multifaceted approach is required to resolve these issues occurring in breast cancer classification. Data augmentation and collaborative databases can boost the size and diversity of datasets [10]. Attaining interoperability in healthcare relies on standardization and the advancement of interoperable systems [11]. Data security and privacy can be maintained if encryption and access control are combined with privacy-preserving AI methods [12]. Options include crowdsourcing and semi-supervised learning to enhance annotation quality and quantity. Model interpretability is facilitated through Explainable AI (XAI) and external interpretation tools. Generalization is improved with regularization techniques and cross-validation. Clinical validation necessitates rigorous trials and collaboration with regulatory bodies. Computational resource challenges are met with cloud computing and model optimization. Given the dynamic nature of breast cancer, models must incorporate continuous learning [13, 14]. Linear Discriminant Analysis (LDA) is a method in ML that helps to separate and classify different groups by finding the most important features. It's often used in pattern recognition and ML to help classify objects or predict categories [15].

Comparing multiple ML classifiers is essential for optimizing their performance in cancer diagnosis. This analysis helps pinpoint the most effective model by assessing metrics such as accuracy and precision. It also provides valuable insights into the reliability of classifiers across various datasets, guiding the selection of robust models. Fine-tuning hyperparameters based on their impact ensures optimal model performance and considers adaptability to diverse datasets [16, 17].

The problem focused in this study is the accurate identification and classification of breast tumors using ML classifiers. This study aims to explore an efficient classifier among various classifiers for accurately classifying breast tumors. Also, the optimal splitting percentage and folding values should be determined to increase the accuracy of the classification model. Table 1 comprises the enumeration of acronyms.

Table 1. Enumeration of acronyms.

2. Literature Review

Among K-NN, ANNs, LR, and RF, SVM was found to be the most accurate ML classifier for predicting breast cancer. On the other hand, ANNs outperformed other approaches with the highest accuracy of 98.57% [19]. A hybrid approach was created for feature selection that combines the advantages of feature selection methods with an enhanced GA (improved Genetic Algorithm). The findings showed that when choosing the best features, the hybrid feature selection approach is better for both single filter methods and PCA [20].

Similarly, genetic programming and ML techniques were used to create a system differentiating between benign and malignant breast malignancies. The objective of the research was to improve the learning algorithm. This study highlights the potential of genetic programming to automatically select the optimal model by combining feature pre-processing strategies and classifier algorithms [21].

A new integration method combining ML with specific selection and survival analysis based on Cox regression was presented in a study. The study aimed to identify the most useful miRNA biomarkers in different types of breast cancer [22]. The wrapper-based feature selection strategy uses PSO, GS, and a greedy step algorithm. The J48 (DT) estimator is the most accurate predictor of breast cancer using ML [23]. Diagnostics of an IoT environment based on machine learning aims to distinguish between normal and malignant tumors. To develop the classification of this method, an iterative feature selection strategy was used to identify the most important features in breast cancer data [24].

Four ML classifiers (kNN, DT, binary SVM, and Adaboost) were compared and contrasted regarding performance on the BCW dataset. The feature selection model used NCA to select and reduce the number of relevant features to reduce model complexity [25]. Using symmetrical CT scan data, several ML classifiers differentiate between images of healthy and tuberculosis-infected lungs. The MLP classifier outperforms other classifiers with 98.83% accuracy and fast execution time [26].

Naive Bayes and KNN were used to classify breast cancer. The findings indicated that the KNN method performed better and achieved high accuracy, 97.51%, and a lower error rate. On the other hand, the Naive Bayes method also showed good results, with an accuracy of 96.19%. Similarly, CNN was used to detect nodules from large numbers of images and has been evaluated to help radiologists diagnose cancer early [27].

Likewise, public data was used to build a DL model for breast cancer diagnosis and classification. The high accuracy highlights the DL model's effectiveness in accurately detecting and classifying breast cancer [28]. A unique hybrid method that integrates traditional handcrafted features with CNNs to improve the effectiveness of segmenting brain tumors [29]. Decision tree (DT) methodologies offer several advantages in medical image analysis. Firstly, their interpretability is a key strength, allowing clinicians and researchers to understand the reasoning behind each decision [30]. DT handles non-linear relationships effectively [31]. DT methods are robust to outliers, which is common in medical datasets [32]. Moreover, DT methods perform implicit feature selection, prioritizing the most informative features [33].

3. Methodology and Techniques

This section presents the data collection, data preparation, and proposed methodology.

3.1 Dataset Collection and Preparation

This study uses a publicly available dataset from GitHub [34, 35]. ARFF format files are used in this study because they are compatible with the Weka software. This dataset was sourced from the Institute of Oncology at the University Medical Centre, provided by physicians Matjaz Zwitter and Milan Soklic. The dataset was donated by Jeff Schlimmer and Ming Tan [35]. The dataset typically contains several hundred instances, each representing a case with a set of features and a class label indicating the presence or absence of breast cancer. This data includes demographic information, tumor characteristics, and medical history details. The pre-processing of the dataset involves handling missing values and encoding categorical variables. The dataset consists of 286 instances, each characterized by 10 attributes. It is noted that there are missing values present within the dataset. As per the class distribution, 201 instances are labeled as 'no-recurrence-events,' while 85 instances are labeled as 'recurrence-events.' The data is divided into 80% for training and 20% for testing of the model.

3.2 Proposed Classification Model

This proposed model presents a detailed examination to classify breast tumor recurrence. This begins with dataset evaluation and resolving data quality issues via pre-processing. We utilized feature selection methods to pinpoint relevant attributes effectively. Weka classifiers, like MLP, Bayesian Networks, J48, AdaBoostM1, and LogitBoost, were employed. To ensure model robustness, we used both 10-fold and 5-fold cross-validation, along with testing different percentage splits. The model evaluation used metrics like precision, recall, F-measure, and ROC curve analysis, followed by parameter optimization in Weka to enhance performance. The refined model was then deployed for prediction on seen datasets. Fig. 2 depicts the proposed model. The version 3.8.3 of WEKA software [36-38] is employed to generate results.

Fig. 2. Proposed model block diagram.

3.3 Performance Evaluation of the Model

The performance evaluation of the proposed classifiers was measured using the following metrics. These parameters were also used in the previous study [26].

\(\begin{align}\text {TP-rate}= \frac{ {\text {TP}}}{\text {(TP+FN)}}\end{align}\),       (1)

\(\begin{align}\text {TN-rate} = \frac{ \text {TN}}{ \text {(TN + FP)}}\end{align}\),       (2)

\(\begin{align}\text {FP-rate} = \frac{ \text {FP}}{ \text {(FP + TN)}}\end{align}\),       (3)

FN − rate = 1 − TP rate,       (4)

\(\begin{align}\text {Accuracy} = (\frac {\text {correctly predicted class} } {\text {total testing class instance}}) \times 100\%\end{align}\),       (5)

\(\begin{align}\text {Precission} = \frac{ \text {TP}}{ \text {(TP+FP)}}\end{align}\),       (6)

\(\begin{align}\text {Recall} = \frac{ \text {TP}} { \text {(TP+FN)}}\end{align}\),       (7)

\(\begin{align}\text {F - measure} = \frac{ 2 \times {\text {Precission}} \times {\text {Recall}}} {{ \text {Precision + Recall}}}\end{align}\).       (8)

In these equations, TP meanstrue positive, TN meanstrue negative, FP meansfalse positive, and FN is false negative. The ROC Area, also known as the AUC, is a performance metric that assesses the accuracy of a binary classification algorithm. Two classes, "No recurrence" and "recurrence events," have been classified. In the context of this study, no recurrence means normal breast tumors. In contrast, recurrence events mean malignant breast tumors. Fig. 3 depicts the confusion matrix for this analysis. In classifying breast cancer cases into two classes of non-recurrence and recurrence events, "True A" denotes the number of non-recurrence marked right cases, and "True B" gives the cases where recurrence events have been mistakenly marked as non-recurrence. False-A counts the class of non-recurrence instances that the machine mistakenly classified. Meanwhile, False-B counts the instances the machine classified to the class of non-recurrence, but that belonged to the class of recurrence.

Fig. 3. Proposed confusion matrix.

4. Results and Discussion

The following are the results of classifiers used for breast tumor classification.

4.1 Performance of MLP Classifier

Table 2 summarizes the MLP classifier, displaying a runtime of 0.89 seconds and utilizing 10 cross-validation folds. The count of instances is 286, where correctly classified instances are 185 and incorrectly classified instances are 101.

Table 2. Summary of MLP classifier.

Table 3 presents the results of a classification model's performance in distinguishing between "no-recurrence-events" and "recurrence-events." It comprehensively assesses the model's accuracy, precision, recall, and other key metrics. The model demonstrates reasonably good performance, with an overall weighted average accuracy of 0.647, indicating its ability to classify instances into these two classes correctly. Additionally, the MCC suggests moderate overall model quality. Collectively, these metrics show that the model has the potential for identifying instances related to breast cancer recurrence, providing valuable insights for medical decision-making and treatment strategies.

Table 3. Detailed accuracy of MLP classifier.

Fig. 4 depicts the confusion matrix and events classification of the MLP classifier. It indicates that out of 150 instances of "no-recurrence-events," the model correctly classified 150 (TP), but it misclassified 51 as "recurrence-events" (FP). Similarly, out of 50 instances of "recurrence-events," the model correctly classified 35 (TP) but misclassified 50 as "no-recurrence-events" (FN).

Fig. 4. Events Classification by MLP Classifier.

4.2 Performance of J48 (Decision Tree) Classifier

The DT classifier required 0.06 seconds for execution, and the cross-validation involved 10 folds. A summary of the J48 classifier can be found in Table 4. The total number of instances is 286, where correctly classified are 216 and incorrectly classified are 70.

Table 4. The summary of 48 (DT) classifier.

Table 5 presents the performance metrics for the J48 classifier and demonstrates the model's high accuracy in identifying "no-recurrence-events" but struggles with "recurrence-events." Finally, the model's quality is moderate, as indicated by the MCC of 0.339.

Table 5. J48 classifier detailed accuracy.

Fig. 5 depicts the confusion matrix and events classification of the J48 classifier. The model accurately predicted 193 instances of "no-recurrence-events" and 23 instances of "recurrence-events." However, it made 8 FP predictions for "no-recurrence-events" and 62 FN predictions for "recurrence-events."

Fig. 5. Events classification by J48 classifier.

4.3 Performance of LogitBoost Classifier

The model was constructed with a 10-fold cross-validation with a building time of 0.03 seconds. Testing the model on the test split took 0 seconds. Table 6 presents a comprehensive summary of the LogitBoot classifier, including various instances and errors.

Table 6. A Summary of LogitBoot Classifier.

Table 7 presents performance metrics for a LogitBoost classifier and shows strong performance in identifying "no-recurrence-events" with a good TP-rate and Precision. However, for "recurrence-events," the model's performance is comparatively weaker. Finally, all, the MCC suggests a moderate model quality.

Table 7. LogitBoost classifier detailed accuracy.

Fig. 6 depicts the confusion matrix and events classification of a LogitBoost classifier. It shows that the model correctly predicted 176 instances of "no-recurrence-events" and 31 instances of "recurrence-events." However, it made 25 FP predictions for "no-recurrence-events" and 54 FN predictions for "recurrence-events."

Fig. 6. Events classification by LogitBoost classifier.

4.4 Performance of AdaBoostM1 Classifier

The AdaBoostM1 classifier was executed in 0.02 seconds with 10 cross-validation folds. The testing of the model on the test split took 0 seconds. Table 8 offers a comprehensive overview of the AdaBoostM1 classifier, encompassing various instances and associated errors.

Table 8. Summary of AdaBoostM1 classifier.

Table 9 presents performance metrics for the AdaBoostM1 classifier, and the model demonstrates moderate accuracy in both classes, as seen in TP-rate, Precision, and Recall. The MCC is 0.257, indicating moderate overall model quality.

Table 9. Accuracy of AdaBoostM1 in detail.

Fig. 7 depicts the confusion matrix and events classification of the AdaBoostM1 classifier. The model correctly predicted 165 instances of "no-recurrence-events" and 36 instances of "recurrence-events." However, it made 36 FP predictions for "no-recurrence-events" and 49 FN predictions for "recurrence-events.

Fig. 7. Events classification of AdaBoostM1 classifier.

4.5 Performance of BayesNet Classifier

Table 10 presents the summary of the BayesNet classifier, including the time taken to build the model (0.04 seconds) and the number of cross-validation folds (10).

Table 10. BayesNet classifier summary.

Table 11 presents the performance metrics for the BayesNet classifier. Also, it indicates a moderate ability of the model to correctly classify instances in both classes, as shown by metrics like TP-rate, Precision, and Recall. The MCC of 0.295 suggests moderate overall model quality.

Table 11. Detailed accuracy assessment for the BayesNet classifier.

Fig. 8 depicts the confusion matrix and events classification of the BayesNet classifier. It correctly predicted 169 instances of "no-recurrence-events" and 37 instances of "recurrence-events." However, it made 32 FP predictions for "no-recurrence-events" and 48 FN predictions for "recurrence-events."

Fig. 8. Events classification by BayesNet classifier.

4.6 Comparison of the Proposed Models

This study implemented various classifiers like MLP, J48, LogitBoost, AdBoostM1, and BeyesNet for pattern recognition. Table 12 presents the performance results for various classifiers based on their F-measure and Accuracy scores. It demonstrates that the J48 (DT) classifier achieves the highest F-measure and accuracy at 0.713 and 71.3%, respectively, indicating its superior performance in classifying instances.

Table 12. A comparative analysis proposed classifiers.

4.7 Further Evaluation of the J48 Classifier

The J48 classifier is further evaluated regarding validation fold and percentage splitting to find better results. Regarding 5-fold validation, Table 13 presents a breakdown of the accuracy metrics for the J48 classifier, considering various parameters and their respective classes.

Table 13. J48 classifier accuracy in detail.

The accuracy of 10 fold is 71.3, whereas the accuracy of 5 folds is 69%, which is reduced. So, 10 folds of J48 are further evaluated with different percentage splits, as presented in Table 14.

Table 14. Evaluation of 10-fold J48 Classifier with different percentage splits.

Table 14 contains detailed accuracy and performance metrics of J48 split percentages 50, 90, 35, 73, and 40. As per split 50, the model excels in identifying "no-recurrence-events" with a high TP-rate, but it faces challenges in classifying "recurrence-events." Regarding the J48 split percentage 90, the model highlights a higher TP-rate for "no-recurrence-events" but challenges in classifying "recurrence-events." The Precision values are somewhat balanced. The MCC of 0.131 indicates a moderate overall model quality.

Regarding the J48 split percentage, the model achieves a relatively high TP-rate for "no-recurrence-events" but faces challenges with "recurrence-events." The Precision values show a reasonable balance between the classes. The MCC of 0.309 suggests moderate overall model quality. In evaluating J48 split percentage 73, the model identifies "no-recurrence-events" with a high TP-rate but struggles with "recurrence-events." The Precision values show a reasonable balance between the classes. The MCC of 0.273 indicates moderate overall model quality. Finally, the model is evaluated with J48 split percentage 40 and observed that the model is relatively proficient at identifying "no-recurrence-events" with a high TP-rate but faces challenges with "recurrence-events." The Precision values suggest a reasonable balance between the classes. The MCC of 0.236 indicates moderate overall model quality. These findings evaluate the model's ability to distinguish between the two classes, with room for improvement in some areas.

Table 15 compares different split percentages on the accuracy of a machine-learning model. It demonstrates that a split percentage of 66% yields the highest accuracy at 71%, indicating that this particular data split ratio is most effective for this model. Other split percentages result in varying levels of accuracy, suggesting the importance of selecting an appropriate data split strategy for optimal model performance.

Table 15. Accuracy analysis of J48 on different split percentages.

5. Discussion

This study implanted different classifiers, but the maximum accuracy (71%) is achieved using J48 (DT). Weka was used to implement the classifiers, and then different classifiers were tried. The results of these classifiers are presented, but the results of some classifiers are not presented if their accuracy is less than 69%. We mentioned some accuracies, but some are ignored, e.g., as the summary of Naïve Bayes classier is presented in Table 16. The detailed accuracy by class is presented in Table 17.

Table 16. Summary of Naïve Bayes classier.

Table 17. Detailed accuracy of Naïve Bayes.

Weka provides the F-measure and ROC curves to analyze the accuracy of the model. The weighted F-measure of J48 is 0.713, and the ROC is 0.58, which indicates the performance of J48 on a given dataset. The accuracies of different classifiers are evaluated again J48 provided the maximum F-measure weighted average value of 71.3 compared to other classifiers.

6. Conclusion

This comprehensive study uses many machine learning classifiers to classify the occurrence or non-occurrence of breast tumors based on various features and data points related to individuals' medical history. The study evaluated several classification methods, including MLP, AdaBoostM1, logitBoost, BayesNet, and J48. The effectiveness of these classifiers is assessed using various performance metrics. The findings indicated that the J48 (DT) classifier outperformed the other classifiers, demonstrating the highest accuracy among the tested methods. With an accuracy of 71%, J48 demonstrated its effectiveness in accurately classifying instances into the appropriate class. It was also found that a split percentage of 66% provided the optimal balance for achieving the highest accuracy.

Furthermore, the impact of fold values on the model's accuracy is explored by modifying the fold value from 10 to 5. However, the results indicated that the 10-fold cross-validation produced the best accuracy results. This research highlights the potential of employing pattern recognition and DT-based classifiers, particularly J48, in accurately classifying cancer-related instances. These findings offer valuable insights for developing cancer assessment models and significantly contribute to the field of computational biology.

7. Limitations and Future Work

In future studies, the following limitations should be considered:

• The class distribution is imbalanced, which may lead to biased model performance towards the majority class.

• The presence of missing values in the dataset can affect the accuracy of the classifiers if not properly handled.

• Limited feature selection may result in models not capturing all relevant patterns in the data.

• The models trained on this dataset may not generalize well to other datasets or populations.

• Using a publicly available dataset instead of real-time clinical data may limit the generalizability of the classification results to real-world scenarios. It may not capture the full range of variability and complexities present in clinical settings.

• These models are suitable for smaller datasets with fewer features. However, deep learning (DL) models better classify complex patterns and high-dimensional data.

Funding

Not applicable.

Competing Interest

The authors declare there are no competing interests.