Skip to main content
Download PDF

Prediction of nonsentinel lymph node metastasis in breast cancer patients based on machine learning

World Journal of Surgical Oncology volume 21, Article number: 244 (2023) Cite this article

Abstract

Background

Develop the best machine learning (ML) model to predict nonsentinel lymph node metastases (NSLNM) in breast cancer patients.

Methods

From June 2016 to August 2022, 1005 breast cancer patients were included in this retrospective study. Univariate and multivariate analyses were performed using logistic regression. Six ML models were introduced, and their performance was compared.

Results

NSLNM occurred in 338 (33.6%) of 1005 patients. The best ML model was XGBoost, whose average area under the curve (AUC) based on 10-fold cross-verification was 0.722. It performed better than the nomogram, which was based on logistic regression (AUC: 0.764 vs. 0.706).

Conclusions

The ML model XGBoost can well predict NSLNM in breast cancer patients.

Introduction

Cancer and cardiovascular diseases are the two main causes of death across the world and seriously harm people’s physical and mental health [1]. According to data from the World Health Organization (WHO), the number of newly diagnosed cancers in 2020 totaled 19.29 million, of which 2.26 million were breast cancers, and approximately 685,000 died from breast cancer [2]. Breast cancer leads the world in morbidity and mortality rates in most countries [2]. At the same time, the treatment regimens of breast cancer are changing over time. In 1985, the results of the National Surgical Adjuvant Breast and Bowel Project (NSABP) B-06 study demonstrated that breast-conserving surgery combined with radiotherapy led to no significant difference in overall survival (OS) and disease-free survival (DFS) of patients with early breast cancer compared with mastectomy, which raised the proportion of breast cancer patients treated with breast-conserving surgery [3], and the safety of breast-conserving surgery was confirmed in the following 20 years of follow-up [4]. In 2010, the results of the NSABP B-32 study showed that for malignant breast tumor patients with negative axillary lymph nodes, the success rate of axillary sentinel lymph node biopsy (SLNB) was 97.2%, and the false-negative rate was only 9.8%. There were no significant differences in OS, DFS, or local recurrence rate (LRR) for patients with negative sentinel lymph nodes but without axillary lymph node dissection (ALND) compared with those who underwent ALND [5, 6]. The risk of lymphedema and reduced range of motion in the upper limbs associated with ALND is not negligible, and it seriously affects the quality of life of patients [7]. The AMAROS study showed that early breast cancer patients who underwent SLNB combined with radiotherapy had similar axillary lymph node recurrence and DFS rates as those who underwent ALND, even if there were 1 or 2 sentinel lymph node metastases (SLNMs) [8]. In 2015, the American College of Surgeons Oncology Group (ACOSOG) Z0011 study confirmed that SLNB combined with radiotherapy could exempt early breast cancer patients with 1 or 2 SLNMs from ALND [9, 10], which further promotes the clinical application of SLNB. However, ALND is required for breast-conserving surgery patients with more than three sentinel lymph node metastases or total mastectomy patients with more than one sentinel lymph node metastasis. Studies have shown that 40–60% of breast cancer patients who undergo SLNB and further undergo ALND have no other lymph node metastases [11,12,13]. With the progress of individualized treatment of breast cancer and patients’ increasing demand for quality of life, axillary lymph node management is more inclined to include the evaluation of tumor staging and prognosis to accurately predict the risk of axillary lymph node metastasis, which can avoid surgical complications caused by overtreatment and thereby improve patients’ quality of life. It can also help reduce the recurrence risk for breast cancer patients with nonsentinel lymph node metastases (NSLNMs) who undergo SLNB but not ALND.

In recent years, machine learning (ML) has been used to manage different medical problems, such as pathologic diagnosis and treatment support, and ML models constructed in previous studies not only have better model performance but also have higher prediction accuracy [14,15,16]. Few models have been constructed to predict NSLNM. Guo Xu and his team constructed a deep learning model to predict NSLNM, but they failed to explain the impacts of different variables in their model [17]. Yang, ZB et al. [18] developed a nomogram to predict NSLNM, which showed an area under the curve (AUC) of 0.718 in the training set and 0.742 in the validation set, but its performance had not been compared with that of ML models.

Lundberg et al. first conceived the SHapley Additive exPlanations (SHAP) framework, which has been applied to machine learning [19]. It can assess the contributions of different features in different ML models, allowing the performance of each model to be reasonably compared [20].

The purpose of this study was to construct an optimal ML model to predict the NSLNM of breast cancer patients by using preoperative and intraoperative clinicopathological and imaging features and to choose the best model by using the SHAP framework. This study also compared its performance with that of a nomogram.

Materials and methods

Patients

A total of 3658 malignant breast cancer patients undergoing surgery at Harbin Medical University Cancer Hospital from June 2016 to August 2022 were retrospectively enrolled. This study was approved by the Ethics Committee of Harbin Medical University Cancer Hospital. It conforms to the 1964 Helsinki Declaration of the World Medical Association and its subsequent revisions. Informed consent from our hospital was signed by each patient before receiving treatment.

The inclusion criteria were as follows: no other breast cancer treatment prior to breast surgery and SLNB and ALND performed during breast surgery.

Exclusion criteria are as follows: Patients who received neoadjuvant therapy before breast surgery in our hospital, patients who received SLNB without ALND or directly received ALND during breast surgery, the pathological type was ductal carcinoma in situ, a distant metastasis, and male breast cancer patient.

Finally, a total of 1005 breast cancer patients were included. Their details are shown in Fig. 1.

Fig. 1
figure 1

The flow chart of patients selection and the flow chart for the development, evaluation, and explanation of models

Full size image

Axillary lymph node status management

Methylene blue dye was injected into the intradermal, subcutaneous, areolar, and glandular areas (peritumor, intratumor, subtumor) 10–15 min before breast cancer surgery (Jichuan Pharmaceutical Group, China), or a carbon nanosuspension was injected into the subserous membrane along the peritumoral site at 4–6 points (Chongqing Lemei Pharmaceutical, China) during surgery to facilitate the localization of SLNB. Sentinel and nonsentinel lymph nodes were evaluated in hematoxylin-eosin (HE)-stained sections that were fixed with 10% formalin and embedded in paraffin. After fixation, successive sections of the lymph nodes were obtained for definitive analysis of lymph node status.

Classification

An estrogen receptor (ER) immunohistochemical (IHC) detection degree of < 1% nuclear staining was interpreted as ER negativity, and an IHC-positive degree between 1 and 10% nuclear staining was interpreted as ER weak positivity. An IHC-positive degree of > 10% nuclear staining was interpreted as ER positivity [21]. Progesterone receptor (PR) was negative if its IHC-positive degree was < 1%, weakly positive if its IHC-positive degree was between 1 and 20%, and positive if its IHC-positive degree was > 20% [22]. A human epidermal growth factor receptor-2 (HER-2) IHC result of 0 was defined as HER-2 negative. Low HER-2 expression was defined as a HER-2 IHC result of 1+ or a HER-2 IHC result of 2+ along with negative fluorescence in situ hybridization (FISH). A HER-2 IHC result of 3+ or a HER-2 IHC result of 2+ with positive FISH was defined as HER-2 positivity [23]. The Ki-67 expression level was divided into the Ki-67 ≤ 14% group and the Ki-67 > 14% group [24]. According to the results of IHC, all patients were divided into luminal A, luminal B, triple-negative breast cancer (TNBC), and HER-2 overexpression groups [22].

According to current American Joint Committee on Cancer (AJCC) standards [25], single tumor cells or maximum tumor diameter < 2 mm in axillary lymph nodes was defined as node negative, and tumor diameter ≥ 2 mm was defined as node positive. Pathological lymph node staging (pN) was determined according to the number of positive axillary lymph nodes. The staging was as follows: pN0 meant no axillary lymph node metastasis, pN1 meant 1–3 axillary lymph node metastases, pN2 meant 4–9 axillary lymph node metastases, and pN3 meant more than 9 axillary lymph node metastases.

Since the patients included in this study were Chinese women with breast cancer, body mass index (BMI) was classified into different groups according to the standards of the Chinese Health Commission. BMI = weight (kg)/height (m2), and the underweight group was defined as BMI < 18.5 kg/m2. BMI between 18.5 and 23.9 kg/m2 was the normal group, BMI between 24 and 27.9 kg/m2 was the overweight group, and BMI ≥ 28 kg/m2 was the obesity group.

Considering the small number of patients with other types of breast cancer included, the patients were divided into infiltrating ductal carcinoma and other types of carcinoma according to pathological type, including invasive lobular carcinoma (18 patients), invasive micropapillary carcinoma (3 patients), ductal carcinoma in situ with microinvasion (3 patients), and mucinous carcinoma (2 patients).

Data preprocessing and feature selection

The k-nearest neighbor imputer (KNNImputer) was used to supplement parameters with missing values less than 30% [26]. Recursive feature elimination was applied to select the best variables (Fig. S1). The best number of variables was 12: age, BMI, pregnancy history, nipple retraction, single/multiple tumors, cT stage, blood flow signal of tumor, cN stage, ultrasound (US) BI-RADS classification, mammography (MG) BI-RADS classification, SLN group, and SLN-positive ratio.

Model development

This study introduced six ML algorithms, including extreme gradient boosting (XGBoost), logistic regression (LR), support vector machine (SVM), k-nearest neighbor (KNN), random forest (RF), and decision tree (DT).

The LR model is usually applied to explore how characteristics influence binary variables [27]. In the face of a regression or classification problem, the cost function is established, the optimal model parameters are iteratively solved by the optimization method, and then the quality of the resulting model is verified by testing.

SVM is applied to classify things with multidimensional attributes into two categories [28]. It is a supervised learning model that is commonly used for pattern recognition, classification, and regression analysis. Based on structural risk minimization theory, it constructs the optimal hyperplane in the feature space so that the learner is globally optimized, and the expectation of the whole sample space satisfies a certain upper bound with a certain probability.

KNN is one of the most commonly used nonparametric classification techniques. Its working premise is that if most of the nearest k samples to a given sample belong to a certain class in the feature space, then they all must belong to the same class. The KNN method is only related to a very small number of adjacent samples in the class decision. Because the KNN method mainly depends on a few neighboring samples, rather than the method of discriminating the class domain to determine the category, the KNN method is more suitable for dividing the sample with more crossover or overlap of class domains [29].

Classifiers that use multiple trees to train and predict samples are called RF classifiers, which reduces training variance and improves integration and generalization capabilities [30]. Its training can be highly parallelized, which has advantages for large-sample training speed in the era of big data. Since the decision tree nodes that divide the features can be randomly selected, the model can still be trained efficiently even when the sample feature dimension is very high.

The DT algorithm can be divided layer by layer according to the characteristics of the data until all the characteristics are divided, so it can be used to solve classification and regression problems [31]. It is a kind of nonparametric supervised learning that is easy to understand, applicable to all kinds of data, and has good performance in solving various problems, especially various integrated algorithms with tree models as the core. It is widely used in various industries and fields.

XGBoost is an ML technique that can process missing data and build accurate prediction models from weak prediction models [32]. It is good at capturing dependencies between complex data, can obtain effective models from large-scale datasets, and supports multiple systems and languages in practical terms.

Statistical methods

All patients were randomly divided into training and testing sets at a 7:3 ratio (Fig. 1). The ML prediction model was developed in the training set and optimized by using 10-fold cross-validation. The AUC, accuracy, recall rate, F1 value, and precision were used to evaluate the ability of each ML model. Brier scores were applied to evaluate the overall performance of the model [33]. Pearson’s χ2 or Fisher’s exact test was used for intergroup analysis. Univariate and multivariate analyses were performed using logistic regression. Based on multivariable logistic regression analysis, a nomogram was built, whose accuracy was determined by calculating its C-index. The internal verification was carried out by the bootstrap method, and the difference between the actual value and the predicted value obtained from the column chart was analyzed graphically. To more intuitively explain the optimal ML model, we introduce the SHAP framework, whose interpretability has been demonstrated in many cancers [18, 34,35,36]. It can demonstrate the contributions of various variables in any ML model to the outcome event [20]. All statistics were performed using Python 3.9 and R language 4.1.2. P< 0.05 was considered statistically significant.

Results

Clinicopathologic features of patients

A total of 1005 breast cancer patients with a median age of 51 years were enrolled in this study, of whom 829 (82.5%) underwent mastectomy and 176 (17.5%) underwent breast-conserving surgery. NSLNM occurred in 338 cases and not in 667 cases (Table 1). Ninety-nine patients (9.9%) were classified as luminal A, 799 patients (79.5%) were classified as luminal B, 47 patients (4.7%) were classified as TNBC, and 60 patients (6.0%) were classified as HER-2 overexpressing. Most of the included patients were patients with stages T1 and T2 (cT1 and cT2), with a total of 466 patients (46.4%) at cT1 and 512 patients (50.9%) at cT2. Most cancers were pN1 (562 cases (55.9%)). There were 340 cases (33.8%) in the Ki-67 ≤ 14 group and 665 cases (66.2%) in the Ki-67 ≥ 14 group. There were 609 cases (60.6%) without lymphatic vascular infiltration (LVI) and 396 cases (39.4%) with LVI. There were 789 (78.5%) patients with one or two sentinel lymph node metastases (SLNMs) and 113 (11.2%) patients with three or more SLNBs. Notably, 103 patients (10.2%) did not develop SLNM, but 21 of them (20.4%) did develop NSLNM. There were 738 patients (73.4%) with a sentinel node-positive ratio (the number of positive SLNs to the total SLN ratio) ≤ 0.5 and 267 cases (26.6%) with an SLN-positive ratio > 0.5. BMI, cT, cN, ultrasonic (US) BI-RADS classification, mammography (MG) BI-RADS classification, pT, pN, SLN status, number of SLNMs, sentinel lymph node-positive ratio, and LVI were correlated with NSLNM (p < 0.05, Table 1).

Table 1 The relationship between characteristics and non-SLN metastasis
Full size table

Univariate and multivariate logistic regression analysis in the training set

A total of 1005 patients were randomly classified into the training set (703 patients) and the test set (302 patients) in a ratio of 7:3. In the training set, univariate analysis showed that cN1/cN2/cN3 was more prone to NSLNM than cN0 (OR = 1.5, 95% CI: 1.07–2.09, p = 0.018). NSLNM was more likely to occur in MG BI-RADS 4 than in MG BI-RADS 3 (OR = 1.77, 95% CI: 1.02–3.05, p = 0.041). Compared with patients with negative SLNs, patients with positive SLNs were more likely to develop NSLNM (OR = 2.38, 95% CI: 1.3–4.36, p = 0.005). Compared with patients with an SLN-positive ratio ≤ 0.5, patients with an SLN-positive ratio > 0.5 were more likely to develop NSLNM (OR = 3.82, 95% CI: 2.69–5.43, p < 0.001). A variance inflation factor (VIF) < 10 indicates that there is no multicollinearity among different parameters [37]. Parameters with p < 0.05 from the univariate analysis were included in the multivariate analysis, and the results showed that the SLN-positive rate was an independent predictor of NSLNM (OR = 3.51, 95% CI: 2.43–5.05, p < 0.001) (Table 2).

Table 2 Relationship between training set characteristics and NSLNM
Full size table

Machine learning model construction and performance comparison

Twelve variables were selected to develop ML models. The relationships between different variables are shown in Fig. 2. Based on the above 12 variables, six ML models were developed on the training set, and learning curves showed that there was no overfitting of these six machine learning models (Fig. 3). Therefore, we further compared the performance of different ML models using the AUC value, accuracy, precision, F1 value, and Brier score. The results show that in the training set with 10-fold cross-validation, the average AUC value of the XGBoost model was the largest (0.722, Fig. 4a), and its accuracy was the highest (0.673, Fig. 4b). Moreover, in both the training set and the test set, the AUC value of the XGBoost model was the largest, at 0.781 (Fig. 4c) and 0.764 (Fig. 4d), respectively. The Brier score was the smallest in the training set and the second smallest in the test set, at 0.196 (Fig. 4e) and 0.191 (Fig. 4f). In the test set, the accuracy and precision of the XGBoost model were the second largest, at 0.752 and 0.723, respectively. The recall rate and F1 value of the XGBoost model were the highest, at 0.728 and 0.726, respectively (Table 3). The positive predictive value and negative predictive value of the XGBoost model were highly consistent with the real values (Fig. 5). In conclusion, of the six ML models tested, the XGBoost model demonstrated the best performance.

Fig. 2
figure 2

The relationship between different variables

Full size image
Fig. 3
figure 3

Learning curves of different machine learning models

Full size image
Fig. 4
figure 4

Performance comparison of different machine learning models

Full size image
Table 3 Results of NSLNM predicted by different ML
Full size table
Fig. 5
figure 5

The confusion matrix of different ML models

Full size image

Establishment of the nomogram and performance comparison with the XGBoost model

Based on multivariate logistic regression analysis, the SLN-positive ratio was an independent predictor of NSLNM. In a previous study, SLN status was also correlated with NSLNM [38]. Therefore, these two variables were applied to develop the nomogram. The C-index of the nomogram in the training set and test set was 0.706 and 0.647, respectively. After internal verification by the bootstrap method, the C-index in the training set and test set was similar, at 0.706 and 0.646, respectively. Figure S2a shows a nomogram for predicting NSLNM based on the SLN-positive ratio and SLN group. Based on the scores from the different states of the nomogram’s variables, the probability of NSLNM for a certain patient can be obtained. The AUC values of this model in the training set and the test set were 0.647 (Fig. S2b) and 0.706 (Fig. S2c), respectively. The deviation between the predicted value and the actual value in the training set and the test set was somewhat large (Fig. S2d, e). In the training and test sets, the AUC value of XGBoost was larger than that of the nomogram (0.781 vs. 0.647; 0.764 vs. 0.706; Table S1). These results showed that the XGBoost model was superior to the nomogram in predicting NSLNM.

Interpretability of the XGBoost model

Based on the above results, XGBoost was the best model to predict NSLNM. To make this model and its prediction easier to understand, this study makes use of the SHAP framework. Figure 6a shows the first ten characteristic parameters affecting NSLNM: SLN-positive ratio, BMI, MG BI-RADS classification, SLN group, cT, number of births, age, cN, US blood flow signal of tumor, and US BI-RADS classification. To explore how these characteristics affect NSLNM, SHAP values are further used for interpretation (Fig. 6b). The SHAP value (X-axis) represents the degree to which the feature influenced NSLNM, and the feature ranking (Y-axis) represents the size of the feature values. Red dots represent higher values, and blue dots represent lower values. The results show that compared with an SLN-positive ratio ≤ 0.5, a sentinel node-positive ratio > 0.5 was more likely to be found along with NSLNM. Compared with the low-BMI group and the normal-BMI group, the overweight and obesity group was more likely to develop NSLNM. Compared with the lower class of MG BI-RADS, the higher class of MG BI-RADS was more likely to be found with NSLNM. NSLNM was more likely to occur in the SLN-positive group than in the SLN-negative group. NSLNM was more likely to occur with higher clinical T stage than with lower clinical T stage.

Fig. 6
figure 6

SHAP summary of XGBoost model

Full size image

This study also individualized the interpretation of the model and took two typical examples to verify the accuracy of XGBoost: one patient with actual NSLNM (Fig. 7a) and one patient without NSLNM (Fig. 7b). Arrows demonstrate the effects of different variables on the outcome prediction. Red and blue arrows show whether the variable was likely to occur (red) or not (blue). The combined effects of all variables provided the final SHAP value, corresponding to the predicted score. The patient with NSLNM had a high SHAP value of 1.57 and a high prediction score of 0.83. The patient without NSLNM had a low SHAP value of −2.05 and a low prediction score of 0.11.

Fig. 7
figure 7

NSLNM prediction of two typical patients

Full size image

Discussion

In this study, we constructed six ML models to predict NSLNM using preoperative and intraoperative clinicopathological features and compared their performance. The XGBoost model showed the best performance, and its predictive ability was also superior to that of the nomogram. The XGBoost model was well explained through the SHAP framework.

In previous studies, LVI, grade, pathological tumor size, and molecular typing of breast cancer were often included in prediction model to predict NSLNM [39,40,41,42]. Although the inclusion of these postoperative parameters improved the prediction accuracy, the difficulty in obtaining these parameters preoperatively and intraoperatively may limit their clinical application. A previous study used clinical tumor size to establish predictive models [43]. Therefore, clinical tumor size was put into the predictive models in this study. Murata, T et al. included 804 patients with operable primary breast cancer and showed that NSLNM was more likely to occur with an SLN-positive ratio of ≥ 0.5 than with an SLN-positive ratio of < 0.5 (p = 0.024) [44]. Wang Nana et al. retrospectively analyzed 495 patients and found that patients in the SLN-positive group were more likely to develop NSLNM than those in the SLN-negative group (p < 0.001) [41]. This study also demonstrated that the SLN-positive rate was an independent predictor of NSLNM.

Some scholars [45] have found that NSLNM was closely related to the ultrasound tumor boundary and blood flow signal (p = 0.038, p = 0.036). This study had similar results, 26 patients (7.7%) with clear ultrasound tumor boundaries had NSLNM, while 83 patients (24.6%) with ambiguous ultrasound tumor boundaries and 229 patients (67.8%) with unclear ultrasound tumor boundaries had NSLNM. Patients with ambiguous or unclear tumor boundaries were more likely to develop NSLNM. In the patients with NSLNM, most patients (84.3%) showed a blood flow signal. The above parameters were not independent predictors of NSLNM, which may be attributed to the fact that SLN-negative patients were also included in this study.

Kuo YL et al. retrospectively analyzed 1496 malignant breast cancer patients and established a nomogram to predict NSLNM. The model showed good predictive performance, and the AUC value of the model was 0.738 [46], but it is not clear whether it was better than the ML model. Mi DU et al. developed an ML model to predict 3-year and 5-year disease-specific survival for oral and pharyngeal cancers and compared its performance with conventional Cox regression, showing that the ML model had better predictive performance [47]. However, no such comparison has been made in breast cancer for predicting NSLNM. In this study, for the first time, the prediction performance of NSLNM was compared between the ML model and nomogram. The results demonstrated that the AUC value of the XGBoost model was larger than that of the traditional nomogram (0.781 vs. 0.647; 0.764 vs. 0.706). Compared with traditional regression models, ML models can more accurately identify and analyze the potential relationships between different variables, and their predictive accuracy is particularly suitable for achieving individualized therapy and predictive medicine [48], which will help us better solve clinical problems.

A deep learning radiomics model has been developed to predict the risk of NSLNM. Its sensitivity for NSLNM was 98.4% (95% CI: 95.6–99.9%), and its negative predictive value was 91.7% (95% CI: 88.8–97.9%) in the validation set [17]. This model has good predictive ability, but the lack of explanation of the model makes it impossible for readers to intuitively understand the prediction results of the ML model, and its complex region of interest (ROI) drawing also limits its application for clinical breast surgeons. In this study, six powerful ML models were developed using clinicopathological features that are easy to obtain, and their predictive performance for NSLNM was compared. All models showed good predictive performance; the XGBoost model is the best. We visualized the optimal model with SHAP values and graphs. The summary charts show the effects of different variables on NSLNM, among which the SLN-positive ratio had the greatest impact on NSLNM. Compared with an SLN-positive ratio ≤ 0.5, an SLN-positive ratio > 0.5 was more likely to produce NSLNM. Two typical patients (one with NSLNM and one without NSLNM) were also explained using force diagrams.

Some studies [11,12,13] showed that 40–60% of breast cancer patients with SLNB further underwent ALND, even if no other lymph node metastasis was found. Some breast cancer patients chose to directly undergo ALND due to poor finances condition. With ALND comes the problem of lymphedema, which limits the upper limb function of breast cancer patients, leading to worse working ability and lower income, creating a vicious cycle. On the other hand, some patients were found to have NSLNM with negative SLNs (6.2% in this study), which could lead to a second surgery of the axilla. Therefore, accurate prediction of NSLNM is necessary. The XGBoost model in this study showed powerful predictive ability, which could help us avoid overtreatment or undertreatment. There is still a long way to go before this model can be applied to real-world medical settings because it still needs to be tested in different populations. In addition, developing a software application (APP) based on this model will be a difficult and time-consuming project.

Although the XGBoost model developed here can well predict NSLNM, this study has some limitations. First, this is a retrospective study conducted at a single institution. The inclusion of multicenter data would be more conducive to model validation. Second, with the exception of the group with breast invasive ductal carcinoma, few patients had other pathological types of breast cancer. If the sample size of patients with other pathological types of breast cancer can be increased, the probability of occurrence of NSLNM in different pathological types can be better compared, and the ML model developed will be more suitable for clinical practice.

Conclusion

The optimal ML model XGBoost was developed using preoperative and intraoperative clinicopathological features and was superior to the traditional nomogram in predicting NSLNM. The SHAP framework can explain how the best model works, intuitively display the influence of characteristic variables on NSLNM, realize the clinical translation of machine learning technology, and assist clinicians in making more individualized and accurate diagnosis and treatment plans.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Bray F, Laversanne M, Weiderpass E, Soerjomataram I. The ever-increasing importance of cancer as a leading cause of premature death worldwide. Cancer. 2021;127:3029–30.

    PubMed  Google Scholar 

  2. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71:209–49.

    PubMed  Google Scholar 

  3. Fisher B, Bauer M, Margolese R, Poisson R, Pilch Y, Redmond C, Fisher E, Wolmark N, Deutsch M, Montague E, et al. Five-year results of a randomized clinical trial comparing total mastectomy and segmental mastectomy with or without radiation in the treatment of breast cancer. N Engl J Med. 1985;312:665–73.

    PubMed  CAS  Google Scholar 

  4. Fisher B, Anderson S, Bryant J, Margolese RG, Deutsch M, Fisher ER, Jeong JH, Wolmark N. Twenty-year follow-up of a randomized trial comparing total mastectomy, lumpectomy, and lumpectomy plus irradiation for the treatment of invasive breast cancer. N Engl J Med. 2002;347:1233–41.

    PubMed  Google Scholar 

  5. Krag DN, Anderson SJ, Julian TB, Brown AM, Harlow SP, Costantino JP, Ashikaga T, Weaver DL, Mamounas EP, Jalovec LM, et al. Sentinel-lymph-node resection compared with conventional axillary-lymph-node dissection in clinically node-negative patients with breast cancer: overall survival findings from the NSABP B-32 randomised phase 3 trial. Lancet Oncol. 2010;11:927–33.

    PubMed  PubMed Central  Google Scholar 

  6. Ashikaga T, Krag DN, Land SR, Julian TB, Anderson SJ, Brown AM, Skelly JM, Harlow SP, Weaver DL, Mamounas EP, et al. Morbidity results from the NSABP B-32 trial comparing sentinel lymph node dissection versus axillary dissection. J Surg Oncol. 2010;102:111–8.

    PubMed  PubMed Central  Google Scholar 

  7. Che Bakri NA, Kwasnicki RM, Khan N, Ghandour O, Lee A, Grant Y, et al. Impact of axillary lymph node dissection and sentinel lymph node biopsy on upper limb morbidity in breast cancer patients: a systematic review and meta-analysis. Ann Surg. 2022;277:572–80.

  8. Donker M, van Tienhoven G, Straver ME, Meijnen P, van de Velde CJ, Mansel RE, Cataliotti L, Westenberg AH, Klinkenbijl JH, Orzalesi L, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981–22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol. 2014;15:1303–10.

    PubMed  PubMed Central  Google Scholar 

  9. Giuliano AE, Ballman K, McCall L, Beitsch P, Whitworth PW, Blumencranz P, Leitch AM, Saha S, Morrow M, Hunt KK. Locoregional recurrence after sentinel lymph node dissection with or without axillary dissection in patients with sentinel lymph node metastases: long-term follow-up from the American College of Surgeons Oncology Group (Alliance) ACOSOG Z0011 randomized trial. Ann Surg. 2016;264:413–20.

    PubMed  Google Scholar 

  10. Giuliano AE, Ballman KV, McCall L, Beitsch PD, Brennan MB, Kelemen PR, Ollila DW, Hansen NM, Whitworth PW, Blumencranz PW, et al. Effect of axillary dissection vs no axillary dissection on 10-year overall survival among women with invasive breast cancer and sentinel node metastasis: the ACOSOG Z0011 (Alliance) randomized clinical trial. JAMA. 2017;318:918–26.

    PubMed  PubMed Central  Google Scholar 

  11. Cho J, Han W, Lee JW, Ko E, Kang SY, Jung SY, Kim EK, Moon WK, Cho N, Park IA, et al. A scoring system to predict nonsentinel lymph node status in breast cancer patients with metastatic sentinel lymph nodes: a comparison with other scoring systems. Ann Surg Oncol. 2008;15:2278–86.

    PubMed  Google Scholar 

  12. Wong SL, Edwards MJ, Chao C, Tuttle TM, Noyes RD, Woo C, Cerrito PB, McMasters KM. University of Louisville Breast Cancer Sentinel Lymph Node Study G: Predicting the status of the nonsentinel axillary nodes: a multicenter study. Arch Surg. 2001;136:563–8.

    PubMed  CAS  Google Scholar 

  13. Sanjuan A, Escaramis G, Vidal-Sicart S, Illa M, Zanon G, Pahisa J, Rubi S, Velasco M, Santamaria G, Farrus B, et al. Predicting non-sentinel lymph node status in breast cancer patients with sentinel lymph node involvement: evaluation of two scoring systems. Breast J. 2010;16:134–40.

    PubMed  Google Scholar 

  14. Komura D, Ishikawa S. Machine learning approaches for pathologic diagnosis. Virchows Arch. 2019;475:131–8.

    PubMed  CAS  Google Scholar 

  15. Ho DSW, Schierding W, Wake M, Saffery R, O’Sullivan J. Machine learning SNP based prediction for precision medicine. Front Genet. 2019;10:267.

    PubMed  PubMed Central  CAS  Google Scholar 

  16. Handelman GS, Kok HK, Chandra RV, Razavi AH, Lee MJ, Asadi H. eDoctor: machine learning and the future of medicine. J Intern Med. 2018;284:603–19.

    PubMed  CAS  Google Scholar 

  17. Guo X, Liu Z, Sun C, Zhang L, Wang Y, Li Z, Shi J, Wu T, Cui H, Zhang J, et al. Deep learning radiomics of ultrasonography: identifying the risk of axillary non-sentinel lymph node involvement in primary breast cancer. EBioMedicine. 2020;60:103018.

    PubMed  PubMed Central  Google Scholar 

  18. Yang ZB, Huang Z, Wang SL, Tang Y, Jing H, Wang JY, Zhang JH, Yang Y, Song YW, Fang H, et al. A nomogram to predict non-sentinel lymph node metastasis for breast cancer patients with positive axillary sentinel lymph node. Zhonghua Zhong Liu Za Zhi. 2020;42:653–9.

    PubMed  CAS  Google Scholar 

  19. Lundberg S, Lee SI: A unified approach to interpreting model predictions. In Nips. 2017

  20. Joseph A. Shapley regressions: a framework for statistical inference on machine learning models. 2019.

    Google Scholar 

  21. Park YH, Karantza V, Calhoun SR, Park S, Lee S, Kim JY, Yu JH, Kim SW, Lee JE, Nam SJ, et al. Prevalence, treatment patterns, and prognosis of low estrogen receptor-positive (1% to 10%) breast cancer: a single’s experience in Korea. Breast Cancer Res Treat. 2021;189:653–63.

    PubMed  CAS  Google Scholar 

  22. Tsang JYS, Tse GM. Molecular classification of breast cancer. Adv Anat Pathol. 2020;27:27–35.

    PubMed  CAS  Google Scholar 

  23. Tarantino P, Hamilton E, Tolaney SM, Cortes J, Morganti S, Ferraro E, Marra A, Viale G, Trapani D, Cardoso F, et al. HER2-low breast cancer: pathological and clinical landscape. J Clin Oncol. 2020;38:1951–62.

    PubMed  CAS  Google Scholar 

  24. Ignatiadis M, Buyse M, Sotiriou C. St Gallen International Expert Consensus on the primary therapy of early breast cancer: an invaluable tool for physicians and scientists. Ann Oncol. 2015;26:1519–20.

    PubMed  CAS  Google Scholar 

  25. Giuliano AE, Edge SB, Hortobagyi GN. Eighth edition of the AJCC cancer staging manual: breast cancer. Ann Surg Oncol. 2018;25:1783–5.

    PubMed  Google Scholar 

  26. AlJame M, Ahmad I, Imtiaz A, Mohammed A. Ensemble learning model for diagnosing COVID-19 from routine blood tests. Inform Med Unlocked. 2020;21:100449.

    PubMed  PubMed Central  Google Scholar 

  27. Nick TG, Campbell KM. Logistic regression. Methods Mol Biol. 2007;404:273–301.

    PubMed  CAS  Google Scholar 

  28. Noble WS. What is a support vector machine? Nat Biotechnol. 2006;24:1565–7.

    PubMed  CAS  Google Scholar 

  29. Salvador-Meneses J, Ruiz-Chavez Z, Garcia-Rodriguez J. Compressed kNN: k-nearest neighbors with data compression. Entropy (Basel). 2019;21:234.

  30. Jiang H, Mao H, Lu H, Lin P, Garry W, Lu H, Yang G, Rainer TH, Chen X. Machine learning-based models to support decision-making in emergency department triage for patients with suspected cardiovascular disease. Int J Med Inform. 2021;145:104326.

    PubMed  Google Scholar 

  31. Jun S. Evolutionary algorithm for improving decision tree with global discretization in manufacturing. Sensors (Basel). 2021;21:2849.

  32. Yuan KC, Tsai LW, Lee KH, Cheng YW, Hsu SC, Lo YS, Chen RJ. The development an artificial intelligence algorithm for early sepsis diagnosis in the intensive care unit. Int J Med Inform. 2020;141:104176.

    PubMed  Google Scholar 

  33. Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol. 2010;63:938–9 (author reply 939).

    PubMed  Google Scholar 

  34. Wang Y, Lang J, Zuo JZ, Dong Y, Hu Z, Xu X, Zhang Y, Wang Q, Yang L, Wong STC, et al. The radiomic-clinical model using the SHAP method for assessing the treatment response of whole-brain radiotherapy: a multicentric study. Eur Radiol. 2022;32:8737–47.

  35. Ladbury C, Li R, Shiao J, Liu J, Cristea M, Han E, Dellinger T, Lee S, Wang E, Fisher C, et al. Characterizing impact of positive lymph node number in endometrial cancer using machine-learning: a better prognostic indicator than FIGO staging? Gynecol Oncol. 2022;164:39–45.

    PubMed  CAS  Google Scholar 

  36. Chen X, Li Y, Li X, Cao X, Xiang Y, Xia W, Li J, Gao M, Sun Y, Liu K, et al. An interpretable machine learning prognostic system for locoregionally advanced nasopharyngeal carcinoma based on tumor burden features. Oral Oncol. 2021;118:105335.

    PubMed  Google Scholar 

  37. Kim JH. Multicollinearity and misleading statistical results. Korean J Anesthesiol. 2019;72:558–69.

    PubMed  PubMed Central  Google Scholar 

  38. Dong LF, Xu SY, Long JP, Wan F, Chen YD. Role of number of sentinel nodes in predicting non-sentinel node metastasis in breast cancer. J Int Med Res. 2018;46:828–35.

    PubMed  Google Scholar 

  39. Wu X, Guo Y, Sa Y, Song Y, Li X, Lv Y, Xing D, Sun Y, Cong Y, Yu H, Jiang W. Contrast-enhanced spectral mammography-based prediction of non-sentinel lymph node metastasis and axillary tumor burden in patients with breast cancer. Front Oncol. 2022;12:823897.

    PubMed  PubMed Central  Google Scholar 

  40. Oz B, Akcan A, Dogan S, Abdulrezzak U, Aslan D, Sozuer E, Emek E, Akyuz M, Elmali F, Ok E. Prediction of nonsentinel lymph node metastasis in breast cancer patients with one or two positive sentinel lymph nodes. Asian J Surg. 2018;41:12–9.

    PubMed  Google Scholar 

  41. Wang NN, Yang ZJ, Wang X, Chen LX, Zhao HM, Cao WF, Zhang B. A mathematical prediction model incorporating molecular subtype for risk of non-sentinel lymph node metastasis in sentinel lymph node-positive breast cancer patients: a retrospective analysis and nomogram development. Breast Cancer. 2018;25:629–38.

    PubMed  Google Scholar 

  42. Tapia G, Ying V, Di Re A, Stellin A, Cai TY, Warrier S. Predicting non-sentinel lymph node metastasis in Australian breast cancer patients: are the nomograms still useful in the post-Z0011 era? ANZ J Surg. 2019;89:712–7.

    PubMed  Google Scholar 

  43. Murata T, Watase C, Shiino S, Jimbo K, Iwamoto E, Yoshida M, Takayama S, Suto A. Development and validation of a preoperative scoring system to distinguish between nonadvanced and advanced axillary lymph node metastasis in patients with early-stage breast cancer. Clin Breast Cancer. 2021;21:e302–11.

    PubMed  Google Scholar 

  44. Murata T, Watase C, Shiino S, Kurita A, Ogawa A, Jimbo K, Iwamoto E, Yoshida M, Takayama S, Suto A. Development and validation of a pre- and intra-operative scoring system that distinguishes between non-advanced and advanced axillary lymph node metastasis in breast cancer with positive sentinel lymph nodes: a retrospective study. World J Surg Oncol. 2022;20:314.

    PubMed  PubMed Central  Google Scholar 

  45. Cong Y, Wang S, Zou H, Zhu S, Wang X, Cao J, Wang J, Liu Y, Qiao G. Imaging predictors for nonsentinel lymph node metastases in breast cancer patients. Breast Care (Basel). 2020;15:372–9.

    PubMed  Google Scholar 

  46. Kuo YL, Chen WC, Yao WJ, Cheng L, Hsu HP, Lai HW, Kuo SJ, Chen DR, Chang TW. Validation of Memorial Sloan-Kettering Cancer Center nomogram for prediction of non-sentinel lymph node metastasis in sentinel lymph node positive breast cancer patients an international comparison. Int J Surg. 2013;11:538–43.

    PubMed  Google Scholar 

  47. Du M, Haag DG, Lynch JW, Mittinty MN. Comparison of the tree-based machine learning algorithms to Cox regression in predicting the survival of oral and pharyngeal cancers: analyses based on SEER database. Cancers (Basel). 2020;12:2802.

  48. Cruz JA, Wishart DS. Applications of machine learning in cancer prediction and prognosis. Cancer Inform. 2007;2:59–77.

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Thanks for the data support provided by Harbin Medical University Cancer Hospital.

Funding

No funding.

Author information

Author notes

  1. Yuting Xiu, Cong Jiang, and Shiyuan Zhang contributed equally.

Authors and Affiliations

  1. Department of Breast Surgery, Harbin Medical University Cancer Hospital, Harbin, 150086, China

    Yuting Xiu, Cong Jiang, Shiyuan Zhang, Xiao Yu, Kun Qiao & Yuanxi Huang

Authors
  1. Yuting Xiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiyuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kun Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuanxi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yuting Xiu and Yuanxi Huang conceptualized and designed the work.Cong Jiang, Shiyuan Zhang and Xiao Yu collected all the data. Yuting Xiu and Kun Qiao drafted and analyzed the manuscript. All authors contributed to the article and approved the submitted version.

Corresponding authors

Correspondence to Kun Qiao or Yuanxi Huang.

Ethics declarations

Ethics approval and consent to participate

It complies with the World Medical Association Declaration of Helsinki in 1964 and subsequently amended versions. An informed consent form was signed before the treatment.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Supplementary figures: Fig. S1. Recursive feature elimination. Fig. S2. The nomogram prediction model based on multi-factor Logistic regression analysis. Supplementary table: Table S1. Performance comparison between XGBoost model and nomogram.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiu, Y., Jiang, C., Zhang, S. et al. Prediction of nonsentinel lymph node metastasis in breast cancer patients based on machine learning. World J Surg Onc 21, 244 (2023). https://doi.org/10.1186/s12957-023-03109-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12957-023-03109-3

Keywords

World Journal of Surgical Oncology

ISSN: 1477-7819

Contact us