Predictive nomograms for risk and prognostic factors in metastatic bladder cancer: a population-based study

Introduction

Bladder cancer is a malignant disease with high morbidity and mortality rate, which ranks among the top ten cancers in the world, and the sixth in male population. Despite recent advances in surgical techniques and drug therapies, according to a global survey in 2020, there were 573,278 new cases of bladder cancer and 212,536 new deaths reported worldwide (1), causing a global economic burden on healthcare industry. It has been widely recognized that the high incidence of bladder cancer is associated with smoking and secondhand smoke exposure, and the degree of smoking in patients is also linked linearly to tumor malignancy (2). Studies have shown that nearly 4% of all patients diagnosed with bladder cancer are diagnosed as metastatic bladder cancer (MBC) at initial diagnosis (3). MBC means a worse prognosis, with a median overall survival (OS) of only 13–15 months, even when treated with rigorous chemotherapy regimens (4).

Tumor metastasis is a process that involves multiple mechanisms in the final stages of the tumor process and also ultimately leads to the death of patients (5). The main metastatic sites of bladder cancer are bone, lung, liver and brain, etc. (6). Bone metastases are the most common site of MBC, the majority of patients with bladder cancer can be evaluated by routine chest , abdomen and pelvic enhancement Computed Tomography (CT), but sometimes there exists inadequate staging, so positron emission tomography-computed tomography (PET-CT) is increasingly used in the detection of bladder cancer, especially in patients with muscle-invasive bladder cancer and recurrence after radical cystectomy (7). In the evaluation of the prognosis of patients with MBC, there have been nomograms constructed for bone metastasis and brain metastasis to predict the prognosis of patients (6,8), however, predictions for a single metastatic site have limitations, such as the difficulty of applying them broadly to patients with bladder cancer, and they lack application of common clinical indicators. Therefore, the use of common clinical indicators to evaluate the risk and prognosis of MBC is currently needed for providing guidelines to clinicians.

We obtained clinical data of bladder cancer from the Surveillance, Epidemiology, and End Results (SEER) database. After screening, a total of 22,788 patients were included in our study. Two practical nomograms were constructed by clinical information, the risk nomogram for predicting the risk of bladder cancer metastasis and the prognosis nomogram for predicting 1-, 2- and 3-year survival in patients with MBC. Two prediction models have been validated to have favorable clinical utility through the receiver operating characteristic (ROC), decision curve analysis (DCA) curves, clinical impact curves (CIC) and Kaplan-Meier (K-M) survival curves. We present this article in accordance with the TRIPOD reporting checklist (available at https://tcr.amegroups.com/article/view/10.21037/tcr-23-1229/rc).

Methods

Ethical statement

The ethical approval of this study was exempted by the Ethics Committee of the First Affiliated Hospital of Nanchang University as the data were from the publicly accessible database, SEER. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Patients

We downloaded clinical information from SEER state software (version 8.4.1) of patients with bladder cancer registered from 2010 to 2019. The inclusion criteria were: (I) bladder cancer was the only primary carcinoma; (II) histology type was known; and (III) definitive site of metastasis. Exclusion principles were: (I) unknown cause of death and survival time; (II) unknown race; (III) unknown tumor size; (IV) undetermined T and N stage and grade; (V) lack of surgical, radiotherapy and chemotherapy information. Finally, there were 22,788 patients included in the cohort to study the risk factors of metastasis in bladder cancer patients and to establish a risk nomogram. For the exploration of prognostic factors for MBC, a total of 1,150 patients were enrolled in the whole cohort.

In the analysis of metastatic risk, we included the following clinical information: age at diagnosis, sex, race, histological type, tumor size, grade, T-stage (AJCC 7th edition), N-stage (AJCC 7th edition), primary site. Within the analysis of prognostic influences, besides the above factors, we included treatment and metastatic information, such as bone, lung, liver and brain metastases, surgery, radiotherapy and chemotherapy information.

Statistical analysis and nomogram construction

All statistical analyses were performed by SPSS (version 25.0), x-tile software (version 3.6.1), and software packages (rms, pROC, ggDCA, ggplot and rmda) in R software (version 4.2.3). We used x-tile software to calculate the optimal cut-off values for tumor size and age that affect the prognosis of bladder cancer patients, the results are shown in Figure 1. Chi-squared test or Fisher exact test were deployed to compare categorical data. Through univariate logistic regression analysis, variables with a P<0.05 were included in multivariate logistic regression analysis, the odds ratios (OR) and 95% confidence intervals (CI) were calculated, then the significant risk factors were screened out. Next, risk factors for prognosis in MBC patients were analyzed. All MBC patients were randomly divided into training (n=806) and validation (n=344) cohorts according to the ratio of 7:3 (9). In the training cohort, variables with P<0.05 in univariate cox regression analysis were included in the multivariate cox regression analysis to identify significant prognostic factors, the hazard ratios (HR) and 95% CI were calculated. Finally, we established two nomograms based on risk factors and prognostic factors to predict the risk and OS of MBC. In the risk nomogram, its accuracy and discrimination were evaluated by ROC curves (10), bootstrapping (1,000 resamples) calibration curves (11), and the area under curve (AUC) of ROC. Its clinical utility was assessed by clinical DCA and CIC (12). In the prognostic nomogram, accuracy and discrimination were assessed by ROC curves and calibration curves for 1-, 2-, and 3-year, respectively, and AUC. Its clinical utility was assessed by DCA curves and KM curves. Two tailed P values ≤0.05 was considered statistically significant.

Figure 1 The appropriate cut-off values of age and tumor size was assessed by x-tile software. (A,B) The appropriate cut-off values of age were 72 and 84 years old; (C,D) the appropriate cut-off value of tumor size was 30 mm.

Results

Patient characteristics

After the screening process, date of a total of 22,788 bladder cancer patients were obtained from the SEER database. Among them, 1,150 patients were diagnosed as MBC, 806 cases were grouped into training cohort and 344 cases as validation cohort. Among bladder cancer patients with and without metastasis, there was a difference in the percentage of all risk and prognostic factors except surgery. Whites (88.9%) accounted for the largest proportion of all bladder cancer patients, with little difference in the proportion of transitional cell carcinoma and papillary transitional cell carcinoma (44.8% vs. 48.9%). In terms of treatment style, the vast majority of patients underwent local tumor excision (LTE). Bone metastasis accounted for the highest percentage (36.3%), brain metastasis accounted for the least (2.1%) of patients with MBC. The specific clinical information of patients are shown in Table 1.

Analysis of independent risk and prognostic factors

In the univariate logistic regression analysis, a total of nine clinical factors were associated with metastasis, while in the multivariate logistic regression analysis, age, histological type, T stage, N stage and tumor size were found to be associated with metastasis, the analysis results are shown in Table 2. In order to construct the prognosis nomogram of MBC, we divided the patients into training cohort and validation cohort, the Chi-squared test and Fisher’s exact test showed that none of the 16 factors were different between the two groups (Table 3). After inclusion of 16 prognostic factors in the training cohort, the prognostic factor of P<0.05 such as age, histological type, tumor size, T stage, N stage, chemotherapy, bone metastasis, liver metastasis and lung metastasis were included in the multivariate cox regression analysis by univariate cox regression analysis. Finally, age, histological type, chemotherapy, bone metastasis, liver metastasis, and lung metastasis were considered to be significantly associated with MBC patient prognosis. The analysis results are shown in Table 4.

Risk nomogram construction and validation

With the results of multivariate logistic regression analysis, we constructed the final risk nomogram of bladder cancer, as shown in Figure 2. The probability of prediction can be obtained by summing up the scores obtained through the projection of each predictive factor. The AUC of the ROC analysis reached 0.80, demonstrating the excellent discriminatory ability of the risk nomogram (Figure 3A). The high overlap between the observed and predicted results in the calibration curve also showed the good reliability of the nomogram (Figure 3B). In the evaluation of clinical effect, both DCA and CIC curves showed that patients could get excellent clinical net benefit by this nomogram (Figure 3C,3D).

Figure 2 Nomogram to estimate the risk of MBC. TCC, transitional cell carcinoma; PTCC, papillary transitional cell carcinoma; MBC, metastatic bladder cancer; AJCC, American Joint Committee on Cancer.

Figure 3 ROC, calibration, DCA curves and CIC of the nomogram for the risk of MBC. (A) The ROC curve of the risk model; (B) the calibration curve of the risk model; (C) the DCA curve of the risk model; (D) the CIC curve of the risk model. ROC, receiver operating characteristic; DCA, decision curve analysis; CIC, clinical impact curves; MBC, metastatic bladder cancer.

Prognostic nomogram construction and validation

Based on the results obtained from the multivariate cox regression analysis of the training cohort, we constructed the prognostic nomogram of MBC (Figure 4). The survival rate of prediction can be obtained by summing up the scores obtained through the projection of each predictive factor. The AUC of the ROC curve analysis of the nomogram revealed 1-, 2- and 3-year OS respectively reached 0.74, 0.71 and 0.71 in the training cohort (Figure 5A), 1-, 2- and 3-year OS respectively reached 0.81, 0.77 and 0.77 in the validation cohort (Figure 5B). The calibration curve of nomogram revealed an excellent consistency between actual observation and prediction both training cohort and validation cohort (Figure 5C-5H). As shown in Figure 6A-6C, the nomogram demonstrated a significant net benefit of 1-, 2- and 3-year OS, indicating its great clinical practical value in predicting OS of MBC in training cohort, the same results were also shown in validation cohort (Figure 6D-6F). Similarly, the prognostic risk factors by nomogram showed differences in the prognosis of all patients with MBC, training cohort and validation cohort in the KM survival analysis, further validated the clinical utility of the prognostic nomogram (Figure 7A-7C).

Figure 4 Nomogram for predicting the OS of patients with MBC. MBC, metastatic bladder cancer; TCC, transitional cell carcinoma; PTCC, papillary transitional cell carcinoma; OS, overall survival.

Figure 5 ROC and calibration curves of the nomogram for predicting the overall survival of MBC. (A) The ROC curves of the prognosis nomogram in training cohort; (B) the ROC curves of the prognosis nomogram in validation cohort; (C-E) the calibration curves of the prognosis nomogram of 1-, 2- and 3-year OS in training cohort; (F-H) the calibration curves of the prognosis nomogram of 1-, 2- and 3-year OS in validation cohort. AUC, area under the curve; OS, overall survival; ROC, receiver operating characteristic; MBC, metastatic bladder cancer.

Figure 6 DCA curves of the nomogram for predicting the overall survival of MBC. (A-C) The DCA curves of the prognosis nomogram of 1-, 2- and 3-year OS in training cohort; (D-F) the DCA curves of the prognosis nomogram of 1-, 2- and 3-year OS in validation cohort. DCA, decision curve analysis; MBC, metastatic bladder cancer; OS, overall survival.

Figure 7 K-M curves of OS for patients in low-risk and high-risk groups. (A) The K-M curve of the whole MBC patients; (B) the K-M curve of training cohort; (C) the K-M curve of validation cohort. K-M, Kaplan-Meier; OS, overall survival; MBC, metastatic bladder cancer.

Discussion

As a highly heterogeneous disease, bladder cancer is mainly classified into non-muscle invasive bladder cancer (NMIBC), muscle invasive bladder cancer (MIBC) and MBC, with different subtypes also imply different clinical outcomes (13). The main treatment for patients with MIBC is neoadjuvant therapy followed by radical cystectomy, and urinary diversion, or a bladder-sparing protocol, such as chemoradiation or partial cystectomy (14). Treatment of MIBC primarily aims at preventing local recurrence (15). Even so, approximately half of the postoperative patients still experience distant metastasis (16,17), necessitating more management and monitoring measures for the treatment of MBC patients and those who experience metastasis. There are currently prognostic nomograms constructed based on bone metastasis (8) and brain metastasis, but comprehensive nomograms constructed based on clinical information may lead to better management of MBC patients. We screened 22,788 patients with bladder cancer from the SEER database, including 1,150 patients with MBC, independent clinical risk factors related to risk and prognosis were identified by logistic and cox regression analyses, finally, age, histological type, tumor size, T stage, and N stage were considered as independent risk factors for MBC, while age, histological type, chemotherapy, bone, liver and lung metastases were independent risk factors for the prognosis of MBC; ROC, CIC and DCA curves, etc. showed the powerful predictive ability and clinical application ability of the prediction model, which enable doctors to provide better clinical consultation and follow-up strategies for patients in clinical practice.

In all patients with bladder cancer, especially MIBC, the possibility of lymph node metastasis is very high (18), and it means a worse prognosis. Although the exact surgical procedure for lymphadenectomy is controversial (19), there is no doubt that Lymphadenectomy can be beneficial to patients. It is very important to evaluate the patient’s lymph node metastasis status by imaging methods before surgery (20), however, a quarter of patients are still found with lymph node metastasis after undergoing surgery (21). Therefore, lymph node metastasis is critical for treatment assessment, and in our study, lymph node staging had the highest score for assessing patients’ risk of metastasis. In addition, histological type, T stage, and tumor size also play a role in predicting the risk of metastasis.

Unfortunately, bladder cancer is diagnosed at an older age than other types of cancer (22). Most elderly patients pose a substantial challenge to treatment because of the high incidence of complications and frail status. With the aging of society, the number of patients with bladder cancer will also increase and it is noteworthy that cardiovascular disease is a common illness among elderly patients, however a study found that cardiovascular disease was an independent protective factor for bladder cancer, but this effect was not observed in high-risk tumors (23), therefore, for such high risk tumor, to construct an effective assessment tool can help physicians make better decisions (24). In our analysis, the best cut-off value of age was obtained by x-tile software and age acted as the predictive factor of prognosis in patients with MBC. It showed the association of older age with worse prognosis. Therefore, geriatric assessment of elderly patients is recommended as a practice, but it has not been validated on a large-scale study and we were unable to obtain relevant information from the SEER database. Similarly, in the study of metastatic patterns of MBC (25), the largest proportion of patients with bone metastases and the poorest prognosis of patients with liver metastases were observed, and few patients with brain metastases were not included in our model to assess the prognosis of MBC.

The prognosis for patients with untreated bladder cancer is very poor, with low survival rates even at 1 year (26), in a multicenter study of bladder carcinoma in situ, 70 years old was used as cut-off value, it was found that patients over 70 years old had an increased risk of recurrence and progression with a poor recurrence free survival (27), but in our study, multivariate logistic regression analysis showed that patients over 84 years old had a reduced risk of metastasis, but multivariate cox regression analysis showed that with increasing age, OS became worsened. According to treatment guidelines, the first-line therapy for MBC is cisplatin-based cytotoxic chemotherapy (28). Vinflunine is the only approved second-line therapy drug in Europe (29), in addition, second-line immunotherapy with programmed cell death protein 1/programmed cell death-ligand 1 (PD-1/PD-L1) checkpoint inhibitors has also been established (30). In general, chemotherapy has brought some benefits to the survival of patients with bladder cancer (31), however, the proportion of patients without chemotherapy in our study population with MBC reached 76.6%, which also had the most significant impact on prognosis, such a low chemotherapy rate also means a worse prognosis for these patients, which is consistent with the above results.

In conclusion, our nomograms are constructed based on clinical information, which have the convenience for clinical application, and their differentiation and validity have been verified by ROC, DCA and other curves, which can provide better reference for patients by judging the risk of metastasis and prognosis. Inevitably, our study has certain limitations. Firstly, this was a retrospective study, excluded many cases with some missing data, which may cause bias. Secondly, there are also clinical factors such as laboratory test results affecting the prognosis of MBC patients, in addition, some diagnostic tools based on artificial intelligence (AI) technology that have shown perfect results in cystoscopy, urine testing, and imaging analysis can be used for MBC patients (32), unfortunately, these data are not available in the SEER database. Thirdly, the efficacy of nomogram has only been internally validated, and we need more data of external clinical application to test its performance in the future.

Conclusions

In summary, we constructed risk and prognosis nomograms for MBC patients based on clinical risk factors and demonstrated their perfect utility through multiple validations. It could be used to provide counseling recommendations for patients and follow-up guidance for clinicians.

Acknowledgments

Funding: This study was supported by the Natural Science Foundation of Jiangxi Province (No. 20202BAB206014).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://tcr.amegroups.com/article/view/10.21037/tcr-23-1229/rc

Peer Review File: Available at https://tcr.amegroups.com/article/view/10.21037/tcr-23-1229/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tcr.amegroups.com/article/view/10.21037/tcr-23-1229/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The SEER Program has a strict policy to protect patient privacy, and all data were de-identified prior to analysis. The ethical approval of this study was exempted by the Ethics Committee of the First Affiliated Hospital of Nanchang University as the data were from the publicly accessible database, SEER. No informed consent was required for this retrospective study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin 2021;71:209-49. [Crossref] [PubMed]
2. Cumberbatch MG, Rota M, Catto JW, et al. The Role of Tobacco Smoke in Bladder and Kidney Carcinogenesis: A Comparison of Exposures and Meta-analysis of Incidence and Mortality Risks. Eur Urol 2016;70:458-66. [Crossref] [PubMed]
3. Lenis AT, Lec PM, Chamie K, et al. Bladder Cancer: A Review. JAMA 2020;324:1980-91. [Crossref] [PubMed]
4. Tsai TH, Su PJ, Huang SY, et al. The prognostic significance of histologic variant on survival outcomes in patients with metastatic urothelial carcinoma receiving immune checkpoint inhibitor therapy. BMC Cancer 2023;23:871. [Crossref] [PubMed]
5. Liu QL, Luo M, Huang C, et al. Epigenetic Regulation of Epithelial to Mesenchymal Transition in the Cancer Metastatic Cascade: Implications for Cancer Therapy. Front Oncol 2021;11:657546. [Crossref] [PubMed]
6. Yao Z, Zheng Z, Ke W, et al. Prognostic nomogram for bladder cancer with brain metastases: a National Cancer Database analysis. J Transl Med 2019;17:411. [Crossref] [PubMed]
7. Bouchelouche K. PET/CT in Bladder Cancer: An Update. Semin Nucl Med 2022;52:475-85. [Crossref] [PubMed]
8. Fan Z, Huang Z, Hu C, et al. Risk factors and nomogram for newly diagnosis of bone metastasis in bladder cancer: A SEER-based study. Medicine (Baltimore) 2020;99:e22675. [Crossref] [PubMed]
9. Lei Z, Li J, Wu D, et al. Nomogram for Preoperative Estimation of Microvascular Invasion Risk in Hepatitis B Virus-Related Hepatocellular Carcinoma Within the Milan Criteria. JAMA Surg 2016;151:356-63. [Crossref] [PubMed]
10. Verbakel JY, Steyerberg EW, Uno H, et al. ROC curves for clinical prediction models part 1. ROC plots showed no added value above the AUC when evaluating the performance of clinical prediction models. J Clin Epidemiol 2020;126:207-16. [Crossref] [PubMed]
11. Kandi S, Savaryn JP, Ji QC, et al. Use of in-sample calibration curve approach for quantification of peptides with high-resolution mass spectrometry. Rapid Commun Mass Spectrom 2022;36:e9377. [Crossref] [PubMed]
12. Vickers AJ, Holland F. Decision curve analysis to evaluate the clinical benefit of prediction models. Spine J 2021;21:1643-8. [Crossref] [PubMed]
13. Kirkali Z, Chan T, Manoharan M, et al. Bladder cancer: epidemiology, staging and grading, and diagnosis. Urology 2005;66:4-34. [Crossref] [PubMed]
14. Patel VG, Oh WK, Galsky MD. Treatment of muscle-invasive and advanced bladder cancer in 2020. CA Cancer J Clin 2020;70:404-23. [Crossref] [PubMed]
15. Nadal R, Bellmunt J. Management of metastatic bladder cancer. Cancer Treat Rev 2019;76:10-21. [Crossref] [PubMed]
16. Adjuvant chemotherapy in invasive bladder cancer: a systematic review and meta-analysis of individual patient data Advanced Bladder Cancer (ABC) Meta-analysis Collaboration. Eur Urol 2005;48:189-199. [Crossref] [PubMed]
17. Stein JP, Lieskovsky G, Cote R, et al. Radical cystectomy in the treatment of invasive bladder cancer: long-term results in 1,054 patients. J Clin Oncol 2001;19:666-75. [Crossref] [PubMed]
18. Del Bene G, Calabrò F, Giannarelli D, et al. Neoadjuvant vs. Adjuvant Chemotherapy in Muscle Invasive Bladder Cancer (MIBC): Analysis From the RISC Database. Front Oncol 2018;8:463. [Crossref] [PubMed]
19. Gschwend JE, Heck MM, Lehmann J, et al. Extended Versus Limited Lymph Node Dissection in Bladder Cancer Patients Undergoing Radical Cystectomy: Survival Results from a Prospective, Randomized Trial. Eur Urol 2019;75:604-11. [Crossref] [PubMed]
20. Mirmomen SM, Shinagare AB, Williams KE, et al. Preoperative imaging for locoregional staging of bladder cancer. Abdom Radiol (NY) 2019;44:3843-57. [Crossref] [PubMed]
21. Meeks JJ, Bellmunt J, Bochner BH, et al. A systematic review of neoadjuvant and adjuvant chemotherapy for muscle-invasive bladder cancer. Eur Urol 2012;62:523-33. [Crossref] [PubMed]
22. VanderWalde NA, Chi MT, Hurria A, et al. Treatment of muscle invasive bladder cancer in the elderly: navigating the trade-offs of risk and benefit. World J Urol 2016;34:3-11. [Crossref] [PubMed]
23. Barone B, Finati M, Cinelli F, et al. Bladder Cancer and Risk Factors: Data from a Multi-Institutional Long-Term Analysis on Cardiovascular Disease and Cancer Incidence. J Pers Med 2023;13:512. [Crossref] [PubMed]
24. Hatakeyama S, Narita S, Okita K, et al. Management of bladder cancer in older patients. Jpn J Clin Oncol 2022;52:203-13. [Crossref] [PubMed]
25. Shou J, Zhang Q, Zhang D. The prognostic effect of metastasis patterns on overall survival in patients with distant metastatic bladder cancer: a SEER population-based analysis. World J Urol 2021;39:4151-8. [Crossref] [PubMed]
26. Dobruch J, Oszczudłowski M. Bladder Cancer: Current Challenges and Future Directions. Medicina (Kaunas) 2021;57:749. [Crossref] [PubMed]
27. Ferro M, Chiujdea S, Musi G, et al. Impact of Age on Outcomes of Patients With Pure Carcinoma In Situ of the Bladder: Multi-Institutional Cohort Analysis. Clin Genitourin Cancer 2022;20:e166-72. [Crossref] [PubMed]
28. Bellmunt J, Petrylak DP. New therapeutic challenges in advanced bladder cancer. Semin Oncol 2012;39:598-607. [Crossref] [PubMed]
29. Bellmunt J, Théodore C, Demkov T, et al. Phase III trial of vinflunine plus best supportive care compared with best supportive care alone after a platinum-containing regimen in patients with advanced transitional cell carcinoma of the urothelial tract. J Clin Oncol 2009;27:4454-61. [Crossref] [PubMed]
30. Bellmunt J, de Wit R, Vaughn DJ, et al. Pembrolizumab as Second-Line Therapy for Advanced Urothelial Carcinoma. N Engl J Med 2017;376:1015-26. [Crossref] [PubMed]
31. Cathomas R, Lorch A, Bruins HM, et al. The 2021 Updated European Association of Urology Guidelines on Metastatic Urothelial Carcinoma. Eur Urol 2022;81:95-103. [Crossref] [PubMed]
32. Ferro M, Falagario UG, Barone B, et al. Artificial Intelligence in the Advanced Diagnosis of Bladder Cancer-Comprehensive Literature Review and Future Advancement. Diagnostics (Basel) 2023;13:2308. [Crossref] [PubMed]