Development and validation of a clinical prediction model to estimate the probability of malignancy in solid pancreatic lesions and explore its value in the atypical diagnostic category after endoscopic ultrasound-guided fine-needle aspiration biopsy (EUS-FNA)

Introduction

According to the imaging features, the pancreatic mass can be divided into solid and cystic lesions (1). During the solid lesions, malignant are more common (67.3%) (2). And many patients with non-malignant lesions such as autoimmune pancreatitis share characteristics, such as jaundice, new-onset diabetes, and elevated serum tumor markers, with patients with pancreatic cancer (3), which makes the identification of nature of pancreatic lesions difficult. For the solid occupancy, the use of endoscopic ultrasound-guided fine-needle aspiration biopsy (EUS-FNA) is recommended. However, due to technical and cost, its current implementation rate in China is still low (4). Even among patients undergone EUS-FNA, 8% to 10% of them still present with atypical diagnosis due to manipulation and cytologists’ factors, etc. (5-7). Mathematical models have the advantages of accurate, replicable, less subjective, and results being easily exchanged. The existing prediction models only focus on atypical diagnostic category (ADC) after EUS-FNA. And, sample used for modeling and validation were cases diagnosed as ADC by cytology and might result in small sample size, less risk factors included into model (2), or lack of subgroup analysis of solid occupancy (8). In light of these issues, our goal was to establish and validate a mathematical model to predict malignancy in solid pancreatic lesions and to explore its rationality and predictive ability when applied in solid ADC lesions.

We presented the following article in accordance with the TRIPOD reporting checklist (available at http://dx.doi.org/10.21037/tcr-20-2208).

Methods

Clinical data

During a third-class hospital, clinical data of hospitalized cases from June 2013 to July 2019 whose computed tomography (CT) scan of the abdomen showed solid pancreatic mass was reviewed. Clinical cases were excluded if: (I) cystic lesions of the pancreas; (II) relevant clinical data of the 16 indicators (showed below) were incomplete; (III) the nature of the lesion could not be defined finally due to reasons such as missing follow-up. Complete-case analysis was used. Sample size: a total of 16 independent variables were included in the analysis, of which sample size should be 10–20 times, therefore there should be at least 160 cases. From June 2013 to April 2018, a total of 266 cases (group A) were included for the establishment of a malignancy prediction model. Clinical data included a total of 16 indicators: gender, age, abdominal CT scan results, dilatation of common bile duct (diameter >1 cm), low lesion density (CT value <40 Hu), dilatation of main pancreatic duct (diameter >3 mm), pancreas atrophy, pancreatic calcification, lesion site, boundary of lesion, swollen lymph nodes around the pancreas/retroperitoneum; serum tumor markers CA19-9 value (cut-off: 6.74 U/mL), carcinoembryonic antigen (CEA) value (cut-off: 5 ng/mL); jaundice, abdominal pain, weight loss (loss more than 5% of original weight within the last 6 months; not intended) and diabetes history (diagnosed in the past or during this time of hospitalization, regardless of the type and management). All these factors were measured before relevant treatment being applied, and all tests were performed by blind method. The same 16 indicators for the 94 patients in the validation group (group B) from April 2018 to July 2019 were also collected. An additional 20 cases with solid pancreatic lesions whose cytological diagnosis were ADC after EUS-FNA were also reviewed to validate. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and the Harmonized Tripartite Guideline for Good Clinical Practice from the International Conference on Harmonization. Because this was a retrospective study and did not interfere with clinical treatment, ethical approval was not required in our center. And individual consent for this retrospective analysis was waived.

The gold standard for final diagnosis

The following conditions were considered to be malignant: (I) the pathological diagnosis of the disease was confirmed by surgery or EUS-FNA. (II) Imaging examination showed extensive metastasis, or the imaging of the pancreatic lesions was found to be worse than previous local invasion. (III) Death caused by pancreatic lesions. These following conditions were considered to be non-malignant: (I) the pathology was clearly diagnosed as a non-malignant lesion. (II) The imaging features were stable at least for twelve months or regressed and patients were in good condition. The results of each imaging diagnosis were confirmed by at least two experienced imaging physicians. Pathology: all pathology/cytology diagnoses were confirmed by at least two experienced pathologists/cytologists. The classification of pathological results referred to WHO classification of tumors of the digestive system published in 2010 (9). The following cytology diagnostic categories: tumor positive, suspected positive, atypical, benign and unsatisfactory sample were used in our hospital. Pathologists and followers were both blind to the predicted results.

Statistical method

Statistical analysis was performed using SPSS 18.0 software (IBM, USA). The main aim of this study was to development a clinical prediction model for predicting malignancy in the pancreatic solid mass. In the single factor analysis, the t-test and chi-square test were used to analyze the date of group A to study potential factors predicting the nature of pancreatic solid mass. Variables whose P value ≤0.10 during the univariate analysis were included in the multivariate logistic regression. Enter method was used and those variables P<0.05 were included in the mathematical prediction model. To enhance the feasibility of the model in other units, the CA19-9 and CEA values are stratified by the multiples of the upper limit of normal value (ULNV). The normal range was level 0, one to two times of ULNV was Level 1, and so on. Based on the multivariate logistic regression results of group A, a mathematical prediction model for malignant lesions was established and tested by Group B. The receiver operating characteristic (ROC) curve was established and the area under the curve was calculated. Univariate analysis was also performed on the characteristics of the malignant cases of the ADC group and those of group A. The differences between them were explored to explain the rationality of our model using for ADC group. For the prediction of malignant lesions in the ADC group, the sensitivity, specificity, positive predictive value and negative predictive value of our model were calculated and compared with other predictive model (2). Study flow diagram was shown in Figure 1.

Figure 1 Flow diagram of this study. ADC, atypical diagnostic category.

Results

During group A and B, the most common malignant lesion was pancreatic cancer. There were 76 cases (53.9%) in group A and 32 cases (64.0%) in group B. Pancreatitis was the most common disease during non-malignant lesions, with 58 cases in group A (46.4%) and 20 cases (45.5%) in group B (Table 1).

As shown in Table 2, there were statistically significant differences (P<0.10) in age, density of lesions, CA19-9 and CEA grade, dilatation of pancreatic duct, swollen lymph nodes around the pancreas/retroperitoneum, pancreas calcification, weight loss, Jaundice, dilation of common bile duct and pancreatic atrophy between the malignant and non-malignant solid mass of pancreas. Consequently, these factors were included in multivariate analysis which revealed that age, density of lesions, CA19-9 grade, CEA grade, dilatation of pancreatic duct, swollen lymph nodes around the pancreas/retroperitoneum, pancreas calcification, and weight loss were all independent predictors for malignancy (P<0.05) (Table 3).

Establishing mathematical model

Based on the results in Table 3, a prediction model of malignant solid lesion in pancreas was established: P=e^x/(1+e^x); X = −3.569 + 0.038 × age + 0.116 × CA19-9 grade + 0.148 × CEA grade + 1.415 × low density of lesions + 1.954 × swollen lymph node + 2.274 × weight loss + 1.453 × dilatation of pancreatic duct − 2.86 × calcification; “e” is the natural logarithm, 0 for yes and 1 for no in low density of lesion, while 1 for yes and 0 for no in the last four elements (i.e., swollen lymph nodes around the pancreas/retroperitoneum, weight loss, dilatation of pancreatic duct, calcification of the pancreas). The area under the ROC curve of the model was 0.942±0.014; 95% CI: 0.915–0.969, and the cut-off point value was P=0.4127852, P>0.4127852 should be considered to be malignant disease, and P<0.4127852 should be considered non-malignant disease. The sensitivity of the model was 89.4% and the specificity was 84.8%. The data of group B were used to validate the model and the ROC curve was established based on the model’s outcomes. The AUC of ROC curve was 0.854±0.042; 95% CI: 0.771–0.936 (Figure 2).

Figure 2 ROC curve of clinical data of the patients in group B generated using our model. ROC, receiver operating characteristic.

With regard to the twenty cases of ADC group: 2 cases were pancreatitis (pathology and imaging), 4 cases of pancreatic cancer (pathology), 1 case of lymphoma (pathology) and thirteen were malignant cases (imaging 9 and 4 deaths). Univariate analysis of characteristics of solid pancreatic masses showed that there was no statistically significant difference between the ADC group and group A (Table 4). It could be concluded that they came from the same “population”, which suggested that it was reasonable to predict the malignant nature of ADC lesions with this model. The study of Yang et al. (2) showed that the risk factor for malignancy of 81 ADC lesions was only high values of CA19-9 in serum. According to the ROC curve, their cut-off of CA19-9 for pancreatic cancer was 39 U/mL, and based on this value, the malignancy of ADC group was predicted. The sensitivity, specificity, NPV and PPV are calculated and compared with our model (Table 5).

Discussion

The present study aimed to establish a mathematical model able to predict malignancy in solid pancreatic lesions. The difference between the characteristics of malignant lesions in ADC and group A was also analyzed in order to explore the rationality and ability for our model to predict the nature of lesions in ADC group.

Previous studies (10-12) had shown that the proportion of malignant lesions is rather high in lesions diagnosed as ADC by cytologists after EUS-FNA. In line with previous studies, the results of the present study showed that most lesions in the ADC group were malignant [18/20].

The incidence of malignant lesions such as pancreatic cancer increases with age. In China, the incidence of pancreatic cancer has a significant upward trend after the age of forty (13,14). Malignant tumors are often accompanied by necrosis, which is accompanied by relative lower density compared to normal pancreatic tissue.

CA19-9: multivariate logistic regression analysis revealed that each grade of CA19-9 had an odds ratio (OR) value of 1.123 compared to its lower level. Previous studies have reported that CA19-9 not only was associated with pancreatic cancer, but its high value was strongly associated with malignant tumors in the pancreas (OR =9.8; 95% CI: 1.2–83.1) (2). Pancreatitis and pancreatic cancer can both cause abnormal elevation of CA19-9; therefore, it is mainly used for the monitoring of progression of pancreatic cancer (15), rather than for the judgment of the nature of lesions. The latest research reports that CA19-9 may not only act as a “bystander/marker” for pancreatic cancer, but also plays an important role in its development (16). So high-levels of CA19-9 in serum should be given more attention.

Weight loss: when controlling for other factors, the risk of malignancy for patients with weight loss was about 8.715 times higher than that of patients without weight loss. The growth of a malignant tumor in the pancreas triggers the consumption of energy, resulting in insufficient nutrition and weight loss. In addition, the exocrine function of nutrient absorption of the pancreas is also affected leading to further weight loss.

Dilatation of main pancreatic duct: evidence suggests that even in absence of obvious pancreatic space-occupying lesion in CT and other imaging, an indirect sign such as dilatation of main pancreatic duct might suggests that the possibility of pancreatic cancer is still high (17-19). A consequence of pancreatic lesions in the head of the pancreas is the obstruction of both the pancreatic duct and common bile duct, which would result in a phenomenon called “double tube sign” referring to the simultaneous expansion of the pancreatic duct and the common bile duct and being considered a characteristic sign of pancreatic cancer. However, double tube signs can also occur in metastatic tumors, chronic pancreatitis and other diseases. Our study suggested that the presence of dilatation of the main pancreatic duct was more predictive in malignancy (OR =4.277; 95% CI: 1.717–10.655).

Peri-pancreatic or retroperitoneal swollen lymph nodes: lymphatic metastasis is the main way of metastasis in early malignancy such as pancreatic cancer. Although the presence of inflammation in the pancreas could also lead to a responsive swelling of the lymph nodes around the pancreas, the risk of malignancy rose about 6.054 times when swollen lymph nodes occurred around the pancreas/retroperitoneum.

CEA level: the elevation of CEA is common in colorectal cancer, pancreatic cancer, stomach cancer and so on. Although other factors that can cause rising of CEA such as smoking, cardiovascular disease, diabetes, rectal polyps and colitis our results suggest that high-level of CEA was a risk factor for malignant solid pancreas lesions.

Calcification: the present study identified calcification as the only protective factor representing a special sign in benign lesions such as chronic pancreatitis (20).

However, some malignant lesions are also accompanied by inflammation or calcification, therefore only calcification is not sufficient to determine the nature.

ADC: the cytological diagnosis of ADC after EUS-FNA leaves clinicians in an uncertain situation despite studies have shown that most lesions in the ADC are finally confirmed to be malignant (2,5,8). Previous studies had demonstrated that amongst possible factors affecting the diagnosis of EUS-FNA there might be EUS-FNA operation related factors, cytologists’ factors, and pathological interferences such as inflammation in the lesion (5). In this study, the univariate analysis yielded no significant differences in characteristics of the malignant lesions between the ADC group and group A among the 16 indexes taken into consideration (P>0.05). This supports the argument that that the ambiguity of the cytological diagnosis of the ADC group might be related to other variables such as: the number of punctures, physician experience and other factors, rather than the lesion. However, we had only discussed the difference of malignant lesions, and factors leading to ADC need to be further explored. Nevertheless, undergoing EUS-FNA and ADC won’t change the fact that solid lesions of ADC are solid pancreatic lesions in the first place. From the perspective of predictors in our model, they seem to come from the same “population”. It therefore follows that the prediction of this model for ADC was reasonable to a certain degree. Perhaps including more predictors to increase the prediction ability and increasing the sample size of model may allow us to focus on “solid pancreatic lesions” rather than only “ADC” as cases in the modeling group. Alston et al. (8) discussed the ADC diagnosis results and potential influencing factors, and established a prediction model. The cases in their modeling group are a more heterogeneous ADC group including cystic pancreas lesions. Their main result was the prediction of tumors rather than malignancy in pancreas. The focus of our study was the solid pancreatic lesion, which reduced the heterogeneity of the modeling group and the main prediction outcome was the malignant lesions, which were clinically useful. For the prediction of malignancy in ADC, our model showed better prediction ability than that established by Yang et al. (2) which contained only one prediction factor, the CA19-9 value.

For patients with solid pancreatic lesions, the decision to proceed with further treatment or management depends on the suspicion of the malignancy. Although the prognosis of pancreatic cancer is poor, surgical treatment or adjuvant therapy can still significantly prolong the survival time of patients (21-23). New treatment methods are also rapidly emerging; for example, a recent study from Columbia University found that PTC596 can be well delivered to pancreatic tissue and is well tolerated, especially when combined with some first-line therapies such as paclitaxel (24). After the diagnosis of pancreatic cancer in China less than 40% of cases would undergo histological examination, most of which are diagnosed by surgical pathology (4), and the final cytological diagnosis in 8–10% of patients undergone EUS-FNA operation won’t be clear (5-7). A number of patients are overly treated with active surgery or adjuvant therapy, while others are not adequately treated (13). Therefore, it is of great clinical significance to judge the nature of the solid occupancy in pancreas comprehensively through various clinical features.

To our knowledge, this is the first study that established a model combining the imaging, serum tumor markers, and clinical features of solid pancreatic lesions in order to predict the malignancy and tested its application in ADC lesions. Its application in ADC lesions was relative reasonable, and demonstrated even higher sensitivity than previous prediction models. However, the present study also has some limitations. Firstly, the gold standard was not completely pathology. The patients who did not have pathological results were followed up. Specifically, patients’ health condition and evaluation of the imaging of the lesions were considered to determine the final diagnoses. Secondly, this study did not include ultrasound endoscopic features of the pancreatic mass. Endoscopic ultrasound is an invasive diagnostic method that requires special training and can only be performed in specialized medical centers. Clinically, it is usually used as a second-line diagnostic method. However, CT has become the preferred method for scanning pancreas due to its high popularity rate and relatively low cost (1). Therefore, the choice of CT rather than EUS features made the model more convenient and practical during clinical work especially for primary health care institutions without the technology of EUS-FNA. A third limitation consisted in the small sample size of the ADC cases used as a subgroup for the validation of the model’s predictive ability. This could have led to the commission of type II error in statistics. In light of that further research should increase the sample size to confirm the model’s predictive power for ADC.

This study established a comprehensive model predicting the nature of solid pancreatic lesions with clinical performances, serum tumor markers, and CT features and tested its ability in ADC lesions. According to the multivariate logistic regression analysis, old age, low density of lesions, high CA19-9 and CEA grade, dilatation of pancreatic duct, swollen lymph nodes around the pancreas/retroperitoneum and weight loss were risk factors for malignancy. The presence of calcification in the pancreas was a protective factor. This model not only showed high efficacy in the prediction of malignancy in solid pancreas lesions, but also had a relative high prediction power on the nature of ADC diagnosed by cytologists after EUS-FNA.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at http://dx.doi.org/10.21037/tcr-20-2208

Data Sharing Statement: Available at http://dx.doi.org/10.21037/tcr-20-2208

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/tcr-20-2208). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Because this study was a retrospective study and did not interfere with clinical treatment, ethical approval was not required in our center, and individual consent for this retrospective analysis was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Putzer D, Jaschke W. Radiological evaluation of focal pancreatic lesions. Dig Dis 2015;33:91-8. [Crossref] [PubMed]
2. Yang D. Predictors of malignancy in patients with suspicious or indeterminate cytology on pancreatic endoscopic ultrasound-guided fine-needle aspiration: a multivariate model. Pancreas 2014;43:922-6. [Crossref] [PubMed]
3. Takuma K, Kamisawa T, Gopalakrishna R, et al. Strategy to differentiate autoimmune pancreatitis from pancreas cancer. World J Gastroenterol 2012;18:1015-20. [Crossref] [PubMed]
4. Long J, Luo GP, Xiao ZW, et al. Cancer statistics: current diagnosis and treatment of pancreatic cancer in Shanghai, China. Cancer Lett 2014;346:273-7. [Crossref] [PubMed]
5. Eloubeidi MA, Jhala D, Chhieng DC, et al. Yield of endoscopic ultrasound-guided fine-needle aspiration biopsy in patients with suspected pancreatic carcinoma. Cancer 2003;99:285-92. [Crossref] [PubMed]
6. Wiersema MJ, Vilmann P, Giovannini M, et al. Endosonography-guided fine-needle aspiration biopsy: diagnostic accuracy and complication assessment. Gastroenterology 1997;112:1087-95. [Crossref] [PubMed]
7. Gress F, Gottlieb K, Sherman S, et al. Endoscopic ultrasonography-guided fine-needle aspiration biopsy of suspected pancreatic cancer. Ann Intern Med 2001;134:459-64. [Crossref] [PubMed]
8. Alston E, Bae S, Eltoum IA. Atypical cytologic diagnostic category in EUS-FNA of the pancreas: follow-up, outcomes, and predictive models. Cancer Cytopathol 2014;122:428-34. [Crossref] [PubMed]
9. Bosman FT, Carneiro F, Hruban RH, et al. WHO classification of tumours of the digestive system. Lyon: IARC, 2010:279-330.
10. Fisher L, Segarajasingam DS, Stewart C, et al. Endoscopic ultrasound guided fine needle aspiration of solid pancreatic lesions: Performance and outcomes. J Gastroenterol Hepatol 2009;24:90-6. [Crossref] [PubMed]
11. Eloubeidi MA, Chen VK, Eltoum IA, et al. Endoscopic ultrasound-guided fine needle aspiration biopsy of patients with suspected pancreatic cancer: diagnostic accuracy and acute and 30-day complications. Am J Gastroenterol 2003;98:2663-8. [PubMed]
12. Eloubeidi MA, Varadarajulu S, Desai S, et al. A prospective evaluation of an algorithm incorporating routine preoperative endoscopic ultrasound-guided fine needle aspiration in suspected pancreatic cancer. J Gastrointest Surg 2007;11:813-9. [Crossref] [PubMed]
13. Lin QJ, Yang F, Jin C, Fu DL. Current status and progress of pancreatic cancer in China. World J Gastroenterol 2015;21:7988-8003. [Crossref] [PubMed]
14. Kuang TT, Jin DY, Wang DS, et al. Clinical epidemiological analysis of the relationship between pancreatic cancer and diabetes mellitus: data from a single institution in China. J Dig Dis 2009;10:26-9. [Crossref] [PubMed]
15. Poruk KE, Firpo MA, Adler DG, et al. Screening for pancreatic cancer: why, how, and who? Ann Surg 2013;257:17-26. [Crossref] [PubMed]
16. Engle DD, Tiriac H, Rivera KD, et al. The glycan CA19-9 promotes pancreatitis and pancreatic cancer in mice. Science 2019;364:1156-62. [Crossref] [PubMed]
17. Chu LC, Goggins MG, Fishman EK. Diagnosis and Detection of Pancreatic Cancer. Cancer J 2017;23:333-42. [Crossref] [PubMed]
18. Ahn SS, Kim MJ, Choi JY, et al. Indicative findings of pancreatic cancer in prediagnostic CT. Eur Radiol 2009;19:2448-55. [Crossref] [PubMed]
19. Gangi S, Fletcher JG, Nathan MA, et al. Time interval between abnormalities seen on CT and the clinical diagnosis of pancreatic cancer: retrospective review of CT scans obtained before diagnosis. AJR Am J Roentgenol 2004;182:897-903. [Crossref] [PubMed]
20. Karasawa E, Goldberg HI, Moss AA, et al. CT pancreatogram in carcinoma of the pancreas and chronic pancreatitis. Radiology 1983;148:489-93. [Crossref] [PubMed]
21. Linder S, Boström L, Nilsson B. Pancreatic carcinoma incidence and survival in Sweden in 1980-2000: a population-based study of 16,758 hospitalized patients with special reference to different therapies. Eur J Surg Oncol 2007;33:616-22. [Crossref] [PubMed]
22. Pasquali C, Sperti C, Filipponi C, et al. Epidemiology of pancreatic cancer in Northeastern Italy: incidence, resectability rate, hospital stay, costs and survival (1990-1992). Dig Liver Dis 2002;34:723-31. [Crossref] [PubMed]
23. Hedberg M, Borgström A, Genell S, et al. Survival following pancreatic carcinoma: a follow-up study of all cases recorded in Malmö, Sweden, 1977-1991. Br J Surg 1998;85:1641-4. [Crossref] [PubMed]
24. Eberle-Singh JA, Sagalovskiy I, Maurer HC, et al. Effective Delivery of a Microtubule Polymerization Inhibitor Synergizes with Standard Regimens in Models of Pancreatic Ductal Adenocarcinoma. Clin Cancer Res 2019;25:5548-60. [Crossref] [PubMed]