Development, Validation and Comparison of Artificial Neural Network and Logistic Regression Models Predicting Eosinophilic Chronic Rhinosinusitis With Nasal Polyps

- ¹Department of Otolaryngology-Head and Neck Surgery, Renmin Hospital of Wuhan University, Wuhan, China.
- ²Research Institute of Otolaryngology-Head and Neck Surgery, Renmin Hospital of Wuhan University, Wuhan, China.
Correspondence to Yu Xu, MD. Department of Otolaryngology-Head and Neck Surgery, Renmin Hospital of Wuhan University, 238 Jiefang Road, Wuhan 430060, China. Tel: +86-2788041911, Fax: +86-2788042292, Email: xuy@whu.edu.cn

Received April 18, 2022; Revised August 18, 2022; Accepted September 02, 2022.

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Purpose

Chronic rhinosinusitis with nasal polyps (CRSwNP) can be classified into eosinophilic CRSwNP (eCRSwNP) and non-eosinophilic CRSwNP (non-eCRSwNP) by tissue biopsy, which is difficult to perform preoperatively. Clinical biomarkers have predictive value for the classification of CRSwNP. We aimed to evaluate the application of artificial neural network (ANN) modeling in distinguishing different endotypes of CRSwNP based on clinical biomarkers.

Methods

Clinical parameters were collected from 109 CRSwNP patients, and their predictive ability was analyzed. ANN and logistic regression (LR) models were developed in the training group (72 patients) and further tested in the test group (37 patients). The output variable was the diagnosis of eCRSwNP, defined as tissue eosinophil count > 10 per high-power field. The receiver operating characteristics curve was used to assess model performance.

Results

A total of 15 clinical features from 60 healthy controls, 60 eCRSwNP and 49 non-eCRSwNP were selected as candidate predictors. Nasal nitric oxide levels, peripheral eosinophil absolute count, total immunoglobulin E, and ratio of bilateral computed tomography scores for the ethmoid sinus and maxillary sinus were identified as important features for modeling. Two ANN models based on 4 and 15 clinical features were developed to predict eCRSwNP, which showed better performance, with the area under the receiver operator characteristics significantly higher than those from the respective LR models (0.976 vs. 0.902, P = 0.048; 0.970 vs. 0.845, P = 0.011). All ANN models had better fits than single variable prediction models (all P < 0.05), and ANN model 1 had the best predictive performance among all models.

Conclusions

Machine learning models assist clinicians in predicting endotypes of nasal polyps before invasive detection. The ANN model has the potential to predict eCRSwNP with high sensitivity and specificity, and is superior to the LR model. ANNs are valuable for optimizing personalized patient management.

Keywords

Machine learning; neural network models; biomarker; eosinophils; nasal polyps

INTRODUCTION

Chronic rhinosinusitis (CRS) is a chronic inflammatory disease that occurs in the nasal mucosa and is fairly prevalent in the otolaryngology department. The overall prevalence of CRS in the general population in Europe is 10.9% (range 6.9%–27.1%), while in China it is 8.0%.1, 2 CRS can be classified into CRS with nasal polyps (CRSwNP) and CRS without nasal polyps (CRSsNP) based on polyp status.3 Research in this area has shown that patients with CRSwNP have worse prognosis than those with CRSsNP.4 CRSwNP is a highly heterogeneous disease. Based on observed eosinophil infiltration, it can be categorized into 2 main endotypes, one is eosinophilic CRSwNP (eCRSwNP) and the other is non-eosinophilic CRSwNP (non-eCRSwNP).5 The prevalence of eCRSwNP varies across the world. In particular, the reported prevalence of eCRSwNP in CRSwNP populations ranged from 65% in Asia to 91% in Europe.6, 7 It has previously been observed that eCRSwNP has shown an upward trend in Asia lately and eosinophilic inflammation has been significantly augmented in CRSwNP patients from central China.8, 9, 10

The eCRSwNP is also characterized by severe symptoms, a high likelihood of resistance to treatment, and frequent disease recurrence.11 Patients with eCRSwNP usually have higher levels of immunoglobulin E (IgE), higher prevalence of atopic disease, and greater prevalence and severity of comorbidities such as allergic rhinitis and asthma.3, 9 Another feature of eCRSwNP is the efficacy of glucocorticoid therapy.12 Therefore, glucocorticoid therapy would be helpful to identify these kinds of patients in an early phase, in order to guide overall long-term management. Tissue analysis remains the gold standard for the classification of eCRSwNP and non-eCRSwNP. However, tissue biopsy is invasive and unbearable within the outpatient setting or before surgery.

ECRSwNP was associated with Th2 and IgE-mediated allergic responses.10 Therefore, clinical parameters, such as blood eosinophilia, total serum IgE, computed tomography (CT) score, erythrocyte sedimentation rate, and C-reactive protein, have been used for predicting eCRSwNP in previous researches.13, 14, 15 Previous studies have demonstrated that peripheral eosinophil absolute count (PEAC) played a strong predictive role in eCRSwNP.14, 16 What is more, asthma and allergic rhinitis are type 2 inflammation-mediated diseases that are closely related to CRS-related inflammation. Studies have found that the comorbidity rate of asthma and allergic rhinitis were higher in the eCRSwNP subgroup than in the non-eCRSwNP subgroup.17, 18 However, Hu et al.19 found that although the association of eCRSwNP with asthma is widely accepted, there was no significant statistical difference between the rate of coexistence of asthma in eosinophilic and non-eCRSwNP. Nitric oxide (NO) is synthesized by L-arginine and oxygen through NO synthase from the upper and lower airways. The fractional concentration of exhaled NO (FeNO) is recommended as a routine test item in the diagnosis and treatment of asthma, since it is a non-invasive biomarker reflecting eosinophilic inflammation in lower airway inflammation.20 Recent research has also shown that FeNO was elevated in patients with eCRS instead of non-eCRS patients, and showed a strong correlation with Lund-Mackay scores in eCRS patients.21 Similarly, a number of studies suggested that nasal nitric oxide (nNO) is higher in eCRSwNP than in healthy people, and it is a predictive clinical marker of eosinophilic upper airway inflammation.22, 23 Whereas Yoshida et al.24 demonstrated that nNO levels were markedly decreased in eCRS and negatively correlated with eosinophil levels and CT score. For the time being, however, the role of nNO in different subtypes of nasal polyps is not yet known and it has not been widely used in the domain of rhinology.

Identifying endotypes of CRSwNP with a noninvasive approach will be of vital importance for precision medicine in the era of targeted biotherapies. A sensitive and specific noninvasive predictive strategy for identifying patients with eCRSwNP will contribute to patients’ prognosis predication and long-term management. Logistic regression (LR) models are frequently used in building predictive models. Artificial neural network (ANN) is a subset of artificial intelligence in the computing system that has been applied in the bio-medical field with splendid results, assisting in the detection and classification of certain types of diseases.25, 26 In the past 5 years, the amount of novel applications of machine learning in the field of otolaryngology has increased sharply; nevertheless, its practical uses in rhinology remain restricted.27 Here, we aimed to assess the diagnostic accuracy of these 2 methodologies (LR and ANN) in predicting eCRSwNP on the basis of clinical and radiological variables.

MATERIALS AND METHODS

Study population

A total of 109 adult patients (> 18 years) diagnosed with CRSwNP according to the European Position Paper on Rhinosinusitis and Nasal Polyps 2020 (EPOS 2020),3 who had undergone functional endoscopic sinus surgery (FESS) for the treatment were enrolled in our study from January 2021 to December 2021. Sixty subjects in the control group, were those who underwent septoplasty surgery during the same period without any other inflammatory sinonasal diseases. At the time of surgery, polyp samples were collected from CRSwNP patients and fixed in formalin, preparing for hematoxylin and eosin (H&E) staining. Patients at age < 18 years, diagnosed with immunodeficiency, granulomatosis with polyangiitis, choanal polyp, cystic fibrosis, allergic fungal sinusitis and coagulation disorder, or pregnancy were excluded. This study was approved by the Ethics Committee of Renmin Hospital of Wuhan University (WDRY2021-K084) and written informed consents were signed by all subjects. All clinical data were anonymized before data analysis.

Clinical data collection

Demographic characteristics were recorded as potential medical variables before surgery, which contained the patient’s age and sex, asthma, allergic rhinitis, patient-reported allergy, and previous sinus surgery. To access the severity of CT findings on the paranasal sinuses, the paranasal sinus CT scanning was performed before surgery and evaluated by a senior otolaryngologist according to the Lund-Mackay CT grading system in a blinded fashion.28 Opacification of each sinus was graded as 0, no opacification; 1, partial opacification; and 2, full opacification. The total ethmoid sinus score (E score), maxillary sinus score (M score), and E/M ratio were calculated. Since scores cannot be divided by 0 for the E/M ratio calculation, one point was added to each sinus score as the basal level.29 An E/M ratio > 1 indicated that opacification of the ethmoid sinus was greater than that of the maxillary sinus. A blood sample was taken before FESS and measured for a certain number of potential clinical parameters including eosinophils count, eosinophil percent, and total IgE levels. Levels of FeNO and nNO were measured with a nanocoulomb NO analyzer (Sunvou, Wuxi, China) based on the American Thoracic Society/European Respiratory Society guidelines.20 FeNO was measured at an aspiration flow rate of 50 mL/s, while nNO was accessed at a aspiration flow rate of 5 mL/s. Both were measured 3 times and the mean of the 3 values was used for analysis. The demographic and clinical characteristics of enrolled subjects are presented in Table 1.

Table 1
Baseline characteristics of the study population

Click for larger image
Click for full table
Download as Excel file

Input variable selection and outcome definition

Demographic and clinical factors were selected as candidate predictors to establish the prediction models. Variables for inclusion were carefully chosen, given the clinical relevance combined with feature importance or statistical significance, to ensure parsimony of the final model. In this work, the Boruta algorithm and univariate LR analysis were utilized to filter features for model building. The former is an all-relevant feature selection wrapper algorithm, while the latter is a traditional statistical analysis-based method. The Boruta algorithm offers variable importance by comparing the Z-score of each variable to that of the “shadow feature,”30 Shadow features were obtained from the random forest model in each iteration by duplicating and shuffling the real features sequentially. The Z-score of each attribute was calculated and any Z-score of a real feature higher than the maximal Z-score of shadow features was considered “important.” Boruta algorithm was performed using the “Boruta” package in R^®.30 Univariate LR analysis was performed in the training cohort to identify the independent determinants of eCRSwNP, variables with a P value < 0.05 on univariate analysis were introduced into a multivariate model. A P value < 0.05 was considered statistically significant.

The output value of the models was the probability of positive biopsy results ranging between 0 (no probability of eCRSwNP) and 1 (100% probability of eCRSwNP). Tissue eosinophil count > 10 eosinophils per high-power field (HPF) on 3 randomly selected HPFs was recorded as a positive biopsy result, as described by EPOS 2020.3 Polyp specimens were processed with H&E staining and assessed by a pathologist blinded to the clinical data.

Model development

The entire data was divided into 2 new sets by simple random sampling, with two-thirds (n = 72) assigned to training sets and one-third (n = 37) assigned to test sets. Selected variables were entered into LR model 1 and ANN model 1, respectively. In addition, all 15 variables were entered into LR model 2 and ANN model 2, respectively. The output variable was the diagnosis of eCRSwNP. Binary LR and ANN models were developed from the training data and tested for diagnostic ability via the receiver operating characteristic (ROC) curve in the test dataset. Sensitivity, specificity, and accuracy were accessed by the confusion matrixes of the 2 models. ANNs were developed with a single hidden layer using sigmoid activation functions without boosting. The learning rate of ANN was set at 0.1 and evaluated by the one-third holdback method.

Assessment of the models

Area under the receiver operator characteristic (AUC) curve was used to assess model discrimination. The confusion matrixes of the 2 models were created based on the optimal cutoff values to evaluate sensitivity, specificity, positive predictive values (PPVs), negative predictive values (NPVs), positive likelihood ratio (PLR), accuracy, precision, and F1-score (F1). The calculation formula of each index was as follows: sensitivity: TP/(TP + FN), specificity; TN/(FP + TN), PPV and precision; TP/(TP + FP), NPV: TN/(TN + FN), PLR; Sensitivity/(1 − Specificity), accuracy; and (TP + TN)/(TP + TN + FP + FN), F1 = 2 Precision×Sensitivity/(Precision+Sensitivity). In the above statements, TP is true positive, FN is false negative, FP is false positive, TN is true negative. The Hanley-McNeil test was used to compare the ROC curves of different models. ROC analyses were computed by MedCalc, version 20.0.22 (MedCalc software, Mariakerke, Belgium).

Statistical analysis

Continuous variables were explored for parametric distribution by the Kolmogorov-Smirnov test. As all data are non-normally distributed, continuous variables were presented as median and interquartile ranges. A Kruskal-Wallis test with the Dunn post hoc test was used to assess significant intergroup variability among 3 groups and the Mann-Whitney U test was used for between-group comparisons. Categorical variables were tested by the χ² test (or Fisher’s exact test, if appropriate). For multiple comparisons among 3 groups, Bonferroni’s correction was used and adjusted P values were provided. Binary LR was evaluated using the likelihood ratio χ² statistic. The odds ratio (OR) and 95% confidence intervals (CIs) were calculated for each parameter. All statistical analyses and machine learning modeling was carried out by JMP Pro, version 16.0.0 (SAS Institute Inc., Cary, NC, USA). Any P value of < 0.05 was considered statistically significant.

RESULTS

Demographic and clinical characteristics of the subjects

A total of 109 adult patients diagnosed with CRSwNP and 60 healthy controls were included in this study; study subjects were identified as having all the input and output variables (Fig. 1). All patients with confirmed CRSwNP were grouped after assessment of mucosal eosinophils by H&E staining (Fig. 2). Based on the histological criteria for eCRSwNP, 60 patients were classified into the eCRSwNP group and 49 patients were classified into the non-eCRSwNP group. The demographic and clinical characteristics of the subjects are given in detail in Table 1. The 3 groups had a similar sex ratio, drinking history, smoking, and patient-reported allergy. Compared with controls, patients with eCRSwNP had higher comorbidity of asthma and allergic rhinitis as well as higher PEAC, peripheral eosinophil percentage (PEP), total serum IgE, and nNO levels. Patients in the non-eCRSwNP group were slightly older than the healthy controls but did not differ from the eCRSwNP group. In addition, non-eCRSwNP patients had higher nNO level as well as higher incidence of AR comorbidity and prior sinus history compared to controls. In the comparison between the eCRSwNP and non-eCRSwNP groups, E/M ratio, PEAC, PEP, serum total IgE, and nNO level of eCRSwNP patient group were all significantly higher than in the non-eCRSwNP patient group, reflecting a Th2 mediated allergic responses of the disease.

Fig. 1
Study flow chart. Schematic illustration of analysis flow for developing and evaluating models.
eCRSwNP, eosinophilic chronic rhinosinusitis with nasal polyps; non-eCRSwNP, non-eosinophilic chronic rhinosinusitis with nasal polyps.

Fig. 2
Representative hematoxylin and eosin staining of nasal polyps in CRSwNP. (A) Nasal polyps in non-eosinophilic CRSwNP; (B) Nasal polyps in eosinophilic CRSwNP. Original magnification: 400×.
CRSwNP, chronic rhinosinusitis with nasal polyps.

Predictors associated with endotypes of CRSwNP

In this study, a total of 15 demographic and clinical factors were selected as candidate predictors of eCRSwNP. The result of feature selection based on the Boruta algorithm is shown in Fig. 3A. In order of Z-scores, 5 “important” variables were nNO, PEAC, E/M ratio, total IgE, and PEP. Meanwhile, the univariate LR analysis revealed that nNO levels (OR, 1.009; 95% CI, 1.004–1.015; P < 0.001), PEAC (OR, 1.003; 95% CI, 1.001–1.005; P = 0.009), PEP (OR, 1.171; 95% CI, 1.028–1.335; P = 0.017), total IgE (OR, 1.005; 95% CI, 1.001–1.009; P = 0.013), E/M ratio (OR, 1.995; 95% CI, 1.151–3.456; P = 0.014), were associated with endotypes of CRSwNP (Table 2). As a result, the outcome of the Boruta algorithm was in accord with that of the univariate analysis. The PEAC and PEP were positively and significantly correlated, with a Spearman correlation coefficient equal to 0.886 (P < 0.001, data not shown). Since PEAC showed better performance, with higher feature importance and a higher AUC than PEP in the training set (0.725 vs. 0.700, P = 0.438, data not shown), we preferred using the PEAC rather than the PEP for inclusion in the subsequent model construction to avoid collinearity. The multivariate LR model confirmed that nNO level (OR, 1.015; 95% CI, 1.006–1.023; P = 0.001), PEAC (OR, 1.003; 95% CI, 1.001–1.005; P = 0.033), and total IgE (OR, 1.009; 95% CI, 1.001–1.017; P = 0.025) were significantly associated with endotypes of CRSwNP (Table 2). For the E/M ratio, it showed an OR of 1.524 with a P value of 0.269 in our study. Many studies have confirmed that it was an indicator of a diagnosis of eCRSwNP.15, 22, 23 The results of our study may be attributed to the small sample size and the indirect correlation of E/M ratio with eCRSwNP. Finally, 4 variables including nNO, PEAC, E/M ratio, and total IgE were included as indicators in the model 1 construction.

Fig. 3
Boruta screening feature results and diagram of artificial neural network models. (A) Feature selection based on the Boruta algorithm. The horizontal axis is the name of each variable and the vertical one is the Z-score of each variable. The green boxplots represent the first 5 important variables and the red represents unimportant variables. Blue boxplots correspond to minimal, average, and maximum Z-score of a shadow attribute, which are automatically generated by the algorithm and are not included in the analysis. (B) Artificial neural network model 1 with 4 input nodes: nNO, PEAC, E/M ratio and total IgE. (C) Artificial neural network model 2 with 15 input nodes. The output nodes of the 2 ANN models were the probability of eCRSwNP on biopsy. x: input value; b: bias; y: output value.
nNO, nasal nitric oxide; PEAC, peripheral eosinophil absolute count; E/M ratio, ratio of bilateral computed tomography scores for the ethmoid sinus and maxillary sinus; IgE, immunoglobulin E; PEP, peripheral eosinophil percentage; FeNO, fractional concentration of exhaled nitric oxide; AR, allergic rhinitis.

Table 2
Univariate and multivariate analyses of clinical characteristics associated with eosinophilic chronic rhinosinusitis with nasal polyps in training dataset

Click for larger image
Click for full table
Download as Excel file

ANN and LR models

The demographic and clinical characteristics of the training and test groups are outlined in Supplementary Table S1. Data from the training and test sets were comparable, with no significant difference in baseline characteristics between the 2 groups (all P > 0.05). Four independent predictors of eCRSwNP including nNO, PEAC, total IgE, and E/M ratio were used as input variables of ANN model 1 and LR model 1, respectively. Table 3 shows the performance of the univariate ROC analysis for the 4 selected variables in the test dataset. The AUC of LR model 1 was 0.902 (95% CI, 0.758–0.975) in the test group. The sensitivity of LR model 1 for the prediction of eCRSwNP was 0.714 and the specificity was 0.813. LR model 1 had a PPV of 0.833, an NPV of 0.684, and an F1 of 0.769. LR model 1 yielded an accuracy of 0.757, indicating the correct prediction of 28 out of the 37 patients (Table 4, Fig. 4A). ANN model 1 with 4 input nodes, 6 nodes in the hidden layer, and one output neuron was established (Fig. 3B). ANN model 1 had an AUC of 0.976 (95% CI, 0.864–1.000) in the test set, with sensitivity and specificity of 0.904 and 0.938, respectively. Moreover, ANN model 1 had a PPV of 0.950, an NPV of 0.882, and the F1 was 0.927, indicating a good predictive power. Meanwhile, 34 out of 37 cases were correctly predicted, which reflected an accuracy rate of 0.919 (Table 4, Fig. 4B).

Table 3
Univariate receiver operating characteristic curve analysis of predictors associated with eosinophilic chronic rhinosinusitis with nasal polyps in the test dataset

Click for larger image
Click for full table
Download as Excel file

Table 4
Model performance of ANNs and LRs in test dataset

Click for larger image
Click for full table
Download as Excel file

Fig. 4
Confusion matrixes of the LR and ANN models in the test dataset. (A) The LR model 1-based confusion matrix. (B) The ANN model 1-based confusion matrix. (C) The LR model 2-based confusion matrix. (D) The ANN model 2-based confusion matrix.
ANN, artificial neural network; LR, logistic regression.

All the 15 clinical and biological parameters were used to build LR model 2 and ANN model 2 to predict eCRSwNP. ANN model 2 consisted of 15 input nodes, 8 nodes in the hidden layer, and one output neuron (Fig. 3C). The performance of model 2 is presented in Table 4. ANN model 2 had a high AUC of 0.970 (95% CI, 0.854–0.999), compared to that of LR model 2, with an AUC of 0.845 (95% CI, 0.689–0.943). The test sensitivity of LR model 2 was 0.810; specificity, 0.688; and accuracy, 0.757, with 28 out of the 37 patients correctly predicted (Fig. 4C). For ANN model 2, the test sensitivity was 0.905; specificity, 0.875; and accuracy, 0.892, with 33 out of the 37 patients correctly predicted (Fig. 4D).

Model performances

ANN and LR models were evaluated in the test datasets. The diagnostic details of all models are shown in Table 4. ROC curves of all models based on the test sets are shown in Fig. 5. Current evidence was quite strong to suggest that the ANN models had better classification performance for the prediction of eCRSwNP. Both ANN model 1 and 2 showed better performance, with AUCs significantly higher than those from the respective LR models (0.976 vs. 0.902, P = 0.048; 0.970 vs. 0.845, P = 0.011; Supplementary Table S2). Plus, ANNs outperformed LRs in all the other evaluation metrics. The F1s of ANNs remained largely higher than the respective LR models (0.927 vs. 0.769; 0.905 vs. 0.791). Meanwhile, the PPV of ANNs was higher than the respective LRs model (0.950 vs. 0.833; 0.905 vs. 0.773) in the test cohort, resulting in a PLR of 14.476 in ANN model 1 and a PLR of 7.238 in ANN model 2, this reflected a good predictive power of ANN models for eCRSwNP patients. For the 2 LR models, PLRs remained < 5, indicating low predictive power. In addition, although ANN and LR models had AUCs greater than 4 univariate models, we found that only the AUC of ANN was statistically higher than any univariate model in the test population according to the Hanley-McNeil test (all P < 0.05) (Supplementary Table S2). However, the AUC for the LR model 1 and univariate analyses using the PEAC and IgE were not statistically different in the test population (P = 0.289 and P = 0.155, respectively), what is worse, there was no significant difference in AUC between LR model 2 and any of the 4 univariate analyses (all P > 0.05) (Supplementary Table S2).

Fig. 5
ROC curves of univariate analyses, LR models, and ANN models in the test dataset. (A) Univariate ROC curves of nNO, PEAC, E/M ratio and total IgE. (B) ROC curves of LR and ANN models working with 4 screened variables (model 1). (C) ROC curves of LR and ANN models working with all 15 variables (model 2).
ROC, receiver operating characteristic; ANN, artificial neural network; LR, logistic regression; nNO, nasal nitric oxide; PEAC, peripheral eosinophil absolute count; E/M ratio, ratio of bilateral computed tomography scores for the ethmoid sinus and maxillary sinus; IgE, immunoglobulin E; AUC, area under the receiver operator characteristic.

Models 1 and 2 were chosen to see whether working with fewer inputs would have the same performance as working with all inputs. Theoretically, dealing with more inputs slows down algorithms, takes too many resources, and is inconvenient for practical model building. In our study, LR model 1 had an AUC higher than LR model 2 (0.902 vs. 0.845, P = 0.222), meanwhile, the ANN model 1 had an AUC slightly higher than the ANN model 2 (0.976 vs. 0.970, P = 0.830). It would thus appear that the inclusion of more inputs did not result in improved model performance. It was probably the incorporation of weakly relevant variables which led to a decrease in the performance of model 2.

DISCUSSION

LR and ANN are 2 of the most commonly used tools for developing predictive models for dichotomous outcomes in medicine.31 The current research is a preliminary study of using machine learning algorithms to identify eCRSwNP with multiple remarkable features. We collected as many clinical markers related to CRSwNP diagnosis as possible. Finally, the predictive significance of nNO, PEAC, total IgE, and E/M ratio was identified by the Boruta algorithm and univariate and multivariate analyses.

Our study demonstrated that nNO levels were lower in CRSwNP patients than in normal controls and that the eCRSwNP group had an elevated level of nNO compared with the non-eCRSwNP group, while there was no difference between patients with CRSwNP and rhinologically healthy population. The Boruta algorithm revealed that nNO had the highest Z-score, indicating that nNO was a powerful predictor for eCRSwNP diagnosis. This result seemed to be in accord with earlier studies.22, 32 Because of compromised ostiomeatal patency and mucosal eosinophilic inflammation, there is an elevated nNO level locally in the nasal cavity, although nNO is less stable than FeNO, it is becoming a popular biomarker of upper airway diseases. In addition, PEAC was incorporated into modeling. Our results are consistent with those of previous reports that PEAC is closely related to CRSwNP, the more eosinophils in peripheral blood; the more likely it is to be diagnosed with eCRSwNP.16 We found that PEAC ranked second in the Boruta screening feature results. When PEAC was 280.0/μL in the test set, ROC analysis achieved an AUC of 0.817, with a sensitivity of 0.667 and a specificity of 0.875. However, PEAC is not convincing enough as a lone predictor, because of its susceptibility to different diseases and health states.

Exposure to inhalant allergens and other microorganisms increases IgE production in CRSwNP patients, exacerbating inflammatory progression. In recent years, omalizumab, a monoclonal antibody that targets IgE, has become more and more widely used in patients with sinusitis and nasal polyps.33 However, Ho et al.14 suggested that there was no significant correlation between tissue eosinophilia and total IgE, nor with any allergen-specific IgE. On the contrary, we identified that the mean serum total IgE levels were much higher in the eCRSwNP population than in the control and non-eCRSwNP. Univariate LR revealed an ideal serum IgE cutoff of 85.6 IU/mL to predict eCRSwNP with an AUC of 0.808 in the test cohort. It had moderate sensitivity (0.667), high specificity (0.938) and PPV (0.933). The feature importance of total IgE was found to be the lowest of the 4 parameters investigated, indicating that IgE had a low predictive value in this study cohort. Overall, the results demonstrated that while total IgE was relevant to eCRSwNP, its utility as a single predictive marker for tissue eosinophilia was still limited.

Several studies have shown that CT scanning may be helpful in the diagnosis of eCRSwNP.15, 34, 35 The opacification of the M usually reflects the pathological changes of non-eCRSwNP, while polyposis and mucosal edema usually appear around the middle turbinate in eCRSwNP, indicating that inflammation mainly occurs in the anterior and posterior ethmoid sinuses, which is usually presented as the opacity of the ethmoid sinuses on CT scans.29 Therefore, a CT scan showing a high E/M ratio would be indicative of a diagnosis of eCRSwNP. In the present study, we identified a higher ratio in the eCRSwNP group instead of the non-eCRSwNP group, which was in accord with the results of previous studies.15, 23 Additionally, the Boruta algorithm confirmed the E/M ratio as an important variable. According to univariate analysis, the E/M ratio had an AUC of 0.653 with a PLR of 1.778, indicating a relatively low predictive power of the E/M ratio in the test cohort.

Although the above 4 variables were closely related to eCRSwNP and univariate analysis demonstrated that they had independent predictive values, they were not sufficiently valuable to predict tissue eosinophils alone due to the heterogeneity of a single variable in different populations and its susceptibility to different health conditions. ECRSwNP has skewed Th2 inflammation, strong allergic responses, and poor prognosis. Categorizing endotypes of CRSwNP is of vital importance due to its relevance to different clinical decisions. A variety of scoring systems and LR models were used to predict mucosal eosinophilic status; however, there was often no reference to verification.23, 35

Machine learning is a state-of-the-art method for developing predictive algorithms in clinical practice; nevertheless, the application of ML in otorhinolaryngology is relatively new, especially in the field of rhinology. ANNs are composed of interconnected groups of artificial neurons and weighted connections.36 Compared to LR, the main advantage of ANN is that it can efficiently model different response surfaces by building enough hidden nodes and layers. The distribution of data in ANN models is not restricted, allowing researchers to make the most use of data information. ANN can be widely used in the fields of prediction and analysis due to its fault tolerance. What is more, the ANN model can deal with large amounts of data, which can help us manage complex clinical situations in clinical practice.37 Many studies have indicated that ANNs are powerful tools for assisting the clinician in the diagnosis and prognosis of various diseases.38, 39, 40 Tong et al.41 stated that ANNs are convenient and reliable models that outperformed LR models in accurately predicting the survival of unresectable pancreatic cancer. However, Kawakami et al.42 indicated that the LR model has a superior accuracy in predicting prostate cancer than the ANN. So far, there are rarely applications of ANN in the field of rhinology.

Our results can be compared with those of the study of Thorwarth et al.43 that explored the eCRS prediction using ANN and LR models. In their study, peripheral eosinophil count, polyp status, and urinary leukotriene E4 (uLTE4) level were used as predictive variables and two-thirds of the participants were allocated to a training set (54 patients), while one-third were assigned to a testing set (26 patients). The predictive power of the 2 approaches was compared, with the results indicating that ANN did not have significantly better predictive value than LR. The results of that study were different from those of our study that ANN model 1 outperformed not only the univariate analyses but also LR model 1. The heterogeneous input variables and patient populations might partly account for the differences between the 2 studies. Instead of using uLTE4 as an input variable, our study utilized commonly used clinical parameters selected from 15 candidate predictors, including parameters reflecting eosinophilic inflammation from peripheral blood, respiratory system, and radiology for building a model, which is more convenient for clinical use at the present stage.

In the present study, we explored the potential of using ANN with appropriate input variables for eCRSwNP. The ANN models had better performance than the respective LR models and any univariate analysis, which suggested that ANN can be used as a powerful predictive tool that supplements clinical judgment. Some studies demonstrated that non-significant variables still play important roles in prediction.42, 44 Therefore, we established 2 sets of models with different numbers of input variables to compare model performance, to help with clinical decision-making. Model 1 was built on 4 strictly selected variables, while model 2 was built based on all 15 variables. ANN model 1 had the best performance, followed by ANN model 2, LR model 1, and LR model 2. We found that carefully selected inputs helped develop better predictive models. This is not surprising as the incorporation of some noisy variables interferes with the whole model. Moreover, a predictive model with fewer features makes it easier for practical model building and clinical practice. ANN model 1 performed the best and the number of characters needed to be collected was acceptable, so we recommend ANN model 1 to clinicians.

The presented study has some advantages as well as limitations. The advantages are as follows: first of all, the variables incorporated in models were non-invasive, repeatable, and readily available in clinical practice without adding additional clinical burden to patients. Secondly, ANN help predict eCRSwNP more accurately, thereby optimizing patients’ precise treatment and achieving personalized management. Thirdly, the baseline characteristics and clinical parameters in the training and test datasets were comparable, and the testing cohort exhibited convincing performance. Several limitations that must be noted as well. Firstly, the ANN is often referred to as a “black box” process,37 which has intermediate layers rather than a direct path from the input variables to the output variables, the results of ANN models are not easily interpretable. Secondly, model testing was only performed on 37 patients at a single center. Further research, including larger-scale and multi-institutional datasets, is needed to validate our model in subtyping endotypes of CRSwNP. Finally, our study was a cross-sectional study rather than a longitudinal one, which inevitably leads to some confounding factors that cannot be eliminated in the study.

In conclusion, nNO, PEAC, total IgE, and E/M ratio were independent predictors for the prediction of tissue eosinophilia in eCRSwNP patients. A convenient and reliable ANN in this study has shown promise in predicting CRSwNP endotypes, and the model testing showed that the prediction accuracy of ANN was superior to that of the LR model. Our model may help clinicians use a non-invasive approach for identifying disease endotypes at early stages.

SUPPLEMENTARY MATERIALS

Supplementary Table S1

Comparison of baseline characteristics by training and test datasets

Click here to view.^{(40K, xls)}

Supplementary Table S2

Values of P for comparison of area under the receiver operator characteristic between all prediction models in the test cohort

Click here to view.^{(38K, xls)}

Notes

Funding:This research was supported by the National Natural Science Foundation of China (No. 82071017), National Natural Science Foundation of Hubei Province (No. 2021CFB125) and the Fundamental Research Funds for the Central Universities (No. 2042021kf0232 and No. 2042021kf0093).

Disclosure:There are no financial or other issues that might lead to conflict of interest.

ACKNOWLEDGMENTS

The authors thank all participants in this study for their enthusiastic cooperation.

References

1. Hastan D, Fokkens WJ, Bachert C, Newson RB, Bislimovska J, Bockelbrink A, et al. Chronic rhinosinusitis in Europe--an underestimated disease. A GA²LEN study. Allergy 2011;66:1216–1223.
  PubMed
  
  CrossRef
1. Shi JB, Fu QL, Zhang H, Cheng L, Wang YJ, Zhu DD, et al. Epidemiology of chronic rhinosinusitis: results from a cross-sectional survey in seven Chinese cities. Allergy 2015;70:533–539.
  PubMed
  
  CrossRef
1. Fokkens WJ, Lund VJ, Hopkins C, Hellings PW, Kern R, Reitsma S, et al. European position paper on rhinosinusitis and nasal polyps 2020. Rhinology 2020;58:1–464.
  CrossRef
1. Sella GC, Tamashiro E, Sella JA, Aragon DC, Mendonça TN, Arruda LK, et al. Asthma is the dominant factor for recurrence in chronic rhinosinusitis. J Allergy Clin Immunol Pract 2020;8:302–309.
  PubMed
  
  CrossRef
1. Ishitoya J, Sakuma Y, Tsukuda M. Eosinophilic chronic rhinosinusitis in Japan. Allergol Int 2010;59:239–245.
  PubMed
  
  CrossRef
1. Wang X, Zhang N, Bo M, Holtappels G, Zheng M, Lou H, et al. Diversity of T_H cytokine profiles in patients with chronic rhinosinusitis: a multicenter study in Europe, Asia, and Oceania. J Allergy Clin Immunol 2016;138:1344–1353.
  PubMed
  
  CrossRef
1. Wang W, Gao Y, Zhu Z, Zha Y, Wang X, Qi F, et al. Changes in the clinical and histological characteristics of Chinese chronic rhinosinusitis with nasal polyps over 11 years. Int Forum Allergy Rhinol 2019;9:149–157.
  PubMed
  
  CrossRef
1. Luo X, Xu Z, Zuo K, Deng J, Gao W, Jiang L, et al. The changes of clinical and histological characteristics of chronic rhinosinusitis in 18 years: was there an inflammatory pattern shift in southern China? World Allergy Organ J 2021;14:100531
  PubMed
  
  CrossRef
1. Wang ET, Zheng Y, Liu PF, Guo LJ. Eosinophilic chronic rhinosinusitis in East Asians. World J Clin Cases 2014;2:873–882.
  PubMed
  
  CrossRef
1. Jiang WX, Cao PP, Li ZY, Zhai GT, Liao B, Lu X, et al. A retrospective study of changes of histopathology of nasal polyps in adult Chinese in central China. Rhinology 2019;57:261–267.
  PubMed
1. McHugh T, Snidvongs K, Xie M, Banglawala S, Sommer D. High tissue eosinophilia as a marker to predict recurrence for eosinophilic chronic rhinosinusitis: a systematic review and meta-analysis. Int Forum Allergy Rhinol 2018;8:1421–1429.
  PubMed
  
  CrossRef
1. Konno W, Kashiwagi T, Tsunemi Y, Goto K, Haruna S. Long-term postoperative control of eosinophilic chronic rhinosinusitis recurrence by inserting a steroid-eluting, sinus-bioabsorbable device reduces the dosage of oral steroid. Auris Nasus Larynx 2019;46:365–373.
  PubMed
  
  CrossRef
1. Ho J, Hamizan AW, Alvarado R, Rimmer J, Sewell WA, Harvey RJ. Systemic predictors of eosinophilic chronic rhinosinusitis. Am J Rhinol Allergy 2018;32:252–257.
  PubMed
  
  CrossRef
1. Ho J, Earls P, Harvey RJ. Systemic biomarkers of eosinophilic chronic rhinosinusitis. Curr Opin Allergy Clin Immunol 2020;20:23–29.
  PubMed
  
  CrossRef
1. Meng Y, Lou H, Wang C, Zhang L. Predictive significance of computed tomography in eosinophilic chronic rhinosinusitis with nasal polyps. Int Forum Allergy Rhinol 2016;6:812–819.
  PubMed
  
  CrossRef
1. Zhong B, Yuan T, Du J, Tan K, Yang Q, Liu F, et al. The role of preoperative blood eosinophil counts in distinguishing chronic rhinosinusitis with nasal polyps phenotypes. Int Forum Allergy Rhinol 2021;11:16–23.
  PubMed
  
  CrossRef
1. Yu L, Jiang Y, Yan B, Fang G, Wang C, Zhang L. Predictive value of clinical characteristics in eosinophilic chronic rhinosinusitis with nasal polyps: a cross-sectional study in the Chinese population. Int Forum Allergy Rhinol 2022;12:726–734.
  PubMed
  
  CrossRef
1. McHugh T, Levin M, Snidvongs K, Banglawala SM, Sommer DD. Comorbidities associated with eosinophilic chronic rhinosinusitis: a systematic review and meta-analysis. Clin Otolaryngol 2020;45:574–583.
  PubMed
  
  CrossRef
1. Hu Y, Cao PP, Liang GT, Cui YH, Liu Z. Diagnostic significance of blood eosinophil count in eosinophilic chronic rhinosinusitis with nasal polyps in Chinese adults. Laryngoscope 2012;122:498–503.
  PubMed
  
  CrossRef
1. Dweik RA, Boggs PB, Erzurum SC, Irvin CG, Leigh MW, Lundberg JO, et al. An official ATS clinical practice guideline: interpretation of exhaled nitric oxide levels (FENO) for clinical applications. Am J Respir Crit Care Med 2011;184:602–615.
  PubMed
  
  CrossRef
1. Kambara R, Minami T, Akazawa H, Tsuji F, Sasaki T, Inohara H, et al. Lower airway inflammation in eosinophilic chronic rhinosinusitis as determined by exhaled nitric oxide. Int Arch Allergy Immunol 2017;173:225–232.
  PubMed
  
  CrossRef
1. Lv H, Liu PQ, Xiang R, Zhang W, Chen SM, Kong YG, et al. Predictive and diagnostic value of nasal nitric oxide in eosinophilic chronic rhinosinusitis with nasal polyps. Int Arch Allergy Immunol 2020;181:853–861.
  PubMed
  
  CrossRef
1. Zhu M, Gao X, Zhu Z, Hu X, Zhou H, Liu J. The roles of nasal nitric oxide in diagnosis and endotypes of chronic rhinosinusitis with nasal polyps. J Otolaryngol Head Neck Surg 2020;49:68.
  PubMed
  
  CrossRef
1. Yoshida K, Takabayashi T, Imoto Y, Sakashita M, Narita N, Fujieda S. Reduced nasal nitric oxide levels in patients with eosinophilic chronic rhinosinusitis. Allergol Int 2019;68:225–232.
  PubMed
  
  CrossRef
1. Imbalzano E, Orlando L, Sciacqua A, Nato G, Dentali F, Nassisi V, et al. Machine learning to calculate heparin dose in COVID-19 patients with active cancer. J Clin Med 2021;11:219.
  PubMed
  
  CrossRef
1. Checcucci E, Autorino R, Cacciamani GE, Amparore D, De Cillis S, Piana A, et al. Artificial intelligence and neural networks in urology: current clinical applications. Minerva Urol Nefrol 2020;72:49–57.
  PubMed
1. Crowson MG, Ranisau J, Eskander A, Babier A, Xu B, Kahmke RR, et al. A contemporary review of machine learning in otolaryngology-head and neck surgery. Laryngoscope 2020;130:45–51.
  PubMed
  
  CrossRef
1. Lund VJ, Kennedy DW. Staging for rhinosinusitis. Otolaryngol Head Neck Surg 1997;117:S35–S40.
  PubMed
  
  CrossRef
1. Sakuma Y, Ishitoya J, Komatsu M, Shiono O, Hirama M, Yamashita Y, et al. New clinical diagnostic criteria for eosinophilic chronic rhinosinusitis. Auris Nasus Larynx 2011;38:583–588.
  PubMed
  
  CrossRef
1. Kursa MB, Rudnicki WR. Feature selection with the Boruta package. J Stat Softw 2010;36:1–13.
  CrossRef
1. Tu JV. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 1996;49:1225–1231.
  PubMed
  
  CrossRef
1. Frendø M, Håkansson K, Schwer S, Ravn AT, Meteran H, Porsbjerg C, et al. Exhaled and nasal nitric oxide in chronic rhinosinusitis patients with nasal polyps in primary care. Rhinology 2018;56:59–64.
  CrossRef
1. Gevaert P, Saenz R, Corren J, Han JK, Mullol J, Lee SE, et al. Long-term efficacy and safety of omalizumab for nasal polyposis in an open-label extension study. J Allergy Clin Immunol 2022;149:957–965.e3.
  PubMed
  
  CrossRef
1. Hopkins C, Browne JP, Slack R, Lund V, Brown P. The Lund-Mackay staging system for chronic rhinosinusitis: how is it used and what does it predict? Otolaryngol Head Neck Surg 2007;137:555–561.
  PubMed
  
  CrossRef
1. Tokunaga T, Sakashita M, Haruna T, Asaka D, Takeno S, Ikeda H, et al. Novel scoring system and algorithm for classifying chronic rhinosinusitis: the JESREC Study. Allergy 2015;70:995–1003.
  PubMed
  
  CrossRef
1. Hu X, Cammann H, Meyer HA, Miller K, Jung K, Stephan C. Artificial neural networks and prostate cancer--tools for diagnosis and management. Nat Rev Urol 2013;10:174–182.
  PubMed
  
  CrossRef
1. Harbaugh RE. Editorial. Artificial neural networks for neurosurgical diagnosis, prognosis, and management. Neurosurg Focus 2018;45:E3
  PubMed
  
  CrossRef
1. Ma RN, He YX, Bai FP, Song ZP, Chen MS, Li M. Machine learning model for predicting acute respiratory failure in individuals with moderate-to-severe traumatic brain injury. Front Med (Lausanne) 2021;8:793230
  PubMed
  
  CrossRef
1. Shi YL, Liu JY, Hu XJ, Tu LP, Cui J, Li J, et al. A new method for syndrome classification of non-small-cell lung cancer based on data of tongue and pulse with machine learning. Biomed Res Int 2021;2021:1337558
  PubMed
1. Chu CS, Lee NP, Adeoye J, Thomson P, Choi SW. Machine learning and treatment outcome prediction for oral cancer. J Oral Pathol Med 2020;49:977–985.
  PubMed
  
  CrossRef
1. Tong Z, Liu Y, Ma H, Zhang J, Lin B, Bao X, et al. Development, validation and comparison of artificial neural network models and logistic regression models predicting survival of unresectable pancreatic cancer. Front Bioeng Biotechnol 2020;8:196.
  PubMed
  
  CrossRef
1. Kawakami S, Numao N, Okubo Y, Koga F, Yamamoto S, Saito K, et al. Development, validation, and head-to-head comparison of logistic regression-based nomograms and artificial neural network models predicting prostate cancer on initial extended biopsy. Eur Urol 2008;54:601–611.
  PubMed
  
  CrossRef
1. Thorwarth RM, Scott DW, Lal D, Marino MJ. Machine learning of biomarkers and clinical observation to predict eosinophilic chronic rhinosinusitis: a pilot study. Int Forum Allergy Rhinol 2021;11:8–15.
  PubMed
  
  CrossRef
1. Wu CF, Wu YJ, Liang PC, Wu CH, Peng SF, Chiu HW. Disease-free survival assessment by artificial neural networks for hepatocellular carcinoma patients after radiofrequency ablation. J Formos Med Assoc 2017;116:765–773.
  PubMed
  
  CrossRef