The median age of these patients was 54.6 years, and the median progression-free survival (PFS) and the overall survival (OS) were 3.4 years and 3.6 years, respectively. Logistic regression is executed using the function glm from R. The optimal cutoff points are determined by two criteria. 1 ~G(2.5, 1),X0~G(1.5,1), and 1 was taken as 0.79, 1.22, 1.97, and 3.82, respectively; for the true cut-points cminP, cJ, cCZ, and cER, the results of Rota and Antolini's were used; for the true cut-point cIU, the empirically estimated objective function is maximized. 2). The approach is based on the area under the ROC curve (AUC), sensitivity, and specificity values. All material on this site has been provided by the respective publishers and authors. Statistically, the AIC values can help researchers with the choice. 20 Nov 2018 Stata's roccomp command is one of Stata's general-purpose programs for computing,. a character string with the name of the categorical covariate according to which optimal cutpoints are to be calculated. Although the previous efforts were targeted on multiple cutoff points, they all assumed that the number of cutoffs had already been known or decided, and a straight-forward way to facilitate choice of an optimal number of cutoffs still remains elusive. Before discharge from the hospital, 107 patients with COVID-19 (31.20%) died. As a result, we designed this study to find new optimal cut points in a larger population of COVID-19 patients, investigate the prognostic value of NT-proBNP in predicting survival time, and collect data on the time-dependent predictive accuracy of NT-proBNP levels. For the unbalanced design under normal homoscedastic distributions, the bootstrap standard deviation, coverage probability, and mean length of the 95% bootstrap CI for the cut-point are given in Table 6. Miller R., Siegmund D. Maximally selected chi square statistics. A cut-point for a biomarker is meaningful for the clinicians when it is clinically interpretable and understandable. Moreover, a low level of NT-proBNP was correlated with the most favorable prognosis, while high levels were associated with the worst prognosis (IR=0.007 for low,=0.025 for medium, and=0.037 for high, log-rank test P=0.012) (Table (Table3).3). As analyzed above, the optimal cutoff points for stromal invasion was 2, at fractions of 0.32 and 0.97. Severe COVID-19 patients died at a higher rate than non-severe patients, with an adjusted HR of 3.32 (95% CI: 1.596.97, P=0.001). Among all the methods, only two of them, the Youden index and the concordance probability, are based on the maximization of this rate. These examples illustrate that as with survival X-tile plots, correlative marker plots can also find biologically meaningful tumor subsets. Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach, GUID:EEEFB3C4-A783-49F9-A043-9FC28082F2AF, Generating data with the same properties given in this manuscript, Applying all methods to the data and estimating cut-points for all methods, Splitting data into two equal subsets, that is, subset I and subset II, Applying all methods to subset I and estimating cut-points for all methods, Assigning each observation in subset II to either one of two groups by using the cut-point obtained in the previous step, Applying all methods to new subset II and estimating cut-points for all methods, Assigning each observation in subset I to either one of two groups by using the cut-point obtained in the previous step, Applying all methods to the combination of these two subsets and estimating cut-points for all methods, Taking the difference between the cut-points obtained at the second step and at the last step. Cox Model Wald P-value selected cutpoints range from 13.5 to 54 pack years (p value <0.0001). Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirusinfected pneumonia in Wuhan. We have no bibliographic references for this item. To measure the diagnostic ability of a biomarker, it is common to use summary measures such as the area under the ROC curve (AUC) and/or the partial area under the ROC curve (pAUC) [1]. PLOS ONE promises fair, rigorous peer review, The second condition is not compulsory, but it is an essential condition when multiple cut-points satisfy the equation. R: A language and environment for statistical computing. The reason behind this idea is to provide the simplicity in defining the point as optimal. Learn more Consistent with previously published articles in our study, patients with higher levels of NT-proBNP and higher death likelihoods were at a higher risk of HTN, CAD, diabetes mellitus, kidney disease and had higher levels of BUN creatinine, leucocytes, and PCT. The following table presents the results. The basic statistic is s c k = i = 1 D ( e i + n i + e i n i) The sum is taken across times with observed events, to D , the largest of these. In determining optimal cut off and any point on ROC curve ie. over all cut-points c; c^J denotes the cut-point corresponding to J. The ePub format uses eBook readers, which have several "ease of reading" features For this scenario the relative bias for the minimum P value approach is larger than the bias given in their work. The analysis results in two gains: diagnostic accuracy of the biomarker and the optimal cut-point value. As the number of cutoff points, or equivalently, the number of risk groups, increases, the flexibility of the model also increases. Validation was performed with a testing cohort of 300 patients (Table 5). revealed that NT-proBNP improved the accuracy of high-sensitivity cardiac troponin T (hs-cTnT), as a prognostic factor of death and the analyzed outcomes [25]. All simulations are done by using R program with the version of 3.2.0. The https:// ensures that you are connecting to the e0176231. The functionality is limited to basic scrolling. The application omits all rows (observations) with NA values. Most of them were at an early stage: 60.6% at FIGO stage IA through IB and 21.8% at IIA through IIB. As shown in Table 6, when one cutoff was made, the testing results were quite unfavorable, while when two cutoffs were made, the results were comparable to Findcut. Our result showed that optimizing patients weight may have positive effect on their survival. This article proposes maximal concordance, a measure similar to the area under an ROC curve, as a metric for selecting an optimal cutpoint with censored endpoints and shows that selecting the cutpoint that maximizes the concordances probability is equivalent to maximizing the Youden index. In order to evaluate the significance of the optimally selected cut-point, twofold cross-validation process [16] is used. To do that you need to know: 1. sensitivity and specificity of the test operated at each cutpoint. Ziaie N, Maleh PA, Ramandi MMA, Pourkia R, Latifi K, Mansouri D. Transient left ventricular clot in COVID-19-related myocarditis is associated with hypereosinophilic syndrome: a case report. Combining natriuretic peptides and necrosis markers in the assessment of acute coronary syndromes. Effect of natriuretic peptides (BNP) gene T-381C polymorphism on the levels of BNP and NT-proBNP in patients with cardiovascular disease. This rectangle is constructed by connecting the intersections points of the lines of x = 1 AUC, y = AUC, x = 1 Sp(c), and y = Se(c). The site is secure. endobj In this study, we wondered what would be the risk if the cervical stroma was completely penetrated by tumor (fraction = 1). The normal homoscedastic unbalanced scenario, Bootstrap standard deviation, coverage probability, and mean length of the 95% confidence interval estimation of all methods. This method defines the optimal cut-point value as the value whose sensitivity and specificity are the closest to the value of the area under the ROC curve and the absolute value of the difference between the sensitivity and specificity values is minimum. . Both methods used here found the second cutoff point between medium and high risk to be at 0.96, a fraction at which the tumor penetrated the cervix stroma almost completely, presenting an alarming risk as the tumor may access other organs. NT-proBNP is a strong prognostic indicator of in-hospital death in the second week of admission. Among these studies, Gao et al. de Lemos JA, Morrow DA. The concordance probability method proposed by Liu [9] defines the optimal cut-point as the point maximizing the product of sensitivity and specificity. The study was approved by the Institutional Review Boards of the participating institutions, Rouhani Hospital and Babol University of Medical Sciences, and conducted in accordance with the guideline of the University Ethics Committee, approval No. Moreover, when BMI distribution was plotted against hazard ratio (HR), shown in Fig 2, the lowest HR occurred when BMI was around 23, and any value bigger or lower than it showed a higher HR. All codes and instructions can be found at https://osf.io/ef7na/#. Hence, further studies are advised for serial sampling from patients with COVID-19 to measure the dependency of NT-proBNP values on admission duration and find the highest predictive value of this biomarker. The IU method provides a cut-point whose sensitivity and specificity are equally high. Table 3 shows the results for the balanced design under normal homoscedastic distributions. The gamma balanced scenario, Bootstrap standard deviation, coverage probability, and mean length of the 95% confidence interval estimation of all methods. Therefore, PCT values increase earlier and get the normal range more rapidly than CRP [25]. Robin X., Turck N., Hainard A., et al. Therefore, the use of the IU method is advised to get more interpretable and better optimized cut-point. The results are shown in Figure1 in Supplementary Material available online at https://doi.org/10.1155/2017/3762651. Some interesting and novel aspects of our study have been emphasized here. According to this method, the optimal cut-point cERis the cthat achieves the minimum of the objective function ER(c)=SD(c)2+(SD(c)1)2over all possible cut-point values cof X(Fig. The coverage probabilities are close to the nominal level for all methods. This method defines the optimal cut-point value as the value whose sensitivity and specificity are the closest to the value of the area under the ROC curve and the absolute value of the difference between the sensitivity and specificity values is minimum. All authors provided comments on the manuscript at various stages of development. studied the dependency of NT-proBNP values on the time interval from the beginning of symptoms. In smash or pass terraria bosses. (1) 1 ~N(1, 1),X0~N(0,1), and 1 was taken as 0.51, 1.05, 1.68, and 2.56, respectively. The half-life of PCT is about 24h. By comparison, CRP takes 1224h to reach the peak and persists for up to 37days. This measure was first introduced to the medical literature by Youden [5]. The discrimination ability of NT-proBNP decreased until day 9, at which it became constant. To define the cut-point with the IU method, some of cut-points with their sensitivity and specificity values and AUC value are given. Discover a faster, simpler path to publishing in a high-quality journal. When the relative bias and MSE values of the IU method are compared with the previous methods, it is seen that the IU method is better than the others. X The upper part of Table 9 shows the cut-points obtained by using the previously proposed methods. For this design (under gamma distributions), the SDB and mean length of 95% CI values for the point closest-to-(0,1) corner method and the IU method are lower than the other investigated approaches (Table 8). The next step is to look for a cut-point from the coordinates of ROC whose sensitivity and specificity values are simultaneously so close or equal to 0.8. Myocardial injury is a common complication among patients with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections, correlated with poor outcomes [2]. Our study presented novel data, which can guide clinicians to better manage patients with COVID-19, based on the NT-proBNP plasma level at the hospital admission time. There are many methods proposed in the literature to obtain the optimal cut-point value. In another study on 138 patients with COVID-19, 7.2% and 16.7% of the subjects suffered from acute cardiac injury and arrhythmia, respectively. Nezhad MS, Seif F, Darazam IA, Samei A, Kamali M, Aazami H, et al. In this line, NT-proBNP levels lower than 331pg/mL and higher than 11,126pg/mL were associated with the longest and the shortest duration from admission to death, respectively. how to find zbrush serial number; code of ethics in project management; heavy duty mattress protector for incontinence; a mountain lake crossword clue; volunteer for paralympics; install monterey on late 2013 macbook pro; Details Package: OptimalCutpoints Type: Package Version: 1.1-5 Date: 2021-10-06 License: GPL In the OptimalCutpoints package all these methods have been incorporated in a way designed to be clear and user-friendly for the end-user. Kaplan-Meier curves for the three risk groups of BMI using the validation cohort of 300 patients. To obtain optimal cutoff points of CYFRA21-1 for these endpoints, a running log-rank statistical method was applied. At the same time, the number of parameters that needs to be estimated also increases. For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . 6 Its principle for finding the optimal cutoff point is to select the minimum P-value according to Kaplan-Meier survival curves and the log-rank test. The optimal cutoff points are defined as the ones with the most significant (likelihood ratio test) split, similar to the survival data. We provided an R function (findcutnum) to calculate AIC values for both survival and binary outcomes, the codes of which can be accessed at http://www.math.nsysu.edu.tw/~cchang/cutoff/findcutnum.txt. This product gets value between 0 and 1. Moreover, troponin is a marker of myocardial necrosis [26]. Based on several lines of evidence, myocardial injury is a common complication among hospitalized COVID-19 patients [2, 12]. To validate the result, we performed a multivariate analysis with the testing cohort of 300 patients, where tumor size was included as a binary variable with 3.25cm as the cutoff point side by side with several other variables, i.e., age, stage and histological types. Despite the lack of clinical meaning, it is shown in the literature that this method is superior to the other methods in estimating the true cut-point [11]. A biomarker with AUC = 1 discriminates individuals perfectly as diseased or healthy. In all scenarios, 1000 samples were generated with sample sizes 50, 100, and 200 for each group and with sample size n1 = 50, n0 = 100; n1 = 50, n0 = 150; and n1 = 50, n0 = 200 (n1 is the number of diseased subjects and n0 is the number of nondiseased subjects). The relative bias and MSE results for the unbalanced design under normal homoscedastic distributions are shown in Table 5. No surprise, another common criterion for choosing the most appropriate cut-off value is selecting the point on the ROC curve with the minimum distance from the . The procedure is then repeated in the resulting groups to obtain two supplementary cutoff values. The coverage probabilities are close to the nominal level for all methods. http://fmwww.bc.edu/repec/bocode/c/cutpt.ado, http://fmwww.bc.edu/repec/bocode/c/cutpt.sthlp, CUTPT: Stata module for empirical estimation of cutpoint for a diagnostic test, https://edirc.repec.org/data/debocus.html. A traditional method like X2 doesnt provide any assistance to decide the optimal number of cutoff points and one must make a decision beforehand, and if that decision is incorrectly made, the cutoff points subsequently located will not be optimal. i)JtWvSpP^(z,0@4_%oks:!>~\K|;4-% E:c*5LQIke? z?0 W GC& 3(`c7yx6c9^3=Ys5GSzC]5= kq7|l}xyd#.3y CG4z5p=ruv@> 1@zx8HVFf)}#Dx-QrWg The above criteria correspond to the following equation: The cut-point c^IU, which minimizes the IU(c) function and the |Se(c) Sp(c)| difference, will be the optimal cut-point value. Accessibility The first scenario is normal homoscedastic scenario with balanced design where all of the methods theoretically identify the same true cut-point. A cut-point will be referred to as optimal when the point classifies most of the individuals correctly [4, 5]. Interestingly, this correlation remained the same in both groups. For the balanced design under normal homoscedastic distributions, the bootstrap standard deviation, coverage probability, and mean length of the 95% bootstrap CI for the cut-point are shown in Table 4. You can help correct errors and omissions. Patients in the low NT-proBNP group were significantly younger with a lower prevalence of hypertension (HTN), coronary artery disease (CAD), diabetes mellitus, and kidney disease, as well as lower blood urea (BUN), procalcitonin (PCT), creatinine, white blood cell (WBC) and a lower level of troponin than those in the medium and high NT-proBNP groups (P<0.0001). All authors read and approved the final manuscript. PLoS ONE 12(4): on. If, for instance, 1 is taken as {0.79, 1.22, 1.97, 3.82}, the corresponding cut-points for each method will be different; that is, for minP approach, cminP = {0.80,1.73,2.54,3.51}, for Youden index, cJ = {1.12,1.79,2.45,3.42}, for the concordance probability, cCZ = {1.35,1.81,2.41,3.38}, and, for the point closest-to-(0,1) corner, cER = {1.38,1.82,2.36,3.24} [11]. Hypoxia stimulates release of ANP and BNP from perfused rat ventricular myocardium. Then, the conditional distribution of the quantitative variable X in group D is FD(c) = P(X cD) for D = 0, 1. KaplanMeier plots were generated and showed statistically significant differences in survival days between patients with low, medium, and high levels of NT-proBNP (Fig. previously reported by lroc;thus, the two commands are equivalent, and in fact, they Using the "R optimal cut-point package . 3. the losses associated with false positive and negative results. Requests for the patient data can be sent to Dr. JENG Shaw-Yeu, a member of the IRB, at wwwfm@vghks.gov.tw. The Cox PH based methods can still be used in survival studies to find multiple cutoff points, and logistic regression based methods in binomial outcomes. Before In Section 3, in order to compare the performance of the previous methods with that of the proposed one, generated data under the assumption of normal distribution and gamma distribution models for the biomarker are used. The Institutional Review Boards approved the study of the participating institutions, Rouhani Hospital and Babol University of Medical Sciences, and conducted by the guideline of the University Ethics Committee, approval No 724133037. We used the maximally selected rank statistics to determine the optimal cut points for NT-proBNP (the most significant split based on the standardized log-rank test). Such dichotomized outcome relies on logistic-regression based algorithms to find the optimal cutoff. These results could be applied to patients with squamous cell carcinoma. Using a time-dependent dynamic ROC curve enabled us to estimate a patients survival time in different time courses. These suggested underlying pathways result in increased ventricular wall stress and subsequent release of NT-proBNP. They recorded detailed histories of the 225 patients, including demographic characteristics, cardiovascular (CV) risk factors, and medication usage. Chung Chang, >> The proposed approach is based on the value of the area under the ROC curve. Perkins and Schisterman [4] stated that the optimal cut-point should be chosen as the point which classifies most of the individuals correctly and thus least of them incorrectly. AUC was 69.64% on day 4 and 59.57 on day 11. The results showed that advance stage and more than 5 metastatic lymph nodes independently associated patients with high risk for disease relapse and the subtype of squamous cell carcinoma independently put patients with low risk for relapse, while in the event of death, only the number of metastatic lymph nodes was significant (Table 2). In agreement with previous articles, the number of troponin-positive patients was significantly higher in our high category of NT-proBNP and associated with a shorter duration from admission to death. By analyzing several risk factors of cervical cancer such as tumor size, body mass index (BMI), number of lymph nodes involved and depth of stromal invasion, in relation to survival and clinical outcome such as lymph nodal metastasis and lymphovascular invasion, we demonstrated that the best choice for BMI and stromal invasion was two cutoff points and one for the others. The present study aimed to find new optimal cut points for NT-proBNP across censored survival failure time outcomes in hospitalized COVID-19 patients. According to the results, for each method, the difference between the optimal cut-points estimated before and after cross-validation is around 0 and the IU method gets the smallest mean absolute difference in all four scenarios. The optimal cut-point for this X-tile plot occurs at 45.7 ( P = 0.0005). Firstly, the algorithm splits the cohort into two groups by estimation of the optimal cutpoint with the highest log-rank statistics. The IU method achieves the smallest SDB value and the narrowest CIs in most of the scenarios. However, none of them evaluated this cardiac biomarker's informative value to estimate a patients survival time. This outcome-oriented method provides a cut point value that corresponds to the most significant relationship with the outcome (here, survival time to in-hospital death). In the following section, first the background methodologies of previous methods are summarized, and, then, the proposed method is introduced. Body mass index (BMI) is a good example of multiple cutoffs. This is a perfect time to use the log-rank test to see if they are actually different. Different cardiac biomarkers, including cardiac troponin I (cTnI), alpha-hydroxybutyrate dehydrogenase (-HBDH), myoglobin (Mb), lactate dehydrogenase (LDH), creatine phosphokinase (CPK), creatinine phosphokinase-muscle/brain (CPK-MB), aspartate aminotransferase (AST), and brain natriuretic peptide (BNP)/N-terminal of the prohormone brain natriuretic peptide (NT-proBNP), increase to a different extent among patients with Coronavirus disease of 2019 (COVID-19). The normal homoscedastic balanced scenarioa. In addition, a comprehensive tool that addresses both survival and dichotomized outcomes are also of interest to researchers. We also investigated our three low, medium, and high patient groups for their admission to the ICU. However, as a general rule, minimizing the total misclassification rates is a good approach. The procedure is shown to perform well in a simulation study. Lymph node metastasis is an important prognostic factor in cervical cancer [10]. . Cervical cancer represents a major public health problem worldwide, as the third most common female cancer ranking after breast and colorectal cancer [9]. Work flow of deciding the optimal number and locations of cutoff points. November 04, 2022 . For poor and poor-moderate classification accuracy (i.e., copt = 0.25 and 0.52), the MSE is the lowest for the IU method, and, for moderate-high and high classification accuracy (i.e., copt = 0.84 and 1.28), both the point closest-to-(0,1) corner method and the IU method get the lowest MSE values. Therefore, although X2 can give similar results to our package, its performance is dependent on whether the number of cutoff points can be correctly guessed prior to analysis. The second method uses X-tile, which is a free software available from Yale University School of Medicine that can determine a cutoff point for continuous data with a time-dependent outcome. According to the results given in the tables, the proposed IU method can be a better alternative for defining the cut-point. Arrow marks position of the lowest HR. mentioned that pneumonia induced by SARS-CoV-2 infection results in critical gas exchange obstruction, causing hypoxemia. We also used a traditional method, contingency table (X2), to perform similar analysis. You may switch to Article in classic view. Hence, it also provides an interpretable cut-point. The relative bias values of the previously proposed methods are similar to the results of Rota and Antolini's work [11] except the relative bias of Youden index. This approach is very practical. For this scenario, the lowest SDB and mean length of the 95% bootstrap CI values are obtained by the point closest-to-(0, 1) corner method and the IU method. We then looked at the AIC values in case of BMI in relation to OS, which were 749.9 with one cutoff and 744.9 with two, also favoring two cutoffs to one. Interestingly, we observed that NT-proBNP predictive information rose during the follow-up time. BMI has been associated with many diseases, and a high BMI may put one at risk for poor prognosis [12,13]. Then the IU function can be written as one of the following forms (according to the difference in the absolute value): That is, IU(c) = F0(c) + F1(c) + where , and are arbitrary (, = 1 or 1, 1 1). AUC, sensitivity, and specificity values are useful for the evaluation of a marker; however they do not specify optimal cut-points directly.