Table 2 - uploaded by Takeshi Umegaki
Content may be subject to copyright.
Source publication
To develop a prediction model of 28-day mortality in adult intensive care units using administrative data.
We obtained data from 33 ICUs in Japan on all adult patients discharged from ICUs in 2007. Three predictive models were developed using (i) the five variables of the Critical Care Outcome Prediction Equation (COPE) model (age, unplanned admiss...
Contexts in source publication
Context 1
... define admission categories, items in the administrative data pertaining to the course of admission were used. The "emergency" Table 2) ...
Context 2
... As Japan is a comparatively small country and development of access to hospitals has occurred through medical care policy, the hospital category (as defined in the COPE model) was assumed to be metropolitan. As International classification of diseases, 10th revision (ICD-10) codes rather than ICD- 10-AM (Australian modification) codes are used in Japan, we translated ICD-10-AM codes into their nearest equiva- lent ICD-10 codes (Table 2). In addition to mechanical ventilation, which is included in the COPE model, dialysis, pressors/vasoconstrictors, and use of fresh frozen plasma or a platelet preparation were considered as life-support interventions. ...
Similar publications
Central line associated bloodstream infections (CLABSIs) are a frequent source of health complication for patients of all ages, including for patients in the pediatric intensive care unit (PICU) and Pediatric Cardiothoracic Intensive Care Unit (PCTU). Many hospitals, including the University of Michigan Health System, currently use the Internationa...
Citations
... To address these issues, several administrative database studies have attempted to develop risk-adjustment models for ICU patients using data on patients' characteristics, comorbidities, and primary diagnoses; these models have shown good performance for predicting mortality. [9][10][11] In addition to data on patients' characteristics, comorbidities, and primary diagnoses, including information on therapeutic procedures for organ failure may improve risk adjustment and prediction of mortality in studies on ICU patients. Previous studies on surgical patients and noncritically ill patients have shown good performance of procedure-based risk adjustment models using administrative databases. ...
... Several administrative database studies have constructed mortality prediction models for critically ill patients using [9][10][11] To our knowledge, the present study is the first to construct a mortality prediction model considering timeseries information on procedures performed on the day of ICU admission, rather than procedures during ICU admission. The prognostic ability of our model was comparable or superior to the AUROC of 0.69 through 0.89 of the models presented in previous studies using administrative databases. ...
... The prognostic ability of our model was comparable or superior to the AUROC of 0.69 through 0.89 of the models presented in previous studies using administrative databases. [9][10][11] Previous prospective studies have demonstrated that SOFA score is a useful predictor of ICU mortality, with the AUROC ranging from 0.61 to 0.88. [23][24][25] The prognostic accuracy of the procedure-based organ failure assessment model in our study was comparable to that of SOFA score. ...
Aim
To develop a procedure-based organ failure assessment model for intensive care unit (ICU) patients and to examine the ability of this model to predict in-hospital mortality, with reference to the Sequential Organ Failure Assessment (SOFA) score.
Methods
Using the Japanese nationwide Diagnosis Procedure Combination database, we identified patients aged ≥15 years who were admitted to the ICUs April 2018–March 2019. Since April 2018, Japanese health care providers have been required to input ICU patients' SOFA scores into this database. We extracted data on the following procedures on ICU admission: oxygen supplementation, invasive mechanical ventilation, blood transfusions, catecholamines, chest compression, extracorporeal membrane oxygenation, and renal replacement therapy. A procedure-based organ failure assessment model (Model 1) for in-hospital mortality was developed using therapeutic procedures for organ failure on the day of ICU admission in the derivation cohort. We also constructed a model using the SOFA score (Model 2). Discriminatory ability was assessed using area under the receiver operating characteristic curve (AUROC) in the validation cohort, and the discriminatory abilities of the models were compared.
Results
In total, 69,019 patients were included. Overall in-hospital mortality was 7.2%. The AUROCs for Model 1 (0.810) and Model 2 (0.817) in the validation cohort did not show a statistically significant difference (P = 0.20).
Conclusion
The models established using procedure-based organ failure assessment showed no statistically significant differences from those using the SOFA score, suggesting that procedure records in administrative databases can be used for risk adjustment in clinical studies on ICU mortality.
... Characteristics of the mortality prediction models and underlying derivation cohorts are presented in Table 1. In all, 19 mortality prediction models (44%) were developed using prospectively collected data specifically gathered for the development of the prediction model, 6,13-27 whereas 24 (56%) were developed using either retrospective data [28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44] or prospective data previously collected for other purposes. [45][46][47][48][49] The start of data collection for the development cohorts spanned 36 years (1979-2015), and the duration of the cohort studies varying from 2 months up to 10 years for each cohort. ...
... Two mortality prediction models (4.7%) did not report the timespan during which their development cohort was assembled. 22,33 In all, 31 mortality prediction models (74%) were developed in a single country, 14,[18][19][20][21][22][23][24][25][26][27]29,31,[33][34][35][36][37][38][39][40][41][42][43][44][45]47,49 six (14%) in neighbouring countries (two or more) 6,13,28,30,32,46 and five (12%) were developed in multiple countries worldwide. [15][16][17]48 The number of patients included in the development databases ranged from 232 to 731 611 patients with a median of 4,895 (IQR 528-35 878). ...
... Hospital mortality was the most frequently used primary outcome in 29 (67%) mortality prediction models. 6,[13][14][15][16][17][18][19]21,22,24,27,28,[30][31][32][33]35,36,38,[41][42][43]45,46 Other primary outcome variables were ICU mortality (7%), 23,26,34 28-day mortality (4.7%), 39,44 90-day mortality (4.7%), 48,49 3-to 28-day mortality (4.7%), 40 30-day mortality (2.3%), 47 180-day mortality (2.3%), 20 6-month mortality (2.3%), 25 15-year mortality (2.3%), 37 and 6-and 12-month mortality (2.3%). 29 Secondary outcomes were 1-month mortality after ICU admission (4.7%), 24,31 hospital mortality (4.7%), 29,34 ICU mortality (2.3%), 45 3-month mortality after ICU admission (2.3%), 31 6-month mortality after ICU admission (2.3%), 31 9-month mortality (2.3%), 47 1-year mortality (2.3%) 45 and length of stay (2.3%). ...
Background:
Mortality prediction models are applied in the Intensive Care Unit (ICU) to stratify patients into different risk categories and to facilitate benchmarking. To ensure that the correct prediction models are applied for these purposes, the best performing models must be identified. As a first step, we aimed to establish a systematic review of mortality prediction models in critically ill patients.
Methods:
Mortality prediction models were searched in four databases using the following criteria: developed for use in adult ICU patients in high-income countries, with mortality as primary or secondary outcome. Characteristics and performance measures of the models were summarized. Performance was presented in terms of discrimination, calibration and overall performance measures presented in the original publication.
Results:
In total, 43 mortality prediction models were included in the final analysis. Fifteen models were only internally validated (35%), 13 externally (30%) and 10 (23%) were both internally and externally validated by the original researchers. Discrimination was assessed in 42 models (98%). Commonly used calibration measures were the Hosmer-Lemeshow test (60%) and the calibration plot (28%). Calibration was not assessed in 11 models (26%). Overall performance was assessed in the Brier score (19%) and the Nagelkerke's R2 (4.7%).
Conclusions:
Mortality prediction models have varying methodology, and validation and performance of individual models differ. External validation by the original researchers is often lacking and head-to-head comparisons are urgently needed to identify the best performing mortality prediction models for guiding clinical care and research in different settings and populations.
... To determine the categories of primary diagnoses, we used the World Health Organization International classification of diseases and related health problems 10th revision, to translate our administrative data into diagnostic categories in accordance with the methods used by Umegaki et al. [7] (Additional file 1: Table S1). There were four categories of time between hospital admission and initiation of MV (days) ( Table 1). ...
... Table 1 and Additional file 1: Table S1 show the variables that could potentially influence hospital mortality rates. These variables were chosen from previous studies [7][8][9]. The results of the multivariate analyses are shown in Table 4. ICU admission was significantly associated with a decrease in the hospital mortality rate (OR 0.713, 95% CI 0.676 to 0.753). ...
... We also performed multivariate data analysis using the administrative data, revealing that ICU admission was significantly associated with a decrease in mortality. The validity of similar data adjustment using administrative data was previously shown to have a similar effect on APACHE II or SOFA scores [7][8][9]. Another limitation is that we were unable to evaluate ventilator settings and ventilator-induced adverse events, because our database did not include information on ventilator management. In hospitals without ICU facilities, there is potential for a delay in the introduction of current standards of care and new technology in MV owing to a potential lack of awareness of updates in the rapidly changing field of critical care. ...
Background
In most countries, patients receiving mechanical ventilation (MV) are treated in intensive care units (ICUs). However, in some countries, including Japan, many patients on MV are not treated in ICUs. There are insufficient epidemiological data on these patients. Here, we sought to describe the epidemiology of patients on MV in Japan by comparing and contrasting patients on MV treated in ICUs and in non-ICU settings. A preliminary comparison of patient outcomes between ICU and non-ICU patients was a secondary objective.
Methods
Data on adult patients receiving MV for at least 3 days in ICUs or non-ICU settings from April 2010 through March 2012 were obtained from the Quality Indicator/Improvement Project, a voluntary data-administration project covering more than 400 acute-care hospitals in Japan. We excluded patients with cancer-related diagnoses. Patient demographic data and the critical care provided were compared between groups.
Results
Over the study period, 17,775 patients on MV were treated only in non-ICU settings, whereas 20,516 patients were treated at least once in ICUs (46.4% vs. 53.6%). Average age was higher in non-ICU patients than in ICU patients (72.8 vs. 70.2, P < 0.001). Mean number of ventilation days was greater in non-ICU patients (11.7 vs. 9.5, P < 0.001). Hospital mortality was higher in non-ICU patients (41.4% vs. 38.8%, P < 0.001). Standard critical care (e.g., arterial line placement, enteral nutrition, and stress-ulcer prevention) was provided significantly less often in non-ICU patients. Multivariate analysis showed that ICU admission significantly decreased hospital mortality (adjusted odds ratio 0.713, 95% CI 0.676 to 0.753).
Conclusions
A large proportion of Japanese patients on MV were treated in non-ICU settings. Analysis of administrative data indicated preliminarily that hospital mortality rates in these patients were higher in non-ICU settings than in ICUs. Prospective analyses comparing non-ICU and ICU patients on MV by severity scoring are needed.
Electronic supplementary material
The online version of this article (10.1186/s13054-018-2250-3) contains supplementary material, which is available to authorized users.
... As far as we known, vast majority of previous studies have primarily used the AUC or C-statistic solely to statistically check the model performance using fixed effect analyses ignoring the multilevel nature of the variance. [54][55][56][57]. On some occasions, researchers have applied mixed effect models considering the multilevel structure of the data [58,59], but they have not used the AUC for evaluating hospital general effects as this study does. ...
Background:
Hospital performance is frequently evaluated by analyzing differences between hospital averages in some quality indicators. The results are often expressed as quality charts of hospital variance (e.g., league tables, funnel plots). However, those analyses seldom consider patients heterogeneity around averages, which is of fundamental relevance for a correct evaluation. Therefore, we apply an innovative methodology based on measures of components of variance and discriminatory accuracy to analyze 30-day mortality after hospital discharge with a diagnosis of Heart Failure (HF) in Sweden.
Methods:
We analyzed 36,943 patients aged 45-80 treated in 565 wards at 71 hospitals during 2007-2009. We applied single and multilevel logistic regression analyses to calculate the odds ratios and the area under the receiver-operating characteristic (AUC). We evaluated general hospital and ward effects by quantifying the intra-class correlation coefficient (ICC) and the increment in the AUC obtained by adding random effects in a multilevel regression analysis (MLRA). Finally, the Odds Ratios (ORs) for specific ward and hospital characteristics were interpreted jointly with the proportional change in variance (PCV) and the proportion of ORs in the opposite direction (POOR).
Findings:
Overall, the average 30-day mortality was 9%. Using only patient information on age and previous hospitalizations for different diseases we obtained an AUC = 0.727. This value was almost unchanged when adding sex, country of birth as well as hospitals and wards levels. Average mortality was higher in small wards and municipal hospitals but the POOR values were 15% and 16% respectively.
Conclusions:
Swedish wards and hospitals in general performed homogeneously well, resulting in a low 30-day mortality rate after HF. In our study, knowledge on a patient's previous hospitalizations was the best predictor of 30-day mortality, and this information did not improve by knowing the sex and country of birth of the patient or where the patient was treated.
... We used a validated mortality prediction model using the Japanese Diagnosis Procedure Combination database (26). In this model, we used the following variables for predicting in-hospital mortality: 1) age, 2) admission category (emergency or scheduled), 3) reasons for ICU admission (scheduled surgery, emergency surgery, or medical treatments), 4) interval from hospital to ICU admission (direct to ICU, 1 d after hospital admission, and > 1 d after hospital admission), 5) mechanical ventilation, 6) renal replacement therapy, and 7) use of vasopressors. ...
... Emergency surgery patients were defined as those who underwent surgery on the day of hospital admission or the following day. Other surgical patients were defined as scheduled surgery patients (26). Hospital volume of ICU patients was defined as the average annual number of eligible patients admitted to the ICU. ...
... Second, the Diagnosis Procedure Combination database did not provide patients' severity scores, such as the Acute Physiology and Chronic Health Evaluation II Score (33) or the Sequential Organ Failure Assessment (34). However, the risk factors used in this study were well validated to predict in-hospital mortality (26). Third, it was not possible to identify several process of care, such as ventilation strategy or prophylaxis for thromboembolism, which affect outcomes in critically ill patients. ...
A volume-outcome relationship in ICU patients has been suggested in recent studies. However, it is unclear whether the ICU-to-hospital bed ratio affects the volume-outcome relationship. The aim of this study is to investigate the relationship between hospital volume and in-hospital mortality of adult ICU patients in relation to the ratio of ICU beds to regular hospital beds.
Retrospective cohort study.
Four hundred seventy-seven Japanese hospitals from 2007 to 2012 in the Japanese Diagnosis Procedure Combination database.
A total of 596,143 patients discharged from acute care hospitals.
None.
We analyzed data from 596,143 ICU patients from 2007 through 2012 using a nationwide administrative database. Patients were categorized into nine subgroups (the tertiles of hospital volume of ICU patients combined with the tertiles of ICU-to-hospital bed ratio). Multivariable logistic regression analyses were performed to examine the concurrent effects of hospital volume of ICU patients and ICU-to-hospital bed ratio on in-hospital mortality, with adjustment for patient and hospital characteristics. Higher hospital volume of ICU patients and a higher ICU-to-hospital bed ratio were independently associated with lower mortality. When patients were stratified by ICU-to-hospital bed ratio categories, in-hospital mortality was significantly lower in the high-volume subgroup (odds ratio, 0.74; 95% CI, 0.58-0.93) compared with the low-volume subgroup in hospitals with a high ICU-to-hospital bed ratio. However, these relationships were not significant in hospitals with low ICU-to-hospital bed ratios (odds ratio, 0.94; 95% CI, 0.59-1.50) or in hospitals with intermediate ICU-to-hospital bed ratios (odds ratio, 0.80; 95% CI, 0.71-1.08).
An inverse relationship between hospital volume of ICU patients and mortality was seen only when the ICU-to-hospital bed ratio was sufficiently high. Regionalization and increasing the number of ICU beds in referral centers may improve patient outcomes.
... Renal replacement therapy and pressors/vasoconstrictors were included in the candidate variables due to their reported association with 28-day hospital mortality in a previous study [20]. Renal replacement therapy included continuous renal replacement therapy, intermittent renal replacement therapy, plasma absorption, and plasma exchange, but excluded peritoneal dialysis due to its rare utilization for ICU patients. ...
Objective:
To develop an equation model of in-hospital mortality for mechanically ventilated patients in adult intensive care using administrative data for the purpose of retrospective performance comparison among intensive care units (ICUs).
Design:
Two models were developed using the split-half method, in which one test dataset and two validation datasets were used to develop and validate the prediction model, respectively. Nine candidate variables (demographics: age; gender; clinical factors hospital admission course; primary diagnosis; reason for ICU entry; Charlson score; number of organ failures; procedures and therapies administered at any time during ICU admission: renal replacement therapy; pressors/vasoconstrictors) were used for developing the equation model.
Setting:
In acute-care teaching hospitals in Japan: 282 ICUs in 2008, 310 ICUs in 2009, and 364 ICUs in 2010.
Participants:
Mechanically ventilated adult patients discharged from an ICU from July 1 to December 31 in 2008, 2009, and 2010.
Main outcome measures:
The test dataset consisted of 5,807 patients in 2008, and the validation datasets consisted of 10,610 patients in 2009 and 7,576 patients in 2010. Two models were developed: Model 1 (using independent variables of demographics and clinical factors), Model 2 (using procedures and therapies administered at any time during ICU admission in addition to the variables in Model 1). Using the test dataset, 8 variables (except for gender) were included in multiple logistic regression analysis with in-hospital mortality as the dependent variable, and the mortality prediction equation was constructed. Coefficients from the equation were then tested in the validation model.
Results:
Hosmer-Lemeshow χ(2) are values for the test dataset in Model 1 and Model 2, and were 11.9 (P = 0.15) and 15.6 (P = 0.05), respectively; C-statistics for the test dataset in Model 1and Model 2 were 0.70 and 0.78, respectively. In-hospital mortality prediction for the validation datasets showed low and moderate accuracy in Model 1 and Model 2, respectively.
Conclusions:
Model 2 may potentially serve as an alternative model for predicting mortality in mechanically ventilated patients, who have so far required physiological data for the accurate prediction of outcomes. Model 2 may facilitate the comparative evaluation of in-hospital mortality in multicenter analyses based on administrative data for mechanically ventilated patients.
... Other studies have been undertaken using study-specific comorbidity weights of the Charlson score [40] and Elixhauser score [41] to examine in-hospital mortality of critically ill populations, and these scores were found to improve prediction. Whereas the administrative-only model developed in this study performed better than those developed to predict in-hospital mortality of intensive care patients in the studies by Quach et al (C = 0.66) [29] and Poses et al (C = 0.67) [33], it performed more poorly than those developed by Duke et al (C = 0.87) [42] and Umegaki et al (C = 0.84) [43]. Although we used similar variables as the Duke et al model, their model was developed to predict inhospital death, which may explain the difference. ...
... Many administrative databases also routinely link to data from death registries to provide information on long-term patient survival. In the field of intensive care, models relying on administrative data have been developed for both in-hospital mortality [47] and 28-day mortality [43]. However, data quality and clinical accuracy risk adjustment methods require further external validation. ...
... [19] The COPE model uses information from standard administrative data and is a robust, risk-adjusted hospital mortality prediction tool. And we showed that the COPE model had good performance for ICU patients in Japan [20]. ...
Objective: The purpose of the study was to investigate associations among intensive care unit (ICU) staffing and care processes in patients with severe sepsis. Design: An observational multicenter cross-sectional study performed from October 2007 to March 2008. Setting: Forty-nine teaching hospitals in Japan. Participants: Patients (n=576) with severe sepsis identified using ICD-10 codes from administrative data. Main outcome measures: Care processes including mechanical ventilation, dialysis, enteral feeding, parentetal nutrition, and antibiotic empirical therapy which were available in administrative data. Results: ICUs were classified as high- or low-intensity based on policies regarding the responsibilities of intensivists. There were no differences in baseline patient characteristics between the ICU groups. In the high-intensity group, ICU stay for survivors was about two days shorter and hospital stay was significantly shorter by three days. Majority of patients had high rates of enteral feeding; however, the high-intensity group had significantly earlier initiation of enteral feeding and a significantly shorter duration of mechanical ventilation. A shorter duration of mechanical ventilation was significantly associated with the ICU structure. Conclusions: The results showed an association between ICU physician and processes of intensive care, and high-intensity ICU was aggressive in mechanical ventilation in patients with severe sepsis.
Objectives:
In a recent scoping review, we identified 43 mortality prediction models for critically ill patients. We aimed to assess the performances of these models through external validation.
Design:
Multicenter study.
Setting:
External validation of models was performed in the Simple Intensive Care Studies-I (SICS-I) and the Finnish Acute Kidney Injury (FINNAKI) study.
Patients:
The SICS-I study consisted of 1,075 patients, and the FINNAKI study consisted of 2,901 critically ill patients.
Measurements and main results:
For each model, we assessed: 1) the original publications for the data needed for model reconstruction, 2) availability of the variables, 3) model performance in two independent cohorts, and 4) the effects of recalibration on model performance. The models were recalibrated using data of the SICS-I and subsequently validated using data of the FINNAKI study. We evaluated overall model performance using various indexes, including the (scaled) Brier score, discrimination (area under the curve of the receiver operating characteristics), calibration (intercepts and slopes), and decision curves. Eleven models (26%) could be externally validated. The Acute Physiology And Chronic Health Evaluation (APACHE) II, APACHE IV, Simplified Acute Physiology Score (SAPS)-Reduced (SAPS-R)' and Simplified Mortality Score for the ICU models showed the best scaled Brier scores of 0.11' 0.10' 0.10' and 0.06' respectively. SAPS II, APACHE II, and APACHE IV discriminated best; overall discrimination of models ranged from area under the curve of the receiver operating characteristics of 0.63 (0.61-0.66) to 0.83 (0.81-0.85). We observed poor calibration in most models, which improved to at least moderate after recalibration of intercepts and slopes. The decision curve showed a positive net benefit in the 0-60% threshold probability range for APACHE IV and SAPS-R.
Conclusions:
In only 11 out of 43 available mortality prediction models, the performance could be studied using two cohorts of critically ill patients. External validation showed that the discriminative ability of APACHE II, APACHE IV, and SAPS II was acceptable to excellent, whereas calibration was poor.
Background
Excess morbidity and mortality following critical illness is increasingly attributed to potentially avoidable complications occurring as a result of complex ICU management (Berenholtz et al., J Crit Care 17:1-2, 2002; De Vos et al., J Crit Care 22:267-74, 2007; Zimmerman J Crit Care 1:12-5, 2002). Routine measurement of quality indicators (QIs) through an Electronic Health Record (EHR) or registries are increasingly used to benchmark care and evaluate improvement interventions. However, existing indicators of quality for intensive care are derived almost exclusively from relatively narrow subsets of ICU patients from high-income healthcare systems. The aim of this scoping review is to systematically review the literature on QIs for evaluating critical care, identify QIs, map their definitions, evidence base, and describe the variances in measurement, and both the reported advantages and challenges of implementation.
Method
We searched MEDLINE, EMBASE, CINAHL, and the Cochrane libraries from the earliest available date through to January 2019. To increase the sensitivity of the search, grey literature and reference lists were reviewed. Minimum inclusion criteria were a description of one or more QIs designed to evaluate care for patients in ICU captured through a registry platform or EHR adapted for quality of care surveillance.
Results
The search identified 4780 citations. Review of abstracts led to retrieval of 276 full-text articles, of which 123 articles were accepted. Fifty-one unique QIs in ICU were classified using the three components of health care quality proposed by the High Quality Health Systems (HQSS) framework. Adverse events including hospital acquired infections (13.7%), hospital processes (54.9%), and outcomes (31.4%) were the most common QIs identified. Patient reported outcome QIs accounted for less than 6%. Barriers to the implementation of QIs were described in 35.7% of articles and divided into operational barriers (51%) and acceptability barriers (49%).
Conclusions
Despite the complexity and risk associated with ICU care, there are only a small number of operational indicators used. Future selection of QIs would benefit from a stakeholder-driven approach, whereby the values of patients and communities and the priorities for actionable improvement as perceived by healthcare providers are prioritized and include greater focus on measuring discriminable processes of care.
