Content uploaded by Anne Elixhauser
Author content
All content in this area was uploaded by Anne Elixhauser on Jan 06, 2014
Content may be subject to copyright.
ORIGINAL CONTRIBUTION
Enhancement of Claims Data to Improve
Risk Adjustment of Hospital Mortality
Michael Pine, MD, MBA
Harmon S. Jordan, ScD
Anne Elixhauser, PhD
Donald E. Fry, MD
David C. Hoaglin, PhD
Barbara Jones, MA
Roger Meimban, PhD
David Warner, MS
Junius Gonzales, MD, MBA
RISK-ADJUSTED HOSPITAL MOR-
tality rates for specified con-
ditions and procedures fre-
quently are used in public
reports and pay-for-performance pro-
grams as indicators of the quality of hos-
pital care.1-3 Risk adjustment often is
based solely on administrative claims
data from uniform bills that hospitals
submit to payers. These data lack clini-
cally important pathophysiological in-
formation and do not distinguish be-
tween conditions that were present on
admission (POA; ie, potential risk fac-
tors) and complications that occurred
during hospitalization. The validity of
risk-adjustment systems that use only
administrative data has been chal-
lenged repeatedly,4-9 and there is gen-
eral agreement10-23 that additional data
are required to predict accurately an in-
dividual patient’s risk of dying.
Physicians are particularly con-
cerned that inadequate risk adjust-
ment penalizes the practitioners and fa-
cilities that care for the sickest patients
and may result in the denial of needed
care to high-risk patients.24-26 Con-
sumer advocates and payers, through
initiatives such as the Consumer-
Purchaser Disclosure Project,27 are at-
tempting to expand administrative data
sets to include clinical information to
ensure that incentives reward high-
quality clinical care. On the other hand,
many hospital administrators have com-
plained that the cost of retrieving
supplementary clinical data from medi-
cal records is prohibitive, and some re-
searchers have argued that risk adjust-
ment using only administrative data can
be made sufficiently accurate to sup-
port valid comparisons among hospi-
tals.28,29
The addition of a POA modifier for
secondary diagnosis codes first was pro-
posed in 199130 and was successfully
adopted in New York State in 1994 and
in California in 1996.31 The planned
implementation in March 2007 of new
standards for hospital claims data in-
cludes nationwide adoption of this
Author Affiliations: Michael Pine and Associates Inc,
Chicago, Ill (Drs Pine, Fry, and Meimban, and Ms
Jones); Department of Medicine, Pritzker School of
Medicine, University of Chicago, Chicago, Ill (Dr Pine);
Abt Associates Inc, Cambridge, Mass (Drs Jordan, Hoag-
lin, Gonzales, and Mr Warner); School of Medicine,
Tufts University, Boston, Mass (Dr Jordan); and Agency
for Healthcare Research and Quality, Rockville, Md
(Dr Elixhauser).
Corresponding Author: Michael Pine, MD, MBA, 1210
Chicago Ave, Suite 503, Evanston, IL 60202 (mpine
@aol.com).
Context Comparisons of risk-adjusted hospital performance often are important com-
ponents of public reports, pay-for-performance programs, and quality improvement
initiatives. Risk-adjustment equations used in these analyses must contain sufficient
clinical detail to ensure accurate measurements of hospital quality.
Objective To assess the effect on risk-adjusted hospital mortality rates of adding present
on admission codes and numerical laboratory data to administrative claims data.
Design, Setting, and Patients Comparison of risk-adjustment equations for in-
patient mortality from July 2000 through June 2003 derived by sequentially adding
increasingly difficult-to-obtain clinical data to an administrative database of 188 Penn-
sylvania hospitals. Patients were hospitalized for acute myocardial infarction, conges-
tive heart failure, cerebrovascular accident, gastrointestinal tract hemorrhage, or pneu-
monia or underwent an abdominal aortic aneurysm repair, coronary artery bypass graft
surgery, or craniotomy.
Main Outcome Measures Cstatistics as a measure of the discriminatory power
of alternative risk-adjustment models (administrative, present on admission, labora-
tory, and clinical for each of the 5 conditions and 3 procedures).
Results The mean (SD) cstatistic for the administrative model was 0.79 (0.02). Add-
ing present on admission codes and numerical laboratory data collected at the time of
admission resulted in substantially improved risk-adjustment equations (mean [SD] c
statistic of 0.84 [0.01] and 0.86 [0.01], respectively). Modest additional improve-
ments were obtained by adding more complex and expensive to collect clinical data
such as vital signs, blood culture results, key clinical findings, and composite scores
abstracted from patients’ medical records (mean [SD] cstatistic of 0.88 [0.01]).
Conclusions This study supports the value of adding present on admission codes
and numerical laboratory values to administrative databases. Secondary abstraction
of difficult-to-obtain key clinical findings adds little to the predictive power of risk-
adjustment equations.
JAMA. 2007;297:71-76 www.jama.com
©2007 American Medical Association. All rights reserved. (Reprinted) JAMA, January 3, 2007—Vol 297, No. 1 71
at Attn Clinical Affairs, on January 3, 2007 www.jama.comDownloaded from
modifier, which distinguishes condi-
tions that develop during hospital stays
(potential complications of care) from
conditions that were present at admis-
sion (potential treatment-indepen-
dent risk factors). Inclusion of POA
codes in administrative data sets should
permit analysts to incorporate impor-
tant predictors of inpatient mortality
into administrative risk-adjustment
equations without improperly desig-
nating patients as having high intrin-
sic risks at admission when their in-
creased vulnerability resulted from
hospital-acquired complications.31
The adoption of Logical Observa-
tion Identifiers Names and Codes32 for
laboratory data and advances in elec-
tronic health data technology have low-
ered the cost of retrieving numerical
laboratory data at many hospitals.33 Be-
cause of substantial differences in the
cost of obtaining various types of clini-
cal data, limited enhancement of ad-
ministrative data sets appears to be both
practical and desirable. Ideally, clini-
cal data elements selected for this pur-
pose will be relatively inexpensive to
obtain and will be useful predictors of
mortality for multiple conditions and
procedures.
This study was designed to test the
hypothesis that the combination of POA
modifiers for secondary diagnoses and
a limited set of numerical laboratory
data would improve risk adjustment of
inpatient mortality for a diverse set of
clinical conditions and procedures. We
also hypothesized that further addi-
tions of highly specific, difficult-to-
obtain clinical data sometimes consid-
ered important predictors of inpatient
mortality by clinicians would add little
to the accuracy of predictive models.
METHODS
Risk-adjustment models were created
and analyzed using data from July 2000
through June 2003 from 188 Pennsyl-
vania hospitals supplied by the Penn-
sylvania Health Care Cost Contain-
ment Council.34 Case-level claims data
were supplemented with clinical data
abstracted from medical records by spe-
cially trained personnel using Medi-
Qual’s proprietary Atlas clinical infor-
mation system.35 This system defines a
broad array of clinical data elements,
including historical information, labo-
ratory results, vital signs, clinical symp-
toms and signs, pathophysiological
abnormalities, and composite patho-
physiological scores, which are col-
lected and stored along with the hos-
pital day on which each clinical finding
was observed.
Risk-adjusted mortality rates were
analyzed for 5 health conditions (acute
myocardial infarction, congestive heart
failure, acute cerebrovascular acci-
dent, gastrointestinal tract hemor-
rhage, or pneumonia) and 3 surgical
procedures (abdominal aortic aneu-
rysm repair, coronary artery bypass graft
surgery, or craniotomy). The Agency for
Healthcare Research and Quality’s In-
patient Quality Indicator software ver-
sion 2.1 was used to identify cases that
met criteria for inclusion in each
group.36
Four models were constructed for
each condition and procedure (1 set of
data for each of the 5 conditions and 1
set of data for each of the 3 proce-
dures). The first model termed admin-
istrative used standard claims data. The
second model termed POA used data ab-
stracted from medical records to deter-
mine whether coded secondary diag-
noses had been present at admission.
The third model termed laboratory used
POA codes and numerical laboratory
data (often available in electronic form;
eg, creatinine, hematocrit level) docu-
mented on the first day of hospitaliza-
tion prior to a procedure requiring gen-
eral or regional anesthesia. The fourth
model termed clinical used the criteria
in the third model plus vital signs, other
laboratory data not included in the third
model (eg, bacterial culture results), At-
las key clinical findings abstracted from
medical records (eg, immunocompro-
mised, lethargy), and composite clini-
cal scores (ie, American Society of An-
esthesiologists classification, Glasgow
Coma Score) documented on the first
day of hospitalization prior to a proce-
dure requiring general or regional an-
esthesia.
The administrative model was based
solely on data from hospital bills (ie,
age, sex, and principal diagnoses, sec-
ondary diagnoses, and procedures
coded according to the International
Classification of Diseases, Ninth Revi-
sion, Clinical Modification [ICD-9-
CM]). To avoid using hospital-
acquired complications as risk factors,
hospital bills from New York and Cali-
fornia (secondary diagnoses were modi-
fied by POA codes in these states) were
used to help identify which secondary
diagnoses were generally present at ad-
mission. Secondary diagnoses were eli-
gible for inclusion as risk factors in the
administrative model only when they
were coded as hospital-acquired com-
plications in fewer than 20% of cases
in which they occurred.
The POA model included addi-
tional secondary diagnoses excluded
from the administrative model be-
cause of their association with unac-
ceptably high rates of complications. Be-
cause Pennsylvania claims data do not
include POA codes, clinical data in the
Atlas database were used to determine
whether coded secondary diagnoses
were present at admission. In the cre-
ation of surrogate POA codes, the At-
las database served as a substitute for
the complete medical record available
to coders in New York and California.
For example, posthemorrhagic ane-
mia was excluded from the adminis-
trative model for congestive heart fail-
ure because hospitals in New York and
California coded it as acquired during
hospitalization in more than 30% of the
cases in which it occurred. However,
posthemorrhagic anemia was eligible
for inclusion as a risk factor in the POA
model for congestive heart failure when
the Atlas database documented that
anemia was present on the day of ad-
mission.
For each condition or procedure,
candidate risk factors were con-
structed from principal diagnosis codes,
up to 8 secondary diagnosis codes, up
to 6 procedure codes, and clinical data
elements associated with higher than
average mortality rates. Infrequently oc-
curring codes were combined with
CLAIMS DATA FOR IMPROVING RISK-ADJUSTED HOSPITAL MORTALITY
72 JAMA, January 3, 2007—Vol 297, No. 1 (Reprinted) ©2007 American Medical Association. All rights reserved.
at Attn Clinical Affairs, on January 3, 2007 www.jama.comDownloaded from
codes for clinically similar conditions
or procedures that had similar mortal-
ity rates. Continuous measures (eg, age,
creatinine level) were transformed into
1 or more categorical variables based
on clinical judgment and empirical
evaluation of associated mortality rates.
For each condition or procedure,
stratified random samples of live dis-
charges and fatalities were combined to
create 3 mutually exclusive data sets:
a training set (50%), a validation set
(25%), and a test set (25%). Partition-
ing the data in this way facilitated the
construction of more robust models.
A preliminary predictive equation
was developed on the training set for
each condition or procedure using only
age categories and individual hospital
identifiers. (Including hospitals as risk
factors during model development is a
standard technique for reducing pos-
sible bias caused by the associations be-
tween the prevalence of potential risk
factors at individual hospitals and the
quality of care provided by those hos-
pitals.) For the administrative, POA,
and laboratory models, additional po-
tential risk factors were added in a se-
quence determined by forward step-
wise logistic regression.37 To avoid
overfitting, variables added after the
minimum value of the Schwarz crite-
rion38 was attained were removed from
models. (This criterion weighs the
trade-off between the fit of a model and
its complexity.) The remaining predic-
tive variables and their coefficients were
evaluated for clinical plausibility. On
rare occasions, clinically problematic
variables were eliminated or modi-
fied. To avoid substituting more ex-
pensive clinical variables for less costly
ones with almost equivalent predic-
tive power, predictive variables se-
lected for the laboratory model were re-
tained and additional clinical risk
factors were added in sequence as de-
scribed above.
For each of the 4 models (adminis-
trative, POA, laboratory, and clinical)
for each condition or procedure, a
nested sequence of models was cre-
ated first with 1 variable selected us-
ing the training data set, then with 2
variables, and lastly with all the vari-
ables. Variables were added to succes-
sive models in the order in which they
were entered in the minimum Schwarz
criterion model. From each nested se-
quence of models, the validation set was
used to select the model with the small-
est average prediction error39 as the fi-
nal validated model. Finally, the coef-
ficients of the variables in the validated
models were retained, the hospital vari-
ables were removed, and the inter-
cepts were recalculated to equate ob-
served and predicted mortality rates.
Case-level discriminatory power (ie,
the ability of a model to distinguish
cases that died from those that sur-
vived) was computed on the test set us-
ing cstatistics.40 All data management
and statistical analyses were per-
formed using SAS software versions 8
and 9.1 (SAS Institute Inc, Cary, NC).
The study design was approved by the
Abt Associates’ institutional review
board.
RESULTS
The numbers of cases ranged from 5309
(abdominal aortic aneurysm repair) to
200 506 (congestive heart failure)
(TABLE). Mortality rates ranged from
3.2% for coronary artery bypass graft
surgery to 10.8% for acute cerebrovas-
cular accident.
Designating secondary diagnoses as
present at admission increased the av-
erage number of secondary diagnosis
variables included from 8.6 in the ad-
ministrative model to 15.4 in the POA
model. This increase occurred be-
cause secondary diagnoses such as acute
renal failure in patients admitted to the
hospital with pneumonia, who also had
elevated creatinine levels on the day of
admission, were eligible for inclusion
as risk factors in the POA model. Com-
parison of the POA model and the labo-
ratory model revealed that the addi-
tion of an average of 11.1 numerical
laboratory values present on the first
hospital day was accompanied by an av-
erage reduction of 4.5 secondary diag-
nosis variables. This reduction re-
flected the substitution of more specific
laboratory values for less specific sec-
ondary diagnosis variables (eg, pH
ⱕ7.25 or pH ⬎7.25 but ⱕ7.35 re-
placed the ICD-9-CM secondary diag-
nosis code for acidosis in acute myo-
cardial infarction). Compared with the
laboratory model, an average of 9 ad-
ditional clinical findings present on the
first hospital day were incorporated into
the clinical model.
The final models included a total of
20 numerical laboratory determina-
tions, 3 other laboratory determina-
tions (eg, blood cultures), 5 vital signs,
22 key clinical findings, and 2 compos-
ite scores. Many individual numerical
laboratory results and vital signs ap-
peared in the clinical models for 4 or
more conditions or procedures (eg, pH
and prothrombin time were risk fac-
tors in the clinical models for all 5 con-
ditions and 3 procedures). On the other
hand, few key clinical findings ap-
peared in the models for more than 2 of
the conditions and procedures. The
Glasgow Coma Score was a risk factor
for 3 of the 5 conditions and 1 of the 3
procedures and the American Society of
Table. Number of Hospitals, Cases, and Fatalities and Mortality Rate for Each Condition and
Procedure
Condition or Procedure
No. of
Hospitals
No. of
Cases
No. of
Deaths
Mortality
Rate, %
Pneumonia 188 176 696 14 552 8.2
Congestive heart failure 187 200 506 8739 4.4
Acute cerebrovascular accident 187 82 682 8960 10.8
Gastrointestinal tract hemorrhage 187 75 392 2507 3.3
Acute myocardial infarction 184 104 110 9821 9.4
Abdominal aortic aneurysm repair 139 5309 557 10.5
Craniotomy 100 16 928 1169 6.9
Coronary artery bypass graft surgery 63 58 879 1890 3.2
CLAIMS DATA FOR IMPROVING RISK-ADJUSTED HOSPITAL MORTALITY
©2007 American Medical Association. All rights reserved. (Reprinted) JAMA, January 3, 2007—Vol 297, No. 1 73
at Attn Clinical Affairs, on January 3, 2007 www.jama.comDownloaded from
Anesthesiologists classification was a risk
factor for the 2 other procedures.
The receiver operating characteris-
tic curves reflecting the average csta-
tistics of alternative models are shown
in the FIGURE. The average cstatistic
increased from 0.50 for no risk adjust-
ment to a mean (SD) of 0.79 (0.02) for
the administrative model, to 0.84 (0.01)
for the POA model, to 0.86 (0.01) for
the laboratory model, to 0.88 (0.01) for
the clinical model.
COMMENT
This study was designed to guide the
selection of a cost-effective set of clini-
cal data elements to improve the valid-
ity of comparisons of risk-adjusted hos-
pital mortality rates. Because these
comparisons often are important com-
ponents of public reports, pay-for-
performance programs, and quality im-
provement initiatives, it is essential that
they accurately reflect the quality of care
provided by each facility.41 Unlike most
previous studies that attempted to de-
rive the most parsimonious or most so-
phisticated risk-adjustment model for
a single condition or procedure or to
compare models based on administra-
tive data to corresponding models based
on clinical data, the principal goal of this
study was to evaluate the relative per-
formances of alternative equations
based on progressively more detailed
data sets and identify one that could
meet the sometimes conflicting needs
of physicians, hospital administrators,
and payers. Therefore, a diverse sample
of conditions and procedures was evalu-
ated, and methodological uniformity
was emphasized to minimize the con-
founding effects of differences in ana-
lytic technique and to obtain precise es-
timates of improvements in the risk
adjustment directly attributable to
changing the type of data available for
use in the predictive equations.
In deriving the administrative and
POA models, care was taken to avoid
using risk factors based on conditions
or procedures that reflected poten-
tially avoidable hospital-acquired com-
plications rather than intrinsic patient
risks at the time of admission. For both
the administrative and POA models, the
use of procedure codes as risk factors
was limited to situations in which they
were found to be irreplaceable surro-
gates for intrinsic patient risk. In the ad-
ministrative model, secondary diag-
noses were eligible for use as risk factors
only when a separate analysis docu-
mented that they only rarely reflected
hospital-acquired complications. In the
POA model, secondary diagnoses in-
eligible for the administrative model
were considered as potential risk fac-
tors only if the Atlas clinical data sub-
stantiated their presence on the first day
of hospitalization.
A national standard for adding a POA
code to administrative claims data in the
UB-04 (the uniform bill used to sub-
mit all hospital claims to payers) is
planned as part of the revised ICD-
9-CM coding modifications for 2007.31,42
In this study, the use of a surrogate for
this code resulted in noteworthy im-
provements in the performance of the
risk-adjustment models, confirming the
value of this new coding convention.
Substantial additional improvements in
the performance of risk-adjustment
models occurred when numerical labo-
ratory values were added to the POA
codes. The sum of all further improve-
ments from adding other clinical data
elements was substantially less than
improvements achieved by adding sur-
rogate POA coding and numerical labo-
ratory values to the standard admin-
istrative data.
Data collected by the Pennsylvania
Health Care Cost Containment Coun-
cil43 demonstrated that when hospi-
tals routinely collect a specified set of
clinical data elements on a large num-
ber of discharges, they can reduce the
cost of retrieving these data by invest-
ing in standardized record formats and
electronic aids to data collection and by
training less expensive personnel to ob-
tain required data quickly and accu-
rately. In addition, a recent study44 by
HIMSS Analytics found that 80.8% of
hospitals had the computerized labo-
ratory systems required to support the
laboratory models. Therefore, many
Figure. Receiver Operating Characteristic Curves for the Models
No Risk Adjustment
Laboratory Model
Administrative Model
POA Model
Clinical Model
100
60
40
80
20
0
0
100 80 60 40 20 0
0%
2%
4%
6%
8%
100%
20 40 60 80 100
100 – Specificity, %
Specificity, %
Sensitivity, %
The 4 receiver operating characteristic curves represent data from each of the 4 risk-adjustment models: stan-
dard administrative model; present at admission (POA) model, standard administrative model with POA modi-
fiers; laboratory model, POA plus numerical laboratory data; and clinical model, laboratory model plus all avail-
able clinical data. The diagonal dotted line indicates no risk adjustment. The data markers represent cut points
of mortality risk predicted by each model in increments of 2%. Mortality rate cut points shown in the plot
include 0% (upper right), 2% (gray), 4% (blue), 6% (pink), 8% (black), and 100% (bottom left). Each model
is based on 8 data sets (1 set of data for each of the 5 conditions and 1 set of data for each of the 3 proce-
dures). Predicted mortality rates from the 8 data sets in each model were averaged and compared with the
observed values to calculate the true positive rate (sensitivity, where positive equals dead) and false-positive
rate (100 minus specificity) at each mortality risk cut point.
CLAIMS DATA FOR IMPROVING RISK-ADJUSTED HOSPITAL MORTALITY
74 JAMA, January 3, 2007—Vol 297, No. 1 (Reprinted) ©2007 American Medical Association. All rights reserved.
at Attn Clinical Affairs, on January 3, 2007 www.jama.comDownloaded from
hospitals currently should be capable
of electronically merging administra-
tive and numerical laboratory data,
thereby reducing their costs of acquir-
ing laboratory data by eliminating the
need for manual abstraction. In con-
trast, only 10.6% of hospitals cur-
rently have computerized nursing docu-
mentation required to support models
that include vital signs and other clini-
cal data, although rapid improve-
ments in health information technol-
ogy are anticipated over the next few
years.33
The present study was limited by its
use of only 1 indicator of hospital qual-
ity (ie, mortality) and by its failure to
evaluate directly the effects of varia-
tions in coding practices and in the
number of secondary diagnosis codes
included in the centralized databases.
Recommendations about the inclu-
sion of specific data elements within
each level of clinical data may not ap-
ply in all circumstances because some
data elements not identified in this
study might prove to be important for
outcomes and conditions outside the
scope of this investigation. In addi-
tion, measures of function were not
available but have been shown to have
value in predicting outcomes.45
In summary, this analysis strongly
supports the value of enhancing ad-
ministrative claims data with POA codes
and a limited set of numerical labora-
tory values obtained at admission.
These data provide information re-
quired to avoid errors in the designa-
tion of hospitals and their medical staffs
as delivering better than average or
worse than average care. On the other
hand, secondary abstraction of difficult-
to-obtain key clinical findings appears
to add little to the risk adjustment of
inpatient mortality rates.
Author Contributions: Dr Pine had full access to all
of the data in the study and takes responsibility for
the integrity of the data and the accuracy of the data
analysis.
Study concept and design: Pine, Jordan, Elixhauser,
Hoaglin.
Acquisition of data: Jordan, Elixhauser, Jones.
Analysis and interpretation of data: Pine, Jordan, Fry,
Hoaglin, Jones, Meimban, Warner, Gonzales.
Drafting of the manuscript: Pine, Jordan, Fry, Jones,
Meimban, Warner.
Critical revision of the manuscript for important in-
tellectual content: Jordan, Elixhauser, Hoaglin,
Gonzales.
Statistical analysis: Pine, Jordan, Hoaglin, Jones,
Meimban.
Obtained funding: Pine, Jordan, Elixhauser.
Administrative, technical, or material support: Pine,
Jordan, Elixhauser, Warner.
Study supervision: Pine, Jordan, Elixhauser.
Financial Disclosures: Dr Hoaglin reported owning
shares of stock in Cardinal Health Inc. None of the other
authors reported any disclosures.
Funding/Support: This study was supported by the
Agency for Healthcare Research and Quality con-
tract 233-02-0088 (task order 13).
Role of the Sponsor: The Agency for Healthcare Re-
search and Quality (AHRQ) specified the overarch-
ing study design in a request for proposal. Data were
collected by Pennsylvania hospitals, which were re-
quired by law to submit these data to the Pennsylva-
nia Health Care Cost Containment Council. Data were
transmitted to the AHRQ by the Council and the AHRQ
transmitted them to the authors. Further data man-
agement, analyses, interpretation, and preparation of
the manuscript were the independent work of the au-
thors. The manuscript was reviewed by the Council
and by the director of the Center for Delivery, Orga-
nizations, and Markets at AHRQ prior to the initial sub-
mission. Complete specifications of data elements, po-
tential risk factors, and risk-adjustment equations are
available in the final report submitted to the AHRQ.
Administrative and Atlas clinical data were provided
by the Council, an independent state agency respon-
sible for addressing the problem of escalating health
costs, ensuring the quality of health care, and increas-
ing access to health care for all citizens regardless of
ability to pay. The Council provided data in an effort
to further its mission of educating the public and con-
taining health care costs in Pennsylvania. The Coun-
cil, its agents and staff, have made no representa-
tion, guarantee, or warranty, express or implied, that
the data are error-free, or that the use of the data will
avoid differences of opinion or interpretation. Analy-
ses reported in this article were not prepared by the
Council.
Disclaimer: The Pennsylvania Health Care Cost Con-
tainment Council, its agents and staff, bear no re-
sponsibility or liability for the results of these analy-
ses, which are solely the opinion of the authors.
REFERENCES
1. Agency for Healthcare Research and Quality. In-
patient quality indicators, technical specifications [Feb-
ruary 20, 2006]. http://www.qualityindicators.ahrq
.gov/downloads/iqi/iqi_technical_specs_v30.pdf.
Accessed November 6, 2006.
2. HealthGrades Inc Web site. http://www
.healthgrades.com. Accessed November 6, 2006.
3. McKay NL, Deily ME. Comparing high- and low-
performing hospitals using risk-adjusted excess mor-
tality and cost inefficiency. Health Care Manage Rev.
2005;30:347-360.
4. Hannan EL, Kilburn H Jr, Lindsey ML, et al. Clini-
cal versus administrative data bases for CABG sur-
gery: does it matter? Med Care. 1992;30:892-907.
5. Jollis JG, Ancukiewicz M, DeLong ER, et al. Dis-
cordance of databases designed for claims payment
versus clinical information systems: implications for out-
comes research. Ann Intern Med. 1993;119:844-850.
6. Iezzoni LI, Ash AS, Coffman GA, et al. Predicting
in-hospital mortality: a comparison of severity mea-
surement approaches. Med Care. 1992;30:347-359.
7. Iezzoni LI, Ash AS, Shwartz M, et al. Predicting who
dies depends on how severity is measured: implica-
tions for evaluating patient outcomes. Ann Intern Med.
1995;123:763-770.
8. Hannan EL, Racz MJ, Jollis JG, et al. Using Medi-
care claims data to assess provider quality for CABG
surgery: does it work well enough? Health Serv Res.
1997;31:659-678.
9. Pine M, Norusis M, Jones B, et al. Predictions of
hospital mortality rates: a comparison of data sources.
Ann Intern Med. 1997;126:347-354.
10. Knaus WA, Zimmerman JE, Wagner DP, et al.
APACHE-acute physiology and chronic health evalu-
ation: a physiologically based classification system. Crit
Care Med. 1981;9:591-597.
11. Brewster AC, Karlin BG, Hyde LA, et al. MEDIS-
GRPS: a clinically based approach to classifying hos-
pital patients at admission. Inquiry. 1985;22:377-387.
12. Daley J, Jencks S, Draper D, et al. Predicting hos-
pital-associated mortality for Medicare patients: a
method for patients with stroke, pneumonia, acute
myocardial infarction, and congestive heart failure.
JAMA. 1988;260:3617-3624.
13. Hannan EL, Kilburn H Jr, O’Donnell JF, et al. Adult
open heart surgery in New York State: an analysis of
risk factors and hospital mortality rates. JAMA. 1990;
264:2768-2774.
14. Smith DW, Pine M, Bailey RC, et al. Using clini-
cal variables to estimate the risk of patient mortality.
Med Care. 1991;29:1108-1129.
15. Higgins TL, Estafanous FG, Loop FD, et al. Strati-
fication of morbidity and mortality outcome by
preoperative risk factors in coronary artery bypass pa-
tients: a clinical severity score. JAMA. 1992;267:2344-
2348.
16. Lemeshow S, Teres D, Klar J, et al. Mortality prob-
ability models (MPM II) based on an international co-
hort of intensive care unit patients. JAMA. 1993;270:
2478-2486.
17. Van Ruiswyk J, Hartz A, Kuhn E, et al. A mea-
sure of mortality risk for elderly patients with acute
myocardial infarction. Med Decis Making. 1993;13:
152-160.
18. Khuri SF, Daley J, Henderson W, et al. The De-
partment of Veterans Affairs’ NSQIP [National VA Sur-
gical Quality Improvement Program]: the first na-
tional, validated, outcome-based, risk-adjusted, and
peer-controlled program for the measurement and en-
hancement of the quality of surgical care. Ann Surg.
1998;228:491-507.
19. Pine M, Jones B, Lou YB. Laboratory values im-
prove predictions of hospital mortality. Int J Qual
Health Care. 1998;10:491-501.
20. Gibbs J, Cull W, Henderson W, et al. Preopera-
tive serum albumin level as a predictor of operative
mortality and morbidity. Arch Surg. 1999;134:36-42.
21. Ghali WA, Faris PD, Galbraith PD, et al. Sex dif-
ferences in access to coronary revascularization after
cardiac catheterization: importance of detailed clini-
cal data. Ann Intern Med. 2002;136:723-732.
22. Grover FL, Shroyer AL, Hammermeister K, et al.
A decade’s experience with quality improvement in
cardiac surgery using the Veterans Affairs and Soci-
ety of Thoracic Surgeons national databases. Ann Surg.
2001;234:464-474.
23. Lee DS, Austin PC, Rouleau JL, et al. Predicting
mortality among patients hospitalized for heart fail-
ure: derivation and validation of a clinical model. JAMA.
2003;290:2581-2587.
24. Schneider EC, Epstein AM. Influence of cardiac
surgery performance reports on referral practices and
access to care: a survey of cardiovascular specialists.
N Engl J Med. 1996;335:251-256.
25. Moscucci M, Eagle KA, Share D, et al. Public re-
porting and case selection for percutaneous coro-
nary interventions: an analysis from two large multi-
center percutaneous coronary intervention databases.
J Am Coll Cardiol. 2005;45:1759-1765.
26. Narins CR, Dozier AM, Ling FS, et al. The influ-
ence of public reporting of outcome data on medical
decision making by physicians. Arch Intern Med. 2005;
165:83-87.
27. National Health Care Purchasing Institute.
Status report: consumer and purchaser disclosure
CLAIMS DATA FOR IMPROVING RISK-ADJUSTED HOSPITAL MORTALITY
©2007 American Medical Association. All rights reserved. (Reprinted) JAMA, January 3, 2007—Vol 297, No. 1 75
at Attn Clinical Affairs, on January 3, 2007 www.jama.comDownloaded from
project [October 2002]. http://www.pbgh
.org/programs/standardization/documents
/DisclosureProjectNewsletter-October2002.pdf. Ac-
cessed November 6, 2006.
28. Krumholz HM, Wang Y, Mattera JA, et al. An ad-
ministrative claims model suitable for profiling hospital
performance based on 30-day mortality rates among
patients with an acute myocardial infarction. Circulation.
2006;113:1683-1692.
29. Krumholz HM, Wang Y, Mattera JA, et al. An ad-
ministrative claims model suitable for profiling hospital
performance based on 30-day mortality rates among
patients with heart failure. Circulation. 2006;113:1693-
1701.
30. Naessens JM, Brennan MD, Boberg CJ, et al. Ac-
quired conditions: an improvement to hospital dis-
charge abstracts. Qual Assur Health Care. 1991;3:257-
262.
31. Coffey R, Milenkovic M, Andrews RM; US Agency
for Healthcare Research and Quality. The case for the
present on admission (POA) indicator: HCUP methods
series report 2006-01 [June 26, 2006]. http://www
.hcup-us.ahrq.gov/reports/2006_1.pdf. Accessed No-
vember 6, 2006.
32. Regenstreif Institute. Logical observation identifi-
ers names and codes (LOINC). http://www.regenstrief
.org/loinc. Accessed November 6, 2006.
33. Berner ES, Detmer DE, Simborg D. Will the wave
finally break? a brief view of the adoption of electronic
medical records in the United States. J Am Med Inform
Assoc. 2005;12:3-7.
34. Pennsylvania Health Care Cost Containment Coun-
cil Web site. http://www.phc4.org. Accessed Novem-
ber 6, 2006.
35. Cardinal Health. Atlas outcomes system. http:
//www.mediqual.com/products/atlas.asp. Accessed No-
vember 6, 2006.
36. Agency for Healthcare Research and Quality. Qual-
ity indicators software for SAS, zipped. http://www
.qualityindicators.ahrq.gov/iqi_download.htm. Ac-
cessed November 6, 2006.
37. Hosmer DW, Lemeshow S. Logistic regression. In:
Applied Logistic Regression. 2nd ed. New York, NY: John
Wiley and Sons; 2000:91-142.
38. Schwarz G. Estimating the dimension of a model.
Ann Stat. 1978;6:461-464.
39. Hastie T, Tibshirani R, Friedman J. The Elements of
Statistical Learning. New York, NY: Springer-Verlag; 2001.
40. Hanley JA, McNeil BJ. The meaning and use of the
area under a receiver operating characteristic (ROC)
curve. Radiology. 1982;143:29-36.
41. Lee TH, Meyer GS, Brennan TA. A middle ground
on public accountability. N Engl J Med. 2004;350:2409-
2412.
42. National Uniform Billing Committee. UB-04 Data
specifications manual (Beta3): present on admission
indicator. http://www.nubc.org/public/whatsnew/POA
.pdf. Accessed November 6, 2006.
43. Pennsylvania Healthcare Cost Containment Council.
Independent accountant’s [Parente Randolph] report
on applying agreed-upon procedures [August 5, 2004].
http://www.phc4.org. Accessed December 1,
2006.
44. Garets D, Davis M. Electronic medical records vs elec-
tronic health records: yes, there is a difference: a HIMSS
analytics white paper [2006]. http://www.himssanalytics
.org/docs/WP_EMR_EHR.pdf. Accessed November 6,
2006.
45. Lee SJ, Lindquist K, Segal MR, Covinsky KE. De-
velopment and validation of a prognostic index for 4-year
mortality in older adults. JAMA. 2006;295:801-
808.
Of all the inanimate objects, of all of man’s creations,
books are the nearest to us, for they contain our very
thought, our ambitions, our indignations, our illu-
sions, our fidelity to truth, and our persistent lean-
ing toward error.
—Joseph Conrad (1857-1924)
CLAIMS DATA FOR IMPROVING RISK-ADJUSTED HOSPITAL MORTALITY
76 JAMA, January 3, 2007—Vol 297, No. 1 (Reprinted) ©2007 American Medical Association. All rights reserved.
at Attn Clinical Affairs, on January 3, 2007 www.jama.comDownloaded from
