Impact of Implementation of Standardized Criteria in the Assessment of Complication Reporting After Robotic Partial Nephrectomy: A Systematic Review

Abstract

The definition of a surgical complication still lacks standardization, hampering evalua- tion of surgical performance in this regard. Over the years, efforts to address this issue have been carried out to improve reporting of outcomes. In 2012, the European Association of Urology (EAU) proposed a standardized reporting tool for urological complications. The aim of this study was to evaluate the impact of those recommenda- tions on complication reporting for patients undergoing robotic partial nephrectomy (RPN). A comprehensive systematic review of all English language publications on RPN was carried out. We followed the Preferred Reporting Items for Systematic Review and Meta-Analyses statement and Agency for Healthcare Research and Quality guidelines in evaluating articles retrieved from the PubMed, Scopus, and Web of Science databases (January 1, 2000 to October 31, 2016; updated June 2017). The quality of reporting and grading complications was assessed according to the EAU recommendations. Temporal comparison revealed an improvement in outcome reporting in terms of mortality rates and causes of death (p = 0.05), definition of complications (p < 0.001), procedure- specific complications (p = 0.02), severity grade (p < 0.001), postoperative complica- tions presented by grade/complication type (p < 0.001), and risk factors (p < 0.001). Our analysis demonstrates an improvement in complication reporting and grading after the EAU recommendation on RPN.

Full text

Mini
Review
–
Kidney
Cancer
Impact
of
Implementation
of
Standardized
Criteria
in
the
Assessment
of
Complication
Reporting
After
Robotic
Partial
Nephrectomy:
A
Systematic
Review
Giovanni
E.
Cacciamani
a,b,
*,
Luis
G.
Medina
a
,
Alessandro
Tafuri
a,b
,
Tania
Gill
a
,
Willy
Baccaglini
b,c
,
Vanessa
Blasic
b,c
,
Felipe
P.A.
Glina
a,d
,
Andre
L.
De
Castro
Abreu
a
,
Rene
´Sotelo
a
,
Inderbir
S.
Gill
a
,
Walter
Artibani
b
a
Urology
Institute
University
of
Southern
California
(USC),
Los
Angeles,
CA,
United
States;
b
Department
of
Urology,
University
of
Verona,
Verona,
Italy;
c
ABC
Medical
School,
Santo
André,
SP,
Brazil;
d
Lusíada
University
Center,
School
of
Medical
Sciences
of
Santos,
Santos,
SP,
Brazil
E
U
R
O
P
E
A
N
U
R
O
L
O
G
Y
F
O
C
U
S
X
X
X
(
2
0
18
)
X
X
X
–
X
X
X
ava
ilable
at
www.sciencedirect.com
journa
l
homepage:
www.europea
nurology.com/eufocus
Article
info
Article
history:
Accepted
December
10,
2018
Associate
Editor:
Malte
Rieken
Keywords:
Robotic
partial
nephrectomy
Complication
Outcome
reporting
European
Association
of
Urology
guidelines
Abstract
The
definition
of
a
surgical
complication
still
lacks
standardization,
hampering
evalua-
tion
of
surgical
performance
in
this
regard.
Over
the
years,
efforts
to
address
this
issue
have
been
carried
out
to
improve
reporting
of
outcomes.
In
2012,
the
European
Association
of
Urology
(EAU)
proposed
a
standardized
reporting
tool
for
urological
complications.
The
aim
of
this
study
was
to
evaluate
the
impact
of
those
recommenda-
tions
on
complication
reporting
for
patients
undergoing
robotic
partial
nephrectomy
(RPN).
A
comprehensive
systematic
review
of
all
English
language
publications
on
RPN
was
carried
out.
We
followed
the
Preferred
Reporting
Items
for
Systematic
Review
and
Meta-Analyses
statement
and
Agency
for
Healthcare
Research
and
Quality
guidelines
in
evaluating
articles
retrieved
from
the
PubMed,
Scopus,
and
Web
of
Science
databases
(January
1,
2000
to
October
31,
2016;
updated
June
2017).
The
quality
of
reporting
and
grading
complications
was
assessed
according
to
the
EAU
recommendations.
Temporal
comparison
revealed
an
improvement
in
outcome
reporting
in
terms
of
mortality
rates
and
causes
of
death
(p
=
0.05),
definition
of
complications
(p
<
0.001),
procedure-
specific
complications
(p
=
0.02),
severity
grade
(p
<
0.001),
postoperative
complica-
tions
presented
by
grade/complication
type
(p
<
0.001),
and
risk
factors
(p
<
0.001).
Our
analysis
demonstrates
an
improvement
in
complication
reporting
and
grading
after
the
EAU
recommendation
on
RPN.
Patient
summary:
Complications
are
unexpected
events
that
could
negatively
impact
a
patient’s
outcomes
after
surgery,
but
there
is
no
agreement
on
the
definition
and
reporting
of
complications.
In
2012,
the
European
Association
of
Urology
proposed
a
standardized
reporting
tool
for
urological
complications.
This
study
shows
an
improve-
ment
in
the
way
physicians
report
complications
after
robotic
partial
nephrectomy.
The
results
underline
the
importance
of
standardization
in
medicine
to
improve
clinical
research.
©
2018
European
Association
of
Urology.
Published
by
Elsevier
B.V.
All
rights
reserved.
*
Corresponding
author.
Urology
Institute,
University
of
Southern
California,
CA,
USA.
Tel.
+1
626
4911531;
Fax:
+1
0458127702.
E-mail
address:
giovanni.cacciamani@med.usc.edu
(G.E.
Cacciamani).
EUF-646;
No.
of
Pages
5
Please
cite
this
article
in
press
as:
Cacciamani
GE,
et
al.
Impact
of
Implementation
of
Standardized
Criteria
in
the
Assessment
of
Complication
Reporting
After
Robotic
Partial
Nephrectomy:
A
Systematic
Review.
Eur
Urol
Focus
(2018),
https://doi.org/10.1016/j.
euf.2018.12.004
https://doi.org/10.1016/j.euf.2018.12.004
2405-4569/©
2018
European
Association
of
Urology.
Published
by
Elsevier
B.V.
All
rights
reserved.

1.
Introduction
A
rigorous
methodology
for
measuring
outcomes
and
com-
plications
following
surgeries
is
mandatory
to
increase
the
quality
and
safety
of
care
in
the
era
of
evidence-based
medi-
cine.
It
is
well
known
that
the
definition
of
complications
still
lacks
standardization,
partly
hindering
a
full
understanding
of
surgical
performance
[1].
In
2002,
Martin
et
al
[2]
identified
major
limitations
in
collection
of
complication
data
in
the
surgical
literature.
With
the
intent
to
improve
outcome
reporting,
they
proposed
the
ten
so-called
Martin
criteria.
In
2004,
the
Clavien-Dindo
classification
came
into
play
and
dramatically
influenced
investigator
behavior
[3].
However,
full
adherence
by
the
scientific
community
to
this
classifica-
tion
scheme
has
not
been
completely
effective,
leading
to
occasional
misapplication
of
these
tools
[2,4].
In
2012,
a
European
Associationof Urology
(EAU)ad hoc
panelproposed
guidelines
to
create
a
14-item
standardized
reporting
tool
that
“focuses
on
patient-centered
outcomes”
[5].
Robotic
partial
nephrectomy
(RPN)
has
emerged
as
a
safe,
effective,
and
even
preferred
PN
surgical
approach
for
the
treatment
of
small
renal
masses
for
which
it
has
been
shown
that
surgical
and
host
factors
have
an
impact
on
perioperative
outcomes
[6–8].
However,
there
is
still
a
lack
of
homogeneity
in
report-
ing
complications.
The
aim
of
this
study
was
to
critically
evaluate
the
impact
of
EAU
guideline
recommendations
on
grading
and
report-
ing
of
complications
for
patients
undergoing
RPN.
2.
Data
acquisition
and
analysis
All
English
language
publications
on
RPN
were
evaluated.
We
followed
the
Preferred
Reporting
Items
for
Systematic
Review
and
Meta-Analyses
statement
and
Agency
for
Healthcare
Research
and
Quality
guidelines
to
evaluate
articles
retrieved
from
the
PubMed,
Scopus,
and
Web
of
269 arcles included
in the qu
antav
e synthesis
114
RPN case
series
155
RPN com
parave s
tudies
Idenficaon
ScreeningEligibility
Inclusion
12 106
arcles ide
nfie
d :
3262 idenfied
fr
om PubMed
4727 idenfied from Scopu
s
4117
idenfied
fr
om Web of Science
10 23
3
All non-roboc
PN arc
les
excl
uded
1873 Arcles on roboc
approac
h iden
fied
987 arcles ide
nfied as being eligibl
e
106
arcles included
aer
updat
ed literature
search
1093 arcles ide
nfied as being
eli
gibl
e
824
excluded with
reas
on *arcles
886 Fu
ll-text arcles
excl
uded
with
reas
on:
Dup
licates
Replies, commentaries
, commentsand editorial
Case
reports
Surgical
technique descripon
s
and meta-analysesReviews
Non-matching rec
ords
Fig.
1
–
Literature
search
strategy.
Preferred
Reporting
Items
for
Systematic
Review
and
Meta-Analyses
flow
chart
for
selection
of
papers
on
robotic
partial
nephrectomy
published
in
English
up
to
June
2017.
*Exclusion
criteria:
replies,
commentaries,
and
editorial
comments;
case
reports;
surgical
technique
descriptions;
reviews
and
meta-analyses;
pediatric
surgery;
nonmatching
articles;
publications
from
the
same
institution
with
overlapping
data;
no
outcome
of
interest;
not
relevant
for
the
key
questions.
PN
=
partial
nephrectomy;
RPN
=
robotic
PN.
E
U
R
O
P
E
A
N
U
R
O
L
O
G
Y
F
O
C
U
S
X
X
X
(
2
0
1
8
)
X
X
X
–
X
X
X
2
EUF-646;
No.
of
Pages
5
Please
cite
this
article
in
press
as:
Cacciamani
GE,
et
al.
Impact
of
Implementation
of
Standardized
Criteria
in
the
Assessment
of
Complication
Reporting
After
Robotic
Partial
Nephrectomy:
A
Systematic
Review.
Eur
Urol
Focus
(2018),
https://doi.org/10.1016/j.
euf.2018.12.004

Science
databases
(January
1,
2000
to
October
31,
2016;
updated
June
2017).
The
study
was
registered
at
PROSPERO
(CRD42017062712).
The
quality
of
the
reporting
and
grad-
ing
of
complications
was
assessed
according
to
the
EAU
ad
hoc
panel
recommendation.
To
establish
a
possible
change
in
attitude
towards
the
reporting
of
complications,
tempo-
ral
and
location
comparisons
were
performed
for
reports
before
and
after
the
introduction
of
the
EAU
guidelines
on
complication
reporting
in
2012.
Data
for
categorical
vari-
ables
are
shown
as
percentages,
and
differences
between
groups
were
analyzed
using
Pearson’s
x
2
test
or
Fisher’s
exact
test
as
appropriate.
All
statistical
analyses
were
per-
formed
using
SPSS
v.24.0
(IBM,
Armonk,
NY,
USA).
All
tests
were
two-sided,
with
p
<
0.05
considered
to
indicate
sta-
tistical
significance.
3.
Results
Our
electronic
search
identified
1093
RPN
publications
for
detailed
review,
which
yielded
114
case
series
and
155
com-
parative
studies
(Fig.
1),
of
which
96
(35.6%)
were
published
before
2012
and
173
(64.3%)
were
published
after
2012
(Supplementary
Table
1).
Overall,
229
papers
(85.1%)
reported
complications
as
outcomes
of
interest.
Compari-
son
of
the
quality
of
reporting
criteria
before
and
after
the
EAU
ad
hoc
panel
recommendations
(Fig.
2
and
Table
1)
revealed
no
differences
in
terms
of
definition
of
methods
for
accruing
data
(75%
vs
92%;
p
=
0.51),
identification
of
who
collected
the
data
(2%
vs
6%;
p
=
0.46),
inclusion
of
follow-
up
duration
(60%
vs
65%;
p
=
0.62),
outpatient
information
(70%
vs
80%;
p
=
0.08),
procedure-specific
complications
(55%
vs
66%;
p
=
0.2),
separate
reporting
of
intra-
and
post-
operative
complications
(58%
vs
59%;
p
=
0.81),
readmis-
sions
and
causes
(17%
vs
18%;
p
=
1),
reoperations,
types,
and
causes
(35%
vs
30%;
p
=
0.61),
procedure-specific
complica-
tions
(55%
vs
66%;
p
=
0.2),
and
percentage
of
patients
lost
to
follow-up
(2%
vs
3%;
p
=
1).
We
observed
improvements
in
outcome
reporting
in
terms
of
mortality
rates
and
causes
of
death
(11%
vs
24%;
p
=
0.05),
definitions
of
complications
(33%
vs
52%;
p
<
0.001),
procedure-specific
complications
(55%
vs
66%;
p
=
0.2),
severity
grade
(42%
vs
92%;
p
<
0.001),
postoperative
complications
tabulated
by
either
grade
or
complication
type
(36%
vs
69%;
p
<
0.001),
and
risk
factors
included
in
analysis
(23%
vs
46%;
p
<
0.001).
4.
Discussion
Our
analysis
revealed
an
improvement
in
reporting
and
grad-
ing
of
complications
after
the
EAU
guideline
recommenda-
tions.
The
importance
of
choosing
the
correct
outcome
reporting
tool
has
been
described:
the
same
data
could
lead
to
two
completely
different
sets
of
information,
highlighting
the
importance
of
standardization
in
medicine
to
avoid
mis-
leading
information.In
the
absence
of
the
correct
tool,
there
is
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00
%
Method of accruing
data de
fine
d*
Who coll
ected the data defined
Duration of foll
ow-up ind
icated
*
Outpatient i
nformation
includ
ed *
Mort
ality r
ate and causes of death listed*
Definition
s of complications provide
d*
Proced
ure-spe
cific compli
cation
s include
d*
Separate r
eport
ing of intr
a-
and postope
rative
compli
cation
s
Severity grade
used*
Postoperative compli cations w
ere presented in a
table either
by grade
or
by compli
cation type
Risk factors includ
ed in analysis*
Readmission
s and cau
ses w
ere includ
ed
Reoperation
s, type
s, and
cau
ses w
ere included
Percen
tage
of patients lost to follow-up w
as
included
Befo
re EAU GL
recommend
aons
Aer EA
U GL
rec
omm
endaons
Fig.
2
–
Comparison
of
quality
assessment
criteria
following
the
European
Association
of
Urology
guideline
(EAU
GL)
recommendations.
*Outcomes
in
common
with
the
Martin
criteria.
E
U
R
O
P
E
A
N
U
R
O
L
O
G
Y
F
O
C
U
S
X
X
X
(
2
0
1
8
)
X
X
X
–
X
X
X
3
EUF-646;
No.
of
Pages
5
Please
cite
this
article
in
press
as:
Cacciamani
GE,
et
al.
Impact
of
Implementation
of
Standardized
Criteria
in
the
Assessment
of
Complication
Reporting
After
Robotic
Partial
Nephrectomy:
A
Systematic
Review.
Eur
Urol
Focus
(2018),
https://doi.org/10.1016/j.
euf.2018.12.004

a
real
risk
of
reporting
of
false
data
[9].
A
systematic
review
by
Mitropoulos
et
al
[4]
revealed
a
weakness
in
the
literature
for
grading
and
reporting
of
complications
following
PN.
The
authors
found
that
only
six
studies
(2.9%)
fulfilled
all
the
criteria;
the
most
underreported
variables
(<50%)
were
the
percentage
of
patients
lost
to
follow-up
(6.9%),
readmission
rates
(12.7%),
who
collected
the
data
(18.6),
follow-up
dura-
tion
(47.1%),
mortality
data
and
causes
of
death
(33.8%),
definition
of
procedure-specific
complications
(39.2%),
sepa-
rate
reporting
of
intra-
and
postoperative
complications
(45.1%),
complication
severity
or
grade
(32.4%),
and
risk
factors
analysis
(44.1%)
[4].
Our
results
revealed
that
after
publication
of
the
EAU
guideline
recommendations
on
outcome
reporting,
there
was
mainly
better
adherence
to
all
the
criteria.
Overall,
there
was
underreporting
(<50%)
for
six
of
the
14
criteria
after
publication
of
the
EAU
guidelines
(Table
1).
More-
over,
we
found
statistically
significant
improvements
in
the
inclusion
of
mortality
rates
and
causes
of
death,
definitions
of
complications,
severity
grade,
postopera-
tive
complications
tabulated
either
by
grade
or
complica-
tion
type,
and
inclusion
of
risk
factors
in
analyses.
As
previously
reported,
the
vast
majority
of
studies
did
not
investigate
who
collected
the
data
and
the
percentage
of
patients
lost
to
follow
up.
It
is
challenging
to
find
a
causal
link
between
publication
of
the
EAU
guidelines
and
the
quality
of
outcome
reporting
after
RPN.
A
debatable
point
is
that
studies
in
European
centers
(ECs)
are
more
likely
to
be
influenced
by
EAU
recommendations,
whereas
those
in
non-European
centers
(NECs)
are
less
likely
to
be
even
aware
of
the
EAU
guidelines.
In
this
context,
we
reviewed
the
locations
of
institutions
reporting
complications.
A
total
of
51
papers
(22.4%)
were
published
by
ECs
and
178
(77.6%)
by
NECs.
Although
there
was
no
difference
in
the
mean
percentage
of
criteria
fol-
lowed
before
2012
(14%
vs
14%;
p
=
0.8),
the
variance
was
higher
after
publication
of
the
EAU
guidelines,
but
did
not
reach
statistical
significance
(51%
vs
48%;
p
=
0.3).
This
finding
could
be
further
proof
that
dissemination
of
the
EAU
guidelines
may
have
broadly
influenced
complication
reporting
outside
Europe
as
well
as
in
ECs.
Further
inves-
tigations
are
necessary
to
determine
if
the
results
presented
here
are
a
consequence
of
guideline
adherence
that
might
have
influenced
investigator
performance,
leading
to
the
wide
spread
of
a
refined
methodology
in
research.
5.
Conclusions
Overall,
our
results
show
that
the
EAU
guideline
recom-
mendations
on
complication
reporting
may
have
had
a
positive
impact
on
outcome
measurement
after
RPN.
How-
ever,
more
effort
is
required
to
improve
complication
mea-
surement,
especially
for
intraoperative
complications
(for
which
a
standard
reporting
system
is
still
lacking)
to
guar-
antee
more
accurate
and
comprehensive
information
on
patients
undergoing
RPN.
Conflicts
of
interest:
The
authors
have
nothing
to
disclose.
Appendix
A.
Supplementary
data
Supplementary
data
associated
with
this
article
can
be
found,
in
the
online
version,
at
https://doi.org/10.1016/j.euf.
2018.12.004.
References
[1]
Dindo
D,
Clavien
PA.
Quality
assessment
of
partial
nephrectomy
complications
reporting:
time
to
get
the
head
out
of
the
sand.
Eur
Urol
2014;66:527–8.
[2]
Martin
2nd
RC,
Brennan
MF,
Jaques
DP.
Quality
of
complication
reporting
in
the
surgical
literature.
Ann
Surg
2002;235:803–13.
[3]
Dindo
D,
Demartines
N,
Clavien
PA.
Classification
of
surgical
com-
plications:
a
new
proposal
with
evaluation
in
a
cohort
of
6336
patients
and
results
of
a
survey.
Ann
Surg
2004;240:205–13.
Table
1
–
Comparison
of
Quality
Assessment
Criteria
before
and
after
European
Association
of
Urology
Guideline
recommendations.
Before
EAU
GLs
Recommendations
After
EAU
GLs
Recommendations
p
value
N.
of
Studies
Reporting
Complications
76
153
Method
of
accruing
data
defined
*
75.00%
92.00%
0.51
Who
collected
the
data
defined
2.00%
6.00%
0.46
Duration
of
follow-up
indicated
*
60.00%
65.00%
0.62
Outpatient
information
included
*
70.00%
80.00%
0.08
Mortality
rate
and
causes
of
death
listed
*
11.00%
24.00%
0.05
Definitions
of
complications
provided
*
33.00%
52.00%
<0.0001
Procedure-specific
complications
included
*
55.00%
66.00%
0.2
Separate
reporting
of
intra-
and
postoperative
complications
58.00%
59.00%
1
Severity
grade
used
*
42.00%
92.00%
<0.0001
Postoperative
complications
were
presented
in
a
table
either
by
grade
or
by
complication
type
36.00%
69.00%
<0.0001
Risk
factors
included
in
analysis
*
23.00%
46.00%
0.002
Readmissions
and
causes
were
included
14.00%
18.00%
1
Reoperations,
types,
and
causes
were
included
35.00%
30.00%
0.6
Percentage
of
patients
lost
to
follow-up
was
included
2.00%
3.00%
1
*
Outcomes
in
common
with
the
Martin
Criteria.
E
U
R
O
P
E
A
N
U
R
O
L
O
G
Y
F
O
C
U
S
X
X
X
(
2
0
1
8
)
X
X
X
–
X
X
X
4
EUF-646;
No.
of
Pages
5
Please
cite
this
article
in
press
as:
Cacciamani
GE,
et
al.
Impact
of
Implementation
of
Standardized
Criteria
in
the
Assessment
of
Complication
Reporting
After
Robotic
Partial
Nephrectomy:
A
Systematic
Review.
Eur
Urol
Focus
(2018),
https://doi.org/10.1016/j.
euf.2018.12.004

[4]
Mitropoulos
D,
Artibani
W,
Biyani
CS,
et
al.
Quality
assessment
of
partial
nephrectomy
complications
reporting
using
EAU
standar-
dised
quality
criteria.
Eur
Urol
2014;66:522–6.
[5]
Mitropoulos
D,
Artibani
W,
Graefen
M,
et
al.
Reporting
and
grading
of
complications
after
urologic
surgical
procedures:
an
ad
hoc
EAU
guidelines
panel
assessment
and
recommendations.
Eur
Urol
2012;61:341–9.
[6]
Cacciamani
GE,
Medina
LG,
Gill
T,
et
al.
Impact
of
surgical
factors
on
robotic
partial
nephrectomy
outcomes:
comprehensive
systematic
review
and
meta-analysis.
J
Urol
2018;200:258–74.
[7]
Cacciamani
GE,
Medina
LG,
Gill
TS,
et
al.
Impact
of
renal
hilar
control
on
outcomes
of
robotic
partial
nephrectomy:
systematic
review
and
cumulative
meta-analysis.
Eur
Urol
Focus.
In
press.
https://doi.org/
10.1016/j.euf.2018.01.012
[8]
Cacciamani
GE,
Gill
T,
Medina
L,
et
al.
Impact
of
host
factors
on
robotic
partial
nephrectomy
outcomes:
comprehensive
systematic
review
and
meta-analysis.
J
Urol
2018;200:716–30.
[9]
Artibani
W.
What
you
measure
depends
on
the
tool
you
use:
a
short
step
from
incorrect
measurements
to
fake
data.
Eur
Urol
2018;74:8–9.
E
U
R
O
P
E
A
N
U
R
O
L
O
G
Y
F
O
C
U
S
X
X
X
(
2
0
1
8
)
X
X
X
–
X
X
X
5
EUF-646;
No.
of
Pages
5
Please
cite
this
article
in
press
as:
Cacciamani
GE,
et
al.
Impact
of
Implementation
of
Standardized
Criteria
in
the
Assessment
of
Complication
Reporting
After
Robotic
Partial
Nephrectomy:
A
Systematic
Review.
Eur
Urol
Focus
(2018),
https://doi.org/10.1016/j.
euf.2018.12.004