Bias and Precision in Magnetic Resonance Imaging‐Based Estimates of Renal Blood Flow: Assessment by Triangulation

Abstract

Background
Renal blood flow (RBF) can be measured with dynamic contrast enhanced-MRI (DCE-MRI) and arterial spin labeling (ASL). Unfortunately, individual estimates from both methods vary and reference-standard methods are not available. A potential solution is to include a third, arbitrating MRI method in the comparison.

Purpose
To compare RBF estimates between ASL, DCE, and phase contrast (PC)-MRI.

Study Type
Prospective.

Population
Twenty-five patients with type-2 diabetes (36% female) and five healthy volunteers (HV, 80% female).

Field Strength/Sequences
A 3 T; gradient-echo 2D-DCE, pseudo-continuous ASL (pCASL) and cine 2D-PC.

Assessment
ASL, DCE, and PC were acquired once in all patients. ASL and PC were acquired four times in each HV. RBF was estimated and split-RBF was derived as (right kidney RBF)/total RBF. Repeatability error (RE) was calculated for each HV, RE = 1.96 × SD, where SD is the standard deviation of repeat scans.

Statistical Tests
Paired t-tests and one-way analysis of variance (ANOVA) were used for statistical analysis. The 95% confidence interval (CI) for difference between ASL/PC and DCE/PC was assessed using two-sample F-test for variances. Statistical significance level was P < 0.05. Influential outliers were assessed with Cook's distance (Di > 1) and results with outliers removed were presented.

Results
In patients, the mean RBF (mL/min/1.73m²) was 618 ± 62 (PC), 526 ± 91 (ASL), and 569 ± 110 (DCE). Differences between measurements were not significant (P = 0.28). Intrasubject agreement was poor for RBF with limits-of-agreement (mL/min/1.73m²) [−687, 772] DCE-ASL, [−482, 580] PC-DCE, and [−277, 460] PC-ASL. The difference PC-ASL was significantly smaller than PC-DCE, but this was driven by a single-DCE outlier (P = 0.31, after removing outlier). The difference in split-RBF was comparatively small. In HVs, mean RE (±95% CI; mL/min/1.73 m²) was significantly smaller for PC (79 ± 41) than for ASL (241 ± 85).

Conclusions
ASL, DCE, and PC RBF show poor agreement in individual subjects but agree well on average. Triangulation with PC suggests that the accuracy of ASL and DCE is comparable.

Evidence Level
2

Technical Efficacy
Stage 2

Full text

RESEARCH ARTICLE
Bias and Precision in Magnetic Resonance
Imaging-Based Estimates of Renal Blood
Flow: Assessment by Triangulation
Bashair A. Alhummiany, MSc,
1
David Shelley, BSc,
1,2
Margaret Saysell, BSc,
1,2
Maria-Alexandra Olaru, PhD,
3
Bernd Kühn, PhD,
3
David L. Buckley, PhD,
1
Julie Bailey, BSc,
2
Kelly Wroe, BSc,
2
Cherry Coupland, BSc,
2
Michael W. Mansfield, DM,
2
Steven P. Sourbron, PhD,
4
*and Kanishka Sharma, PhD
4
Background: Renal blood flow (RBF) can be measured with dynamic contrast enhanced-MRI (DCE-MRI) and arterial spin
labeling (ASL). Unfortunately, individual estimates from both methods vary and reference-standard methods are not avail-
able. A potential solution is to include a third, arbitrating MRI method in the comparison.
Purpose: To compare RBF estimates between ASL, DCE, and phase contrast (PC)-MRI.
Study Type: Prospective.
Population: Twenty-five patients with type-2 diabetes (36% female) and five healthy volunteers (HV, 80% female).
Field Strength/Sequences: A 3 T; gradient-echo 2D-DCE, pseudo-continuous ASL (pCASL) and cine 2D-PC.
Assessment: ASL, DCE, and PC were acquired once in all patients. ASL and PC were acquired four times in each HV. RBF
was estimated and split-RBF was derived as (right kidney RBF)/total RBF. Repeatability error (RE) was calculated for each
HV, RE =1.96 SD, where SD is the standard deviation of repeat scans.
Statistical Tests: Paired t-tests and one-way analysis of variance (ANOVA) were used for statistical analysis. The 95% confi-
dence interval (CI) for difference between ASL/PC and DCE/PC was assessed using two-sample F-test for variances. Statis-
tical significance level was P< 0.05. Influential outliers were assessed with Cook’s distance (D
i
> 1) and results with outliers
removed were presented.
Results: In patients, the mean RBF (mL/min/1.73m
2
) was 618 62 (PC), 526 91 (ASL), and 569 110 (DCE). Differences
between measurements were not significant (P=0.28). Intrasubject agreement was poor for RBF with limits-of-agreement
(mL/min/1.73m
2
)[687, 772] DCE-ASL, [482, 580] PC-DCE, and [277, 460] PC-ASL. The difference PC-ASL was signifi-
cantly smaller than PC-DCE, but this was driven by a single-DCE outlier (P=0.31, after removing outlier). The difference
in split-RBF was comparatively small. In HVs, mean RE (95% CI; mL/min/1.73 m
2
) was significantly smaller for PC
(79 41) than for ASL (241 85).
Conclusions: ASL, DCE, and PC RBF show poor agreement in individual subjects but agree well on average. Triangulation
with PC suggests that the accuracy of ASL and DCE is comparable.
Evidence Level: 2
Technical Efficacy: Stage 2
J. MAGN. RESON. IMAGING 2022;55:1241–1250.
High renal perfusion is essential for sustaining stable glo-
merular filtration and blood flow autoregulation is vital
for protecting the kidney from elevated arterial pressure that
causes glomerular capillary injury.
1
Impairment in the
autoregulatory mechanism of the kidney is known to contrib-
ute to diabetic nephropathy
1
and chronic kidney disease
View this article online at wileyonlinelibrary.com. DOI: 10.1002/jmri.27888
Received May 12, 2021, Accepted for publication Aug 3, 2021.
*Address reprint requests to: S.P.S., Department of Imaging, Infection, Immunity and Cardiovascular Disease, The University of Sheffield, UK.
E-mail: s.sourbron@sheffield.ac.uk
From the
1
Department of Biomedical Imaging Sciences, University of Leeds, Leeds, UK;
2
Leeds Teaching Hospitals NHS Trust, Leeds, UK;
3
Siemens Healthcare
GmbH, Erlangen, Germany; and
4
Department of Imaging, Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, UK
Additional supporting information may be found in the online version of this article
© 2021 International Society for Magnetic Resonance in Medicine. 1241

(CKD) has been associated with reduced renal perfusion.
2
Thus, the measurement of renal perfusion can serve as a prog-
nostic biomarker in CKD and aid in the diagnosis of renal
dysfunction or in the identification of progressive disease.
MRI enables quantification of renal perfusion
(expressed in mL/min/100 mL) either by dynamic contrast
enhanced (DCE)
3
or by arterial spin labeling (ASL)
4
based
approaches. Both MRI methods have demonstrated promis-
ing results for detecting kidney dysfunction and evaluating
disease progression.
5,6
However, resting perfusion measure-
ments from healthy volunteers reported in the literature vary
between the two methods by as much as a factor of 2 (range
between 172 and 427 mL/min/100 mL for cortical perfu-
sion using ASL and between 244 and 443 mL/min/100 mL
for the whole parenchyma using DCE, see
Supplementary File S1), undermining clinical confidence in
MRI perfusion measures. While differences in demo-
graphics, subject preparation, and physiological variations
play a role,
7
the effect of measurement error cannot be
excluded. Unfortunately, due to the lack of clinical reference
standard methods, it is not currently possible to distinguish
these effects.
In the absence of a reference standard, comparisons
within MRI methods can be performed as an alternative
approach. Since DCE and ASL measure renal perfusion based
on different mechanisms, it is unlikely that their errors are
related. Hence, a strong agreement between the two would
offer confidence on the relative accuracy of the techniques. In
case of disagreement, however, the technical validation will be
inconclusive as this could be due to an error in one of the
two methods, or in both. Unfortunately, the latter has proven
to be the case
8,9
; although DCE and ASL have agreed well on
average, the limits of agreement are large.
10
Other technical
validation studies have focused on comparing repeatability
and reproducibility of the measurements.
7
However, while
repeatability is a necessary condition for any assay to be valid,
a separate assessment of bias is necessary to avoid promoting
methods that are less reliable.
11
A possible approach to assessing bias in the absence of
a reference standard is to involve a third, arbitrating,
method. Phase-contrast (PC) MRI of the renal arteries is a
good candidate as it is built on different physical principles
and produces a measurement of renal blood flow (RBF)
that can also be derived from ASL and DCE by integrating
perfusion values over the parenchyma. Moreover, PC is a
well-established method and readily available in clinical
practice.
12
If PC agrees better with ASL than with DCE for
instance, this will offer evidence that ASL is more reliable
than DCE (and vice versa). Conversely, if PC agrees equally
well with DCE and ASL, this will indicate that DCE and
ASL have similar levels of error. Previous work in healthy
volunteers has confirmed a significant correlation between
ASL and PC,
13
but this question has not been addressed in
patients and a comparison of all three methods has not yet
been performed.
The primary aim of this study was to compare RBF esti-
mates from ASL, DCE, and PC in patients with type-2 diabe-
tes. Split RBF (relative to the right kidney, as used in nuclear
medicine studies) was also compared between the three
methods. In order to support the interpretation of the data, the
repeatability of PC and ASL was assessed in healthy volunteers.
TABLE 1. Summary of Acquisition Parameters for PC,
ASL, and DCE MRI Sequences
MRI Sequence Scanning Parameters
Phase contrast
(PC)
Acquisition plane: sagittal;
Acquisition mode: free
breathing with ECG triggering;
Pulse sequence: 2D gradient
echo; FOV 350 241 mm;
TR 40.48 msec; TE 2.74 msec;
FA 25

; voxel size
0.6 0.6 mm; Slice thickness
6 mm; velocity encoding
120 cm/sec; Acquisition time:
1:40 minutes
Arterial spin
labeling (ASL)
a
Acquisition plane: Coronal-
oblique; Acquisition mode: free
breathing; Readout: 3D TGSE;
FOV 300 150 mm; TR
5000 msec; TE 19.28 msec;
excitation FA 90; slice
thickness 5 mm; Number of
slices 16
Labeling scheme: pCASL;
labeling duration 1500 msec;
postlabeling delay 1500 msec
Dynamic contrast
enhanced (DCE)
Acquisition plane: eight coronal-
oblique slices and one
transverse slice; acquisition
mode: free breathing; pulse
sequence: 2D Turbo-FLASH;
FOV 400 400 mm; TR
179 msec; TE 0.97 msec; FA
10

; TI 85 msec; slice thickness
7.5 mm; temporal resolution
1.6 seconds; parallel imaging:
GRAPPA 2; acquisition time:
7:07 minutes
a
Work-In-Progress package, the product is currently under
development and is not for sale in the United States and in
other countries. Its future availability cannot be ensured.
ECG =echocardiogram; FA =flip angle; FLASH =fast low
angle shot; FOV =field of view; pCASL =pseudo-continuous
arterial spin labeling; TE =echo time; TI =inversion time;
TR =repletion time; TGSE =turbo gradient spin echo.
1242 Volume 55, No. 4
Journal of Magnetic Resonance Imaging
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

Methods
Subjects
The study was approved by the institutional research ethics com-
mittee and written informed consent was obtained from all sub-
jects. The study included 25 consecutive patients with type-2
diabetes (estimated glomerular filtration rate [eGFR] ≥30 mL/
min/1.73 m
2
) from the imaging biomarkers enterprise to attack
diabetic kidney disease (iBEAt) study cohort.
14
To assess the
repeatability of ASL and PC techniques, five healthy volunteers
(HVs) underwent four MRI scans. All volunteers were normo-
tensive with no history of diabetes or renal disease.
The same subject preparation and MRI protocol was
employed for both patients and HVs, except that DCE
was not performed on healthy volunteers. The scans were per-
formed in the morning (between 8 and 11 am) after an over-
night fast (>8 hours). A standard breakfast (containing two
slices whole bread and butter) and 250 mL of water were pro-
vided immediately before the scan. Full details on the iBEAt
study including MRI protocol, patient preparation and
recruitment criteria have been published previously.
14
Imaging Protocol
MRI data were acquired on a 3 T MRI scanner
(MAGNETOM Prisma, Siemens Healthcare GmbH,
Erlangen, Germany) using an 18-channel phased array body
coil combined with inbuilt spine coil for signal reception.
The acquisition involved 2D cine PC-MRI, a prototypical
sequence for 3D renal ASL (pseudo-continuous arterial spin
labeling [pCASL]) and 2D DCE-MRI sequences acquired in
the same order. An overview of the MRI acquisition param-
eters is listed in Table 1.
PHASE CONTRAST. The renal arteries were depicted using a
combination of a coronal survey scan and an axial half-
Fourier acquisition single-shot turbo spin-echo (HASTE)
sequence. The imaging plane for PC-MRI was positioned
perpendicular to the renal artery close to its origin from the
descending aorta.
ARTERIAL SPIN LABELING. Data were recorded using a
pseudo-continuous arterial spin labeling (pCASL) sequence with
a slice selective labeling pulse. The prototype sequence is
implemented by the vendor and acquires label, control images,
and a reference proton density weighted (M0) image within the
same package (sequence parameters are provided in Table 1).
The labeling plane was positioned perpendicular to the abdomi-
nal aorta 10 cm above the center of the kidneys. Background
suppression was employed to reduce signal from static tissue.
Perfusion maps were generated on the scanner using inline soft-
ware, preceded by retrospective 2D motion correction that is
applied to the acquired MR datasets using 2D elastic registra-
tion. Maps of perfusion rate (in mL/min/100 mL) were derived
using the general kinetic model.
15
DYNAMIC CONTRAST ENHANCED. Data were acquired
continuously using T1-weighted sequence. Gd-DOTA
(Dotarem, Guerbet Group, France) was injected intrave-
nously using a quarter dose (0.025 mmol/kg) at a rate of
2 mL/sec followed by a 20 mL saline flush. The injection was
given using an automatic injector, 20 seconds after the acqui-
sition started.
Postprocessing
All postprocessing were performed by a radiographer (B.A.)
with 2 years of experience in renal MRI analyses.
PHASE CONTRAST. Analysis was performed on a Syngo.Via
workstation (Siemens Healthcare GmbH, Erlangen, Germany).
The renal arteries were defined, and threshold adjusted on the
anatomical images using an elliptical shape region-of-interest
(ROI). The regions were adjusted on each time frame according
to renal vessel movement in the cardiac cycle. The ROIs were
then propagated automatically into the velocity map images
FIGURE 1: Representative image showing the region-of-interest (ROI) placement on phase contrast (PC) magnitude and velocity
images (a and b, respectively), arterial spin labeling (ASL) perfusion map (c), and dynamic contrast enhance (DCE) apparent
extracellular volume map (d).
April 2022 1243
Alhummiany et al.: Triangulation of MRI-based RBF methods
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

(Fig. 1(a,b)). Renal blood flow (RBF) was computed by multi-
plying average blood velocity with average vessel cross-sectional
area reported in units of mL/min.
ARTERIAL SPIN LABELING. Using a prototypical sequence
for kidney ASL, perfusion maps were reconstructed on the
scanner and exported for analysis. The analysis was performed
using PMI 0.4 software (Platform for research in Medical
Imaging).
16
The renal parenchyma ROIs were segmented on
proton density-weighted (M
0
) images, using threshold tech-
nique based on pixel intensities and transferred to the perfu-
sion map (Fig. 1(c)). Kidney volume (mL) was derived by
multiplying the total number of voxels in the ROI by voxel
volume and RBF was derived by multiplying the average per-
fusion of the ROI by its volume.
DYNAMIC CONTRAST ENHANCED. Data were also ana-
lyzed using PMI 0.4. Breathing motion was corrected using
FIGURE 2: Box and whisker plots depicting distribution of renal blood flow (RBF) (a) and split RBF (b) measured with phase contrast
(PC), arterial spin labeling (ASL) and dynamic contrast enhanced (DCE). The Pvalues for pairwise comparisons of means are given
above the plots.
FIGURE 3: Bland–Altman plots comparing phase contrast, arterial spin labeling (PC-ASL) (a), phase contrast, dynamic contrast
enhanced (PC-DCE) (b) and DCE-ASL (c) for renal blood flow (RBF) (top panel) and split-RBF (bottom panel). Dashed lines indicate
upper and lower 95% confidence intervals (CI) calculated as: mean difference 1.96 (SD). Solid lines represent the mean difference
between two techniques. Measurement of ASL-PC in healthy volunteers (▲) was plotted for visual reference.
1244 Volume 55, No. 4
Journal of Magnetic Resonance Imaging
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

deformable model-driven registration.
17
The aorta was
defined semi-automatically by setting a user-defined lower
threshold on an axial slice of a maximum signal enhancement
map. An arterial-input-function (AIF) was derived by
averaging the signal values over the ROI. The renal paren-
chyma was outlined on a map of apparent extracellular vol-
ume (mL/100 mL) derived by model-free deconvolution.
18
The parenchyma was first coarsely outlined by a manual
selection of a lower threshold on this map, followed by man-
ual exclusion of voxels in the renal pelvis, collecting system or
outside the kidney (Fig. 1(d)). A fit was performed to the
concentration time curve of the whole parenchyma, which
was derived from the signals using a nonlinear signal model
with a literature-based T1 value of 1200 msec. Tracer-kinetic
model fitting was performed with a two-compartment-
filtration model.
3
Reporting
To account for differences in body size between participants,
RBF was normalized to body-surface-area (BSA) in units of
1.73 m
2
according to the Du Bois formula.
19
RBF reported
in this study represents the total RBF of both kidneys
obtained as the sum of the right and left RBF. Split RBF was
calculated as the ratio of right RBF to total RBF. Measure-
ments are presented as the mean 95% confidence interval
(CI), unless indicated otherwise.
In HVs, the repeatability error (RE) was calculated for
each HV as RE =1.96 SD where SD is the standard devia-
tion over the four repeat scans. The relative repeatability error
(RRE) was calculated as RRE =(RE/mean) 100% where
the mean is the average over repeat scans in the same subject.
RE and RRE were subsequently averaged over all five
FIGURE 4: Scatterplots for phase contrast (PC) and arterial spin labeling (ASL) (a); PC and dynamic contrast enhanced (DCE) (b); ASL
and DCE (c), for renal blood flow (RBF) (top panel) and Split-RBF (bottom panel) for diabetic patients (). Pairwise Pearson’s
correlation (r) and P-value for the significance of the correlation are shown in the plots. The dotted line is the identity.
Measurements of ASL-PC in healthy volunteers (▲) were plotted for visual reference.
TABLE 2. The Mean and 95% CI and Pearson’s
Correlation Coefficient Calculated With and Without
Outliers for Renal Blood Flow Measurement in
Diabetic Patients
MRI
Technique
Outlier
Included (n=25)
Outlier
Removed
(n=24)
Mean 95% CI (mL/min/1.73 m
2
)
PC 618 62 618 64
ASL 526 91 544 88
DCE 569 110 525 72
Pearson’s correlation coefficient (r)
PC-ASL 0.57
a
0.61
a
PC-DCE 0.34 0.57
a
DCE-ASL 0.05 0.41
a
a
Indicates significant correlation, P< 0.05.
ASL =arterial spin labeling; DCE =dynamic contrast
enhanced; PC =phase contrast.
April 2022 1245
Alhummiany et al.: Triangulation of MRI-based RBF methods
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

volunteers. With these definitions, RE and RRE represent the
absolute and relative 95% CI for a measured value in an indi-
vidual subject, respectively. Population heterogeneity was
calculated for the RBF of each method as the standard devia-
tion across the population expressed as a percentage of the
population mean.
FIGURE 5: Bar chart for renal blood flow (RBF, ml/min/1.73 m
2
)(top row) and split-RBF (bottom row) for all patients (1–25, horizontal
axis) and each of the three methods (color coded).
TABLE 3. Summary Statistics of Repeatability Results in RBF (mL/min/1.73 m
2
) and Split RBF Measured with ASL
and PC on Healthy Volunteers
RBF (mL/min/1.73 m
2
) Split RBF
P**
Mean 95%
CI
RE 95%
CI
RRE
(%) 95% CI
mean 95%
CI
RE 95%
CI
RRE
(%) 95% CI
PC 709 130 79 41 11 6 0.54 0.03 0.04 0.02 6 2.7 0.31
ASL 508 202 241 85 61 19 0.50 0.05 0.06 0.04 11 7.6 0.03
P*0.13 0.02 0.06 0.22 0.35 0.22
*Pvalue for PC vs ASL.
**Pvalue for the difference in RRE between RBF vs split RBF.
ASL =arterial spin labeling; CI =confidence interval; PC =phase contrast; RBF =renal blood flow; RE =repeatability error;
RRE =relative repeatability error.
1246 Volume 55, No. 4
Journal of Magnetic Resonance Imaging
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

Statistical Analysis
In patients, repeated measures one-way analysis of variance
(ANOVA) and paired t-tests were conducted to determine the
significance of the difference in RBF (and split RBF) across the
techniques. Influential outliers in the measurements were identi-
fied using Cook’s distance, with a cut-off value of D
i
>1.Where
an outlier was deemed influential, the results with outliers
removed were also provided. The intrasubject agreement in RBF
(and split RBF) was assessed using Bland and Altman method.
20
Atwo-sampleF-test for equal variances was performed to deter-
mine the significance of the difference between ASL/PC and
DCE/PC. Linear correlations between techniques were assessed
using Pearson’s correlation coefficient. Statistical significance was
defined at P< 0.05 for this study.
Results
Participant Demographics
Twenty-five patients (16 males and 9 females, mean age
(interquartile range) of 65 (56–72) years) were analyzed and
included for comparison. Five healthy volunteers (one male
and four females, mean age [interquartile range] of 39 [31–
40] years) completed the four visits at an average of 4 months
apart (range between 2 and 7 months).
Comparison of ASL, DCE, and PC in Diabetic
Patients
In patients, whole parenchyma perfusion, averaged over both
kidneys, was 146 22 mL/min/100 mL using ASL and
214 31 mL/min/100 mL using DCE.
Figure 2 shows the distribution of RBF and split RBF values
derivedfromPC,ASL,andDCEinthisstudy.Thedifferencein
the means was not significant for RBF (F(1, 32) =1.26,
P=0.28) or for split RBF (F(2, 36) =0.14, P=0.87). Using
Cook’s distance measures, RBF measurement of the subject num-
ber 23 was deemed strongly influential (D
i
=4.3). There was no
influential outlier for split RBF parameter (D
i
<1 for all data
points).
Figure 3 shows Bland Altman analysis for RBF and split
RBF pairwise between techniques. For RBF, the plots con-
firmed a small bias between techniques, but the 95% CI of
the differences was large compared to the mean, indicating
substantial differences at the subject level. Specifically, the
limits-of-agreements for RBF (mL/min/1.73 m
2
) were
[687, 772] for DCE-ASL, [482, 580] for PC-DCE, and
[277, 460] for PC-ASL. The 95% CI for the difference was
smaller for PC-ASL than PC-DCE, but only marginally sig-
nificant (P=0.049). Excluding the extreme DCE-RBF out-
lier (subject 23), the difference was no longer significant
(P=0.315), and the limits-of-agreement were reduced to
[220, 407] for PC-DCE, [272, 422] for PC-ASL, and
[448,411] for DCE-ASL. For split RBF, the plots showed
no systematic bias, with the limits-of-agreement of the order
[0.1, 0.1] for each comparison.
Figure 4 shows scatterplots for RBF and split RBF
pairwise between techniques.
Correlations were significant only between PC-ASL for
RBF (r=0.57) and between DCE-ASL for split-RBF
(r=0.80). Removing the DCE outlier had a small effect on
FIGURE 6: Repeat measurements of renal blood flow (RBF) (top row) and split RBF (bottom row) for each of the five healthy
volunteers (HV 01 to HV 05 color coded) derived from phase contrast (PC) (left) and arterial spin labeling (ASL) (right).
April 2022 1247
Alhummiany et al.: Triangulation of MRI-based RBF methods
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

the average RBF measurements but improved correlations for
all techniques (r≥0.41) as shown in Table 2.
Figure 5 shows individual RBF and split RBF values for
all patients across the three techniques. An example case is
presented in the Supplementary Material Figure S2 shows a
typical source of error related to the DCE outlier.
Repeatability of PC and ASL
The average perfusion in the whole parenchyma estimated
using ASL in HVs was 155 50 mL/min/100 mL.
Table 3 summarizes the repeatability results of RBF and
split RBF derived from PC and ASL. For RBF, the mean RE
of an individual PC measurement was significantly (three
times) lower than ASL. The difference in RE between PC
and ASL was not significant for split RBF (P=0.35).
When measured with PC, the difference in repeatability
between split-RBF and RBF was not significant (RRE 6% vs.
11%, P=0.31). Whereas for ASL, split RBF had five times
better repeatability than RBF (RRE 11% vs. 61%).
The difference in the HV population means between
PC-RBF and ASL-RBF was relatively large (33%) but not sta-
tistically significant (P=0.13). The difference in split-RBF
between PC and ASL was much smaller in relative terms
(7.8%) and also not significant (P=0.22). The heterogeneity
of the RBF population in ASL (45%) was twice that of
PC (21%).
Figure 6 shows the repeat measurements of PC and
ASL for each HV. An example case of renal perfusion maps
obtained on four repeat scans for the same HV is shown in
the Supplementary Material Fig. S3.
Discussion
This study compared RBF values between ASL, DCE, and
PC in diabetic patients. The key finding is that RBF values
agree well on average, but intrasubject agreement is poor
between all methods. The uncertainty in split RBF is compar-
atively small, which is consistent with the fact that this metric
is a ratio and therefore any scaling errors related to left and
right kidney will be eliminated. Therefore, the improved
agreement in split RBF suggests that subject-specific scaling
errors are responsible for the poorer precision in RBF. The
repeatability assessment in healthy volunteers further supports
the value of PC as a reference method.
With regards to the direct comparison of DCE and
ASL, previous studies are not entirely aligned. Cutajar et al
9
showed that renal perfusion derived from ASL and DCE in
healthy humans agreed well on average, but agreement on an
individual level was poor with limits-of-agreement of
[150, 200] mL/min/100 g for an individual kidney. Wu
et al
8
on the other hand reported that renal perfusion mea-
surements were correlated but not entirely comparable
between ASL and DCE. Our study supports the observation
in the study by Cutajar et al
9
—strengthening the hypothesis
that DCE and ASL are unbiased but that one of the two, or
both, show poor precision relative to PC.
Comparison with PC in our study indicates that mea-
surement error in both ASL and DCE plays a role, and in an
approximately equal measure. In DCE, a possible source of
error is inflow effect in the aorta, which varied from case to
case. The arterial input function (AIF) of the DCE outlier
showed very strong signal pulsations, an effect that can be
minimized by placing the arterial ROI on the descending
aorta below the origin of the renal arteries (see
Supplementary File S1). In ASL, a possible source of error is
imperfect labeling in the aorta, leading to a signal drop in the
label images. The patient data show several cases with very
low RBF measurement in ASL, and HV’s data show signifi-
cant fluctuations between follow-ups in the same subject (see
Supplementary File S1). Inspection of the scans confirmed a
global signal dropout in perfusion maps in these cases, consis-
tent with the effect of reduced labeling efficiency due to B
0
inhomogeneity near the inversion plane.
21
To mitigate this
effect, a separate B
0
shimming at the labeling site will be con-
sidered for future acquisitions.
As noted earlier, both types of error can be addressed by
improvements in the acquisition and/or analysis. An alterna-
tive strategy would be to combine all three methods, either
by reporting the median value or by performing a joint opti-
mization.
22
Alternatively, PC can be used to derive global
RBF values, while ASL and DCE can be employed to deter-
mine spatial heterogeneity in perfusion but not absolute
values.
For the repeatability study, we reported RRE that
relates to the coefficient of variation (CV) reported in other
studies as CV ¼RRE =1:96. The repeatability of PC was in
agreement with previous studies in the literature. In spite of
the relatively long interval to complete four repeat scans
(4 months on average), it was encouraging to find RRE of
11% (CV =6%). This indicates that possible sources of vari-
ability such as difficulties in positioning the PC plane or the
effect of free breathing were relatively small. The repeatability
of PC-RBF in this study is comparable to the results reported
by Dambreville et al and Khatir et al where two scans were
acquired within 1 week with RRE of 17% and 16%,
respectively.
Studies reporting repeatability for ASL using pCASL are
very limited. One study reported short-term RRE of 27% for
cortical perfusion using pCASL based on two scans acquired
within the same visit.
23
The RRE of ASL in our study
(CV 31%) is in agreement with the results reported by
Harteveld et al using the same pCASL labeling.
24
Repeatability
error in ASL of our study, however, is higher compared to previ-
ous studies using flow-sensitive alternating inversion recovery
(FAIR) labeling approach.
25,26
Besides the sensitivity to B
0
inho-
mogeneity, pCASL labeling can also be influenced by the maxi-
mum pulsatility and blood flow velocity of the descending
1248 Volume 55, No. 4
Journal of Magnetic Resonance Imaging
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

aorta.
27
Previous studies have reported a difference in RBF
between healthy controls and patients with impaired renal func-
tion in the range between 90 and 470 mL/min.
28,29
Thus, the
current uncertainty in pCASL labeling would indicate that sub-
tle changes in RBF (due to disease progression or follow-up
responses after interventions) may go undetected.
Absolute RBF values in this study are comparable to
those reported by Khatir et al
30
in healthy subjects and Jin
et al using breath hold PC,
31
Gillis et al
28
and Shirvani
et al
32
using ASL with FAIR labeling and Rankin et al.
33
using the same pCASL labeling used in our study. Neverthe-
less, it is surprising that mean RBF of our healthy participants
was substantially lower than reference values in the literature.
Based on early work using the clearance of para-
aminohippurate (PAH), normal RBF was determined to be
1129 mL/min/1.73 m
2
in healthy young subjects.
34
These
estimates agree with those of other studies using radioiso-
topes.
35
The population of HVs in our study was predomi-
nantly female, but while this will likely explain a small part of
the difference with male cohorts,
36
the effect of sex alone is
insufficient to explain the large difference. Smaller RBF values
are expected in diabetic patients,
37
but the cohort in our
study was at an early disease stage and had values comparable
to the HVs. A systematic underestimation is theoretically pos-
sible, but it is highly unlikely that ASL, DCE and PC would
produce a broadly similar systematic error. A possible expla-
nation lies in the effect of subject preparation. For reasons
related to the blood sampling in the iBEAt study, study par-
ticipants were scanned after an overnight fast followed by a
standardized breakfast and a drink of water immediately
before the MRI. The content of the meal was chosen to avoid
protein-/salt-rich food that is known to influence RBF, but a
reduced RBF is consistent with the effect of fasting.
38
A future study is currently being planned for to help deter-
mine whether this effect alone is sufficient to explain the
lower RBF values found in this study.
Limitations
The acquisition order of PC, ASL, and DCE was not ran-
domized, due to the use of gadolinium contrast agent. PC
was always acquired first as part of a multiparametric MRI
protocol, resulting in a time difference of approximately
20 minutes between sequences. However, it is unlikely that
this time difference could explain the differences observed
between methods. While employing PC as a reference
method has enriched the results of this study, there is a clear
advantage of using other available methods such as
15
O-
labeled water positron emission tomography (PET) imaging.
This will be addressed in a separate cohort of iBEAt partici-
pants.
14
The use of kidney volume to derive RBF from ASL
and DCE may have introduced an additional source of vari-
ability to the measurements. This effect is expected to be
small, however, since volume measurements derived from the
same ROI were used for each technique. The number of par-
ticipants is small (n=25), and this is likely insufficient to
detect the relatively small differences in the mean values
between the three methods. Patient repeatability data were
not available for this study, and repeatability values from
healthy volunteers do not necessarily translate to disease due
for instance to reduced cortico-medullary differentiation.
Finally, despite efforts in standardizing patient preparation,
subtle physiological changes between the repeat scans could
not be excluded.
Conclusions
Comparing RBF derived from ASL, DCE, and PC showed
little bias, but poor precision reflecting substantial differences
at the individual level. Triangulation with PC indicated that
ASL and DCE can achieve a comparable level of accuracy.
Acknowledgments
iBEAt study is part of the BEAt-DKD project. The BEAt-
DKD project has received funding from the Innovative Medi-
cines Initiative 2 Joint Undertaking under grant agreement
No 115974. This Joint Undertaking receives support from
the European Union’s Horizon 2020 research and innovation
program and EFPIA with JDRF. For a full list of BEAt-DKD
partners, see www.beat-dkd.eu
References
1. Burke M, Pabbidi MR, Farley J, Roman RJ. Molecular mechanisms of
renal blood flow autoregulation. Curr Vasc Pharmacol 2014;12(6):
845-858.
2. Cai Y-Z, Li Z-C, Zuo P-L, et al. Diagnostic value of renal perfusion in
patients with chronic kidney disease using 3D arterial spin labelling.
J Magn Reson Imaging 2017;46(2):589-594.
3. Sourbron SP, Michaely HJ, Reiser MF, Schoenberg SO. MRI-
measurement of perfusion and glomerular filtration in the human kid-
ney with a separable compartment model. Invest Radiol 2008;43(1):
40-48.
4. Odudu A, Nery F, Harteveld AA, et al. Arterial spin labelling MRI to
measure renal perfusion: A systematic review and statement paper.
Nephrol Dial Transplant 2018;33(suppl_2):ii15-ii21.
5. Su M-YM, Huang K-H, Chang C-C, et al. MRI evaluation of the adaptive
response of the contralateral kidney following nephrectomy in patients
with renal cell carcinoma. J Magn Reson Imaging 2015;41(3):822-828.
6. Brown RS, Sun MRM, Stillman IE, et al. The utility of magnetic reso-
nance imaging for noninvasive evaluation of diabetic nephropathy.
Nephrol Dial Transplant off Publ Eur Dial Transpl Assoc - Eur Ren Assoc
2020;35(6):970-978.
7. Selby NM, Blankestijn PJ, Boor P, et al. Magnetic resonance imaging
biomarkers for chronic kidney disease: A position paper from the
European Cooperation in Science and Technology action
PARENCHIMA. Nephrol Dial Transplant 2018;33(suppl_2):ii4-ii14.
8. Wu W-C, Su M-Y, Chang C-C, Tseng W-YI, Liu K-L. Renal perfusion 3-T
MR imaging: A comparative study of arterial spin labelling and dynamic
contrast-enhanced techniques. Radiology 2011;261(3):845-853.
9. Cutajar M, Thomas DL, Hales PW, Banks T, Clark CA, Gordon I. Com-
parison of ASL and DCE MRI for the non-invasive measurement of renal
April 2022 1249
Alhummiany et al.: Triangulation of MRI-based RBF methods
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

blood flow: Quantification and reproducibility. Eur Radiol 2014;24(6):
1300-1308.
10. Zimmer F, Zöllner FG, Hoeger S, et al. Quantitative renal perfusion
measurements in a rat model of acute kidney injury at 3T: Testing
inter-and intramethodical significance of ASL and DCE-MRI. PLoS One
2013;8(1):e53849.
11. Hockings P, Saeed N, Simms R, et al. MRI biomarkers. In: Seiberlich N,
Gulani V, Calamante F, et al., editors. Quantitative magnetic resonance
imaging. 1st ed. Elsevier Academic Press; Elsevier Science; 2020. p Iiii-
Ixxxvi.
12. Villa G, Ringgaard S, Hermann I, et al. Phase-contrast magnetic reso-
nance imaging to assess renal perfusion: A systematic review and state-
ment paper. Magn Reson Mater Physics, Biol Med 2020;33(1):3-21.
13. Eckerbom P, Hansell P, Cox E, et al. Multiparametric assessment of
renal physiology in healthy volunteers using noninvasive magnetic res-
onance imaging. Am J Physiol Renal Physiol 2019;316(4):F693-F702.
14. Gooding KM, Lienczewski C, Papale M, et al. Prognostic imaging bio-
markers for diabetic kidney disease (iBEAt): Study protocol. BMC
Nephrol. 2020;21(242):1-11.
15. Buxton RB, Frank LR, Wong EC, Siewert B, Warach S, Edelman RR. A
general kinetic model for quantitative perfusion imaging with arterial
spin labelling. Magn Reson Med 1998;40(3):383-396.
16. Sourbron S. plaresmedima/PMI-0.4-SUGAR: Outcomes [Internet]. 2020.
Available from: https://zenodo.org/record/4086055
17. Buonaccorsi GA, O’Connor JPB, Caunce A, et al. Tracer kinetic model–
driven registration for dynamic contrast-enhanced MRI time-series data.
Magn Reson Med 2007;58(5):1010-1019.
18. Dujardin M, Sourbron S, Luypaert R, Verbeelen D, Stadnik T. Quantifi-
cation of renal perfusion and function on a voxel-by-voxel basis: A fea-
sibility study. Magn Reson Med 2005;54(4):841-849.
19. Du Bois D, Du Bois EF. Clinical calorimetry: Tenth paper a formula to
estimate the approximate surface area if height and weight be known.
Arch Intern Med. 1916;XVII(6_2):863-871.
20. Bland JM, Altman DG. Statistical methods for assessing agreement
between two methods of clinical measurement. Lancet (London,
England) 1986;1(8476):307-310.
21. Wu W-C, Fern
andez-Seara M, Detre JA, Wehrli FW, Wang J. A theoret-
ical and experimental investigation of the tagging efficiency of
pseudocontinuous arterial spin labelling. Magn Reson Med 2007;58(5):
1020-1027.
22. Robson PM, Madhuranthakam AJ, Dai W, Pedrosa I, Rofsky NM,
Alsop DC. Strategies for reducing respiratory motion artifacts in renal
perfusion imaging with arterial spin labelling. Magn Reson Med 2009;
61(6):1374-1387.
23. Kim DW, Shim WH, Yoon SK, et al. Measurement of arterial transit time
and renal blood flow using pseudocontinuous ASL MRI with multiple
post-labelling delays: Feasibility, reproducibility, and variation. J Magn
Reson Imaging 2017;46(3):813-819.
24. Harteveld AA, de Boer A, Franklin SL, Leiner T, van Stralen M, Bos C.
Comparison of multi-delay FAIR and pCASL labelling approaches for
renal perfusion quantification at 3T MRI. Magn Reson Mater Physics,
Biol Med. 2020;33(1):81-94.
25. Cox EF, Buchanan CE, Bradley CR, et al. Multiparametric renal mag-
netic resonance imaging: Validation, interventions, and alterations in
chronic kidney disease. Front Physiol 2017;8:696.
26. de Boer A, Harteveld AA, Stemkens B, et al. Multiparametric renal MRI:
An Intrasubject test–retest repeatability study. J Magn Reson Imaging
2020;53(3):859-873.
27. Amanuma M, Mohiaddin RH, Hasegawa M, Heshiki A, Longmore DB.
Abdominal aorta: Characterisation of blood flow and measurement of
its regional distribution by cine magnetic resonance phase-shift velocity
mapping. Eur Radiol 1992;2(6):559-564.
28. Gillis KA, McComb C, Patel RK, et al. Non-contrast renal magnetic res-
onance imaging to assess perfusion and corticomedullary differentia-
tion in health and chronic kidney disease. Nephron 2016;133(3):
183-192.
29. Buchanan CE, Mahmoud H, Cox EF, et al. Quantitative assessment of
renal structural and functional changes in chronic kidney disease using
multi-parametric magnetic resonance imaging. Nephrol Dial Transplant
2019;35(6):955-964.
30. Khatir DS, Pedersen M, Jespersen B, Buus NH. Reproducibility of MRI
renal artery blood flow and BOLD measurements in patients with
chronic kidney disease and healthy controls. J Magn Reson Imaging
2014;40(5):1091-1098.
31. Jin N, Lewandowski RJ, Omary RA, Larson AC. Respiratory self-gating
for free-breathing abdominal phase-contrast blood flow measurements.
J Magn Reson Imaging 2009;29(4):860-868.
32. Shirvani S, Tokarczuk P, Statton B, et al. Motion-corrected multi-
parametric renal arterial spin labelling at 3 T: Reproducibility and effect
of vasodilator challenge. Eur Radiol 2019;29(1):232-420.
33. Rankin AJ, Allwood-Spiers S, Lee MMY, et al. Comparing the inter-
observer reproducibility of different regions of interest on multi-
parametric renal magnetic resonance imaging in healthy volunteers,
patients with heart failure and renal transplant recipients. Magma 2020;
33(1):103-112.
34. Davies DF, Shock N. Age changes in glomerular filtration rate, effective
renal plasma flow, and tubular excretory capacity in adult males. J Clin
Invest 1950;29(5):496-507.
35. Botti RE, Razzak MA, MacIntyre WJ, Pritchard WH. The relationship of
renal blood flow to cardiac output in normal individuals as determined
by concomitant radioisotopic measurements. Cardiovasc Res 1968;2(3):
243-246.
36. Bax L, Bakker CJG, Klein WM, Blanken N, Beutler JJ, Mali WPTRM.
Renal blood flow measurements with use of phase-contrast magnetic
resonance imaging: Normal values and reproducibility. J Vasc Interv
Radiol 2005;16(6):807-814.
37. Mora-Gutierrez JM, Garcia-Fernandez N, Slon Roblero MF, et al. Arte-
rial spin labelling MRI is able to detect early hemodynamic changes in
diabetic nephropathy. J Magn Reson Imaging 2017;46(6):1810-1817.
38. Hostetter TH. Human renal response to meat meal. Am J Physiol 1986;
250(4 Pt 2):F613-F618.
1250 Volume 55, No. 4
Journal of Magnetic Resonance Imaging
15222586, 2022, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jmri.27888 by <Shibboleth>-member@leeds.ac.uk, Wiley Online Library on [25/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License