ArticlePDF Available

Transfer function restoration in 3D electron microscopy via iterative data refinement

IOP Publishing
Physics in Medicine & Biology
Authors:

Abstract and Figures

Three-dimensional electron microscopy (3D-EM) is a powerful tool for visualizing complex biological systems. As with any other imaging device, the electron microscope introduces a transfer function (called in this field the contrast transfer function, CTF) into the image acquisition process that modulates the various frequencies of the signal. Thus, the 3D reconstructions performed with these CTF-affected projections are also affected by an implicit 3D transfer function. For high-resolution electron microscopy, the effect of the CTF is quite dramatic and limits severely the achievable resolution. In this work we make use of the iterative data refinement (IDR) technique to ameliorate the effect of the CTF. It is demonstrated that the approach can be successfully applied to noisy data.
Content may be subject to copyright.
INSTITUTE OF PHYSICS PUBLISHING PHYSICS IN MEDICINE AND BIOLOGY
Phys. Med. Biol. 49 (2004) 509–522 PII: S0031-9155(04)69260-8
Transfer function restoration in 3D electron
microscopy via iterative data refinement
C O S Sorzano
1,2
, R Marabini
2,3
, G T Herman
4
, Y Censor
5
and J M Carazo
2,3
1
Escuela Polit
´
enica Superior, Universidad San Pablo-CEU, Campus Urb Montepríncipe,
s/n, 28668 Boadilla del Monte, Madrid, Spain
2
Centro Nacional de Biotecnolog
´
ıa, Campus Universidad Aut
´
onoma s/n, 28049 Cantoblanco,
Madrid, Spain
3
Escuela Polit
´
ecnica Superior, Universidad Aut
´
onoma de Madrid, Campus Universidad
Aut
´
onoma s/n, 28049 Cantoblanco, Madrid, Spain
4
The Graduate Center, The City University of New York, 365 Fifth Avenue, New York,
NY 10016-4309, USA
5
Department of Mathematics, University of Haifa, Mt Carmel, Haifa 31905, Israel
Received 12 September 2003
Published 27 January 2004
Online at stacks.iop.org/PMB/49/509 (
DOI: 10.1088/0031-9155/49/4/003)
Abstract
Three-dimensional electron microscopy (3D-EM) is a powerful tool for
visualizing complex biological systems. As with any other imaging device,
the electron microscope introduces a transfer function (called in this field
the contrast transfer function, CTF) into the image acquisition process that
modulates the various frequencies of the signal. Thus, the 3D reconstructions
performed with these CTF-affected projections are also affected by an implicit
3D transfer function. For high-resolution electron microscopy, the effect of
the CTF is quite dramatic and limits severely the achievable resolution. In
this work we make use of the iterative data refinement (IDR) technique to
ameliorate the effect of the CTF. It is demonstrated that the approach can be
successfully applied to noisy data.
1. Introduction
The analysis of macromolecular complexes and their dynamics is one of the most interesting
challenges in molecular biology. A promising future is awaiting the electron microscopist due
to the possibilities of visualizing molecular machines, reconstructing unique (as opposed to
averaged) objects and imaging dynamic processes. The road to achieving these possibilities is
via three-dimensional reconstruction from electron-microscopic images of the macromolecular
complexes.
There are many methods for reconstructing a three-dimensional object from its line
integrals (Herman 1980, Natterer and W
¨
ubbeling 2001). Typically, the line integrals are
estimated for a set of parallel lines from a projection image that is obtained by some instrument.
0031-9155/04/040509+14$30.00 © 2004 IOP Publishing Ltd Printed in the UK 509
510 C O S Sorzano et al
-1.5
-1
-0.5
0
0.5
1
1.5
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16
CTF
Frequency (1/A)
Figure 1. Radial profile of the CTF used in the cryomicroscopy simulations.
Figure 2. Amplitude of a typical astigmatic CTF.
A difficulty that arises in electron microscopy is that the image that is produced by the
instrument corresponds to the convolution of the ideal projection image with a point spread
function (PSF). The PSF is usually described by its Fourier transform that is commonly called
the contrast transfer function (CTF); for example, see figures 1 and 2.
The CTF severely limits the achievable resolution in the three-dimensional reconstruction.
In particular, it filters both the high and the low frequencies, introduces zones of alternate
contrast and eliminates all information at certain frequencies. It is, therefore, desirable to
Transfer function restoration in 3D electron microscopy via iterative data refinement 511
replace the reconstruction obtained by a ‘real’ microscope by a reconstruction that would be
obtained from images that would be produced by an ideal, aberration-free microscope. In order
to achieve this goal several methods have been proposed: Frank and Penczek (1995) applied
Wiener filtering in the three-dimensional space to the reconstructed volume; Zhu et al (1997)
incorporated a three-dimensional PSF into the data model and used a regularized steepest-
descent technique; Stark et al (1997) applied inverse CTF filtering to the reconstructed volume;
Skoglund et al (1996) incorporated a two-dimensional CTF, particular to each projection to the
projection, model in a maximum-entropy reconstruction algorithm; Grigorieff (1998) provided
a Fourier reconstruction algorithm in which the CTF for each projection is considered in a
Wiener-like fashion; Ludtke et al (1999) proposed a CTF correction applied to the individual
projections with a weighting function in the Fourier space computed from a set of images
sharing a common CTF; Ludtke et al (2001) added a Wiener filter to the weighting function
defined in Ludtke et al (1999). An alternative is to explicitly introduce the effect of the CTF
in the reconstruction equations; this was done by Zubelli et al (2003), who then reformulated
the problem so that Chahine’s method became applicable to it. The existence of these multiple
approaches is indicative of the fact that there is no agreed standard technique for the correction
of CTF effects in three-dimensional electron microscopy (3D-EM) of single particles and the
search for superior methods is still active.
In this work we apply the technique of iterative data refinement (IDR)—introduced in
Censor et al (1985) and further studied in Herman (1989), Herman and Ro (1990), Losada and
Navarro (1998) and Ro et al (1989)—to reduce the effect of the CTF and, thus, to obtain high-
resolution structural information about the macromolecules under study. As opposed to many
of the approaches discussed in the previous paragraph, our proposed algorithm can handle the
case of differing CTFs in the projections and does not require estimation of the signal-to-noise
ratio (SNR). The potential benefit of the method is illustrated by an experiment that involves
realistic simulation of the electron microscopic imaging of a biological macromolecule.
2. Mathematical background
2.1. Contrast transfer function
Image formation by an electron microscope is due to several physical processes of electron
interaction with the specimen. These effects combine to produce a single CTF, see Frank
(1996, ch 2.II). A parametric model of this transfer function has been used in the simulations
presented in this work. This parametric model accounts for the various effects involved in the
CTF (Zhu et al (1997) and Frank (1996,ch2.II))andwasalsousedinZhouet al (1996).
Basically, the microscope transfer function is a real-valued function in Fourier space
formed by a damped harmonic function. The ‘sine’ part of this function comes from the phase
change that electrons undergo when interacting with the sample specimen. A detailed study
of the electronic interaction in the image formation plane shows that the transfer function of
an electron microscope can be usefully approximated by
CTF(ω) = E(ω)
sinf |ω|
2
) Q
0
cosf
|
ω
|
2
)
(1)
where ω is the spatial frequency, f is the defocus and Q
0
is a factor accounting for the
loss of electrons during the image formation process. Usually Q
0
is a small number, which
implies that the DC component of the projection Fourier transform is nearly removed and, thus,
the absolute density values in the projections are not meaningful. For this reason, usually only
the relative values are taken into consideration when interpreting a 3D-EM reconstruction. The
damping envelope, E(ω), models microscope imperfections such as chromatic aberration,
512 C O S Sorzano et al
spherical aberration, current and voltage instabilities, angular aperture, etc; see, e.g.,
Zhou et al (1996)orFrank(1996,ch2.II).
The model explained so far defines the shape of the profile of the CTF (for a typical
example, see figure 1). Many studies assume that this profile is radially symmetric, although
this is not necessarily true. Astigmatism is a well-known effect which turns the circles
produced by the radial symmetrization of the CTF profile into ellipses (see figure 2). This
results in a different defocus along every radial line of the Fourier space.
2.2. Phase flipping
Notice (in figure 1) that the sign changes at the zero-crossings result in a contrast inversion
in the projection image and cause the complete elimination of the information at certain
frequencies. This is a very limiting factor in electron microscopy, since without CTF correction
all reconstructions are unreliable at frequencies beyond the one where the CTF first becomes
zero. Nevertheless, biologically useful results can sometimes be obtained even without CTF
correction (B
´
arcena et al 2001, Sorzano et al 2001). However, this can only be the case if the
important biological information is not in the high-frequency part of the reconstruction, since
(as illustrated in our experiment reported below) in a reconstruction without CTF correction,
the information regarding frequencies beyond the first zero-crossing of the CTF is incorrect.
A simple method to alleviate this problem consists of multiplying the projection Fourier
transform by the sign of the CTF (this approach is named phase flipping (Frank 1996, p 45))
to produce the corrected projection data to which the reconstruction algorithm is then applied.
Thus the correction of the CTF sign is simple, as it only needs to adjust the sign at those
frequencies where it is flipped. However, amplitude correction is more difficult, as it requires
either dividing by the transfer function (avoiding zeros by using, for instance, a Wiener filter)
or the incorporation of the CTF operator into the reconstruction algorithm, allowing each
projection to have its own CTF. The IDR approach of this paper addresses this more difficult
problem. It will be set up based on the assumption that the data had been already corrected by
phase flipping.
2.3. Computational representation of volumes and projections
For the computational procedures of this paper we need to establish conventions for
representing volumes and projections by finite sets of numbers. In this subsection we present
our conventions and explain the operators which are incorporated into our algorithms.
We approximate arbitrary volumes by finite series expansions of the general form
J
j=1
c
j
b(r r
j
). (2)
In this formula r is the point at which the volume is being approximated, the r
j
are fixed points
in space, b is a function of three variables and the c
j
are the coefficients of the expansion. In
any application, b and the r
j
are fixed, it is the c
j
that distinguish one volume from another.
Following Lewitt (1990, 1992) and Matej and Lewitt (1995, 1996), we use a generalized
Kaiser–Bessel window function (also called a blob)forb and a finite subset of the body
centered cubic grid for the points r
j
. Such a representation was found useful in electron
microscopy applications; see B
´
arcena et al (2001), Marabini et al (1997, 1998) and Sorzano
et al (2001). The specific choice that we adopt for the blob and the grid is the one referred to
as the ‘standard blob’ in Matej and Lewitt (1996).
We approximate a projection by a two-dimensional array of numbers, each representing a
projection value at a point of a square grid. To bring this into the electron microscopy context,
Transfer function restoration in 3D electron microscopy via iterative data refinement 513
we think of the square grid as lying on a projection plane that is perpendicular to the direction
of the electrons. Assuming that we have m different projection planes, we use g
i
(1 i m)
to denote the array of numbers associated with the ith projection. We use g to denote the
complete set of the m projections, meaning that g is the concatenated vector of all the g
i
s.
Given a volume representation as in (2) it is easy to calculate the ideal projection (line
integrals along lines perpendicular to the projection plane and passing through the points of
the square grid). This is so because the integration can be brought inside the summation and
can be analytically evaluated for the known blob b and grid point r
j
.For1 i m,we
define an ideal projection operator P
i
that associates with the J -dimensional vector c (whose
jth component is c
j
) the vector representing the ideal projection of the volume onto the ith
projection plane. Note that, in practice, P
i
is a matrix of J columns and as many rows as the
number of grid points in the ith projection plane.
Each projection also has its own CTF operator that we denote by H
i
. In practice, given
the projection g
i
, H
i
g
i
is computed by taking the discrete Fourier transform of g
i
, multiplying
it point-wise by the phase-flipped CTF associated with the ith projection and then taking the
inverse discrete Fourier transform.
2.4. Iterative data refinement
The measuring device (the electron microscope) provides data that only approximate what we
intend to measure. The discrepancy between the actual data (under our assumptions, corrupted
by the phase-flipped CTF) and data that are idealized (uncorrupted by the CTF, henceforth
called ideal data) can be estimated from the actual data and knowledge of the measuring
process, leading to a better approximation of the ideal data. This new approximation can then
be used to estimate the new discrepancy, and the process can be repeated. Our knowledge
of the measurement process is insufficient to obtain the ideal data exactly, but the original
discrepancy is significantly reduced by just a few of such iterative steps. This process is
accomplished by the iterative data refinement (IDR) methodology of Censor et al (1985).
Here we briefly review the fundamentals of IDR and describe our specific implementation of
it for the CTF removal problem. We then supply a short discussion that puts the approach in
perspective and relates it to the current literature.
In the following we use R to denote a recovery operator (in our case a three-dimensional
reconstruction algorithm) that produces, for a complete set of projections g, a vector c that
represents a volume using (2). For now it is not important to specify our choice of R;we
will do so below. An important assumption about R (well justified by the known behaviour
of reconstruction algorithms (Herman 1980)) is that if it is applied to the ideal data
ˆ
g (that
is, the concatenated vector of all the P
i
cs), then
ˆ
c = R
ˆ
g is an acceptable approximation to c.
The problem is that, in practice, the actual data
˜
g are corrupted by the CTF and so R
˜
g is not
a satisfactory approximation to c. The IDR approach aims at estimating the ideal data
ˆ
g from
the actual data
˜
g, assuming knowledge of the CTFs.
IDR produces a sequence of vectors g
k
(k = 0, 1, 2,...); the aim is that they should be
improving estimates of the ideal data. We denote by g
k
i
the part of g
k
that is associated with
the ith projection. With this notation, our version of the IDR algorithm is formulated as the
following iterative process. (Figure 3 depicts a block diagram of the algorithm.)
Algorithm: iterative data refinement (IDR) for 3D electron microscopy
Initialization: Take g
0
=
˜
g, the actual data (i.e., the experimentally measured data corrected
by phase flipping).
514 C O S Sorzano et al
Figure 3. Block diagram for the IDR algorithm (based on the one originally published by Censor
et al (1985)).
Iterative step: Given the current iterate g
k
=
g
k
i
m
i=1
, calculate the next iterate g
k+1
=
g
k+1
i
m
i=1
by using, for all i = 1, 2,...,m,the formula
g
k+1
i
= µ
k
g
0
i
+ (P
i
µ
k
H
i
P
i
)Rg
k
(3)
where {µ
k
}
k=0
is a sequence of the so-called relaxation parameters.
This algorithm generates an iterative sequence {g
k
}
k=0
which is guaranteed to converge
to the ideal data
ˆ
g under some stringent conditions, see Censor and Zenios (1997, proposition
10.5.8). However, in practice, it has been shown (see Censor et al (1985) and references
therein) that, even when convergence cannot be guaranteed, the early iterates produced by the
IDR algorithm are closer to the ideal data vector
ˆ
g than g
0
is. This property of IDR is the basis
of the present study.
The underlying idea of IDR is to refine the data iteratively in a way that bridges the gap
between an accurate model of data collection (but one for which we do not have a reconstruction
algorithm) and an approximate model that leads to a reconstruction algorithm (which would
Transfer function restoration in 3D electron microscopy via iterative data refinement 515
work if the model were correct). This is quite different from just being another reconstruction
method, since it iterates on the data rather than on the unknowns of the reconstruction problem.
The IDR method is a member of the family of ‘iterative defect-correction methods’ much used
in the field of differential equations, see, e.g., B
¨
ohmer et al (1984) and Stetter (1978). For
additional applications of the IDR approach, consult section 2 of Censor et al (1985), where
beam hardening correction in x-ray computerized tomography, attenuation correction in single
photon emission computed tomography (SPECT) and image reconstruction with incomplete
data are described along with references to the original studies. See also Herman and Ro
(1990) for a study of the connection between IDR and phase retrieval algorithms.
2.5. Reconstruction algorithm
We now return to the choice of the recovery operator R that was left unspecified in the previous
subsection. We emphasize that the basic approach of IDR is independent of this choice; any
good reconstruction algorithm could be used. In our study we used for the recovery operator R
the reconstruction algorithm called block-ART with blobs in which each block corresponds to
one projection. We note that this algorithm has been found efficacious for 3D reconstruction
from electron microscopic data (B
´
arcena et al 2001, Marabini et al 1997, 1998, Sorzano et al
2001).
Block-ART with blobs is an iterative algorithm. Given a data vector g, it produces a
sequence of iterates {c
k
}
k=0
, each of which defines a volume using (2).
Algorithm: iterative block-ART for volume recovery
Initialization: Take c
(0)
= 0, the zero vector.
Iterative Step: Given the current iterate c
k
, calculate the next iterate c
k+1
by the formula
c
k+1
= c
k
+ P
T
i
k
(g
i
P
i
c
k
) (4)
where i=k mod m + 1.
Here P
T
i
is the transpose of the ideal projection operator P
i
, and each
k
is a relaxation
matrix whose exact nature is explained by Eggermont et al (1981), who also provide
convergence results for the iterative block-ART algorithm (their theorem 1.3). In the
experiments, reported below, we do not run the algorithm to convergence, but use only
one full cycle of it; i.e., we define Rg to be c
m
. (This is justified by previous experience in this
application area; see, e.g., Marabini et al (1998).) Also, we selected each
k
to be a diagonal
matrix, the value of each entry on the diagonal is a constant λ divided by the square of the
norm of the corresponding row of P
i
; see (2.18) in Eggermont et al (1981). (This is also
justified by previous experience. It was also found that for the data collection that we used for
the experiments reported below, λ = 0.05 is a good choice.)
The recovery operator R is used not only in the IDR process, but also to produce the final
reconstruction from the (possibly corrected) projection data. For this purpose, it needs to be
extended by an additional step, since further analysis of the reconstructions requires that they
should be evaluated at the points of a cubic grid. Such an evaluation is done using (2), yielding
a three-dimensional array of numbers that we consider to be the reconstruction.
There is an interesting alternative to be considered here: why not incorporate the effect
of the CTF into the model and then use iterative block-art for volume recovery directly on
the actual data? (In practice, this means that in (4) P
i
has to be replaced by H
i
P
i
in both
places where it occurs, with corresponding changes in the calculations of the entries of the
diagonal matrix
k
. Computationally this is some additional burden, but not too much: since
516 C O S Sorzano et al
H
i
is symmetric, H
T
i
= H
i
and the computation can be carried out by the method described at
the end of the subsection on computational representation of volumes and projections.) In the
current paper the choice of the recovery operator is not essential: it can be replaced in the
IDR process by any good reconstruction algorithm. We leave to the future the investigation
of the efficacy of IDR relative to applying directly to the actual data either iterative block-art
for volume recovery or the alternative algebraic approach proposed by Zubelli et al (2003).
3. Evaluation methodology
To compare the performance of IDR with no CTF correction and with phase flipping, we
adapted the methodology proposed in Furuie et al (1994) and previously applied to electron
microscopy in Marabini et al (1997, 1998) and Sorzano et al (2001). In the experiments
described below, many sets of simulated electron microscopy projections of a particular
molecule are taken and reconstructions are produced from each of these sets using the
approaches to be compared. The success of the approaches is determined by using figures of
merit (FOMs). We now provide details of this outline.
3.1. Projection data generation
The volume used in our tests was created from an atomic structure deposited in the protein data
bank (PDB), see Berman et al (2000), namely the Halobacterium halobium bacteriorhodopsin
(PDB id: 1BRD, Henderson et al (1990)). For comparison purposes this volume was evaluated
for points of a 64 ×64 ×64 cubic grid, with distance 3.5
˚
A between neighbouring grid points.
We refer to the resulting three-dimensional array as the phantom.
Several sets of 2,000 projections were created with signal-to-noise ratio 1/3; this resembles
cryomicroscopy conditions. This noise in the measurements was combined with other sources
of inconsistency in the form of random translations (by moving the projection plane parallel to
itself by a distance randomly selected from a zero-mean Gaussian distribution with standard
deviation 7
˚
A) and random rotations (by adding a zero-mean 5
standard deviation Gaussian
noise to each of the Euler angles that defined the orientation of the projection planes).
In addition to noise, the projections were convolved with a CTF. The parameters for a
circularly-symmetric simulated CTF, see (1), were given values typically found in experimental
conditions: Q
0
=−0.06 and f =−20 000
˚
A, and the factors that influence the damping
envelope E(ω) were selected as acceleration voltage =100 kV, spherical aberration =5.5 mm,
chromatic aberration =6 mm, energy loss =9.9 eV, convergence cone =0.2 mrad, longitudinal
displacement = 80
˚
A(seeVel
´
azquez-Muriel et al (2003) for a description of these parameters).
The radial profile of this CTF is shown in figure 1. The same CTF was used in all projections,
since this is the worst case that can occur because it makes it difficult to compensate for
missing information in one projection by data from other projections.
A surface rendering of the ideal volume and a selection of projections are shown in
figures 4 and 5, respectively.
3.2. Figures of merit
Figures of merit (FOMs) are numerical measures of the reconstruction quality that are based on
specific aspects. A simple but often used measure is the sum of the squares of the differences
between the individual values in the reconstructions and the corresponding values in the
phantom. In Sorzano et al (2001), this measure was divided by the number of points in the
cubic grid and the result was called the FOM scL2.
Transfer function restoration in 3D electron microscopy via iterative data refinement 517
Figure 4. Side and top view of the isosurface of the bacteriorhodopsin phantom.
Figure 5. A selection of projections simulating cryomicroscopy images from the bacteriorhodopsin
phantom.
More sophisticated measures are provided by the Fourier shell correlation (FSC), as
described in equation (3.65) of Frank (1996) with F
1
the phantom and F
2
the reconstruction.
The FSC indicates, for every shell of frequencies (determined by a frequency k and a shell
thickness k), how well the reconstruction correlates with the phantom for all frequencies
within that shell. Thus, the FSC provides a separate FOM for every shell. The FSC can also
be used to provide the additional single FOM of resolution by defining it as the frequency at
which the FSC falls below 0.5.
The presence of many reasonable FOMs (Sorzano et al (2001) lists 24 of them) makes
exhaustive comparisons difficult. We have developed a methodology that applies multivariate
518 C O S Sorzano et al
Figure 6. Difference in the scL2 FOM between the reconstruction after kth and (k 1)th IDR
iterations for 30 tests.
statistics to obtain a single FOM that in some senses captures the essence of what is provided
by the full set of FOMs; it is described in detail in the doctoral dissertation of Sorzano (2002).
Here we give a brief description of the use of this methodology for selecting an optimal
range for a parameter µ, such as one of the relaxation parameters µ
k
in (3). To do this, a
number of training data sets are created and each one of them is processed using a number
of values of the parameter µ. Then the method for producing a single representative FOM
proceeds in five steps: first, each of the FOMs that we wish to consider in producing the
single FOM is normalized to have mean 0 and standard deviation 1; second, those FOMs
that show no dependency (as indicated by a 1-way analysis of variance, ANOVA for short)
with µ are removed; third, all FOMs showing a similar dependency with µ are clustered
by a hierarchical classification; fourth, the dimensionality of the clusters is reduced using a
principal component analysis (PCA) and a single representative is selected for each cluster and
fifth, the cluster representatives are combined into a single FOM. We refer to it in this paper
as the training FOM.Theoptimal range for µ is considered to be the maximal range within
which the performance (as measured by the training FOM applied to the results obtained from
the training data sets) is not statistically significantly different from the optimal performance.
3.3. Training
We first give the details of the training methodology for selecting the optimal range of µ
0
.
Preliminary tests indicated that we should not be looking outside the range [1.4, 2.4]. Within
this range we investigated values of µ
0
at 0.1 increments. For each of this 11 values, ten
complete actual (noisy) data sets were generated by the method described above and a training
FOM was produced based on these 110 reconstructions. We found that the corresponding
optimal range for µ
0
is [1.7, 1.9].
To select µ
1
, we essentially repeat this process, but now the evaluation is based on the
reconstructions produced from g
2
. (For each of the 110 data sets, µ
0
was randomly selected
from its optimal range.) The results were similar, namely the optimal range for µ
1
turned out
Transfer function restoration in 3D electron microscopy via iterative data refinement 519
Figure 7. From top to bottom, slices corresponding to central sections of: bacteriorhodopsin
phantom, reconstruction without CTF correction, reconstruction with phase flipping, reconstruction
with IDR after one and six iterations.
to be the same as for µ
0
. In fact, repeating this process for µ
k
,for2 k 15, the same
optimal range was found each time.
To determine the stopping criterion, 30 new complete actual data sets were generated
and the IDR algorithm was run for 15 iterations. For each of the 30 actual data sets
˜
g,
reconstructions were produced from g
k
,for0 k 15. The FOM scL2 was calculated for
each reconstruction and, for 1 k 15, the difference between scL2 values for that iteration
and the previous iteration (from the same data set) was calculated. The results, plotted in
figure 6, show that there appear to be no significant changes produced by the IDR iterations
beyond the sixth one. In this case, the scL2 value to which the algorithm converged was 0.998.
4. Results
Figure 7 illustrates our results: four central slices of the phantom and of reconstructions
(from a new data set generated by the previously described rules) are shown. In figure 8 we
plot the associated Fourier shell correlations (see section 3) for assessing the reconstruction
quality. The improvement by any of the corrections over the uncorrected reconstruction is
highly noticeable, even in the case of only phase flipping. Note that using only the FOM
‘resolution’ (as defined in section 3), there is no significant difference between phase flipping
and IDR (regardless of the number of iterations), since all the CTF corrected reconstructions
520 C O S Sorzano et al
0
0.2
0.4
0.6
0.8
1
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07
Fourier Shell Correlation
Frequency (1/A)
No CTF correction
Phase correction
IDR iteration 1
IDR iteration 6
Figure 8. Fourier shell correlation curves for the reconstructions without CTF correction, with
phase flipping and IDR after one and six iterations.
have a resolution at around 0.06
˚
A
1
(as opposed to the resolution of the reconstruction
from uncorrected data that is around 0.035
˚
A
1
). However, examining the full FSC curves
instead of concentrating only on resolution reveals a clear improvement in the IDR-corrected
reconstructions over the phase-flipped reconstruction in the range in which the CTF is inverted
(between 0.037
˚
A
1
and 0.052
˚
A
1
). Furthermore, we can see the improvements produced by
additional IDR iterations. All this is visually confirmed in figure 7.
The discussion in the previous paragraph is anecdotal: it is based on reconstructions from
a single data set. To be able to assign statistical significance to our claim of superiority of the
IDR reconstruction, over the phase-flipped reconstruction, we generated 30 additional actual
data sets (using the same rules as before) and compared the phase-flipped reconstructions with
the reconstructions after the sixth iteration of the IDR algorithm. The results are summarized
in figure 9, which plots (for 19 shells) the average (over the 30 data sets) of the FSC value
for IDR after six iterations less the FSC value for phase flipping. Standard deviations of these
differences over the 30 experimental outcomes are also indicated. The standard deviation of
the average difference is 1/
30 times the standard deviation indicated in the figure. Hence it
is clear that, for each of the 19 shells, one can extremely confidently reject the null hypothesis
that the expected value of the FSC for the IDR reconstruction after six iteration is the same as
the expected value of the FSC for the phase-flipped reconstruction in favour of the alternative
hypothesis that the expected value of the FSC is higher for the IDR reconstruction. In fact, for
each of the 19 shells, the value of the average difference is more than ten times the standard
deviation of the average difference (thus providing us with a P value less than 10
23
).
IDR’s ability of removing the effect of an instrumental transfer function was first
demonstrated by Ro et al (1989) for magnetic resonance imaging. We have shown here
that IDR is also efficacious in electron microscopy, which presents much nastier transfer
functions and extremely noisy images. Our results demonstrate that it is possible to recover
much of the information that is lost near the zeros of the CTF and that the quality of the
Transfer function restoration in 3D electron microscopy via iterative data refinement 521
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
0.045
0.05
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07
Difference in Fourier Shell Correlation
Frequency (1/A)
Average
Average-Std.Deviation
Average+Std.Deviation
Figure 9. Plot of the values for 19 shells of the average over 30 actual data sets (±1 standard
deviation) of the FSC for IDR after six iterations less the FSC for phase flipping.
reconstruction from electron microscopic data can be significantly improved by iterative data
refinement.
Acknowledgments
Partial support is acknowledged to the Comisi
´
on Interministerial de Ciencia y Tecnología
of Spain through projects BIO98-0761 and BIO2001-1237 and to National Institutes of
Health through grant HL70472. The work of Y Censor was done in part at the Center
for Computational Mathematics and Scientific Computation (CCMSC) at the University of
Haifa and supported by Research Grant 592/00 from the Israel Science Foundation founded
by the Israel Academy of Sciences and Humanities.
The authors would like to thank DrJJFern
´
andez for fruitful discussions and advice on
the manuscript.
References
B
´
arcena M, Donate L, Ruiz T, Dixon N, Radermacher M and Carazo J M 2001 The Dnab-Dnac complex: a structure
based on interactions among assymetric dimers EMBO J. 20 1462–8
Berman H, Westbrook J, Feng Z, Gilliland G, Bhat T, Weissig H, Shindyalov I and Bourne P 2000 The protein data
bank Nucleic Acids Res. 28 235–42
B
¨
ohmer K, Hemker P and Stetter H 1984 The defect correction approach Comput. Suppl. 5 1–32
Censor Y, Elfving T and Herman G T 1985 A method of iterative data refinement and its applications Math. Methods
Appl. Sci. 7 108–23
Censor Y and Zenios S A 1997 Parallel Optimization: Theory, Algorithms, and Applications (New York: Oxford
University Press)
Eggermont P P B, Herman G T and Lent A 1981 Iterative algorithms for large partitioned linear systems with
applications to image reconstruction Linear Algebr. Appl. 40 37–67
Frank J 1996 Three Dimensional Electron Microscopy of Macromolecular Assemblies (San Diego, CA: Academic)
522 C O S Sorzano et al
Frank J and Penczek P 1995 On the correction of the contrast transfer function in biological electron microscopy
Optik 98 125–9
Furuie S S, Herman G T, Narayan T K, Kinahan P E, Karp J S, Lewitt R M and Matej S 1994 A methodology for
testing for statistically significant differences between fully 3D PET reconstruction algorithms Phys. Med. Biol.
39 341–54
Grigorieff N 1998 Three-dimensional structure of bovine NADH: Ubiquinone oxidoreductase (complex I) at 22
˚
Ain
ice J. Mol. Biol. 277 1033–46
Henderson R, Baldwin J M, Ceska T A, Zemlin F, Beckmann E and Downing K H 1990 Model for the structure of
bacteriorhodopsin based on high-resolution electron cryo-microscopy J. Mol. Biol. 213 899–929
Herman G T 1980 Image Reconstruction from Projections: The Fundamentals of Computerized Tomography (New
York: Academic)
Herman G T 1989 Standard image recovery methods in the iterative data refinement framework Proc. SPIE 1199
1501–11
Herman G T and Ro D 1990 Image recovery using iterative data refinement with relaxation Opt. Eng. 29 513–23
Lewitt R M 1990 Multidimensional digital image representations using generalized Kaiser–Bessel window functions
J. Opt. Soc. Am. A 7 1834–46
Lewitt R M 1992 Alternatives to voxels for image representation in iterative reconstruction algorithms Phys. Med.
Biol. 37 705–16
Losada M and Navarro R 1998 Point spread function of the human eye obtained by a dual double-pass method Pure
Appl. Opt. 7 L7–13
Ludtke S J, Baldwin P R and Chiu W 1999 EMAN: semiautomated software for high-resolution single-particle
reconstructions J. Struct. Biol. 128 82–97
Ludtke S J, Jakana J, Song J, Chuang D T and Chiu W 2001 A 11.5
˚
A single particle reconstruction of GroEL using
EMAN J. Mol. Biol. 314 253–62
Marabini R, Herman G T and Carazo J M 1998 3D reconstruction in electron microscopy using ART with smooth
spherically symmetric volume elements (blobs) Ultramicroscopy 72 53–65
Marabini R, Rietzel E, Schr
¨
oder R, Herman G T and Carazo J M 1997 Three-dimensional reconstruction from reduced
sets of very noisy images acquired following a single-axis tilt schema: application of a new three-dimensional
reconstruction algorithm and objective comparison with weighted backprojection J. Struct. Biol. 120 363–71
Matej S and Lewitt R M 1995 Efficient 3D grids for image reconstruction using spherically-symmetric volume
elements IEEE Trans. Nucl. Sci. 42 1361–70
Matej S and Lewitt R M 1996 Practical considerations for 3-D image reconstruction using spherically symmetric
volume elements IEEE Trans. Med. Imaging 15 68–78
Natterer F and W
¨
ubbeling F 2001 Mathematical Methods in Image Reconstruction (Philadelphia: SIAM)
Ro D W, Herman G T and Joseph P M 1989 Resolution enhancement of magnetic resonance images using an iterative
data refinement technique Proc. SPIE 1199 952–62
Skoglund U, Ofverstedt L G, Burnett R and Bricogne G 1996 Maximum-entropy three-dimensional reconstruction
with deconvolution of the contrast transfer function: a test application with adenovirus J. Struct. Biol. 117
173–88
Sorzano C O S 2002 Algoritmos iterativos de reconstrucci
´
on tridimensional en microscop
´
ıa electr
´
onica de transmisi
´
on
PhD Thesis E.T.S.I. Telecomunicaci
´
on (Univ. Polit
´
ecnica de Madrid)
Sorzano C O S, Marabini R, Boisset N, Rietzel E, Schr
¨
oder R, Herman G T and Carazo J M 2001 The effect of
overabundant projection directions on 3D reconstruction algorithms J. Struct. Biol. 133 108–18
Stark H, Orlova E V, Rinke-Appel J, Junke N, M
¨
uller F, Rodnina M, Wintermeyer W, Brimacombe R and
van Heel M 1997 Arrangement of tRNAs in pre- and posttranslocational ribosomes revealed by electron
cryomicroscopy Cell 88 19–29
Stetter H 1978 The defect correction principle and discretization methods Numer. Math. 29 425–43
Ve l
´
azquez-Muriel J A, Sorzano C O S, Fern
´
andez J J and Carazo J M 2003 A method for estimating the CTF in
electron microscopy based on ARMA models and parameter adjusting Ultramicroscopy 96 17–35
Zhou Z H, Hardt S, Wang B, Sherman M B, Jakana J and Chiu W 1996 CTF determination of images of ice-embedded
single particles using a graphics interface J. Struct. Biol. 116 216–22
Zhu J, Penczek P A, Schr
¨
oder R and Frank J 1997 Three-dimensional reconstruction with contrast transfer function
correction from energy-filtered cryoelectron micrographs: procedure and application to the 70S Escherichia
coli ribosome J. Struct. Biol. 118 197–219
Zubelli J P, Marabini R, Sorzano C O S and Herman G T 2003 Three-dimensional reconstruction by Chahine’s method
from electron microscopic projections corrupted by instrumental aberrationsInverse Problems 19 933–49
... Another versatile component is PickerView, implemented in emvis, which is instantiated by the em-viewer program to display the results of particle picking. The underlying PickingModel allows the easy support of different output formats from many programs such as Xmipp (Sorzano et al., 2004;Scheres et al., 2008;de la Rosa-Trevín et al., 2013), RELION (Scheres, 2012;Kimanius et al., 2016;Zivanov et al., 2018), Scipion (de la Rosa-Trevín et al., 2016), EMAN (Tang et al., 2007), crYOLO (Wagner et al., 2019) and Topaz (Bepler et al., 2019). It also facilitates the addition of new programs by providing a minimal amount of code to parse from a specific format. ...
... (Sorzano et al., 2004;Scheres et al., 2008;de la Rosa-Trevín et al., 2013), EMAN2(Tang et al., 2007), Bsoft(Heymann & Belnap, 2007) and RELION(Scheres, 2012;Kimanius et al., 2016;Zivanov et al., ...
Article
Full-text available
Image-processing software has always been an integral part of structure determination by cryogenic electron microscopy (cryo-EM). Recent advances in hardware and software are recognized as one of the key factors in the so-called cryo-EM resolution revolution. Increasing computational power has opened many possibilities to consider more demanding algorithms, which in turn allow more complex biological problems to be tackled. Moreover, data processing has become more accessible to many experimental groups, with computations that used to last for many days at supercomputing facilities now being performed in hours on personal workstations. All of these advances, together with the rapid expansion of the community, continue to pose challenges and new demands on the software-development side. In this article, the development of emcore and emvis , two basic software libraries for image manipulation and data visualization in cryo-EM, is presented. The main goal is to provide basic functionality organized in modular components that other developers can reuse to implement new algorithms or build graphical applications. An additional aim is to showcase the importance of following established practices in software engineering, with the hope that this could be a first step towards a more standardized way of developing and distributing software in the field.
... Due to it being theoretically optimal, the problem of full CTF correction is frequently addressed in the community of 3DEM methods research (e.g. [11,[24][25][26][27][28][29]). ...
... Other ways of doing a full CTF correction have been tried as well, such as the iterative method given by Penczek et al. [32], the iterative data refinement (IDR) technique [25], and the Chahine's method [27]. Differing in important details, these methods all attempt finding an approximation of the original image by iterative refinement or minimization of a residual function. ...
Article
The typical resolution of three-dimensional reconstruction by cryo-EM single particle analysis is now being pushed up to and beyond the nanometer scale. Correction of the contrast transfer function (CTF) of electron microscopic images is essential for achieving such a high resolution. Various correction methods exist and are employed in popular reconstruction software packages. Here, we present a novel approximation method that corrects the amplitude modulation introduced by the contrast transfer function by convoluting the images with a piecewise continuous function. Our new approach can easily be implemented and incorporated into other packages. The implemented method yielded higher resolution reconstructions with data sets from both highly symmetric and asymmetric structures. It is an efficient alternative correction method that allows quick convergence of the 3D reconstruction and has a high tolerance for noisy images, thus easing a bottleneck in practical reconstruction of macromolecules.
... Matej and Lewitt [5,6] provided a careful investigation of how the blob basis functions should be chosen when they are used in the context of 3D image reconstruction. Since then blobs have been used extensively for 3D image reconstruction in x-ray computed tomography [7], positron emission tomography [8,9], single photon emission computed tomography [10][11][12], and electron microscopy [13][14][15][16]. ...
Article
Full-text available
A technique for optimizing parameters for image representation using blob basis functions is presented and demonstrated. The exact choice of the basis functions significantly influences the quality of the image representation. It has been previously established that using spherically symmetric volume elements (blobs) as basis functions, instead of the more traditional voxels, yields superior representations of real objects, provided that the parameters that occur in the definition of the family of blobs are appropriately tuned. The technique presented in this paper makes use of an extra degree of freedom, which has been previously ignored, in the blob parameter space. The efficacy of the resulting parameters is illustrated.
... Matej and Lewitt [11,12] provided a careful investigation of how the blob basis functions should be chosen when they are used in the context of image reconstruction from projections. Since then blobs have been used extensively for image reconstruction in X-ray computerized tomography [13], positron emission tomography [14][15][16][17], single photon emission computerized tomography [18][19][20], optoacoustic tomography [21] and electron microscopy [22][23][24][25][26]. ...
Article
Full-text available
The series expansion approaches to image reconstruction from projections assume that the object to be reconstructed can be represented as a linear combination of fixed basis functions and the task of the reconstruction algorithm is to estimate the coefficients in such a linear combination based on the measured projection data. It is demonstrated that using spherically symmetric basis functions (blobs), instead of ones based on the more traditional pixels, yields superior reconstructions of medically relevant objects. The demonstration uses simulated computerized tomography projection data of head cross-sections and the series expansion method ART for the reconstruction. In addition to showing the results of one anecdotal example, the relative efficacy of using pixel and blob basis functions in image reconstruction from projections is also evaluated using a statistical hypothesis testing based task oriented comparison methodology. The superiority of the efficacy of blob basis functions over that of pixel basis function is found to be statistically significant.
... The IPR method can be mathematically identified (with appropriate nonessential changes of notations dictated by the problem at hand here) with the Iterative Data Refinement (IDR) method of [5], see also [ [20], and in positron emission tomography (PET) by Crepaldi and De Pierro [8]. Comparing IDR with the IPR outline proposed here the necessary adjustments to the IDR which yield the IPR and the identification of quantities can be observed as follows. ...
Article
We formulate the method of iterative prescription refinement for inverse planning in any fully discretized model of radiation therapy. The method starts out from an ideal dose prescription and repeatedly refines it into a refined dose prescription. This is done computationally without human interaction until a prespecified stopping rule is met, at which point the refined dose vector and the accompanying beamlet intensities vector are evaluated and presented to the planner. The algorithmic regime is general enough to encompass various physical models that may use different particles (photons, protons, etc.) It is formulated for a general inversion operator thus different objective functions or approaches to the optimization problem (such as DVH, gEUD, or TCP and NTCP cost functions) may all be applied. Although not limited to this model, we demonstrate that the approach at all works on two exemplary cases from photon intensity-modulated radiation therapy.
... However, it is not always the case that there are just two groups with opposing tendencies. For instance, the method of iterative data refinement (IDR) [40] has been recently applied in electron microscopy to correct for microscope aberrations [41,42]. That algorithm also has a relaxation factor to be optimized. ...
Article
Many algorithms require the tuning of parameters in order to achieve optimal performance. Usually the best values of these parameters depend on both the particular conditions under which the experimental data have been acquired and the kind of information that we aim to obtain. The performance of an algorithm can be measured by means of numerical observers called Figures of Merit (FOMs). Usually there are no analytical formulas expressing the dependence of the FOMs on the parameters, but the nature of such dependence can be observed by the use of computational experiments. This article proposes a methodology for assigning values to the algorithmic parameters in the presence of a high number of FOMs. A multiobjective optimization framework is provided that identifies a set of optimal parameter values whose performance, from several points of view based on the initial FOMs, is statistically indistinguishable. This methodology is illustrated by applying it to the three-dimensional reconstruction (using an algebraic reconstruction technique) of single particles in electron microscopy.
Article
Fourier Shell Correlation, Spectral Signal-to-Noise Ratio, Fourier Neighbour Correlation, and Differential Phase Residual are different measures that have been proposed over time to determine the spatial resolution achieved by a certain 3D reconstruction. Estimates of B-factors to describe the reduction in signal-to-noise ratio with increasing resolution is also a useful parameter. All these concepts are interrelated and different thresholds have been given for each one of them. However, the problem of resolution assessment in 3DEM is still far from settled and preferences are normally adopted in order to choose the "correct" threshold. In this paper we review the different concepts, their theoretical foundations and the derivation of their statistical distributions (the basis for establishing sensible thresholds). We provide theoretical justifications for some common practices in the field for which a formal justification was missing. We also analyze the relationship between SSNR and B-factors, the electron dose needed for achieving a given contrast and resolution, the number of images required, etc. Finally, we review the consequences for the number of particles needed to achieve a certain resolution and how to analyze the Signal-to-Noise Ratio for a sequence of imaging operations.
Chapter
We discuss and illustrate defocus-gradient and attenuation effects that are part of the image formation models of microscopy of biological specimens. We demonstrate how they affect the projection data and in turn the 3D reconstructions. Biologically meaningful results can be obtained ignoring both of these effects, but using image processing techniques to incorporate corrections for them into reconstruction methods provides more accurate reconstructions, with potential for creating higher-resolution models of the biological specimens.
Article
The Protein Data Bank (PDB; http://www.rcsb.org/pdb/ ) is the single worldwide archive of structural data of biological macromolecules. This paper describes the goals of the PDB, the systems in place for data deposition and access, how to obtain further information, and near-term plans for the future development of the resource.
Article
We present an experimental method to obtain the point spread function (PSF) of human eyes, based on two techniques: the conventional symmetric double-pass method and the recently introduced asymmetric one-and-a-half-pass method. The symmetric double-pass method provides the autocorrelation of the retinal PSF and thus, the modulation transfer function (MTF). Here, we show that the double-pass MTF can be combined with partial phase information provided by the one-and-a-half-pass method to estimate the PSF. Two reconstruction methods were used, depending on the relative contribution of diffraction and aberrations. When aberrations are dominant (large pupils), the PSF can be obtained directly by combining the modulus and phase. When aberrations are small (small to medium pupils), however, a partial phase-retrieval algorithm has to be applied. In agreement with previous findings, the PSFs show substantial intersubject variability and for large pupils, they display the characteristic ray-pattern of star images.
Article
This is an introductory survey of the defect correction approach which may serve as a unifying frame of reference for the subsequent papers on special subjects.
Article
Iterative Data Refinement (IDR) is a general procedure for estimating data that would have been collected by an ideal measuring device from data that were collected by an actual measuring device. An example is in Computerized Tomography (CT), where we have a mathematical procedure to reconstruct the x-ray attenuation coefficient at individual points inside the human body from (the ideal) data obtained by passing monoenergetic x-rays through the body and measuring the percentage of energy that gets through. Unfortunately, x-ray tubes deliver polyenergetic x-rays and the actual measurements only approximate what is assumed by the mathematics of CT, resulting in images whose quality is noticeably worse that those reconstructed from ideal data. This is one of the applications where the efficacy of IDR has been demonstrated: it can be used to estimate data that would be obtained from the ideal monoenergetic x-ray tube from the data that are obtained from the actual polyenergetic x-ray tube. In fact, IDR is general enough to encompass such well-accepted image recovery methods as the Gerchberg-Saxton algorithm and the error reduction and hybrid input-output methods of Fienup. The generalizations provided by IDR give new insights into the nature of such algorithms and, in particular, allow us to introduce the notion of relaxation into them, resulting in many cases in a much improved computational behavior.
Article
Iterative data refinement (IDR) is a general procedure for estimating data that would have been collected by an ideal measuring device from data that were collected by an actual measuring device. We show that IDR is general enough to encompass such well-accepted image recovery methods as the Gerchberg-Saxton algorithm and the error reduction and hybrid input-output methods of Fienup. The generalizations provided by IDR give new insights into the nature of such algorithms and, in particular, allow us to introduce the notion of relaxation into them. Along similar lines, IDR provides a common framework within which new algorithms can be developed for improved magnetic resonance imaging (MRI). We apply the approach of IDR to a specific problem in MRI, namely, to the correction of spatially dependent blurs due to transverse relaxation.
Article
Scitation is the online home of leading journals and conference proceedings from AIP Publishing and AIP Member Societies
Article
We present a unifying framework for a wide class of iterative methods in numerical linear algebra. In particular, the class of algorithms contains Kaczmarz's and Richardson's methods for the regularized weighted least squares problem with weighted norm. The convergence theory for this class of algorithms yields as corollaries the usual convergence conditions for Kaczmarz's and Richardson's methods. The algorithms in the class may be characterized as being group-iterative, and incorporate relaxation matrices, as opposed to a single relaxation parameter. We show that some well-known iterative methods of image reconstruction fall into the class of algorithms under consideration, and are thus covered by the convergence theory. We also describe a novel application to truly three-dimensional image reconstruction.
Article
Iterative Data Refinement (abbreviated IDR) is a general procedure which encompasses many special procedures for image reconstruction and for related problems. It is a procedure for estimating data that would have been collected by an idealized measuring device from data that were collected by an actual measuring device. Such approaches have been applied successfully in areas of reconstruction in x-ray tomographic radiology. In fact, IDR is general enough to encompass standard approaches to data recovery, such as the Error-Reduction and the Hybrid Input-Output methods. Along similar lines, IDR provides a common framework within which new algorithms can be developed for improved magnetic resonance imaging (MRI). We have applied and implemented the approach of IDR to a specific problem in MRI, namely to the correction of spatially-dependent blurs due to short local transverse relaxation (T2) values. The algorithm is designed to reconstruct T2-weighted spin density images with improved spatial resolution. The practical computational significance of using the IDR approach will be illustrated by the reconstruction of mathematical phantoms. We have found that over-relaxation of the algorithm improves computational speed by up to a factor of five.