Content uploaded by Fulvio Saccoccia
Author content
All content in this area was uploaded by Fulvio Saccoccia
Content may be subject to copyright.
2
What we will see
●Quick overview (slide 3-6)
●The XDS.INP file (slide 7-8)
●Running XDS (slide 10-20)
●Optimization (slide 21-31)
3
The XDS package
●XDS: main program
indexing, integrating
●XSCALE: scaling
program to merge
several data sets
●(1) XDS-Viewer
to visualize single frames
(2) XDSSTAT
to generate plot
●XDSCONV:
conversion from .HKL
to ccp4, shelx, cns
format
4
XDS
●Integration program for
monochromatic X-ray
data
●Single frames
(compressed)
●3-dimensional
integration (different
from DENZO and
mosflm)
●Fine-tuned detector parameter
(readout noise, blank regions, etc.)
●Correction for radiation damage
●Optimised for “fine-sliced” data
(Pilatus - be happy!)
●Command-line program (do not be
afraid!)
●Multi-CPU usage: fast
●Lot of documentation available
●27 years of development and
improvements
5
Availability
●XDS (main program, XSCALE, XDSCONV)
http://xds.mpinf-heidelberg.mpg.de/
●Xds-viewer http://xds-viewer.sourceforge.net
●XDSSTAT: ftp://turn5.biologie.uni-konstanz.de/pub/xdsstat.bz2
●Tips and tricks (a lot of stuffs, example, FAQ, tutorial...)
http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Main_Page/
●Google XDS + some other words: you will find something about data raduction!
●This presentation of course
Useful Documentation
6
Codes for this presentation
●Green
File
Program
●Yellow
XDS subroutine
steps
●Red
To be remind
●UPPER CASE
KEYWORDS_=VALUE
7
All you need is the XDS.INP file!
Templates: http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html
Generated by your frames:
http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Generate_XDS.INP
●XYCORR: positional corrections of the
detector
●INIT: background
●COLSPOT: locates strong spots to use in
indexing (IDXREF) step
●IDXREF: indexing (unit cell dimensions
and crystal orientation)
●DEFPIX: masks blank regions of detector
(Pilatus is a challenge for mosflm users)
●XPLAN: strategy - optional
●INTEGRATE: determines intensity of
reflections
●CORRECT: applies correction, scaling,
data analysis and statistics
The JOB routine tells XDS the step to do
8
.LP and .cbf files are generated
by each step during XDS run
●.LP contains useful information
about the step – is the .log file
●.cbf can be visualized by
xds-viewer – is the .img file
IDXREF.LP: output from indexing step. Here you can choose the symmetry and cell
constants. Otherwise, XDS will choose by itself in the CORRECT (see slide 12)
CORRECT.LP: output of scaling. Check your data for completeness, Rs, CC1/2 ...
http://xds.mpimf-heidelberg.mpg.de/html_doc/xds
_files.html
Kabsch W. XDS. Acta Crystallogr D Biol Crystallogr. 2010, 66:125-32
Free online http://journals.iucr.org/d/issues/2010/02/00/dz5179/dz5179.pdf
Every step requires the file generated in the
previous step
You can run IDXREF only after
1) XYCORR
2) INIT
3) COLSPOT;
You can run CORRECT
only after all previous steps
9
Most often the only files you have to
see/check
●IDXREF.LP – indexing; cell assignement and
symmetry
●CORRECT.LP – statistics from scaling; correctness
for autoassignment parameters
●FRAME.cbf – goodness of predictions of spot
position and integration
10
Running XDS
11
A simple run 1/3
Edit your XDS.INP
●JOB=XYCORR INIT COLSPOT IDXREF
●Choose the
DATA_RANGE (how many frames to integrate?)
SPOT_RANGE (how many frames to index?)
BACKGROUND_RANGE (how many frames for
background evaluation?)
●OSCILLATION_RANGE (∆φ)
●X-RAY_WAVELENGTH
●NAME_OF_TEMPLATE (file locations with
_????.img – beware with lenght!)
●DETECTOR_DISTANCE
●check ORGX, ORGY (usually correct)
Default params: can be fine-tuned
12
A simple run 2/3
●Integration step
!SPACE_GROUP
!UNIT_CELL_CONSTANTS
RESOLUTION_SHELLS
FRIEDEL'S_LAW
REFINE
MAX_NUM_OF_PROCs
If SG and cell constants are not specified (commented - !),
INTEGRATE step will be performed always in P1 in the first run.
Then CORRECT will choose best SG that gives best scaling.
Usually, XDS requires at least 2 runs of INTEGRATE and CORRECT
to achieve best statistics and data reduction.
Optimization or fine-tuning
13
Now, inspect IDXREF.LP...
14
A simple run 3/3
●Now run XDS.INP with JOB= DEFPIX
INTEGRATE CORRECT
●Integration in batch of 5 frames (3D profile fitting
- see DELPHI – slide 27)
●Mosaicity estimates
●Refinement parameters
●Look at :
1) STANDARD DEVIATION OF SPOT
POSITION (PIXELS) it should be about 1.0 or
less
2) STANDARD DEVIATION OF SPINDLE
POSITION (DEGREES) it depends on ∆φ; it
should be 0.1 to 0.5.
●Averaged mosaicity used in integration
●Look at distance!
●No setting for SG: integration in P1
15
INTEGRATE and CORRECT
●INTEGRATE
–determines spot intensities based on
parameters saved in XPARM.XDS
–write INTEGRATE.LP with spot profile
fitting, refined parameters, mosaicity,
divergence...
●CORRECT
- experimental corrections (e.g. Lorentz- and
Polarisation-correction), refine parameters
and determine space group (saved in
GXPARM.XDS, ready for second run)
- write FRAME.cbf: check correctness of
predictions
16
Now, inspect CORRECT.LP...
17
After INTEGRATE and CORRECT
●XDS_ASCII.HKL
h,k,l,I,σ(I)
●Conversion by:
(1) xdsconv – command line
XDSCONV.INP can be edit in order to
generate .mtz (I or F), .cv (CNS), .hkl (shelx)
(2) Aimless to .mtz – GUI (within ccp4i-6.3.0 –
upgrade your distribution!)
False for MAD/SAD
18
xdsconv
●Edit XDSCONV.INP
–Requires XDS_ASCII.HKL from CORRECT
–Self-explaining
–Choose among several formats and labels
assignement (useful for MAD/SAD)
●Edit to include
Wilson statistics
Rfree flag
Friedel pairs (FALSE for MAD/SAD)
SCALEPACK sigma-cutoff-like option
For ccp4, a
further conversion to .mtz
is required:
just follow the instructions
19
XDS and XDSCONV
MOSFLM XDS HKL2000
SCALA XDSCONV SCALEPACK
Pointless (ccp4)
Scalepack2mtz (ccp4)
SHELX CNS
(XDS_ASCII.HKL)
20
Xscale for SAD/MAD user
●Separate Bijvoet pairs before
CORRECT step
To scale and merge reflections from
peak, inflection and remote(s) use
xscale and its XSCALE.INP
and apply correction for decaying
With FRIEDEL'S_LAW=FALSE, run CORRECT twice,
with STRICT_ABSORPTION_CORRECTION=TRUE
and STRICT_ABSORPTION_CORRECTION=FALSE,
and compare the results.
21
XDSSTAT – a simple data plotter
Run xdsstat > XDSSTAT.LP
It takes the XDS_ASCII.HKL
and generate a XDSSTAT.LP
that can be opened by
loggraph (ccp4) from terminal
If you are interested on a particular resolution range, simply type
xdsstat 20 3 > XDSSTAT_low.LP
22
Optimization
23
The finer things about XDS integration -
recycling
●Rename GXPARM.XDS in XPARM.XDS (params for lambda, distance, cell, symm...)
●Rename GX-CORRECTIONS.cbf and GY-CORRECTIONS.cbf in
X-CORRECTIONS.cbf and Y-CORRECTIONS.cbf (detector corrections)
●From INTEGRATE.LP, copy
in XDS.INP
●Edit XDS.INP: JOB=INTEGRATE CORRECT
●Do not refine nothing during INTEGRATE step (see slide 28)
●Re-run xds
This second run will impose the SG both in
INTEGRATE and CORRECT step
Mosaicity is kept costant during integration
but it can be override (see slide 32)
24
First and second run comparison – 1
25
First and second run comparison – 2
anomalous signal
SAD data kindly provided by Dr. Giovanna Avella
26
Some insights about IDXREF step
●INDEX_ORIGIN
can be changed if something goes wrong
–Distance issues
–Incident beam position issues
●SPOT_RANGE
–Use wide range
●REFINE(IDXREF)=ALL
●Otherwise, REFINE(IDXREF)=ORIENTATION
CELL AXIS
Kabsch W. XDS. Acta Crystallogr D Biol Crystallogr. 2010, 66:125-32
Free online http://journals.iucr.org/d/issues/2010/02/00/dz5179/dz5179.pdf
XDS output files are friendly!
They are self-explaining and frequently
give useful informations for most common
issues; read carefully at least once...
Check for STANDARD DEVIATION OF SPOT POSITION (PIXELS)
STANDARD DEVIATION OF SPINDLE POSITION (PIXELS)
27
Fine tuning 1/2 – XDS.INP
●MINIMUM_NUMBER_OF_PIXELS_IN_A_
SPOT (def 6)
- decrease if weak diffraction
- increase if diffraction is noisy
- increase if multiple lattice
- probably to decrease for Pilatus (e.g. 3 - try it!)
●STRONG_PIXEL
A 'strong' pixel to be included in a spot must
exceed the background by more than the
given multiple of standard deviations
- increase if twinning is present (multiple
lattices)
●DELPHI
If there are too few strong spots which
could be used for learning spot profiles, it
may be useful to specify a larger value
for DELPHI=
–Usually, mosaicity increases
Documentation
http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_parameters.html
28
Fine tuning 2/3 – XDS.INP
●REFINE(IDXREF)= DISTANCE BEAM ORIENTATION CELL AXIS
Usually are OK, but only if COLSPOT has seen all (a significant fraction of the) frames. If only a small
SPOT_RANGE was used, use REFINE(IDXREF)= AXIS BEAM ORIENTATION CELL.
●REFINE(INTEGRATE)= BEAM ORIENTATION AXIS CELL !DISTANCE
Usually OK in the first run. In the second run, do not refine nothing: REFINE(INTEGRATE)= ! If only a small
SPOT_RANGE was used, try also REFINE(INTEGRATE)= BEAM ORIENTATION CELL AXIS
●REFINE(CORRECT)=DISTANCE BEAM ORIENTATION CELL AXIS
Usually is OK. For low resolution datasets the distance refinement may not be stable (i.e. refined distance differs more
than a few mm from expected distance). In this case one should remove DISTANCE from the list.
29
Fine tuning 3/3 – parameters correctness
●Labelit.index *.img --index_only (within Phenix suite –
command line)
the most reliable method to determine beam position;
–it can generate XPARM.XDS!
If you know beam position from other sources
(DENZO, mosflm, .img header) try
x(beam)/QX and y(beam)/QY
Adxv http://www.scripps.edu/~arvai/adxv.html
Header file information (distance, λ, beam center)
Able to visualize *.cbf file
Edit ORGX ORGY with new values
30
Most common errors
reported after IDXREF
“!!! ERROR !!! SOLUTION
IS INACCURATE”
1) wrong ORGX, ORGY
2) Poor data quality
3) twinning (satellite spot)
a1) check beam position
a2-3) decrease
MINIMUM_NUMBER_OF_P
IXELS_IN_A_SPOT
a2) increase SPOT_RANGE
“!!! ERROR !!! INSUFFICIENT
PERCENTAGE (< 70%) OF
INDEXED REFLECTIONS”
do not be afraid: less than 70% of
strong pixels indexed (XDS is
almost too careful!)
Usually XDS.INP can be re-run
with JOB=DEFPIX
INTEGRATE CORRECT
without further complications
otherwise increase number of
SPOT_RANGE
31
Checking your FRAME.cbf – 1/2
●Off-centered profiles
–Mis-indexing
–Crystal slippage
–Change in beam direction
see slide 26 to 29
●The uncircled spots are not close to the
spindle, or
●The uncircled spots do not belong to ice rings
Parameters in XPARM.XDS are wrong
(mis-indexing? See slide 26))
Estimated mosaicity too high/low: try
varying DELPHI or override automatic
assignement (see slide 23)
The last data image written by INTEGRATE
To visualize use xds-viewer
32
Checking your FRAME.cbf – 2/2
●Profiles extending to the borders
●Incorrect estimate of spot profile parameters
–Based on Gaussian profile of peak (estimated in COLSPOT)
–Appropriate values are determined automatically
–REFLECTING_RANGE δM
–BEAM_DIVERGENCE δM
–REFLECTING_RANGE_E.S.D σM
–BEAM_DIVERGENCE_E.S.D σD
–δM δD > 6-10 times σM σD
Override automatic assignment of mosaicity (usually divergence refinement
works fine and not need to be changed)
–See slide 23
33
Resolution cut/off
●XDS uses profile fitting (similar
to DENZO, mosflm):
1) calculate an expected spot
shape from STRONG spots
2) scale the shape at the
calculated spot position to the
pixel values
3) subtract the average
background (possibility of
negative intensities!)
●XDS collect 3D profiles from several
frames
●Divide frame into 3x3
●Fit strong spots within each square to
3D Gaussian curve
●Fit weak spots to this Gaussian
CONSEQUENCES
●Processing programs will integrate noise when the
resolution cut-off is set too high.
●Including the noise is going to negatively
influence the “real” data.
●Set the resolution limit during integration (after
first run) instead of e.g. Cutting the poor quality
part after integration.
From: http://shelx.uni-ac.gwdg.de/~tg/teaching/ggnb/A106/pdfs/A106-day1.pdf
34
Now you are ready to run XDS
Do not forget to copy in a different directory
all files you want to hold on
Otherwise XDS will overwrite them!
Rename *.LP files before a new run
Do not forget compress images (.bz2, .zip)
Do not forget to parallelize (xds_par, xscale_par)
35
Thank you for your attention
and for your interest
Enjoy with XDS
Fulvio