HKL2000 Tutorial

Author: Dr. Miroslaw Gilski

Purpose of this exercise.
In this exercise you will learn how to process raw diffraction images and how to go through data reduction to a set of intensities or structure factor modules

The HKL suite is a package of programs intended for the analysis of X-ray diffraction data collected from single crystals. It consists of three parts:
X-ray diffraction data analysis performed by the HKL package (Otwinowski,1993; Otwinowski & Minor, 1997) or similar programs, is used to obtain the following results Other results, such as indexing of the diffraction pattern, are in most cases only intermediate steps to achieve the above goals. The HKL system also has tools for validating the results by self-consistency checks.
The fundamental stages of data analysis are:
  1. visual inspection of the diffraction images;
  2. (auto)-indexing;
  3. refinement of diffraction geometry parameters;
  4. integration of the diffraction peaks;
  5. conversion of the data to a common scale;
  6. symmetry determination and merging of symmetry related reflections;
  7. statistical summary and estimation of errors.

    Recommended Reading

  1. International Tables for Crystallography Volume F: Macromolecular Crystallography
  2. Z. Otwinowski and W. Minor, " Processing of X-ray Diffraction Data Collected in Oscillation Mode ", Methods in Enzymology, Volume 276: Macromolecular Crystallography, part A, p.307-326, 1997,C.W. Carter, Jr. & R. M. Sweet, Eds., Academic Press (New York).
  3. HKL-2000 Online Manual

Your diffraction data

In this exercise you will be working with an X-ray diffraction data set collected for a single crystal of trypsin at DESY (Deutches Elektronen Synchrotron) in Hamburg by dr Szymon Krzywda. The data directory contains 113 raw diffraction images recorded to the maximum resolution of 2.15 Å. The wavelength of the X-ray radiation used was 0.81600 Å, and the detector was a MAR Research CCD device with 165 mm diameter. The experiments were carried out at beamline X11 belonging to the EMBL (European Molecular Biology Laboratory).
  1. How to launch the HKL2000 package ?

    To start the HKL2000 program type:
    # /home/nfs/xtal/hkl2000/hklbin/HKL2000
    In detector type window select the following detector: DESY-X11.
  2. "HKL Main" window - select the files with images you wish to integrate.

    • in the “Directory Tree” window highlight the directory with the images: /home/nfs/xtal/data/TR_tryg
      HKL2000 uses a special naming scheme for files containing diffraction images. Each image file name is constructed accordingto the following template: datasetname_###.ext, where datasetname is the name of our data set (usually project name), ### is a three-digit sequence number of the consecutive images and .ext defines detector type (e.g. mccd for MAR CCD, img for ADSC Quantum, etc.).
    • click on the “>>” button in the “New Raw Data Dir” field.
    • in the same way choose the output directory in “New Output Data Dir”
    • select the “Show All Files” button and click the “Load Data Sets” button.
    • Click on any image, the program will set the whole range of images from this dataset. In our case they range from 1 to 113. Click “Done”.

    • In the “Experiment Geometry”, “Frame Geometry” fields check the crystal-to-detector Distance, for this dataset 200 mm, as well as “Oscillation range”, “Frame Width”. This information is retrieved from the headers of the images. Also note the wavelength of the X-ray beam.

  3. Indexing of the data (to figure out the Laue class, Bravais lattice and cell dimensions)

    • Now you can go to the “Summary” tab page where you can inspect and edit the parameters for each data set.

    • After that navigate to the “Index” tab
    • Hit the “Display” button to see the first image and evaluate its quality.
    • An image appears in a new pop-up window. How does the image look? Are the spots distinct or smudged. Are they very close to each other or overlapped ? Can you see the shadow of the x-ray beam stop? The program for visualization of the diffraction pattern (XdisplayF) has a number of useful options

      You can try some of them, for example “Dim” and “Bright” change the image brightness. “Floor Up” and “Floor Down” change the displayed background level. Can you see the water ring ? Can you explain it?
    • Click on the "Peak Search” button to select the peaks/reflections you want the program to index. Red selections circles should appear. Is the number of selected peaks sufficient for indexing?
    • Now hit the "Index” button. The program will try to fit the reciprocal lattice defined by these reflections to a corresponding direct lattice ("Bravais Lattice Table" window). The result is a table of all possible Bravais lattices with a deformation (or embarrassment) indicator (in %) showing how much the theoretically required constraints on the lattice would be violated. For instance, if all the angles of the cryatal alttice were different from 90 deg, then it would be very difficult to fit it to an orthogonal system of coordinates (high % of distortion). Optimally, you should choose the highest symmetry lattice with a low distortion parameter. HKL2000 will suggest the unit cells that can index the diffraction pattern in the best way (green color) - the lowest percentage indicate the best fit of your data and lattice.
    • The HKL2000 program will always select the lowest symmetry space group for a particular system during "Bravais Lattice" selection process (P2 for monoclinic, P222 for orthorhombic, P3 for hexagonal system, etc.) - the correct space group can be selected during the Scaling process.
    • Select your lattice and click “Apply & Close”.
    • Now you need to refine your indexing parameters to adjust the detector and unit cell parameters to the data.
      Look at the “Refinement Options” field on the right. The basic subset of the parameters (selected by default) is a good way to start refinement.
    • Inspect the reflection predictions on the image window (in yellow).
    • Click on the “Refine” button.
      The refined parameters are visible in the upper right portion of the screen. In the xdisp window all reflections should be present in the yellow prediction circles. Click on the “Refine” button and check again.
    • Click on “Fit All” and then “Refine” - look at the “Refinement Information” field.
    • Click on the “Refine” button several times till the resulting numbers converge (no change in the last cycle).

    • Click on the “Zoom wind” button. Press the middle mouse button on the main window in the desired area to see a magnified view of the detector in the Zoom window. You can “Zoom in” and display the Integration box (“Int. Box”). This is the area within which the program will analyze the pixels in order to determine the size and intensity of each reflection. In the Zoom window you should be able to see the individual pixels of the detector, each with its own degree of blackness, proportional to the number of X-ray quanta accumulated during the exposure. At very high magnification, you can even read the numerical intensity values of each pixel. Zoom in a different area of the image and inspect the integration boxes. Now you are ready to integrate the data.

  4. Integration

    • In this process each reflection intensity is evaluated and integrated according to a spot profile and pixel density. In this process, the program analyzes each image to locate the reflections and determine their intensities. The reflection position (in 2D space), the intensity, and background will be stored in a working directory as a series of files with the extension .x
    • From the Index window, click on the “Integrate” button. During the integration calculations you should closely monitor the &chi² statistics of the x and y positions of the reflections, the cell parameters, the crystal orientation, and the distance parameter. The mosaicity of crystal is monitored in the lower-left histogram. This plots indicates the partiality of the reflections on an image, and (indicated by an arrow) the expected rotation (in degrees) required to record a full reflection.
    • The &chi² values are error estimate. The higher the &chi² number, the poorer the fit of the predictions to the real data on the image, and thus the worse the data quality.
    • In the display window (lower right window) you can see how the different parameters change as the crystal rotates from image to image during the exposure (large fluctuations may indicate experimental problems).
      Integrating the full dataset will require a couple of minutes.
  5. Scaling

    • After the integration step you will have in the working directory a series of .x files which define the intensities of the reflections found on each image. You now need to scale the data so that all the equivalent reflection can be merged together. Reflections will be merged not only from the same image (when they are symmetry-equivalent) but also between different images. When a reflection is not complete on a single image is it called a partial, and continues on a the following image. Sometimes, during the experiment the crystal can move slightly in the beam and may diffract more or less strongly as it rotates. Also, the X-ray beam intensity fluctuates with time. In addition, the diffraction power may decay over time due to radiation damage, or due to poor cryo-protection as ice rings develop. The scaling attempts to normalize these effects.
      Finally, the scaling averages all equivalent reflections to provide a unique set of data.
    • To start scaling go to the "Scale" tab (on the top "HKL2000" window) and click the "Scale Set" button. Next scroll up the window with scale results and look at the "Crystal - Global Refinement" and "Global Statistics". Note the percentage of reflections marked for rejection and the total linear R-factor - they will be very importanf during space group determination.
    • Select "Use rejections on next run" and hit the "Scale Set" button again.
    • Click on "Show Log File" (if you see a red warning window, you should go to the "Options" menu, choose "Editor" and select the "vi" editor). In a new editor window scroll to the end of the log file and locate the "Summary of reflection intensities and R-factors by shells" table. Look at the "Chi**2" column - all values in each resolution shell should ideally be equal to 1.0 (in practice from 0.9 to 1.1). If they are very different it is neccesary to "Adjust Error Model". In the pop-up window "Errors" adjust the Error Model for each zone (increase or decrease errors). Each time when you change the Error Model, close the log file window and hit the "Scale Set" button to run the Scalepack program again.
    • Space Group Determination: How to do this with Scalepack?

      Scalepack can be used to determine the space group of your crystal. What follows is a description of how you would continue from the lattice type given by Denzo to determine your space group. This whole analysis assumes that you have a crystal of an enantiomorphic compound, such as protein, where only proper symmetry (proper rotation) is possible. Since the decision-making process is based on comparison (merging) of potentially equivalen reflections, it will only work if you have enough data, i.e. if the redundancy of the data set is sufficiently high. You also need enough data for the analysis of systematic absences.
        To determine your space group, follow these steps:
      • Scale by the primary space group in Scalepack. The primary space groups are the first space groups in each Bravais lattice type in the table below. Note the χ2 statistics. Now try a higher symmetry space group (next down the list) and repeat the scaling, keeping everything else the same (remember to "Delete Reject File" and run Scalepack - "Scale Set" button - twice, every time you change the "Space Group")
        If the χ2 is about the same, then you know that this is OK, and you can continue. If the χ2 are much worse, then you know that this is the wrong space group, and the previous choice represented the correct Laue class.
        The exception is primitive hexagonal, where you should try P61 after failing P3121 and P3112.
      • Examine the bottom of the log file or simulated reciprocal lattice picture for the systematic absences. If this was the correct space group, all of these reflections should be absent and their values should be very small. Compare this list with the listing of reflection conditions by each of the candidate space groups. The set of absences seen in your data which corresponds to the absences characteristic of the listed space groups identifies your space group or a pair of enantiomorphic space groups (differing in screw axis handedness). Note that you cannot do any better than this (i.e. get the handedness of screw axes) without phase information.
      • If it turns out that your space group is orthorhombic and contains one or two screw axes, you may need to reindex the data to align the screw axes with the standard definition. If you have one screw axis, your space group should be P2221, with the screw axis along c. If you have two screw axes, then your space group is P21212, with the screw axes along a and b. If the Denzo indexing is not the same as these, then you should reindex the reflections using the Reindex button.

        Bravais Lattice

        Primary assigned Space Groups


        Reflection Conditions along screw axes

        Primitive Cubic


        195   P23




        198   P213




        207   P432




        208   P4232




        212   P4332




        213   P4132


        I Centered Cubic


        197   I23




        199   I213




        211   I432




        214   I4132


        F Centered Cubic


        196   F23




        209   F432




        210   F4132


        Primitive Rhombohedral


        146   R3




        155   R32


        Primitive Hexagonal


        143   P3




        144   P31




        145   P32




        149   P312




        151   P3112




        153   P3212




        150   P321




        152   P3121




        154   P3221




        168   P6




        169   P61




        170   P65




        171   P62




        172   P64




        173   P63




        177   P622




        178   P6122




        179   P6522




        180   P6222




        181   P6422




        182   P6322


        Primitive Tetragonal


         75   P4




         76   P41




         77   P42




         78   P43




         89   P422




         90   P4212




         91   P4122




         95   P4322




         93   P4222




         94   P42212




         92   P41212




         96   P43212


        I Centered Tetragonal


         79   I4




         80   I41




         97   I422




         98   I4122


        Primitive Orthorhombic


         16   P222




         17   P2221




         18   P21212




         19   P212121


        C Centered Orthorhombic


         20   C2221




         21   C222


        I Centered Orthorhombic


         23   I222




         24   I212121


        F Centered Orthorhombic


         22   F222


        Primitive Monoclinic


          3   P2




          4   P21


        C Centered Monoclinic


          5   C2


        Primitive Triclinic


          1   P1


    • The mathematical fit between symmetry-related reflections is indicated by the Rmerge residual of the data. A well behaved dataset will have a low Rmerge (below 10%).
    • When you scroll the scalepack output window from top to bottom, you will have the following graphs:
      • Scale and B vs Frame:
        The scale factor was allowed to vary during this run and you can see how the scale factor changed during of data collection.
      • Completeness vs Resolution:
        The completeness is plotted versus intensity of the reflections. Strong reflections in yellow and all reflections in blue.
      • χ2 vs Frame or vs Resolution:
        We are watching the χ2 values to determine the quality of the error estimate in the data. We would like our average χ2 value to equal 1.00 and the Rfactor to be as small as possible (lower than 8-10%).
      • I/sigma vs resolution:
        Due to the scattering of an atom we can see how the reflection intensity decreases in higher resolutions.
      • Low resolution vs completeness:
        It is important to have both high and low resolution data complete.

      The "Scaling window" also allows access to various parameters (number of zones, Error scale factor, etc.) and various scaling options (B restrain, Anomalous, etc.
      These options are fully documented in the HKL2000 manual.