EQSANS Data Reduction With drtsans

Quick Reference for script mode

Editors: Changwoo Do, Gergely Nagy, William Heller

Date: 07/16/2020

Introduction

EQ-SANS data reduction is typically performed through two equivalent means:

The present document details the former method. Your local contact will work with you to configure the reduction for the various instrument configurations that you use during your experiment.

The content of the document is also explained in a [video tutorial](<https://youtu.be/1XU_40Z8RuY).

Below find the step by step description of the procedure.

  1. Visit analysis.sns.gov

    a. Data reduction can be performed both from inside and outside of ORNL

    b. Log in with your ucams ID and password

    c. The same credentials can be used for easy file transfer to personal computer via e.g. WinSCP

  2. Populate the shared directory of your IPTS with the required reduction tools and activate the reduction environment

    a. open a terminal

    b. go to the shared folder of your IPTS

    cd /SNS/EQSANS/IPTS-****/shared

    c. copy eqsans_setup.sh from /SNS/EQSANS/shared/usertools/

    cp /SNS/EQSANS/shared/usertools/* ./

    Alternatively, one can directly run the shell script by typing /SNS/EQSANS/shared/usertools/eqsans_setup.sh and click enter.

    d. run eqsans_setup.sh by typing

    ./eqsans_setup.sh

    e. run eqsans_activatedrtsans.sh by typing

    source ./eqsans_activatedrtsans.sh

  3. Edit the template reduction script, which is written in python 3

    a. Use gedit or pluma to edit reduce_template.py

    gedit reduce_template.py & or pluma reduce_template.py &

    b. The script contains examples for:

  4. Example of writing and running a script for a single configuration reduction

    a. Update output directory

    It is recommended for the target directory for the reduced files to be empty or to not to contain reduction results from previous reductions in EQSANS_runnumber_*.nxs format because drtsans will over-write previous results without asking the user if they wish to do so.

    b. Update common reduction parameters

    c. Update run numbers and names for individual scattering and transmission runs

    d. to look up run numbers from your experiment please refer to

    e. Below is a complete example of how to specify data reduction for a standard sample. Note that the loop could be used to reduce data from multiple samples if the samscatt, samtrans and fn_list arrays contained more than one element. If used to reduce multiple data sets, these three arrays must have the same number of elements.

    f. comment all unnecessary sections of the file by using either a # to remove a single line or a pair of ''' at the start and end (i.e. enclosing) of sections that you wish to comment out and save the document.

    This is advised to prevent time consuming re-run of already completed reduction runs

    g. Save the file and run the reduction script from the terminal. For the example template file provided, type the following.

    python reduce_template.py

    Depending on the size of the data files, which can be quite large for strongly scattering samples, each data set may require several minutes to reduce because the instrument saves data in "event mode", which results in large files. If you are reducing a large number of files, such as after an overnight script, you may want to get a cup of coffee.

    h. Reduced files can be found in the output folder in a variety of formats.

  5. Stitching of individual runs

    a. The template script also contains a section entitled "stitching" that allows data sets collected in different instrument configurations for a sample to be stitched into a single intensity profile. While the example presents stitching of two data sets, it is possible to stitch data from several different configurations together. Then, the overlap array takes the form overlap = [MergeAB_min, MergeAB_max, MergeBC_min, MergBC_max, ...], having 2·(Num_Configurations -1) pairs of minimum and maximum Q-values for the merge regions.

    b. Update the section with names of files to be stitched, with output name, stitching range etc.

    c. Unless another name is provided for the merged_fn variable in the save_iqmod command, the stitched file will have an ending of _merged.dat. In the example above, the output file for the stitched data is "porasil_merged.dat".

  6. Reducing and stitching two configurations in one go

    a. Example to be found under "# example of one-step reduction"

    Comment out all sections not intended to be used

    b. Define necessary parameters. In the example below, there are two different configurations: low Q and high Q. Note that the two configurations have their own variables, which makes it possible to reduce the data from two configurations with a single script.

    c. Provide scattering and transmission run numbers for the samples and the overlap range for stitching

    d. Run the reduction script and observe the results as describe above in section 4.

  7. Data reduction with added flexibility

    a. The template file provided serves as a reduction example for simple cases. The data reduction software is highly configurable through the various reduction parameters that can be seen in the json files created during script execution. Consult with an instrument scientist about their usage. Data reduction with more specialized needs can be achieved through modifying json parameters. A separate document describes the various parameters.

    b. The json files can be directly edited with gedit or pluma, but can be viewed easily with the prettyjson.sh script that is created during the steps described in section 2.

    From the output subdirectory within the shared directory the script can be run as follows:

    ./prettyjson.sh <name_of_jsonfile>

    c. Modify the selected json parameters by adding new lines to the reduction script. In the example below, a different sensitivity file is specified on the last line.

    d. All parameter names are to be written in lowercase letters.

Disclaimer

The data reduction scripting method presented here was developed by Dr. Changwoo Do (doc1@ornl.gov). Responsibility for its use, bug fixes and further development lie solely with the EQ-SANS team. All questions and concerns should be directed to your local contact, who will either help debug your reduction script or provide feedback to the Research Software Engineering group about bugs in the drtsans package that performs the reduction.


Up: Index