Editors: Changwoo Do, Gergely Nagy, William Heller
Date: 07/16/2020
EQ-SANS data reduction is typically performed through two equivalent means:
The present document details the former method. Your local contact will work with you to configure the reduction for the various instrument configurations that you use during your experiment.
The content of the document is also explained in a [video tutorial](<https://youtu.be/1XU_40Z8RuY).
Below find the step by step description of the procedure.
Visit analysis.sns.gov
a. Data reduction can be performed both from inside and outside of ORNL
b. Log in with your ucams ID and password
c. The same credentials can be used for easy file transfer to personal computer via e.g. WinSCP
Populate the shared directory of your IPTS with the required reduction tools and activate the reduction environment
a. open a terminal
b. go to the shared folder of your IPTS
cd /SNS/EQSANS/IPTS-****/shared
c. copy eqsans_setup.sh
from
/SNS/EQSANS/shared/usertools/
cp /SNS/EQSANS/shared/usertools/* ./
Alternatively, one can directly run the shell script by typing
/SNS/EQSANS/shared/usertools/eqsans_setup.sh
and click
enter.
d. run eqsans_setup.sh
by typing
./eqsans_setup.sh
e. run eqsans_activatedrtsans.sh
by typing
source ./eqsans_activatedrtsans.sh
Edit the template reduction script, which is written in python 3
a. Use gedit or pluma to edit reduce_template.py
gedit reduce_template.py &
or
pluma reduce_template.py &
b. The script contains examples for:
reduction of data for a single configuration
reduction of data from 2 configurations in a single script
Example of writing and running a script for a single configuration reduction
a. Update output directory
It is recommended for the target directory for the reduced files to be empty or to not to contain reduction results from previous reductions in EQSANS_runnumber_*.nxs format because drtsans will over-write previous results without asking the user if they wish to do so.
b. Update common reduction parameters
c. Update run numbers and names for individual scattering and transmission runs
multiple samples can be inserted with comma separation
multiple runs collected from a single sample that you wish to sum together are specified as a comma-separated list enclosed in quotation marks
d. to look up run numbers from your experiment please refer to
the catalog method presented in the other tutorial document.
e. Below is a complete example of how to specify data reduction for a standard sample. Note that the loop could be used to reduce data from multiple samples if the samscatt, samtrans and fn_list arrays contained more than one element. If used to reduce multiple data sets, these three arrays must have the same number of elements.
f. comment all unnecessary sections of the file by using either a
#
to remove a single line or a pair of '''
at
the start and end (i.e. enclosing) of sections that you wish to comment
out and save the document.
This is advised to prevent time consuming re-run of already completed reduction runs
g. Save the file and run the reduction script from the terminal. For the example template file provided, type the following.
python reduce_template.py
Depending on the size of the data files, which can be quite large for strongly scattering samples, each data set may require several minutes to reduce because the instrument saves data in "event mode", which results in large files. If you are reducing a large number of files, such as after an overnight script, you may want to get a cup of coffee.
h. Reduced files can be found in the output folder in a variety of formats.
Images in png format for both 1D and 2D reduced data can be displayed via the "display" command ([analysis] display filename.png), or can be opened using the Caja file browser by double clicking on the file.
I(q) files can be also displayed via launching eqsans_rungui.sh created in paragraph 2, clicking on the "Display IQ" button and select the data of interest.
Analysis cluster has sasview software, which can also display and perform fittings. To launch sasview, open a terminal and type the following.
sasview
Stitching of individual runs
a. The template script also contains a section entitled "stitching"
that allows data sets collected in different instrument configurations
for a sample to be stitched into a single intensity profile. While the
example presents stitching of two data sets, it is possible to stitch
data from several different configurations together. Then, the overlap
array takes the form
overlap = [MergeAB_min, MergeAB_max, MergeBC_min, MergBC_max, ...]
,
having 2·(Num_Configurations -1) pairs of minimum and maximum Q-values
for the merge regions.
b. Update the section with names of files to be stitched, with output name, stitching range etc.
The stitching ranges are generally determined by examining the data files. Feel free to ask your local contact or another instrument scientist for advice.
The stitching ranges have to be listed in order of increasing momentum transfer (from low-Q to high-Q)
During scaling of the independent curves target_profile_index will determine the curve to scale to. Note that index of the first curve is 0.
The applied scale factor will be displayed in the terminal window during script execution.
c. Unless another name is provided for the merged_fn
variable in the save_iqmod command, the stitched file
will have an ending of _merged.dat
. In the example above,
the output file for the stitched data is "porasil_merged.dat".
Reducing and stitching two configurations in one go
a. Example to be found under "# example of one-step reduction"
Comment out all sections not intended to be used
b. Define necessary parameters. In the example below, there are two different configurations: low Q and high Q. Note that the two configurations have their own variables, which makes it possible to reduce the data from two configurations with a single script.
c. Provide scattering and transmission run numbers for the samples and the overlap range for stitching
d. Run the reduction script and observe the results as describe above in section 4.
Data reduction with added flexibility
a. The template file provided serves as a reduction example for simple cases. The data reduction software is highly configurable through the various reduction parameters that can be seen in the json files created during script execution. Consult with an instrument scientist about their usage. Data reduction with more specialized needs can be achieved through modifying json parameters. A separate document describes the various parameters.
b. The json files can be directly edited with gedit or pluma, but can be viewed easily with the prettyjson.sh script that is created during the steps described in section 2.
From the output subdirectory within the shared directory the script can be run as follows:
./prettyjson.sh <name_of_jsonfile>
c. Modify the selected json parameters by adding new lines to the reduction script. In the example below, a different sensitivity file is specified on the last line.
d. All parameter names are to be written in lowercase letters.
Disclaimer
The data reduction scripting method presented here was developed by Dr. Changwoo Do (doc1@ornl.gov). Responsibility for its use, bug fixes and further development lie solely with the EQ-SANS team. All questions and concerns should be directed to your local contact, who will either help debug your reduction script or provide feedback to the Research Software Engineering group about bugs in the drtsans package that performs the reduction.