Tutorial 3: Raw Image Loading

Before starting, make sure that instances of pygid.ExpParams and pygid.CoordMaps have been successfully created (see Tutorial 1 and Tutorial 2).

from pygid.datasets import get_dataset

# Download example dataset from Zenodo
try:
    files = get_dataset("tutorial_03")
    poni_path = files["poni_for_tiff"]
    # several files for batch processing
    data_path1 = files["data1_tiff"]
    data_path2 = files["data2_tiff"]
    data_path3 = files["data3_tiff"]
except:
    print("Dataset download skipped on Read the Docs.")

Dataset download skipped on Read the Docs.

import pygid

params = pygid.ExpParams(
    poni_path=poni_path,          # path to the PONI file
    ai=0.01                       # angle of incidence (degrees)
)

matrix = pygid.CoordMaps(
    params  # pygid.ExpParams
)

After creating the pygid.CoordMaps instance, raw detector images can be loaded. You can provide either the path to a raw data file (or list of files) or a 2D/3D numpy array.

Load Image from File (EDF, TIFF, CBF)

Single file:

analysis = pygid.Conversion(
    matrix=matrix,                   # pygid.CoordMaps
    path=data_path1                  # path to the raw data file
)
print(f"raw data shape {analysis.img_raw.shape}")

raw data shape (1, 2048, 2048)

Multiple files:

analysis = pygid.Conversion(
    matrix=matrix,
    path=[data_path1, data_path2, data_path3] # several files
)
print(f"raw data shape {analysis.img_raw.shape}")

raw data shape (3, 2048, 2048)

Note: If the number of images exceeds batch_size, batch processing mode activates (see Tutorial 8).

Load Dataset from File (HDF5, NXS)

HDF5 and NXS files can contain multiple images with internal structure. Specify the dataset path (e.g., 'measurement/eiger4m') and the frame number (frame_num):

• None – load all images

• int – load a single frame

• list – load a list of frames

from pygid.datasets import get_dataset

# Download example dataset from Zenodo
files = get_dataset("tutorial_03")
poni_path = files["poni_for_h5"]
mask_path = files["mask_for_h5"]
# several files for batch processing
data_path = files["data_h5"]

import pygid

params = pygid.ExpParams(
    poni_path=poni_path,          # path to the PONI file
    mask_path=mask_path,
    ai=0.01,                       # angle of incidence (degrees)
    fliplr=True,
    flipud=True
)

matrix = pygid.CoordMaps(
    params  # pygid.ExpParams
)

Load all frames from the dataset:

analysis = pygid.Conversion(
    matrix=matrix,                   # pygid.CoordMaps
    path=data_path,                  # path to the raw data file
    dataset='/entry_0000/ESRF-ID10/eiger4m/data'  # dataset path
)
print(f"raw data shape {analysis.img_raw.shape}")

raw data shape (13, 2162, 2068)

Load a specific frame from the dataset:

analysis = pygid.Conversion(
    matrix=matrix,                   # pygid.CoordMaps
    path=data_path,                  # path to the raw data file
    dataset='/entry_0000/ESRF-ID10/eiger4m/data',  # dataset path
    frame_num = 7,
)
print(f"raw data shape {analysis.img_raw.shape}")

raw data shape (1, 2162, 2068)

Load several frames from the dataset:

analysis = pygid.Conversion(
    matrix=matrix,                   # pygid.CoordMaps
    path=data_path,                  # path to the raw data file
    dataset='/entry_0000/ESRF-ID10/eiger4m/data',  # dataset path
    frame_num = [5,6,7],
)
print(f"raw data shape {analysis.img_raw.shape}")

raw data shape (3, 2162, 2068)

Pass the Loaded Image (numpy.array)

If the image is already loaded, pass it using the img_raw parameter. Supports:

• 2D numpy array

• 3D numpy array (axis 0 = frames)

Orientation must match the mask and PONI coordinates.

import h5py
with h5py.File(data_path, mode='r') as root:
    img_raw = root['/entry_0000/ESRF-ID10/eiger4m/data'][()]

analysis = pygid.Conversion(
    matrix=matrix,                   # pygid.CoordMaps
    img_raw = img_raw
)
print(f"raw data shape {analysis.img_raw.shape}")

raw data shape (13, 2162, 2068)

Other Parameters

• average_all / sum_all – average or sum all frames

• number_to_average / number_to_sum – partial averaging or sum

• roi_range=[y_min, y_max, x_min, x_max] – select region of interest (in pixels of raw image before flipping/rotation)

analysis = pygid.Conversion(
    matrix=matrix,
    path=data_path,
    dataset='/entry_0000/ESRF-ID10/eiger4m/data',  # dataset path
    average_all=True,
    sum_all=False,
    roi_range=[0, 500, 0, 500]
)
print(f"raw data shape {analysis.img_raw.shape}")

raw data shape (1, 500, 500)

analysis = pygid.Conversion(
    matrix=matrix,
    path=data_path,
    dataset='/entry_0000/ESRF-ID10/eiger4m/data',  # dataset path
    number_to_sum=5,
)
print(f"raw data shape {analysis.img_raw.shape}")

raw data shape (3, 2162, 2068)

D:\PhD\mlgid\pygid\pygid\conversion.py:374: UserWarning: 3 images left, averaging them separately.
  warnings.warn(f"{remaining} images left, averaging them separately.", UserWarning)

After loading, images are preprocessed: masked, flipped (from ExpParams), and normalized using intensity correction matrices (CoordMaps).

Any other manipulations with raw data can be done using analysis.img_raw.

Visualization of Raw Images

Plot images in detector pixel positions (0,0 = direct beam). Ensure Detector_config==3. Orientation of PONI, mask, and raw image must match.

Parameters

• frame_num – frame to plot (int)

• clims – (vmin, vmax) for color scale

• return_result – return (x, y, img) if True

• plot_result – display the plot (True/False)

• xlim, ylim – axis limits, e.g., (0, 500)

• save_fig – save the figure (True/False)

• path_to_save_fig – path to save figure, default "img.png"

analysis = pygid.Conversion(
    matrix=matrix,                   # pygid.CoordMaps
    path=data_path,                  # path to the raw data file
    dataset='/entry_0000/ESRF-ID10/eiger4m/data'  # dataset path
)

x, y, img = analysis.plot_img_raw(
    frame_num=7,
    clims=(4e2,1e5),
    return_result=True,
    plot_result=True,
    save_fig=False,
    path_to_save_fig="img.png"
)

print(f"x-axis shape {x.shape}")
print(f"y-axis shape {y.shape}")
print(f"raw image shape {img.shape}")

x-axis shape (2068,)
y-axis shape (2162,)
raw image shape (2162, 2068)

Plot parameters can be changed using pygid.set_plot_defaults function:

analysis.set_plot_defaults(cmap = 'jet')
analysis.plot_img_raw(
    frame_num=7,
    clims=(4e2,1e5),
    return_result=False,
    plot_result=True,
    save_fig=True,
    path_to_save_fig="img.png"
)

INFO - Saved figure in D:\PhD\mlgid\pygid\docs\tutorials\img.png

Full list of adjustable parameters in set_plot_defaults

help(analysis.set_plot_defaults)

Help on method set_plot_defaults in module pygid.conversion:

set_plot_defaults(font_size=14, axes_titlesize=14, axes_labelsize=18, grid=False, grid_color='gray', grid_linestyle='--', grid_linewidth=0.5, xtick_labelsize=14, ytick_labelsize=14, legend_fontsize=12, legend_loc='best', legend_frameon=True, legend_borderpad=1.0, legend_borderaxespad=1.0, figure_titlesize=16, figsize=(6.4, 4.8), axes_linewidth=0.5, savefig_dpi=600, savefig_transparent=False, savefig_bbox_inches=None, savefig_pad_inches=0.1, line_linewidth=2, line_color='blue', line_linestyle='-', line_marker=None, scatter_marker='o', scatter_edgecolors='black', cmap='inferno') class method of pygid.conversion.Conversion
    Sets the default settings for various parts of a Matplotlib plot, including font sizes, gridlines,
    legend, figure properties, and line styles. The function configures the default style for future
    plots created with Matplotlib.

    Parameters:
    - font_size (int): Default font size for text elements (e.g., title, labels, ticks).
    - axes_titlesize (int): Font size for axes titles.
    - axes_labelsize (int): Font size for axes labels (x and y).
    - grid (bool): Whether or not to display gridlines (True/False).
    - grid_color (str): Color of the gridlines (e.g., 'gray', 'black').
    - grid_linestyle (str): Line style of the gridlines (e.g., '--', '-', ':').
    - grid_linewidth (float): Width of the gridlines.
    - xtick_labelsize (int): Font size for x-axis tick labels.
    - ytick_labelsize (int): Font size for y-axis tick labels.
    - legend_fontsize (int): Font size for the legend text.
    - legend_loc (str): Location of the legend (e.g., 'best', 'upper right', 'lower left').
    - legend_frameon (bool): Whether to display a frame around the legend.
    - legend_borderpad (float): Padding between the legend's content and the legend's frame.
    - legend_borderaxespad (float): Padding between the legend and axes.
    - figure_titlesize (int): Font size for the figure title.
    - figsize (tuple): Size of the figure in inches (e.g., (6, 6)).
    - savefig_dpi (int): DPI for saving the figure (higher DPI means better quality).
    - savefig_transparent (bool): Whether the saved figure should have a transparent background.
    - savefig_bbox_inches (str): Defines what part of the plot to save (e.g., 'tight' to crop extra whitespace).
    - savefig_pad_inches (float): Padding added around the figure when saving.
    - line_linewidth (float): Line width for plot lines.
    - line_color (str): Color of the plot lines (e.g., 'blue', 'red').
    - line_linestyle (str): Line style (e.g., '-', '--', ':').
    - line_marker (str): Marker style for plot lines (e.g., 'o', 'x').
    - scatter_marker (str): Marker style for scatter plots (e.g., 'o', 'x').
    - scatter_edgecolors (str): Color for the edges of scatter plot markers (e.g., 'black').
    - cmap (str): Image colormap