Tutorial 6: Experimental and Sample Metadata

pygid supports metadata handling through the ExpMetadata and SampleMetadata classes.

Experimental Metadata

ExpMetadata stores additional information about the experiment that is not used for coordinate calculations, such as experiment dates and source details. The structure is dictionary-like.

This metadata is saved in the entry/instrument group of the NeXus file.

If multiple pygid.Conversion results are saved into the same NeXus entry, the experimental metadata will be overwritten, except for fields listed in extend_fields.

import pygid

exp_metadata = pygid.ExpMetadata(
    start_time="2024-03-29T15:51:41.343788",
    end_time="2024-07-12T08:26:22Z",
    source_type="synchrotron",
    source_name="ESRF ID10",
    instrument_name="ID10-surf",
    detector="eiger4m",
    monitor=1.1e5,                 # optional
    extend_fields=["monitor"]      # fields appended instead of overwritten
)

Sample Metadata

Sample-related information is handled by the SampleMetadata class. The input format is a nested dictionary, and the same structure is written to the entry/sample group in the NeXus file.

We strongly recommend that the metadata include the sample name, structure, preparation description and experimental conditions via the SampleMetadata class. Users may further extend the sample group with custom fields, for example chemical formula, temperature, pressure, mass etc., as proposed by the DAPHNE4NFDI initiative in accordance with the FAIR. Both metadata classes support formats such as strings, lists, integer or float values, and NumPy arrays

data = {
    "name": "240306_DIP",
    "structure": {
        "stack": "air | DIP 0-25 | SiOx 1 | Si",
        "materials": {
            "DIP": {
                "name": "Diindenoperylene DIP",
                "thickness": 25e-9,     # optional
                "cif": "DIP.cif",       # optional
                "type": "gradient film"
            },
            "SiOx": {
                "name": "native SiOx",
                "thickness": 1
            },
            "Si": {
                "name": "Si wafer"
            }
        }
    },
    "preparation": "gradient thin film prepared by thermal evaporation",
    "experimental_conditions": "standard conditions, on air"
}

smpl_metadata = pygid.SampleMetadata(data=data)

Saving and Loading Sample Metadata

Sample metadata can be stored as a YAML file and reused for other conversions.

Saving:

smpl_metadata = pygid.SampleMetadata(
    path_to_save="sample.yaml",
    data=data
)

INFO - Saved sample metadata to /home/docs/checkouts/readthedocs.org/user_builds/pygid/checkouts/latest/docs/tutorials/sample.yaml

Loading:

smpl_metadata = pygid.SampleMetadata(
    path_to_load="sample.yaml"
)

Saving the Metadata in a NeXus file

Define the paths:

from pygid.datasets import get_dataset

# Download example dataset from Zenodo
try:
    files = get_dataset("tutorial_06")
    poni_path = files["poni"]
    mask_path = files["mask"]
    # several files for batch processing
    data_path = files["data"]
except:
    print("Dataset download skipped on Read the Docs.")

Dataset download skipped on Read the Docs.

Make the conversion as described in Tutorials 1-5:

import pygid

params = pygid.ExpParams(
    poni_path=poni_path,          # path to the PONI file
    mask_path=mask_path,
    ai=0.01,                       # angle of incidence (degrees)
    fliplr=True,
    flipud=True
)

matrix = pygid.CoordMaps(
    params,  # pygid.ExpParams
    vert_positive=True,
    hor_positive=True,
)

analysis = pygid.Conversion(
    matrix=matrix,                   # pygid.CoordMaps
    path=data_path,                  # path to the raw data file
    dataset='/entry_0000/ESRF-ID10/eiger4m/data',  # dataset path
    frame_num = 0,
)

analysis.det2q_gid(
    plot_result=False,
    save_result=True,
    path_to_save="result.h5",
    smpl_metadata=smpl_metadata,
    exp_metadata=exp_metadata,
)

INFO - Saved in D:\PhD\mlgid\pygid\docs\tutorials\result.h5 in group entry_0000

or after conversion using DataSaver:

# Run conversion without saving
analysis.det2q_gid(save_result=False)

# Save results to HDF5
pygid.DataSaver(
    analysis,                    # pygid.Conversion instance
    path_to_save='result.h5',    # path to save
    overwrite_file=True,         # Whether to overwrite an existing HDF5 file
    overwrite_group=True,      # Whether to overwrite an existing HDF5 entry
    smpl_metadata=smpl_metadata, # pygid.SampleMetadata
    exp_metadata=exp_metadata,   # pygid.ExpMetadata
)

INFO - Saved in D:\PhD\mlgid\pygid\docs\tutorials\result.h5 in group entry_0000

<pygid.datasaver.DataSaver at 0x25af2282570>