Speakers
Description
In recent years, data processing has held an increasingly important role in the toolkit of a scanning probe microscopist. From simple parabolic baseline correction in a topography image, to utilising correlations between images to adjust for distortions between scans, external programming platforms have allowed the extraction of information well beyond the raw data.
This information revolution has also yielded new challenges for the modern researcher. Complex processing protocols require conversion from proprietary formats into user-defined alternatives. These alternatives may lose key physical metadata in the process, such as scale, scan rate, or applied tip voltage. Furthermore, as data-processing programs are often written by individual users or lab groups, potential deviations can appear when comparing results with other researchers. Such issues are exacerbated if the user attempts to collate data from independent sources, where a process may have to be completely rewritten to operate on another data type.
Here, we present Hystorian, a cross-platform Python library that is capable of loading, merging, and operating on data from arbitrary sources [1], by transforming the data into n-dimensional arrays in a hierarchal dataset format (HDF5) file. All metadata is stored in raw text as well as in attributes associated with the primary dataset. Processing functions can therefore utilise multiple datasets in a single datafile, in addition to their associated metadata. The history of process outputs is also saved within the file, allowing a user to operate on intermediate processes, trace a history of these operations, and access all outputs without reapplying computationally intensive algorithms.
Hystorian is also packaged with tools to assist in the application of custom functions to the HDF5 files, along with a range of useful scientific operations. This allows for rapid prototyping and accessibility to any user with a working understanding of Python. The source-agnostic nature of the HDF5 file also allows similar functions to be used across different data types. For example, feature identification in scanning probe microscopy images may be directly transferred to track changes in peak positions in x-ray diffraction reciprocal space maps. In this poster, Hystorian is used to process a series of 30 piezoresponse force microscopy images to track the motion of domain walls over a 20-hour period.
[1] Musy, L., Bulanadi, R., et al. "Hystorian: A processing tool for scanning probe microscopy and other n-dimensional datasets." Ultramicroscopy 228 (2021): 113345.
