Xi-CAM: Details

General Design:



Remote compute resources:


Xi-cam follows a server/client design model. Data processing operations can be built into a workflow/macro which may be executed as a batch locally or pushed to a remote processing server. A seamless integration of HPC resources allows users to make effective use of computing resources allocated to synchrotron facilities without the learning technicalities associated with underlying high performance hardware.



Flexible for advanced use:


To test and extend the broad variety of more specialized analysis, an integrated Jupyter/IPython console is provided. This console exposes all of the data loaded in Xi-cam for manipulation in Python. Users may import additional Python packages to extend the functionality of Xi-cam ad-hoc or script custom analysis procedures within Xi-cam. Data manipulated in memory can then be re-examined in the main Xi-cam interface by loading it back into one of the plugins. Experienced and programming-oriented users may opt to implement their custom analysis procedures or visualizations as their own Xi-cam plugin, which can be dynamically loaded from the Plugins menu.





In cooperation between multiple beamlines and multiple synchrotron facilities, Xi-cam aims to be relevant across a variety of hardware, software, and data formats. Xi-cam is cross-platform, and can be made available as a compiled executable or installed via Python’s “pip” package manager. In the context of SAXS data, beamline hardware and configuration can be automatically extrapolated by a fully automated calibration procedure, parsing the data format, shape, and header information.




Clean, consistent interface novices users:


For novice users Integration of multiple data modes in a common interface also provides the benefit of reducing learning barriers with multiple software packages. Beamline users who collect data with a variety of modes (i.e. SAXS and NEXAFS) will find clear similarities between Xi-cam’s plugin interfaces. By partitioning the interface into a set of defined elements, a novice user can expect similarities in interface design across modes. As best as can be enforced, metadata and configurable properties are displayed in the right panel; data is loaded from the left panel; data is viewed in the center panel; data reduction is plotted in the lower panel.






Automated calibration:

Ability to calibrate scattering images is the most basic feature of any analysis suite. This is done automatically in Xi-CAM. So far Silver-Behenate has been implemented, as it is the most used on LBL scattering beamline. We are in process of implementing others such as lab6 and hexadecanol. The process involves locating the direct-beam position with respect to detector, calculation of sample detector distance and determination of image coordinates in reciprocal space. A user can also input parameters determined a priori.




Data formats:

The broadly diverse field of synchrotron science has proliferated a variety of data formats. The data loading elements of Xi-CAM are built on the Fable IO (FabIO) 2D X-ray detector I/O library. FabIO supports a wide variety of detectors (including Mar, Dectris, ADSC, Hamamatsu, Oxford, …), and file formats (like CBF, EDF, TIFF, …). This library effectively generalizes data I/O, and is even extensible to include more unique data formats.


Read more about FabIO: http://pythonhosted.org/fabio/




Fast azimuthal integration:

A necessary element of SAXS data analysis is the transformation of detector-space data into q-[endif]--> space, in which most analysis and processing is applied. The Python Fast Azimuthal Integration (PyFAI) library is invoked from Xi-cam to efficiently perform this transformation. PyFAI features a robust experimental geometry definition, including detailed definitions for a variety of X-ray detectors. PyFAI includes parallel architecture optimized processes for multi-core CPU and GPU based processing for fast high-volume data throughput.


Read more about PyFAI: http://pythonhosted.org/pyFAI/




GIWAXS remeshing:

Data from the Grazing Incidence experiments has to be corrected for Ewald sphere. This results in a “missing wedge”. This is more prominent in the GIWAXS than the GISAXS. This is done in Xi-cam using the inverse mapping of reciprocal coordinates to pixels, i.e. [endif]-->. This inverse mapping itself is quite efficient, but we developed a C++ module to further improve the speed to ~(200ms) per image.




Crystal space group identification:

Xi-CAM can overlay Bragg peaks allowed by the symmetry, given the space group and lattice spacing. Xi-cam distinguishes between transmission and reflection peaks. This can be particularly useful in identifying the symmetries in WAXS/GIWAXS datasets.




Remote data interface:

With increasing data volumes and acquisition rates at synchrotron facilities, many beamlines have adapted by streaming data to remote storage facilities. Often, these storage facilities have overlapping resources with HPC. The remote data interface in Xi-CAM allows users to pull/transfer data from these resources seamlessly. In future releases, user workflows may be executed remotely for large batch jobs.




Workflow design:

Batch processing is a common strategy for reducing high volume data sets. Similar to macros, Xi-CAM ‘workflows’ can be designed by arranging data processing functions graphically. When parsed, these workflows are translated into a python script which may be exported, run locally, or run remotely. Unlike macros, users can interactively modify/rearrange/delete functions from the workflow, providing a more robust user experience.




Dynamic data slicing:

Large data series can be difficult to manage and reduce. In Xi-CAM, the dynamic data slicing plugin, “Timeline”, provides an interface to create visualizations by slicing data by values from their headers. The simplest use case is to display time-series data; users can interactively scroll through the parameter space to explore the set. This can be generalized to apply to any header values. Unique data reduction also allows key data frames to be identified automatically. Custom parameter space aware data reduction may also be applied to data series in this interface.




Symmetry operators:

Gaps between detector modules and masked data regions  may be visually disrupting when interpreting data, or can interfere with computer vision analysis procedures. By applying symmetry operations (radial symmetry for SAXS; mirror for GISAXS), these disruptions can be diminished. Further, data regions that cannot be populated via symmetry may be filled artificially by solving the heat diffusion equation or linear interpolation.




IPython console:

Advanced users may find a need to manipulate data in memory with custom processing. An IPython console is available for this purpose. Modified data may be directly reloaded into a viewing plugin. For more involved processing or specialized visualization tools, a custom plugin may be developed.




Plugin infrastructure:

The Xi-CAM plugin infrastructure allows developers to add specialized analysis modes, taking advantage of its shared interface features and widgets. Enforcing consistent design in development allows users to quickly adapt to new plugin interfaces.

On-going development Xi-CAM:


Xi-CAM is under continual development. If interested in collaborative development or to receive development beta releases, please contact Ron Pandolfi (ronpandolfi@lbl.gov) and Alex Hexemer (ahexemer@lbl.gov).