Details: Ptychographic Imaging

The need – When new drugs are synthesized, dust particles are brought back from space, or new superconductors are discovered, a variety of sophisticated x-ray microscopes, spectrometers and scattering instruments are often summoned to characterize their structure and properties.  However, a 3D microscope capable of diffraction limited resolution and spectroscopic contrast has so far been well beyond reach, while the need for such a microscope has become ever more urgent.  In battery research or catalysis for example, nanoscale internal structures hold the key to achieving very high reactivity, in photovoltaics, charge separation happens through nanometer (nm) scale interfaces. In CO2 sequestration by porous rock, the finest pore, often at nm scale, is often rate-limiting.  


The opportunity - Progress in science often occurs in bursts following the invention of new experimental techniques or technological advances. With ever brighter x-ray light sources, fast parallel detectors,the ability to transform the data tsunami into the sharpest images ever recorded will help scientists from around the world understand ever more complex nano-materials, self-assembled devices, or to study different length-scales involved in life, from macro-molecular machines to bones, and whenever observing the whole picture is as important as recovering local atomic arrangement of the components.


Background - The principal problem in scattering or diffraction experiments is that it is, in general ,very difficult to reconstruct the object from the measurement, as the information is not directly related to the sample. For an analogy, imagine tossing some grains of sand against an object and trying to understand how the object from the distribution of the grains of sand on the ground, without knowing from which direction they came. It turns out that the vast majority of experiment in x-ray science involve a similar process, whereby x-rays are scattered, absorbed or slowed down as they encounter an object, and we only measure their distribution on a two-dimensional CCD. Knowing the direction (or the phase of the wavefront for coherent light) from which every x-ray came would enable us to propagate back to the sample and provide an image, but measuring this property is a difficult task. The ability to reconstruct a sample from their diffraction pattern has been a standing issue for over 100 years, and arises in fields as  varied as optics, astronomy, X-ray crystallography, tomographic imaging, holography, electron microscopy and particle scattering generally.


Diffraction measurements using short wavelength (such as X-ray, neutron, or electron wavepackets) have been at the foundation of some of the most dramatic breakthrough in science - such as the first direct confirmation ,100 years ago, of the existence of atoms by Bragg, the structure of DNA, RNA  and over 100,000 proteins or drugs involved in human life. However, the solution to the scattering problem for a general object was generally thought to be impossible for many years. Nevertheless, numerous experimental techniques that employ forms of interferometric/holographic measurements, gratings, and other phase mechanisms like random phase masks, sparsity structure, etc to help overcome the problem of phase-less measurements have been proposed over the years.


Progress has been made in solving the phase problem for a single diffraction pattern recorded from a non-periodic object. Such methods, referred to as coherent diffractive imaging (CDI), attempt to recover the complete complex-valued wave scattered from the object, giving phase contrast and a way to overcome depth-of-focus limitations of regular optical systems. Scientists around the world have used a shrink-wrap algorithm developed by our group in 2003  for experiments at synchrotrons and X-ray laser sources worldwide. Although ideal for ultrafast single-shot imaging, recovering the phases in the diffraction patterns is  computationally intensive, non-convex, and requires specific geometric constraints to be satisfied in order to obtain a solution.


The idea to overcome the limitation of diffraction techniques was proposed over 40 years ago, to improve the resolution in electron or x-ray microscopy by combining microscopy with scattering measurements. Initially, technological problems made ptychography impractical. Now thanks to advances in source brightness, detector speed, x-ray optics, and nanoscale position control, research institutions around the world are rushing to develop hundreds of ptychographic microscopes. Experimentally, ptychography works by retrofitting a scanning microscope with a parallel detector.  In a scanning microscope, a small beam is focused onto the sample via a lens, and the transmission is measured in a single- element detector. The image is built up by plotting the transmission as a function of the sample position as it is rastered across the beam. In such microscope, the resolution of the image is given by the beam size. In ptychography, one replaces

the single element detector with a two-dimensional array detector such as a CCD and measures the intensity distribution at many scattering angles, much like a radar detector system for the microscopic world. Each recorded diffraction pattern contains short- spatial Fourier frequency information  about features that are smaller than the beam-size, enabling higher resolution. At short wavelengths however it is only possible to measure the intensity of the diffracted light. To reconstruct an image of the object, one needs to retrieve the phase. The phase retrieval problem is made tractable in ptychography by recording multiple diffraction patterns from the same region of the object, compensating phase-less information with a redundant set of measurements. 

Since the reconstruction of ptychographic data is a non-linear problem, there are still many open questions, for example until 2014[1], there were no theoretical guarantees that the image produced by popular algorithms would be a faithful representation of our specimen. Moreover, existing algorithms do not work well on large data sets. Common iterative methods intrinsically operate by interchanging information between nearest neighbor frames (diffraction patterns) at each step, so it might take many iterations for frames far apart to communicate. Other open questions include the noise influence on the convergence behavior. Experimental uncertainties include not only photon-counting statistics but also perturbations of the lens, illumination scheme (positions), incoherent measurements, detector response and discretization, time dependent fluctuations, etc. Design questions include determine how to construct the best lens and illumination scheme to obtain accurate reconstruction for an arbitrary

Given a detector, with a limited rate, dynamic range and response function, what is the best scheme to encode more information per detector channel?



We addressed several of these issues recently.  We approach the inverse problem in high dimensional data space (we view every pixel of every frame as a dimension) by trying to make the phases of all pixels consistent with each other under some constraints. In other words, we form a ``relationship network'' from the given diffractive images dataset. The largest eigenvector of the connection graph laplacian built up from the relationship network contains the most aligned phases which well approximate the ground truth. In addition, as a spectral method, it enables scalable as well as robust algorithms The approach achieves accelerated convergence for large scale phase retrieval problems spanning multiple length-scales. We also show that this approach can recover experimental fluctuations over a large range of time-scales. 

A practical issue in ptychographic reconstruction are the strict requirements of the experimental geometry to achieve high quality data. For example, the need for stable, well controlled coherent illumination of the sample, limited detector speed and response function all contribute to limit the specifications of a ptychographic microscope. New methods to work with unknown illuminations were proposed. They are now used to calibrate high quality x-ray optics and space telescopes. More recently, position errors, background, noise statistics partially coherent illumination, detector response or vibrations have been added to the nonlinear optimization to fit the data.

A new approach that we developed is to account for the relationship network by forming pairwise comparisons between neighboring frames and update the unknown parameters of each diffraction frame so that each frame is consistent with each other.  We also show that this approach can recover experimental fluctuations over a large range of time-scales, enabling nanoscale resolution over macroscopic objects. As new instruments come online with ptycho-tomographic hyper spectral capabilities and polarization control, ever larger data sets and new unexpected sources of bias, the work described here and in the examples, will have a lasting impact on x-ray science; Times are very exciting, and there are many more things we can do, that are useful and ambitious. 


 Alternating Projection, "Ptychographic Imaging and Phase Synchronization", S. Marchesini, Y-C Tu, H-T Wu,