Now in 3D: Deep learning techniques help visualize X-ray data in three dimensions

That's the challenge a group of scientists from the U.S. Department of Energy's (DOE) Argonne National Laboratory is working to overcome. Artificial intelligence has emerged as a versatile solution to the issues posed by big data processing. For scientists who use the Advanced Photon Source (APS), a DOE Office of Science User Facility at Argonne, to process 3D images, it may be the key to turning X-ray data into visible, understandable shapes at a much faster rate. A breakthrough in this area could have implications for astronomy, electron microscopy and other areas of science dependent on large amounts of 3D data.

The research team, which includes scientists from three Argonne divisions, has developed a new computational framework called 3D-CDI-NN, and has shown that it can create 3D visualizations from data collected at the APS hundreds of times faster than traditional methods can. The team's research was published in Applied Physics Reviews, a publication of the American Institute of Physics.

CDI stands for coherent diffraction imaging, an X-ray technique that involves bouncing ultra-bright X-ray beams off of samples. Those beams of light will then be collected by detectors as data, and it takes some computational effort to turn that data into images. Part of the challenge, explains Mathew Cherukara, leader of the Computational X-ray Science group in Argonne's X-ray Science Division (XSD), is that the detectors only capture some of the information from the beams.

But there is important information contained in the missing data, and scientists rely on computers to fill in that information. As Cherukara notes, while this takes some time to do in 2D, it takes even longer to do with 3D images. The solution, then, is to train an artificial intelligence to recognize objects and the microscopic changes they undergo directly from the raw data, without having to fill in the missing info.

To do this, the team started with simulated X-ray data to train the neural network. The NN in the framework's title, a neural network is a series of algorithms that can teach a computer to predict outcomes based on data it receives. Henry Chan, the lead author on the paper and a postdoctoral researcher in the Center for Nanoscale Materials (CNM), a DOE Office of Science User Facility at Argonne, led this part of the work.

"We used computer simulations to create crystals of different shapes and sizes, and we converted them into images and diffraction patterns for the neural network to learn," Chan said. "The ease of quickly generating many realistic crystals for training is the benefit of simulations."

This work was done using the graphics processing unit resources at Argonne's Joint Laboratory for System Evaluation, which deploys leading-edge testbeds to enable research on emerging high-performance computing platforms and capabilities.

Once the network is trained, says Stephan Hruszkewycz, physicist and group leader with Argonne's Materials Science Division, it can come pretty close to the right answer, pretty quickly. However, there is still room for refinement, so the 3D-CDI-NN framework includes a process to get the network the rest of the way there. Hruszkewycz, along with Northwestern University graduate student Saugat Kandel, worked on this aspect of the project, which reduces the need for time-consuming iterative steps.

"The Materials Science Division cares about coherent diffraction because you can see materials at few-nanometer length scales -- about 100,000 times smaller than the width of a human hair -- with X-rays that penetrate into environments," Hruszkewycz said. "This paper is a demonstration of these advanced methods, and it greatly facilitates the imaging process. We want to know what a material is, and how it changes over time, and this will help us make better pictures of it as we make measurements."

As a final step, 3D-CDI-NN's ability to fill in missing information and come up with a 3D visualization was tested on real X-ray data of tiny particles of gold, collected at beamline 34-ID-C at the APS. The result is a computational method that is hundreds of times faster on simulated data, and nearly that fast on real APS data. The tests also showed that the network can reconstruct images with less data than is usually required to compensate for the information not captured by the detectors.

The next step for this research, according to Chan, is to integrate the network into the APS's workflow, so that it learns from data as it is taken. If the network learns from data at the beamline, he said, it will continuously improve.

For this team, there's a time element to this research as well. As Cherukara points out, a massive upgrade of the APS is in the works, and the amount of data generated now will increase exponentially once the project is complete. The upgraded APS will generate X-ray beams that are up to 500 times brighter, and the coherence of the beam -- the characteristic of light that allows it to diffract in a way that encodes more information about the sample -- will be greatly increased.

That means that while it takes two to three minutes now to gather coherent diffraction imaging data from a sample and get an image, the data collection part of that process will soon be up to 500 times faster. The process of converting that data to a usable image also needs to be hundreds of times faster than it is now to keep up.

"In order to make full use of what the upgraded APS will be capable of, we have to reinvent data analytics," Cherukara said. "Our current methods are not enough to keep up. Machine learning can make full use and go beyond what is currently possible."

In addition to Chan, Cherukara and Hruszkewycz, authors on the paper include Subramanian Sankaranarayanan and Ross Harder, both of Argonne; Youssef Nashed of SLAC National Accelerator Laboratory; and Saugat Kandel of Northwestern University.