AI technique 'decodes' microscope images, overcoming fundamental limit

The deep learning algorithm developed by researchers at the University of Illinois Urbana-Champaign is trained to remove the effects of the probe's width from AFM microscope images. As reported in the journal Nano Letters, the algorithm surpasses other methods in giving the first true three-dimensional surface profiles at resolutions below the width of the microscope probe tip.

"Accurate surface height profiles are crucial to nanoelectronics development as well as scientific studies of material and biological systems, and AFM is a key technique that can measure profiles noninvasively," said Yingjie Zhang, a U. of I. materials science & engineering professor and the project lead. "We've demonstrated how to be even more precise and see things that are even smaller, and we've shown how AI can be leveraged to overcome a seemingly insurmountable limitation."

Often, microscopy techniques can only provide two-dimensional images, essentially providing researchers with aerial photographs of material surfaces. AFM provides full topographical maps accurately showing the height profiles of the surface features. These three-dimensional images are obtained by moving a probe across the material's surface and measuring its vertical deflection.

If surface features approach the size of the probe's tip -- about 10 nanometers -- then they cannot be resolved by the microscope because the probe becomes too large to "feel out" the features. Microscopists have been aware of this limitation for decades, but the U. of I. researchers are the first to give a deterministic solution.

"We turned to AI and deep learning because we wanted to get the height profile -- the exact roughness -- without the inherent limitations of more conventional mathematical methods," said Lalith Bonagiri, a graduate student in Zhang's group and the study's lead author.

The researchers developed a deep learning algorithm with an encoder-decoder framework. It first "encodes" raw AFM images by decomposing them into abstract features. After the feature representation is manipulated to remove the undesired effects, it is then "decoded" back into a recognizable image.

To train the algorithm, the researchers generated artificial images of three-dimensional structures and simulated their AFM readouts. The algorithm was then constructed to transform the simulated AFM images with probe-size effects and extract the underlying features.

"We actually had to do something nonstandard to achieve this," Bonagiri said. "The first step of typical AI image processing is to rescale the brightness and contrast of the images against some standard to simplify comparisons. In our case, though, the absolute brightness and contrast is the part that's meaningful, so we had to forgo that first step. That made the problem much more challenging."

To test their algorithm, the researchers synthesized gold and palladium nanoparticles with known dimensions on a silicon host. The algorithm successfully removed the probe tip effects and correctly identified the three-dimensional features of the nanoparticles.

"We've given a proof-of-concept and shown how to use AI to significantly improve AFM images, but this work is only the beginning," Zhang said. "As with all AI algorithms, we can improve it by training it on more and better data, but the path forward is clear."

The experiments were carried out in the Carl R. Woese Institute for Genomic Biology and the Materials Research Laboratory at the U. of I.

Support was provided by the National Science Foundation and the Arnold and Mabel Beckman Foundation.