Hydrogels can play Pong by 'remembering' previous patterns of electrical simulation

"Ionic hydrogels can achieve the same kind of memory mechanics as more complex neural networks," says first author and robotics engineer Vincent Strong of the University of Reading. "We showed that hydrogels are not only able to play Pong; they can actually get better at it over time."

The researchers were inspired by a previous study that showed that brain cells in a dish can learn to play Pong if they are electrically stimulated in a way that gives them feedback on their performance.

"Our paper addresses the question of whether simple artificial systems can compute closed loops similar to the feedback loops that allow our brains to control our bodies," says corresponding author and biomedical engineer Yoshikatsu Hayashi of the University of Reading. "The basic principle in both neurons and hydrogels is that ion migration and distributions can work as a memory function that can correlate with sensory-motor loops in the Pong world. In neurons, ions run within the cells; in the gel, they run outside."

Hydrogels are complex polymers that become jelly like when hydrated -- gelatin and agar are natural examples. In this case, the researchers used an "electro-active polymer," meaning a hydrogel that can respond to electrical stimulation thanks to the presence of ions (charged particles) in the media surrounding its polymer matrix. When the hydrogel is electrically stimulated, the ions move, dragging water molecules with them, and this movement causes the hydrogel to temporarily change shape.

"The rate at which the hydrogel de-swells takes much longer than the time it takes for it to swell in the first place, meaning that the ions' next motion is influenced by its previous motion, which is sort of like memory occurring," says Strong. "The continued rearrangement of ions within the hydrogel is based off of previous rearrangements within the hydrogel, continuing back to when it was first made and had a homogeneous distribution of ions."

To test whether the hydrogel's physical "memory" could enable it to play Pong, the researchers used electrodes to connect the hydrogel to a virtual game environment and started up the game by sending the ball in a random direction. They used electrical stimulation to inform the hydrogel of the ball's position and measured the movement of ions within the hydrogel to determine the position of its paddle.

As the Pong games played out, the researchers measured the gel's hit rate and examined whether its accuracy improved. They showed that, with more experience, the hydrogel was able to hit the ball more frequently, resulting in longer rallies. Whereas the Pong-playing neurons achieved their optimal ball-skills within around 10 minutes, the hydrogel took closer to 20 minutes to reach its maximum Pong potential.

"Over time, as the ball moves, the gel gathers a memory of all motion. And then the paddle moves to accommodate that ball within the simulated environment," says Strong. "The ions move in a way that maps a memory of all motion over time, and this "memory" results in improved performance."

Because most existing AI algorithms are derived from neural networks, the researchers say that hydrogels represent a different kind of "intelligence" that could be used to develop new, simpler algorithms. In the future, the researchers plan to further probe the hydrogel's "memory" by examining the mechanisms behind its memory and by testing its ability to perform other tasks.

"In our follow-up projects, we are thinking about how to extract the algorithm from the hydrogels that allows memory acquisition," says co-author William Holderbaum of the University of Reading.

"We've shown that memory is emergent within the hydrogels, but the next step is to see whether we can also show specifically that learning is occurring," says Strong.

This research was supported by Process Vision Ltd.