Augmented and virtual reality will involve human senses in verifying the operations of information systems

In the future, machines and AI systems will have a deeper understanding of the actions of their human users. Even now, AI is able to generate an image of what a human is watching on the screen just by recording brain activity or deduce the emotions of people from microexpressions taken from their faces.

In the Human Verifiable Computing project, VTT used augmented and virtual reality to develop solutions for building trust between people and systems and facilitating the verification of information security. This is a vital aspect of the digital future, in which interaction between people and computers will be an effortless part of everyday life. "Augmented and virtual reality technologies let us make fuller use of our senses and enable the constant mutual evaluation of reliability between humans and machines," says Senior Scientist Kimmo Halunen of VTT.

Making cryptographically verifiable computing available to human users was a key part of the project.

The project demonstrated functionalities involving computing verified with human senses. For example, augmented reality was utilized to distribute single-use passwords, which could then be used through voice recognition. Augmented reality was also utilized to give multisensory feedback by showing visual instructions to a maintenance worker who turns a valve and receives an error message if the valve is operated incorrectly. The message can be implemented as an interactive image and also presented through audio on the user's smart glasses. In addition, haptic feedback can be provided by making the user's smart watch or other mobile device vibrate.

The results of the project indicate that the basic technology required for the verification of computing with the human senses is already available. The combination of augmented reality and safety information will also enable new services. Current cryptographic methods and protocols are nearly always applied to communication between machines. Including the user in the interaction will nevertheless require more research and system and application development, as well as more study of human behaviour.