Lack of adherence to usability testing standards for electronic health record products

Many EHRs have poor usability, leading to user frustration and safety risks. The U.S. Department of Health and Human Services' Office of the National Coordinator for Health Information Technology (ONC) has established certification requirements to promote usability practices by EHR vendors as part of a meaningful use program. To develop a certified EHR, vendors are required to attest to using user-centered design (UCD), a process that places the needs of the frontline user at the forefront of software development, and to conduct formal usability testing on 8 different EHR capabilities to ensure the product meets performance objectives.

EHR vendors are required to provide a written statement naming the UCD process they used, and the results of their usability tests. The ONC has endorsed guidelines from the National Institute of Standards and Technology stipulating that usability testing should include at least 15 representative end-user participants, according to background information in the article.

Reports must be made public once the product is certified. Raj M. Ratwani, Ph.D., of MedStar Health, Washington, D.C., and colleagues analyzed reports meeting the 2014 certification requirements for the 50 EHR vendors with the highest number of providers (hospitals and small private practices) attesting to meeting meaningful use requirements with that product between April 2013 and November 2014. From each report, the authors extracted the stated UCD process and the number and clinical background of usability test participants.

Of the 50 certified vendor reports, 41 were available for review (82 percent); the remaining 9 (18 percent) were not publicly available. Of 41 vendors, 34 percent had not met the ONC certification requirement of stating their UCD process, 46 percent used an industry standard, and 15 percent used an internally developed UCD process.

There was variability in the number of participants enrolled in the usability tests. Of the 41 vendors, 63 percent used less than the standard of 15 participants and only 22 percent used at least 15 participants with clinical backgrounds. In addition, 1 of the 41 vendors used no clinical participants, 17 percent used no physician participants, and 5 percent used their own employees. Of the 41 vendor reports available, 12 percent lacked enough detail to determine whether physicians participated and 51 percent did not provide the required demographic details.

"The lack of adherence to usability testing may be a major factor contributing to the poor usability experienced by clinicians. Enforcement of existing standards, specific usability guidelines, and greater scrutiny of vendor UCD processes may be necessary to achieve the functional and safety goals for the next generation of EHRs," the authors conclude.