Patient Safety Demands More Robust Testing of EHR Usability

Pew Charitable Trusts says not enough attention is being paid to electronic health record usability from a safety point of view. And given that federal certification requirements don’t address two key safety factors, it’s offering EHR developers and provider organizations a toolset to help boost patient protections. For all the benefits that EHRs bring, variations in their design, customization and use can “lead to inefficiencies or workflow challenges and can fail to prevent – or even contribute to – patient harm,” according to a new study from Pew. On the flip side, proper EHR optimization can be a boon to patient safety. The challenge is that proper assessment and testing of how the systems are actually used in the wild can be hard to do.

Yes, ONC has exacting certification requirements for health IT, including provisions that say vendors have to conduct usability testing and bring clinical end users into their design and development process. But the testing rules “fall short in two ways when it comes to assessing whether the use of products contributes to patient harm,” according to Pew’s Project Director for Health Information Technology Ben Moscovitch. First, federal testing criteria focus on vendors’ development and design but “do not address circumstances in which customized changes are made to an EHR as part of the implementation process or after the system goes live,” he explained. “The second key challenge is the absence of requirements and guidance on how to test clinician interaction with the EHR for safety issues.”

But use of such clinical test cases, with scenarios that mimic real-world clinical workflows and patient conditions, can be hugely valuable in detecting design or customization quirks that might have an adverse impact on safety. As part of its new report, Pew, in partnership with MedStar Health’s National Center for Human Factors in Healthcare and the American Medical Association, has made available a group of model test cases and list of best practices to help providers make safety assessments of their own post-install EHRs, and spot any “usability-related risks to patients throughout the life cycle of these products.” Earlier this summer, we highlighted human factors research from Pew, MedStar and AMA, where researchers examined how EHRs were used at four health systems: two that use Cerner and two that use Epic. They tracked how emergency physicians at each location handled six specific scenarios, collecting keystroke, mouse click and video data.

“There was wide variability in task completion time, clicks and error rates,” according to the report. “For certain tasks, there were an average of a nine-fold difference in time and eight-fold difference in clicks.” To better help hospitals and health systems ensure their EHRs are deployed and configured for optimal usability and safety, the new Pew report outlines scenarios and advice to help providers get a handle on how its clinical staff is putting the IT to work. “The sample scenarios in the report provide thorough tests covering seven types of usability issues that clinicians may face, including unclear settings, data entry obstacles, and confusing system alerts,” said Moscovitch. “Each scenario was designed so that both health IT vendors and purchasers of EHR products can conduct realistic, safety-focused usability tests. The scenarios can be used directly by vendors or providers and can help build additional tests.” He emphasized that successful scenarios should “include expected users of the system with varying levels of computer expertise; represent realistic clinical care processes; be shaped around a clinically oriented goal; contain clear, quantitative measures of success and failure; and include known risk areas and challenging processes, such as ordering that a patient’s drug dose be tapered.”

The hope, Moscovitch said, is that more EHR vendors adopt such test cases and incorporate them not just in design and development, but also in subsequent iterations as the technology matures. He added that the test scenario criteria included in the Pew report “can serve as a potential standard for the EHR accrediting bodies and a resource for developers” – and ideally would eventually be incorporated into future updates to ONC’s own certification requirements. Health systems, meanwhile, “can use the test criteria and sample cases to evaluate the usability and safety of their product during the implementation phase, after changes are made, and to inform customization decisions,” said Moscovitch. “Organizations can immediately leverage the example test cases to quickly evaluate system safety to identify challenges and prevent harm.” – Healthcare IT News