Expert panel reliable and accurate in identifying injuries in young children

In a study led by University of Louisville biostatistician Doug Lorenz, Ph.D., associate professor, School of Public Health and Information Sciences, the reliability and accuracy of a nine -- member expert panel was measured in determining the likelihood of abuse in more than 2,000 cases. The results of the study that published recently in the Journal of Pediatrics found nearly perfect reliability and accuracy of the panel both individually and as a composite.

"Decision rules for identifying injuries from abuse are valuable for settings such as pediatric emergency departments -- in these cases, reliability and accuracy of human judgement is needed since victims of child abuse are at high risk of future abuse and death," Lorenz said.

The study examined injuries involving children less than 4-years-old who visited pediatric emergency departments between 2011 and 2016 at five children's hospitals in the United States. The expert panel included four child abuse pediatricians, four emergency medicine physicians and one bioengineer with expertise in pediatric injury.

Each panelist received de-identified case information, including current and past data of the patient's visit to a pediatric emergency department. The panelists independently rated the likelihood of abuse on a 5-level classification scale as "definite abuse," "likely abuse," "indeterminate," "likely accident," and "definite accident." Each individual also provided a "yes" or "no" answer to a question on whether to report the case to state child protective services.

At least two panelists reviewed each of the 2,166 cases. There was unanimous agreement in 852 cases (39-percent). An additional 1,048 cases (48-percent) exhibited partial agreement, where all panelists provided classifications of abuse (definite or likely) or accident (definite or likely). There were 44 instances (2-percent) of abuse-accident disagreement, none of which were in the definite category. In the remaining 222 cases (10-perecent), all panelists gave classifications of indeterminate.

The study tested the accuracy of the expert panel based on 584 of the 2,166 cases that could be more definitively classified as abuse or accidental based on other information learned after the child's visit to the emergency department. This included video of an event, a third party account, a confession of abuse or criminal conviction of abuse, to name a few examples. Individual panelists accurately classified between 95 and 98-percent of the abuse cases and between 99 and 100-percent of the accident cases.

"We have demonstrated the reliability and accuracy of the expert panel approach, and suggest this method provides a suitable set of criteria for the future evaluation of clinical decision rules for identifying child abuse," Lorenz said.

Lorenz collaborated with other UofL colleagues in the School of Medicine Department of Pediatrics and J.B. Speed School of Engineering. The study also involved researchers at the Northwestern University, University of Pittsburg, University of Utah, University of Washington and Yale University.