Expert panel diagnosis for diagnostic test poorly described, experts not blinded to test under study

The authors evaluated 81 diagnostic studies published up to May 31, 2012, including studies of diagnostic tests for psychiatric disorders (30 of 81 papers, 37%), half of which pertained to dementia, cardiovascular diseases (17 papers, 21%), and respiratory disorders (10 papers, 12%). They found that reporting was often incomplete, with 83% of studies missing at least some important information about the expert panel. In 75% of studies the panel consisted of three or fewer members, and panel members were blinded to the results of the test results being evaluated in only 31% of studies. Blinding is important because knowledge of the index text results could influence the panelists' decision as to whether the patient had the disease. Reproducibility of the decision process was assessed in only 21% of studies.

The authors state, "Complete and accurate reporting is a prerequisite for judging potential bias in a study and for allowing readers to apply the same study methods. In total, only 14 (17%) papers reported complete data on key issues such as the panel constitution, the information presented to the panel and the exact decision process to determine the final diagnosis." They also found that despite publication of reporting guidelines, the completeness of reporting did not improve over time, perhaps because the reporting guidelines do not include specific criteria for expert panel diagnoses. The authors make a number of recommendations to improve reporting of expert panel diagnosis. They conclude, "Our review revealed a large variation in applied methods as well as major deficiencies in the reporting of key features of the panel diagnosis process…The results of our review may serve as a starting point in the development of formal guidelines on methodology and reporting of panel diagnosis."