Zinc effects on common cold duration illustrate problems of routine statistical analyses

According to a paper published in BMC Medical Research Methodology, when the effect estimates (average shortening of colds in days) were applied to the duration of colds in the respective placebo groups, negative and zero cold durations were predicted, i.e. impossible and unrealistic predictions. In contrast, when the corresponding percentage effect estimates, 43% and 25%, respectively, were applied to the placebo group common cold durations, no impossible durations were predicted.

In the standard statistical analysis of continuous outcomes such as common cold duration, two scales are widely used for measuring the treatment effect. The absolute scale indicates the calculation of effect as the number of days by which the colds were shortened. The second scale widely used has the standard deviation as the unit of the scale, which leads to the standardized mean difference scale (SMD scale). Both of these approaches are available as options in popular meta-analysis software such as the RevMan program of the Cochrane collaboration.

If the absolute effects of a 4.0 day and a 1.77 day reductions in common cold duration are assumed to be uniform effects of taking zinc lozenges, then they should be valid for both short and long lasting colds. However, Dr. Harri Hemilä from the University of Helsinki, Finland, showed that the reduction in the duration of short colds was less than the average, whereas the reduction in the duration of long colds was much more than the average in the two studies that were analyzed. The implication of this finding is that the use of 4.0 and 1.77 day zinc effects for all colds is misleading and illogical. In contrast, the percentage effects of 43% and 25% realistically described the effects of zinc lozenges on both short duration and long duration colds.

The SMD scale is quite confusing for ordinary readers of research reports. For example, the 2011 Cochrane review on zinc and the common cold stated in its abstract that the "intake of zinc is associated with a significant reduction in the duration (standardised mean difference (SMD) -0.97)." Research reporting should always show the unit of measurement, and this sentence should have been written more accurately as: "zinc shortened the duration of colds by 0.97 standard deviation units." However, such accurate reporting would reveal the main problem of using the SMD scale: What does the "standard deviation unit" of common cold duration mean in practical terms? Most physicians and patients can easily understand whether 42% or 25% is a small or a large effect, but few of them can readily comprehend whether an effect of 0.97 standard deviation units is small or large. In this respect, the percentage effect scale is much more tangible and far superior in the communication of findings to physicians, and between physicians and patients, since the population as a whole are familiar with percentage effects.

Although the goal of the study was not to estimate the overall effect of zinc lozenges on common cold duration, the findings of the two trials analyzed are consistent with the percentage effects of zinc lozenges observed in four other trials on zinc lozenges.

Dr. Hemilä states that "The common cold is by itself a clinically relevant topic and the percentage scale leads to much more effective information about the size of the treatment effect than the absolute scale and the SMD scale. It seems evident that the percentage scale should be used in the analysis of many other continuous study outcomes, such as the duration of other diseases, and the duration of hospital stay and so on." Dr. Hemilä is concerned about certain situations when the Cochrane Collaboration has not allowed the usage of the percentage effect scale in the analysis of continuous outcomes even though it is much more informative than the absolute and SMD scales.