John S. Allen's Response to Rodgers (1996)

I am responding to the reply (BF #38, p. 10 ff.) by Dr. Gregory Rodgers of the U.S. Consumer Product Safety Commission to my article on the CPSC's Bicycle Use And Hazard Patterns in the United States (BF #37, p. 12 ff.)...[1]

Dr. Rodgers recalculates the mile/hours comparison using median (middle-of-the-group) rather than average values. He reports a median speed of 2.1 mph with very little variation among age groups.[2] Most people can't even balance a bicycle at this speed, but does the small variation at least make time and mileage data comparable, as Dr. Rodgers suggests?

My own age group comparison using averages shows an upward trend from the 18-30 age group to the 51+ group.[3] Using medians rather than averages decreases the contribution of people who ride frequently, and measuring time rather than mileage further decreases it.

John Forester gives the example of two cyclists who make the same trip. Once rides twice as fast as the other. The faster cyclist's accident rate is doubled if we apply the CPSC hours-of-riding approach. This approach, even if the numbers were right, would still favor anything that slows cyclists down: lack of skill, poor equipment, poor facilities -- note exactly a recipe to encourage bicycle use.

Dr. Rodgers has made some useful points about the methodology of the CPSC study as well as others I cited, and he has corrected a few errors in my speculations about research results. Yet he has not undercut my basic question: why does the CPSC study disagree so strongly with others, and with itself?

Dr. Rodgers has not disputed my assertion that CPSC and USDOT data show an impossibly high level of bicycle use. Also, the CPSC's reported accident rate for adults, about 8 times as high on streets as on bicycle paths or unimproved surfaces, defies explanation, since there is no factor other than motor vehicles which creates greater hazards for cyclists on roads than on paths.

The CPSC itself contradicts this result when it reports that only 15 percent of bicycle accidents on streets involve a motor vehicle. This latter figure agrees well with other studies -- for all kinds of cyclists.

Finally, let me step back and suggest that there is a rift between expert cyclists and scholarly researchers which sorely needs mending. I might have worded my article more kindly, had it reviewed more extensively before publication and checked my math better. In his response, on the other hand, Dr. Rodgers suggests repeatedly that expert cyclists do not understand other cyclists, implying that the CPSC understands them better. We all have things to learn, but speaking to myself, I have not forgotten my early cycling experiences. Rather, I am reminded of them, sometimes forcefully, every day as I encounter other cyclists in the streets. If I did not understand other cyclists, I would have little credibility as a writer, Effective Cycling Instructor, cyclist advocate, consultant on cycling, or as the father of a 4 year old who rides with training wheels. Rather, it is the CPSC which -- despite extensive resources, detailed technical analysis, and the ability to make scholarly corrections -- was unable to determine that its research method was not producing reliable or meaningful data.

The best research unites analytical power with real-world savvy. More tightly focused studies which combine direct measurement and extensive cross-checking of data may provide us only with pieces of the big picture of national bicycle use, but, in my opinion, will produce more useful results.

FURTHER THOUGHTS (2001)

Now that the CPSC study is online, I am making my analysis of it available online as well. I have made every attempt to be fair in doing this, by accepting some of the  criticisms by Dr. Gregory Rodgers and posting the remaining ones.

I do have some disagreements with Dr. Rodgers's comments beyond those which I mentioned in my 1996 reply. For example, a large subset of the respondents in the Kaplan study used odometers. Many of the others kept training diaries, and in any case, as bicycling enthusiasts, they were likely to have a much more accurate idea of their mileage than casual bicyclists would. Kaplan performed an analysis of the mileage reported by those who did and did not use odometers, and found no statistically significant difference. Rodgers's dismissal of the Kaplan study's mileage data is therefore unfounded.

Rodgers's comments about the less reliable recording of mileage in some other studies are well taken, though it should be noted that the method of asking "when was the last time you rode" and "how far" minimizes the errors of recollection which occur when asking people to describe a month's or year's riding and, if responses are accurate, is capable of producing accurate results if the statistical sample is large.

Rodgers's complaint that preexisting studies did not provide a national survey does not make the CPSC study, or its methodology, better by comparison. Taken together, the smaller studies I cited do in fact cover the major segments of the bicycling population, and their more focused methods led to more reliable and consistent data. And the comment that the studies I cited were old doesn't carry much weight: new studies published since the CPSC study have confirmed the findings of the ones I cited.

I do grant that some of the data in the CPSC study provides useful information about accident causation, equipment use and demographics of the bicyclist population. When the CPSC asked questions whose answers depended on things people could clearly remember, it obtained some good data.

Still, the CPSC study's credibility as an official U.S. Government report, fuels dangerous misconceptions, because its figures about relative risks are so wildly incorrect. Roads simply can not be 8 times as hazardous as off-road environments when only 15 percent of the accidents on roads involve motor vehicles. Yet there are altogether too many people ready to believe this figure. Such beliefs lead to attempts to restrict bicycle use on the roads and to construction of off-road facilities which are, in fact, less safe.

The CPSC study provides an object lesson on how bicycling research can go wrong. Highly trained statisticians in a government agency and a contract research firm with a fine reputation did not overcome the problems with the CPSC's approach, which I categorize as follows:

• Failure to heed the lessons of the preexisting literature.

• Use of a research method -- the telephone questionnaire --  which has been shown again and again not to produce accurate data about bicycle use.

• The researchers' failure to overcome common misconceptions about bicycling (in particular, about the relative safety of different facility types), leading to unquestioning acceptance of inconsistent data and to the forming of unsupported conclusions.

• Failure to make the most obvious comparisons between received data sets to determine whether they were consistent with one another.

• Failure to consider the criticisms of experts in the field who pointed out problems with the draft version of the report.

• And, I suspect, denial of these problems due to bureaucratic momentum to publish a report.

Next

Endnotes

1) I have here omitted comments on corrections which I accepted and which are included in my analysis as republished.]

2) Rodgers's calculation of the median spedd is based on an assumption of 8 months of riding, and would be around 1.7 mph if the months of use figures from the Rodale data are taken into account.

3) This variation also occurs in my revised calculation which accounts for the months of use figure.