Controversy with Kristein

Marvin M. Kristein was an economist specialising in health care costs. In policy papers such as Wanted: smoking policies in the workplace he variously estimated the costs of smoking to the US economy as $36 billion per year and “$16 plus zillion a year” (this latter in testimony to Congress). His paper 40 Years of US Cigarette Smoking purported to refute the conclusion of Burch’s Smoking and Mortality in England and Wales by a computer-assisted calculation. An Acknowledgement, in very small print at the end of Kristein’s paper, informs us that the computer calculations were performed by L. Mihalko. A controversy followed, published in three issues of the Journal of Chronic Disease.

The controversy largely concerned what calculation Kristein had actually performed and what he claimed to have done and proved.

Smoking and Mortality in England and Wales studied trends in the overall death rate regardless of cause (a strategy deliberately chosen to eliminate the issue of misdiagnosis). Kristein’s paper studies US data on yearly death rates broken down into two broad categories, respiratory cancer and heart disease, in relation to smoking rates so many years previously, using two different measures of smoking. The output of the calculations reported in the Results section can be summarised in a table:

These are the results of an operation which Kristein describes as “correlation” and “regression”, presumably simple linear regression and the associated measure of goodness of fit, R². Kristein seems to have performed it for all the permutations of time lag and measure of smoking, presenting those results which gave the highest values of R² and omitting the rest. According to Burch

The paper originally submitted by Kristein and on which the Editors first invited my comments contained none of the data used by him in his correlation and regression analyses. The selected results quoted by Kristein did not inspire confidence and accordingly I requested, via the Editors, a copy of the data used by him. These were duly provided and they now appear in Kristein’s Table: I find their derivation obscure and highly unsatisfactory.

On Kristein’s account, the calculation was performed “with various leads and lags” and the results presented are those with the most significant values of R².

So was the calculation simple linear regression? And did Kristein follow the rules given in textbooks for calculating it, one of which is stated by Burch: “the scatter of points around the regression line should be random, with an approximately normal distribution”. If this criterion is not satisfied, the calculation is invalid. Burch:

I had suggested that scatter diagrams would be helpful, a viewpoint rejected by Kristein: “I don’t believe that ‘scatter diagrams’ would contribute to the statistical findings, but Professor Burch may attempt it with the heart attack data, if he wishes.” The permission is gratefully accepted and assumed to extend to the data for lung cancer.

Those who know what the issue is here will find the figures in Burch’s Commentary and his dry comments on them very entertaining. On Figure 1: “… a tenuous argument might be advanced, and Kristein advances it”. On Figure 2: “ Kristein omits the units but I presume “year” is intended”. On Figure 3: “most of us, I suspect, have seen more convincing examples of linearity”. (The figures display the scatter which Kristein should have examined and did not, with the data points linked in order of time, a technique Burch developed during this period.) Burch’s deadliest point is this

In the U.S., the mortality ratio for a smoking rate by men and women of 20 cigarettes per day (7300 per year) is in the region of 10 or lower, but from Kristein’s regression equation it is 83.2

A woolly, structureless Author’s Reply from Kristein followed: “I propose first to discuss… Therefore one may state that… One would have to argue that…” The most important passage is:

Unfortunately, few time series give us random observations. I have studied the autocorrelation in my analysis. I believe that, in this case, the regression analysis remains interesting and valid.

I do not understand Professor Burch’s calculation of the implied relative risk of respiratory cancer of smoker to non-smoker. The regression equation states that for every cigarette per capita (smokers plus non-smokers together) consumed per year 20 years previously, the annual age-adjusted mortality rate for respiratory cancers in the U.S. rose on average by 0.0076… we are speaking of incremental respiratory cancer mortality rates.

It is true that successive data points in a time series such as Kristein’s table are seldom independent of each other. That is what autocorrelation means and why simple linear regression can be the wrong approach to them. As for the figure of 0.0076, Kristein now claims that it is the year on year increase in lung cancer rather than the ratio of lung cancer deaths to cigarettes smoked, as implied in his original paper, where it appears to be the slope of a regression line.

Burch and Kristein were both tenacious men, and Burch fought back with Cigarette smoking and mortality rates. After a paragraph showing that Kristein had misunderstood the entire thesis of Smoking and Mortality in England and Wales, he returns to the question of what Kristein’s calculation actually was (for completeness, 55.48 is 7300 · 0.0076).

I had not supposed that my derivation would present any difficult but, evidently, I owe Kristein an explanation.

According to the above equation, respiratory cancer mortality (RCM) in non-smokers, for whom X = 0 is 0.6746 – the units omitted by Kristein being, presumably, deaths per 10⁵ (age adjusted?) per year. (I shall not dwell on the advisablity of Kristein’s use of four significant figures in his equation.) For smokers of 20 cigarettes a day, X = 7300 cigarettes per years and RCM = 0.6746 + 55.48 = 56.15. Hence, according to Kristein, the mortality ratio for smokers of 20 cigarettes per day vs nonsmokers is 56.15 / 0.6746 = 83.2

Kristein now states in his Reply that he misled me “…by writing, respiratory cancer = …” and that he is “…speaking of incremental respiratory cancer mortality rates.” We may readily check what he (or, more strictly, L. Mihalko) actually did by regressing respiratory cancer mortality rates, as given in his Table I, against cigarette consumption 20 years earlier (X cigarettes per person per year) also as given in his Table 1. Quoting the same number of significant figures I find, as did Mihalko, RCM = 0.6746 + 0.0076 X.

At four significant figures, it defies belief that the two results derive from two different calculations. Mihalko had performed a simple linear regression and Kristein did not know what he was talking about. Burch finished up:

Kristein seeks to influence public policy makers. When public policy seeks to restrict the freedom of the individual we must hope that it will be based on an informed and rigorous scientific analysis of the available evidence.

If Kristein’s intellectual standards had been the same as Burch’s, matters would have rested there, but they were not. In a Response, he wrote that

Burch appears to see his mission as one of exonerating smoking as a major causal and exacerbating factor in heart diseases and respiratory cancers. He is able to do this in 1984, the 20th anniversary of the U.S. Surgeon General’s Report, by a series of sophisticated abuses of scholarship. His techniques include: making irrelevant points, misusing and distorting references, ignoring logic and evidence, dodging criticisms, misusing data, concentrating on minor (confounding) points, using “hit and run” tactics, using “straw man” arguments, and insisting that anything short of absolute (statistical) perfection is valueless.

Burch left it at that, but Richard Hickey submitted a letter accusing Kristein of resorting to personal attack – quite apart from the above, Kristein had compared Burch to Gilbert and Sullivan and Monty Python – and rehearsing the case against the Surgeon General’s methodology. He contrasted that methodology with Berkson’s invocation of Bertrand Russell: “Morally, a philosopher who uses his professional competence for anything except a disinterested search for truth is guilty of a kind of treachery.”

In a Response to Hickey, Kristein claimed to be motivated by the same lofty spirit, “viz. using professional competence except for finding the truth is a kind of treachery. Also, the Talmud tells us that consciously teaching an error is a sin.” This is interesting. Judaism is indeed a scholarly religion. It also attaches huge importance to the preservation of life, and gives considerable weight to the judgement of a doctor on medical matters. Can that be the reason why Kristein regards the pronouncements of the Surgeon General as unchallengeable and unarguable? There were, of course, Jews on both sides of the debate on smoking and health.

Sources

A Half-life of Burch