Thursday, July 31, 2008

New Research Article Concludes that Smoking Ban in Scotland Caused a 17% Reduction in Acute Coronary Events; Comparing Apples to Oranges

A study published in the current issue of the New England Journal of Medicine reports that the smoking ban in Scotland resulted in a 17% decline in hospital admissions for acute coronary syndrome (including myocardial infarctions [heart attacks] and unstable angina) (see : Pell JP et al. Smoke-free legislation and hospitalizations for acute coronary syndrome. N Engl J Med 2008; 359:482-491).

The study compared the number of admissions for acute coronary syndrome in nine hospitals in Scotland (representing 63% of admissions for acute coronary syndrome in the country) during the 10-month period prior to the smoking ban and the corresponding 10-month period the following year. The number of admissions declined from 3235 to 2684, a drop of 17%.

This 17% drop was compared to the trend in overall hospital admissions in all of Scotland during the preceding 10 years. According to the study, "the trend during the 10 years before legislation was a 3% mean annual reduction, with a maximum reduction of 9% in 2000."

Because the observed 17% reduction in admissions for acute coronary syndrome was much higher than the annual reduction during the 10 previous years and exceeded the highest annual decline between any two years, the study concludes that the observed reduction is attributable to the smoking ban.

The Rest of the Story

The problem with this article is that its conclusion is based on a comparison of apples to oranges. In order to compare the change in heart attacks in Scotland from 2006-2007 to the trend in heart attacks during the preceding ten-year period, one needs to use the same data source to compare these trends.

In this article, the researchers use one source of data to estimate the change in heart attacks from 2006-2007 (observed changes in admissions for nine hospitals representing a portion of the country) and a different source of data to estimate the trend in heart attacks from 1996-2006 (national data from the Scottish National Health Service).

A critical basis for the article's conclusion is that the year-to-year decline in heart attacks in Scotland never exceeded 10%, while the researchers found a 17% decline in heart attacks during the year following the smoking ban.

However, the relevant question is not what the national health service data show, but what changes in heart attack admissions would have been found using the same methods employed by the researchers to count heart attack admissions for 2006-2007. What would the annual changes have been using the same 9 hospitals and using the same method of counting heart attack admissions?

It is important to note that:

(1) The diagnosis of acute coronary syndrome in 2006-2007 was based on an assay for cardiac troponin (a component of cardiac muscle which is released into the blood following heart injury), which is a very sensitive test for cardiac injury. For the period 1996-2006, the diagnosis was likely made based on less sensitive measures, since the use of troponin to diagnose coronary syndrome has greatly increased in recent years.

(2) The random variation and secular trends in coronary syndrome for the 10-year period prior to the smoking ban are based on standardized, national data which include the entire country of Scotland. Thus, the variation is likely to be much lower than the variation in the data from a sample of just 9 hospitals.

The correct way to conduct this analysis would be to examine the trends in heart attacks in all of Scotland for the entire ten-year period using a single, standardized and consistent data source and then to examine the degree of random variation in year-to-year changes in heart attacks and see if the observed change associated with the smoking ban is inconsistent with the magnitude of observed year-to-year changes during the years preceding the smoking ban.

Fortunately, the annual data on heart attack admissions in Scotland is available online, so we can examine the magnitude of year-to-year changes in heart attacks in the past decade and see how the change associated with the smoking ban compares.

Remember that the smoking ban was implemented in March 2006, so changes from 2005 to 2006 would reflect the smoking ban, as would changes from 2006 to 2007.

Between 2005 and 2006, the number of heart attack admissions in all of Scotland declined by 4.2%. Between 2006 and 2007, the number of heart attack admissions in Scotland dropped by 8.0%.

That might sound like a big drop, large enough that we would conclude it was due to the smoking ban.

However, look at the year-to-year declines in heart attacks in Scotland in years prior to the smoking ban.

Between 2003 and 2004, heart attack admissions declined by 4.6%. This is greater than the observed heart attack decline from 2005 to 2006.

Between 1999 and 2000, heart attack admissions in Scotland declined by 10.2%. This is much greater than even the 8.0% decline observed from 2006 to 2007.

Even if we look at the 2-year decline in heart attacks from 2005 to 2007, it is about the same as the 2-year decline observed bewteen 1999 and 2001 (11.9% compared to 10.7%).

If I present the data this way, it makes it clear that the observed change in heart attacks associated with the smoking ban is not at all out of the range of normal declines in heart attacks from year to year in Scotland observed in the absence of the smoking ban.

2005-2006: -4.2% 2003-2004: -4.6%
2006-2007: -8.0% 1999-2000: -10.2%

My point here is not that these data prove there was no decline in heart attacks in Scotland attributable to the smoking ban. My point is merely that there is no way one can conclude that the observed decline in the year following the smoking ban was different from the magnitude of the declines observed in previous years.

The analysis in this paper assumes that the entire observed change in heart attacks is attributable to the smoking ban. However, it is clear that a 10.2% decline in Scotland from 1999-2000 occurred in the complete absence of a smoking ban. Clearly, there are other factors which are contributing to a decline in heart attacks, there is a secular trend of substantially declining heart attacks over time, and in fact, the magnitude of the decline associated with the smoking ban is less than the magnitude of the decline observed in some recent years preceding the smoking ban.

In other words, one cannot rule out the very plausible alternative hypothesis that the observed decline in heart attacks is explained by random variation in the data and the already existing secular trend of declining heart attacks in Scotland.

No comments: