AHRQ: Policy-based non-evidence evidence?

June 17th, 2013 by MorganDowney Leave a reply »


Summary: Preventing weight gain among adults is a national health care priority. So I read with interest a recent AHRQ effectiveness review which found scant evidence of any strategy to prevent weight gain working. AHRQ went on to conclude that, “there was no evidence that not adopting a strategy to prevent weight gain is preferable.” In addition, contrary to the implication, AHRQ reviewers did not look for evidence when or if weight gain might be desirable. They seemed to put in that conclusion to buttress the Healthy People 2020 policy goals. AHRQ’s reviews are meant to inform policy; not for policy to spin research reviews. Here’s the story.

Now I concede to no one my respect for the Agency for Healthcare Quality and Research (AHRQ).  I have always felt they were a very reliable source for objective evaluations of research data, often used to drive policy. In fact their mission includes the goal to “Improve health care outcomes by encouraging the use of evidence to make informed health care decisions.”

I believe we have their literature reviews and assessments some twenty times or more on this site. When policy-makers say a recommendation is “evidence-based” they usually mean an AHRQ review of the literature supports it. And AHRQ likes it that way.

Recently AHRQ published Strategies to Prevent Weight Gain Among Adults, one of a series of Comparative Effectiveness Reviews, developed by the Johns Hopkins Evidence Based Practice Center,

This review examined 58 publications from 51 studies on maintenance of weight or prevention of weight gain among adults involving over a half-million patients. Studies targeting a combination of weight loss with weight maintenance or weight loss exclusively were considered outside the scope of the review. Studies had to have at least one year of follow-up with a weight outcome.

The results,given the billions of dollars invested by governments at all levels and by private companies and individuals, are pretty grim. Only two interventions were found to have moderate strength evidence of effectiveness. The first was a workplace intervention involving both individual diet and physical activity with environmental intervention that resulted in significant and meaningful prevention of BMI increases at one year and weight gain at 24 months compared with no intervention. The second intervention involved aerobic and resistance exercise performed at home by women with cancer compared to no intervention.

Additional analysis revealed, “Potentially effective interventions with low strength of evidence include a clinic-based program to teach heart rate monitoring, a combination intervention for mothers of young children, small group sessions to educate college women, and physical activity among individuals at risk of cardiovascular disease and diabetes. Potentially effective approaches described in observational studies having low strength of evidence include eating meals prepared at home among college graduates and less television viewing among individuals with colorectal cancer. When reported, adherence to interventions tended to be below 80% percent. There were no adverse events among the few trials that reported on adverse events. Trial study quality tended to be poor due to knowledge of the intervention by the study personnel who measured the weight of the participants or lack of reporting on this item. This lack of blinding the outcome assessor along with inclusion of studies that were not designed to prevent weight gain resulted in a low strength of evidence for the majority of comparisons.”

The authors conclude, “The literature provides some, although limited, evidence about interventions and approaches that may prevent weight gain. Although there is not strong evidence to promote a particular weight gain prevention strategy, there is no evidence that not adopting a strategy to prevent weight gain is preferable.”(Emphasis in original.)

I must admit to a certain whiplash reading that last sentence. ‘No evidence on not adopting a strategy?’  On the critical national issue of prevention of weight gain, limited evidence that anything works but “no evidence” that not doing something is better? Eh?

  1. So, my first question was, “Did they look for evidence of ‘not adopting a strategy”?

Well, yes and no. Yes: they had excluded from their review studies involving three classes of patients in whom strategies preventing weight gain are commonly not recommended: (a) pregnant women, (b) patients with wasting diseases, such as cancer, HIV/AIDs, eating disorders, and, (c)  lean persons with a BMI of 18.5 kg/m2 or less. See inclusion and exclusion criteria. So, they knew strategies to prevent weight gain were not ‘preferable’ for these rather sizeable groups. (FYI, weight loss for cancer patients is a current research issue in the cancer field, see Downey Obesity Report, May 5, 2012.

But also the answer was No. They did not look for other categories for whom prevention of weight gain might not be indicated. The description of the review states, “We aimed to compare the effectiveness, safety, and impact on the quality of life of strategies to prevent weight gain among adults. Self-management dietary, physical activity, orlistat and combinations of these strategies were considered.” The reviewers developed six Key Questions. By starting with the strategies, as opposed to asking, “For whom is prevention of weight gain contra-indicated?” they missed some key groups.

A quick search of PubMed revealed several, rather sizeable groups, such as:

Older women, one study found higher fat mass was associated with better survival.

The frail elderly, for whom weight loss may be associated with adverse outcomes. (In one study, weight loss among elderly Mexican Americans was shown to predict death. Of those who had lost 5% of weight, 28% died, compared to 19.7% whose weight had remained stable and 15.2% of those who gained weight after 5 years.) See also, The danger of weight loss in the elderly.

Patients with COPD, for whom weight loss is a significant driver of costs and mortality. Overweight and obesity is protective of mortality in patients with COPD.

Patients with Cystic Fibrosis for whom higher BMI is associated with improved lung function.

Patients with Parkinson’s Disease for whom weight loss reduces quality of life.

Nursing Home Patients for whom weight loss can be an indicator of dysphagia, depression and malnutrition.

Patients with psychiatric disorders for whom weight loss is often contraindicated because of low adherence to program protocols and because many anti-psychotic drugs cause weight gain. (However, there are studies and views to the contrary,  see http://well.blogs.nytimes.com/2013/04/15/a-battle-plan-to-lose-weight/)

Patients with ALS who have a high risk of malnutrition as well as patients with multiple sclerosis.

Patients hospitalized with pneumonia for whom obesity is associated with better short term survival.

The point is not when weight gain should be avoided or when overweight or obesity is protective. The point is that contrary to AHRQ’s conclusion that “there is no evidence that not adopting a strategy to prevent weight gain in preferable” (Emphasis in original), there is ample preliminary evidence, at least, that, in significant populations, not adopting a strategy to prevent weight gain is not only preferable, it may be protective.

  1. Is there a problem with recommending pursuit of weight prevention strategies which are likely to be futile?

Well, there are several problems on both an individual and societal level.

With repeated weight loss attempts, weight gain is likely to follow. Dr. N. John Bosonworth argued in 2012 article that, “sustained weight loss is achieved by a small percentage of those intending to lose weight. Mortality is lowest in the high-normal and overweight range. The safest body-size trajectory is stable weight with optimization of physical and metabolic fitness. With weight loss, there is evidence for lower mortality in those with obesity-related comorbidities. There is also evidence for improved health-related quality of life in obese individuals who lose weight. Weight loss in the healthy obese, however, is associated with increased mortality.” He advises weight loss only for those with obesity-related comorbidities.

Ross and Janiszewski, among others, argue that exercise, independent of weight loss, is preferable as a strategy to reduce cardiovascular disease risk. Indeed, one recent study out of Europe of over 250,000 men and women found that high levels of physical activity reduced waist circumference in both men and women, reducing the odds of becoming obese by 7% and 10% respectively.

Perhaps more importantly (to me at least), the statement that “Although there is not strong evidence to promote a particular weight gain prevention strategy, there is no evidence that not adopting a strategy is preferable,” is a tortured linguistic effort to continue to blame individuals for their personal failure to avoid weight gain, even though the recommendations of the experts have failed. This is simply a perpetuation of stigma without evidence from the federal agency whose raison d’etre is evidence-based conclusions. That such personal blame and stigmatization is harmful and counterproductive is now beyond conjecture.

(As a point of personal privilege, I have to note that I have observed over the years that, when strategies proposed by ‘experts’ in the field run aground (rather predictably I might add) the blame falls on the persons who are overweight or obese whose lack of adherence or whose hedonistic tendencies are seen as undercutting the recommendations of the professional. I cannot recall one instance where reviewer(s) concluded that the recommendations might be wrong or at least should be re-evaluated.)

For many years a chorus of critics have seen a sinister conspiracy between the government and weight loss industry. According to their view, the conspiracy promotes an unrealistic and unattainable body image goal on the public, especially on women and young women, leading to obsessive attention to fatness in the society. In turn, this can lead to dangerous or ineffective weight loss efforts, body image dissatisfaction and loss of self-esteem. (This concern receives passing acknowledgement in a single paragraph on p.120 of the review.)

Unfortunately, the conspiracy theorists might find support in this review. Who benefits, they could well ask,  from pursuit of ineffective weight prevention strategies? Not the person trying to prevent weight gain. The review already established that no particular strategy is particularly effective. So, pursuing some strategy only can mean the transfer of time but mainly money to programs, services or products to assist in the weight gain prevention effort and the diversion of such time/money away from other activities, which might be more useful to the individual or society, such as expanding one’s education, time caring for children or one’s parents, volunteering at schools, nursing homes, etc.  As we have seen with the debate over employer wellness programs, more and more employers are imposing sizeable financial penalties on workers who fail to achieve a weight maintenance target or weight loss goal which could be up to 30% of one’s individual or family health insurance premium. (See Downey Obesity Report.) So the costs of the loss or prevention activity can be substantial and the cost of failure can be substantial as well.

Finally, there is the issue of the credibility of the country’s public health authorities and confidence in their recommendations, that when they say there is “no evidence” it is fair for readers to assume they looked for evidence and found none.

I emailed one of the authors. I wrote, “On reflection, it seemed that there were a number of plausible reasons why not adopting a strategy for some individuals might be reasonable, e.g. diversion of time and money from a potentially more productive area into a less productive area for one. Likewise, on a societal level, some might argue that the attention to preventing adult weight gain diverts public and private funding into largely futile areas, contributes to anti-fat attitudes, eating disorders, etc. It may also be thought to contribute to the idea that preventing weight gain is relativity easy and failure to do so reflects a character flaw, thus diverting funding from research on prevention in adults into programs.On looking over the review, I couldn’t see where this issue was a key question or was part of the study design. So I was wondering how this conclusion was reached. Could you let me know?”

The author admitted, “We did not specifically examine whether weight gain prevention should be attempted or not. I think that the statement at the end of the report that you refer to is trying to reflect the national priorities regarding preventing weight gain set forth in Healthy People 2020 in light of the obesity epidemic. (Emphasis added.) There are potential pros and cons with supporting weight gain prevention. You have listed out several potential cons in your email, but I do think that we need to weigh them against the potential benefits such as reduction of or improved control of weight-related conditions like hypertension, diabetes, joint pain, etc. that cause substantial morbidity in our country. Since we do not have good evidence that any particular program successfully leads to weight gain prevention, we cannot really know yet whether these programs lead to benefits or harms.

Ultimately, I think the take away from the report is that we need to do more research to better understand how best to prevent weight gain, and to determine what are the benefits and harms of successful weight gain prevention programs. Such information might make it easier to understand whether this should continue to be a national priority.”

Fair enough, but that’s not what the review’s conclusion states. Supporting Healthy People 2020 is not AHRQ’s job. While we are not even half-way between Healthy People 2010 and Healthy People 2020, we know that the adult population with obesity is increasing and the adult population at normal weight is decreasing. This might be reversed but there are not any secular trends out there providing much hope.

At some point in the near future, a policy-maker (or the head of a supermarket chain) is going to point to this report as ‘evidence’ that preventing gaining weight is achievable and everyone can do it. AHRQ cannot control how their reports are used. But they need to keep straight that their evidence is meant to inform policy decisions; not that policy decisions should spin evidence reports.

Recently, Francis Collins, Director of the National Institutes of Health and Griffin Rogers, Director of the National Institute on Diabetes, and Kidney Diseases stated in an article, The next generation of obesity research: No time to waste:

Americans spend more than $60 billion annually on weight-loss programs and products, yet scant evidence exists that these expenditures translate into lasting weight loss. Given the health consequences of obesity, the United States needs rigorous data on what approaches can help achieve and maintain healthy body weights over the long term.

Indeed, research has provided—and will continue to provide—the foundation of evidence needed to confront the obesity crisis in the most effective and efficient manner. Among the many questions to address are: Why are some individuals more susceptible to obesity? Can the knowledge of biology and behavior be used to develop and better target intervention strategies? What current strategies really work? For whom? Can these approaches be scaled up?

To address this need, research must proceed swiftly on 2 parallel fronts. The first is to devise practical and effective strategies for intervention, with special emphasis on preventive strategies that can be rapidly implemented in health care and community settings.

Likewise, in 2011, Luckner, Moss and Gericke wrote, “responding to the obesity epidemic requires robust evidence to help prioritize the allocation of scarce resources to preventive interventions.”

In this critical area, AHRQ needs to stick to the rigorous evaluation of scientific studies; not engage in buttressing failing public policies.



1 comment

  1. Ricardo says:


    That is a lot of very info that you have on here. How in the world did you manage to assimilate this kind of data? I am sure you must be doing a lot of research on the subject.

    Thanks for sharing.

    Best wishes and regards.


Leave a Reply