Bad Science - Good Publicity

That is how Dr. Barry Sears characterized the study published in JAMA yesterday in the Atlanta Journal Constitution article, Atkins Diet Vindicated? Well Maybe.

In the research paper detailing the A to Z Weight Loss Study, his dietary approach (Zone) went head-to-head against Atkins, LEARN and Ornish and had the least influence on weight.

In the AP article, found on MSN, he said the "study had a good concept and incredibly pathetic execution."

Sour grapes?

Let's see what others thought of the same study.

Dr. Dean Ornish has an entire column in Newsweek - Why I Disagree with New Diet Study - where he voiced his concern that "many people may go on a diet that harms them based on inaccurate information;" and complained, as Dr. Sears did in numerous quotes, that study subjects assigned the Ornish diet weren't following the Ornish diet.

In the Forbes article about the study, Dr. Ornish summed up his belief that those on Atkins did better because "[i]t's a lot easier to follow a diet that tells you to eat bacon and brie than to eat predominantly fruits and vegetables."

Two diets, two authors, two not-so-happy campers.

What did Yale University food policy researcher and creator of the LEARN diet, Kelly Brownell, have to say? The study "shows that nothing works very well, [...] it just screams out for the need to prevent obesity."

One thing I do totally agree with - the study was a good concept...but it was poorly executed (Dr. Sears' main gripe).

That was one reason why, yesterday, I decided to forego detailing the data - it wasn't earth-shattering, it wasn't all that exciting and it really only proved - pardon my French - when you do something half-assed you get half-assed results.

I just could not get excited about it. The mouse study - A High Fat, Ketogenic Diet, Induces a Unique Metabolic State in Mice - in the American Journal of Physiology, Endocrine Metabolism; now that was exciting!

But I digress..

While Sears, Ornish and Brownell are all pointing out the lack of adherence to their respective plans, the same can be said of those assigned the Atkins diet; they didn't exactly follow the diet, as written, either.

In fact, at the two-month mark they were already consuming what is known as "pre-maintenance" or "maintenance" level carbohydrate - more than 60g per day on average.

Keep in mind this was during the initial eight week period that was designed to provide intensive support and teach participants about their assigned diets.

Those assigned the Atkins diet were not the only ones who failed to comply with the defined limits of their dietary approach - none of the groups seemed able to follow their diets correctly in the first eight weeks. Those assigned Ornish were to reduce dietary fat to 10% of calories - they ate 20%; those assigned the Zone were to eat 40:30:30 (carbohdyrate:fat:protein) and ate 42:35:23; and those assigned the LEARN diet were to eat what is similar to the Food Pyramid, around 55/60:30:10/15 and ate 50:30:20.

The question then must be asked - what happened in this study that subjects weren't able to follow the basic guidelines of their diets during the period of intense teaching and support? It's our tax dollars - this was an NIH funded study - so, how did a $2-million trial fail to achieve measurable compliance?

Forget about over the long-term, this study failed to achieve compliance out of the gate!

We have to ask, what went wrong?

If don't ask this question and probe the details, we'll just continue to waste money and get nowhere in answering the billion-dollar question - how do we help people lose weight and keep it off?

First let's look at the issue of compliance to an assigned diet protocol in other studies to see if this study is just one more with poor adherence or one more with a problem to resolve to achieve compliance in future studies.

If we look at other published studies for each of the above dietary approaches we find:

ORNISH

In the American Journal of Cardiology, the study - Comparison of coronary risk factors and quality of life in coronary artery disease patients with versus without diabetes mellitus - was published in May 2006. This trial included intensive support from the start to enable patients to modify lifestyle, diet, activity and stress management. Compliance was reported as good with the subjects "able to adhere to the recommended lifestyle."

ZONE - SEARS

In the American Journal of Clinical Nutrition, the study - Ketogenic low-carbohydrate diets have no metabolic advantage over nonketogenic low-carbohydrate diets - was published in May 2006. In the six-week trial we find good food intake was strictly controlled, so good compliance was built into the protocol.

LEARN (FOOD PYRAMID TYPE DIET 55/60:30:10/15)

In JAMA, the study, Low-Fat Dietary Pattern and Risk of Cardiovascular Disease - was published in February 2006. In the seven year trial we find the one year dietary data provided. In addition, we learn from the Dietary Modification Trial data, published in January 2006 (JAMA) that subjects in the intervention group received intensive support to learn their new diet over the first year and the first year data reported shows those in the intervention group consumed, on average, 24.3% of calories from dietary fat, 8.1% of calories from saturated fat, 58.3% of calories from carbohydrate, and the remaining calories - 17.7% from protein.

ATKINS

In the Annals of Internal Medicine, the study - A Low-Carbohydrate Ketogenic Diet versus a Low-Fat Diet to Treat Obesity and Hyperlipidemia - was published in May 2004. Both groups received printed materials and instruction from registered dietitians about their assigned diet protocol. The researchers reported that at the end of six months, the low-carbohydrate group consumed 29.5g of carbohydrate on average and the low-fat group (protocol less than 30% of diet from fat with a calorie deficiet of 500-1000 calories a day) averaged 29% of their calories from dietary fat and an average calorie intake of 1502-calories a day, a level within the desired deficit.

With the above studies, and many others with similar good compliance, one thing is abundantly clear - those tasked with instructing participants about their assigned diets knew the rules of the dietary approach well and were able to communicate, clearly, how to implement the necessary dietary changes to participants in the studies.

So what went wrong with this one?

As much as I hate to say it - if you ask me, it comes down to, as Dr. Sears said "pathetic execution" of the study concept because of a poorly designed protocol.

The researchers recruited women, aged 25 to 50, who had a BMI between 27 and 40, and who wanted to lose weight. This is good because the researchers started with a motivated population of participants. Motivation and desire to lose weight among study participants is a definite plus at the get-go of a study.

Then the researchers, following the gold-standard randomization protocol, blindly randomized participants into one of four groups, each assigned a popular diet book to use - Dr Atkins' New Diet Revolution, Enter the Zone, The LEARN Manual for Weight Management, or Eat More, Weigh Less by Ornish.

This is where I think things started to "go wrong" - those recruited into the study merely wanted to lose weight. Exclusions from the study included those whose weight was not stable over the last two months before the trial (already potentially dieting), those whose medications had changed in the last three months, and those who "self-reported hypertension (except for those whose blood pressure was stable using antihypertension medications); type 1 or 2 diabetes mellitus; heart, renal, or liver disease; cancer or active neoplasms; hyperthyroidism unless treated and under control; any medication use known to affect weight/energy expenditure; alcohol intake of at least 3 drinks/d; or pregnancy, lactation, no menstrual period in the previous 12 months, or plans to become pregnant within the next year."

Unlike other randomized trials with good compliance, these subjects did not have the additional motivation to reverse or improve a medical condition that would over-ride any doubts about a particular dietary pattern assigned to them.

In fact, if we examine the literature, we find when weight loss is the primary objective, compliance within randomly assigned diets is much lower than when subjects choose their diet or have the additional motivation to reverse or improve a medical condition.

We need to learn from this and not chalk it up as one more example that people can't follow a diet.

It's clear to me that blind randomization of subjects, in dietary modification trials, is less effective than allowing subjects to choose their diet if weight loss is the primary objective without specific disease management as a secondary objective.

Next we find in the A to Z Weight Loss Trial that participants did receive some instruction; specifically A) they were provided the book for their assigned diet, B) attanded a 1-hour classes led by a registered dietitian once per week for 8 weeks, and C) each class covered approximately one eighth of their respective books per class.

Curiously, "[t]he same dietitian taught all classes to all groups in all 4 cohorts."

I don't know about you, but I know me...and there is NO WAY I could objectively present and teach four vastly different dietary approaches without some bias seeping into my advice. It wouldn't be intentional either - I could definitely read and regurgitate Ornish's very low-fat, near vegan diet to an audience; I could definitely discuss how to plan a menu, shop and prepare dishes; and I could offer up meaningful suggestions about making good choices.

But....and here is the "but"....what participants would easily pick up on throughout my presentations is my lack of enthusiasm for the dietary approach. They'd also quickly realize I have no appreciable experience myself with eating that way or practical advice since I have not "been there, done that" but expect them to do it.

In this study, the same dietitian was tasked with and expected to provide expert instruction on each diet to each cohort - it's obvious from the abyssimal compliance at two months this approach to instuction about each diet did not work well.

We need to learn from this and not chalk it up as one more example that people can't follow a diet.

It's clear to me, from this study's method of instruction compared with other studies with better compliance, that it is not just knowledge of a diet that is necessary to teach someone how to follow a diet, but actual expertise of the dietary approach is critically important.

The person tasked with instructing a cohort must know not only the diet rules, but also how to plan menus, cook, shop, dine out, overcome objections from family and friends, and have practical advice to overcome obstacles that come along with any diet. Ideally they themself also eat a diet similar to that being taught and therefore have the "been there, done that" enthusiasm that does motivate someone that they can do it too.

After that intial eight weeks of instuction through group classes, we find the only additional communication was to motivate people to show up for follow-up - this communication with participants included telephone calls and emails reminding them of their follow-up appointments and monetary incentives to make their follow-up appointments. From week nine to the end of month twelve, there does not appear to be any additional support by way of classes, group sessions, or consultation to answer questions or provide advice.

We need to learn from this and not chalk it up as one more example that people can't follow a diet.

It's clear to me, from this study's method of follow-up support - or should I say lack thereof - that if we look at studies with good long term compliance, support over the long term goes hand-in-hand with choosing a diet you think you can do and expert instruction about how to do it.

Ask almost anyone who has followed a carbohydrate restricted diet for a long period if they did it without any support and they'll tell you support was critical along the way - whether by online forum, having access to someone expert in the diet rules or doing the diet with a spouse or friend - support got them through frustrations along the way as they learned their new diet.

Each of us, in some small way, through our taxes, paid for this study - a study that cost us $2-million dollars.

A study that tells us that you can lose weight in any diet you're told to follow for a year. Heck, it even tells us you can lose weight if you totally miss the boat with regard to how you're supposed to eat in the assigned diet as long as you don't continue to eat as you did in your habitual diet (baseline diet).

The media is trumpeting this as proof Atkins is better. Sorry, but this wasn't Atkins. It also was not Ornish, Zone or LEARN for that matter.

However, even with this criticism, I think it's safe to say that the study did meet it's primary objective to "test whether any of the 4 diets, representing a spectrum of carbohydrate intake, was more effective than any other in 12-month weight loss."

The carbohydrate intake of those assigned the Atkins diet - even though they did not reduce carbohydrate to Atkins level of intake - did in fact consume significantly less carbohydrate than the other groups while consuming a similar restricted level (without being told to do so) of calories. At each point in time, the Atkins cohort ate less carbohydrate, that correlates with greater weight loss observed at each data collection point. At the period between two months into the diet and one year, the Atkins cohort more than doubled their intake of carbohydrate from 61g at two months to 138g at one year. Even with this higher than encouraged level of carbohydrate, they lost weight - more weight than the other groups consuming more carbohydrate even with similar calorie intakes.

So before we write this one off as one more example of diets not working in the long-term, let's remember the participants did not have an opportunity to review each diet and choose one they felt they could adopt for the long term.

Let's also be aware that while the participants did recieve some instruction about how to follow their assigned diets, it's also fair to say the instruction was probably adequate, but certainly not from the perspective of "domain expertise" that may be critical to learning a new dietary approach in the long-term.

Lastly, let's not lose sight of the fact we'll continue to fail in our quest to find a method of weight loss and health risk improvement if we do not seriously begin to evaluate and understand what leads to good complicance or poor compliance.

Before we spend millions more on poorly executed diet studies, let's seriously evaluate the data and develop a set of best practices to help researchers avoid the pitfalls that are an inherent part of changing someone's diet.

People need motivation and a desire to lose weight.

People need to know their options, have information and make an educated decision about the type of diet they feel they'll do well with based on their dietary preferences and past experiences with different diets.

People need support to get started on a new dietary approach.

People need quality, expert instruction to guide them as they initiate a dietary modification, ideally from someone who is an enthusiastic expert.

People need support throughout the first year - support from experts and from others in the same boat as they are; online support forums, group face-to-face sessions, telephone access and personal consultations all contribute to success and must be considered as a critical component for compliance in dietary trials.

We have before us is a study that really does indicate carbohydrate restriction can work well over a period of one year. Without sub-group analysis to evaluate results tied to compliance (hey, some of the participants had to be doing the various diet right, dontcha think?) we can't know just how effective doing Atkins or any of the diets is with good compliance though since the researchers didn't take their data to that level of analysis in this paper.

What this study cannot tell us is how much better the outcome could have been - for any of the four diets in the trial - if participants had a chance to really have solid instruction and support.

Maybe next time we (taxpayers) fund a diet study, we'll include in the millions spent enough to include important and critical elements for success in the long-term?