I threw a gross simplification into a post the other day.
?Behavior design interventions have a 10% chance of working.?
Is that the exact, precise success rate? I?ll let you know by the bottom of this post.
A behavior design intervention is something like a class, coach or book. Basically, it?s any tool that you choose to help better yourself.
For example, the Coach.me app is a collection of interventions ranging from self-coaching tools to group coaching to individual coaching. Those are all examples of interventions.
The reason I think this industry-wide success rate is important is because it tells the story of what strategy you should use in your self-improvement regimes.
If the success rate is low, like around 10%, then you need to use the try and try and try again strategy.
Because self-improvement is a competitive field, the sales pitches are all wildly over-confident.
We?re all told that success rates are 100%, which implies your strategy should be try this one thing and if it fails then you?re doomed.
The tension between honesty and effective marketing runs deep. Overconfidence produces long term problems ? you stop trying after the first failure. But under-confidence also produces problems ? why bother trying something so hard if it?s not going to work out?
When you ask if someone?s advice is good, are you asking if it?s easy to follow or if it works?
When we ran our diet study in 2014, we found that 75% of people who finished the month had lost weight.
That sounds great, right?
Here?s a less optimistic way of measuring success rates.
Imagine I propose a diet for you that involves drinking water but not eating any calories. Let?s call it the all-water diet. Thankfully, this is just a 5-day diet.
I guarantee that every single person who completes this diet will have lost weight.
I also guarantee that more than 99% of people who start the diet will fail.
That?s the problem with just answering ?will it work?? The above diet is a stupid failure ? nobody should take that advice.
Instead, we need to consider adherence.
Adherence means that you took the advice and stuck to it (most people don?t stick to the advice).
There?s some wiggle room about where you define the starting line. Generally the right starting point begins with an expression of intent. That way you aren?t measuring from ?the subway preacher said to do something and nobody did it.?
In Coach.me, we measure from when someone signed up for a goal or hired a coach. For books, I?d start with when the person bought the book.
In our diet study example, we found that 12% of people who joined the study had lost weight by the end of the month.
That 12% success rate sounds quite a bit different than the 75% number I led with. It sounds bad, right? It turns out, lots of people didn?t feel like sticking to their diet.
The study had a legit academic experimental design with randomized assignment and control groups, so that 12% number does reflect a positive experimental result. Getting assigned to a diet was definitely more effective than getting assigned to a control group.
Here?s the positive: the minor act of joining a study and filling out a 1-minute survey had a 12% chance of helping that person lose weight. That?s a pretty good return.
Plus, if you actually felt like sticking with the study, your odds went up to 75%.
My standard, dad-joke-style observation about the self-improvement industry goes like this:
I?ve got good news and I?ve got bad news. The good news is that this advice works. The bad news is that you?re going to have to get off the sofa.
There are no silver bullets and that?s fine
However, critics and skeptics are able to cherry pick the lower number to make their criticism sound hyperbolically bad.
The general public thinks popular advice works 90% of the time and then the critics make a headline saying it only works 10% of the time. It?s easy to make something great sound terrible, simply because most people?s expectations are too high.
For example, Alcoholics Anonymous (AA).
Search for ?AA success rates? and the first Google result will say both that the success rate is between 5 and 10% and also that AA is pseudo-science.
Opponents of 12-step programs jump on this 5?10% number as evidence of failure. They?ll go as far as to call the program fraudulent just because it doesn?t work 100% of the time. Never mind that the most popular in-patient alternatives are expensive (AA is free), have waiting lists (AA is universally available) and also suffer from high failure rates:
?The therapeutic community claims a 30% success rate, but they only count people who complete the program.? The other 70?80 percent have dropped out by the 3?6 month marker.
I?ll do the math on that quote for you. That?s a 6% success rate for people that walk in the door.
The above AA debate mostly comes down to a marketing battle. You don?t need to get involved in that. Instead, here?s a third framing.
The problem is the belief in silver bullets. There are no silver bullets.
If AA is presented as the only solution, than that?s a problem with the person doing the presenting. Some courts mandate AA in sentencing as if it?s guaranteed to work. That?s a problem.
This AA example shows something that?s true everywhere, from Cross Fit to Intermittent Fasting to GTD. All of these are both great and also miles away from being guaranteed magic success.
Picking yourself up to try again
So, what?s the actual industry-wide success rate? Buster Benson told me once that he thought 15% was the upper limit for a goal tracking tool like Coach.me. At our peak, we got to 14.5%.
The success rate of someone with a personal habit coach is much higher, approaching 45% with the top coaches.
Any scientist will tell you that those aren?t apples-to-apples comparisons. Only the most motivated people in our community hire coaches. So of course these people are more successful.
Instead of taking away a precise number, I would encourage everyone to view the world as if success rates are closer to 10% rather than to 100%.
In that framing of the world, you have an obvious strategy:
- Try what appeals to you. The steepest drop off on the pathway to success is just you giving up.
- If you fall off, try it again or try something new.
The world has a magical characteristic that most people don?t know about. We are judged mainly by the size of our accomplishments rather than the number of our failures.
It really barely matters how often you fail. So fail. Then pick yourself back up. Repeat.
So try to improve yourself in a 100 ways so that you succeed in 10 of those ways. You?ll be judged as 10 steps ahead of someone who never tried and never failed.
