Frederick Reichheld's latest effort to enlighten CEOs and other business leaders is at its best mildly entertaining, but at its worst it is misleading and could result is some very costly and wrong decisions by potential users.
There are several critical weaknesses of this work-I will only mention a few.
First, there are many contradictions, reversals and logical inconsistencies throughout the book. Examples abound and can be discovered by anyone who spends a modicum of time with the book. Among the biggest is the reinterpretation of the satisfaction measure used by Enterprise Rental Car as a measure of net promoters (p.63). This is very confusing because earlier in the book the reader is led to believe that one needs to measure "recommendation" not "satisfaction" because Mr. Reichheld alleges that satisfaction is unrelated to revenue or profit growth. So why does the satisfaction measure works for Enterprise? More astounding Mr. Reichheld continually uses the Enterprise case throughout the book as justification for using the NPS measure.
Second, the entire premise of the Net Promoter approach is unsupported by third party peer-reviewed research articles in psychology, marketing research, or social science journals. All of the support provided in the book is based upon Mr. Reichheld's claims of research conducted by the firms he works with (Bain and Satmetrix) none of which has been reported in the aforementioned scientific publishing outlets. In fairness, the Net Promoter idea was originally promoted in a Harvard Business Review article, but HBR is not a research journal and its articles are not peer reviewed. Publication in HBR is somewhat equivalent to publication in Business Week or Fortune, and certainly does not qualify as scientific review.
Third, Mr. Reichheld confuses cause and effect with correlation. Recommendation is an effect not a cause. It occurs because something else (like a satisfactory experience) causes it to occur. Yet throughout the book, Mr. Reichheld continuously claims that recommendation's correlation with sales growth proves that it is a driver of growth. Correlation is simply a measure of association that says nothing about cause and effect. Consider the correlation between the number of churches in a community and beer sales. They are probably correlated but does one cause the other? More likely there is a third factor that is causing both to move together-like population growth. The same is true of the Net Promoter measure-it is likely being caused by something else-like satisfaction. Its correlation with sales growth is spurious and is not causal. If one examines the evidence provided by Mr. Reichheld in Appendix A this confusion of cause and effect is even more apparent-in every case shown, the time periods for the sales data predates the time periods when the Net Promoter Scores were collected. So what is causing what?
Fourth, the recommendation measure advocated by Mr. Reichheld is not a measure of "word-of-mouth" despite his claims to the contrary. Anyone who spends a nanosecond reading the question can see an obvious flaw in the interpretation of the measure. Reichheld's recommendation scale is basically a "unidirectional" scale-the scale is bounded by a positive (+) position (the "extremely likely" label) and a neutral (0) position (the "not at all" label) not a negative position. Nevertheless he interprets the scale as though it was actually measuring recommendation in a bi-directional manner by assuming that those who answer 0-6 are "detractors" who will spread negative "word-of-mouth" comments to others-but do they? Perhaps some of the respondents are detractors who answer at the lower end of the scale because there is nowhere else for them to answer, but it is also likely that some are truly advocates, just not extreme advocates. Mr. Reichheld claims this is a logical interpretation of what respondents mean-but is it true?
One final point concerns the claimed accuracy of the Net Promoter measure. In his classification of respondents Mr. Reichheld basically rescales an 11 point scale (0-10) into a three point scale (-1, 0, +1). By doing this the information content of the measure is reduced. The net effect of this, as any elementary statistics student can tell you, is that your confidence intervals are increased and your statistical power is reduced dramatically. This means that if the hapless reader of this book were to use the Net Promoter measure to assess the true value of their customer base they would be unable to detect any changes that would occur in an accurate way. For instance for a sample of about 750 customers, a user of the Net Promoter measure would be able to detect a %5 increase less than 10% of the time. A decision maker contemplating million dollar investments would do better by flipping a coin than relying upon a measure with these kinds of properties.
