Net Promoter Scores and Sampling Bias

Originally posted as a Twitter thread on November 06, 2019


Net Promoter Score (NPS) is, when used properly, a great metric for many businesses. But many startups are trying to tout high scores, not *accurate scores* which is both dangerous and intellectually dishonest.

The classic question is “On a scale of 0 to 10, how likely are you to recommend this company’s product or service to a friend or a colleague?” — so if you have no organic traffic but a high NPS, something doesn’t jive / one of those measurements is wrong

Many companies only (whether by accident or on purpose!) survey their satisfied customers, those who made it through a giant funnel…a form of denominator dishonesty that makes it seem like things are rosy when they are not.

10% NPS might sound terrible, but it could mean you surveyed 10 customers and got nine 8s and one 10. NPS *should* be both a leading and lagging indicator of retention and organic growth, but it loses all meaning when it’s gamed to look rosy. Survey everyone! Face the data!