In Defense of Anecdotal Evidence

During the last two decades traditional retail business has experienced a disruption similar to an earthquake delivered by the proliferation of ecommerce. That earthquake caused tsunami-like floods of online customer reviews describing personal experiences with specific products. Those retailers, who embraced this wave of untamed customer feedback, surfed it to higher “visit to conversion rates”, growth and profitability.   The way I feel isToday millions of customers share their experiences online about a wide variety of products and services, both personal and business related. Based on multiple studies, the trust other consumers give to these reviews is increasing from year to year.   While the flood of experiential information provided by customers and its influence continues to grow, many marketing researchers still question its value to business. To be fair, there were some interesting studies conducted that found correlations between the quantitative aspect (star rating) of customer reviews and the restaurants’ revenue. However, qualitative research of the actual reviews is being sneered upon and labeled “anecdotal”.

“The expression anecdotal evidence refers to evidence from anecdotes. Because of the small sample, there is a larger chance that it may be unreliable due to cherry-picked or otherwise non-representative samples of typical cases. Anecdotal evidence is considered dubious support of a claim; it is accepted only in lieu of more solid evidence. This is true regardless of the veracity of individual claims.”  The underscore is mine.

Interestingly, the above quote comes from Wikipedia, that itself has been attacked by status quo defenders as “inaccurate”. Yet, this quote is the best definition I could find online, after checking more “official” sources like Oxford and Merriam-Webster.   Since Customer Experience is a perception, there is no more meaningful evidence to communicate it than an anecdote. Based on the definition, two primary reasons for not using it to form strategic decisions are the size and quality of the samples in terms of representation. When it comes to customer reviews, the available volume (sample size) often exceeds the size of samples collected by most quantitative marketing research projects. Mining of these anecdotes produces very meaningful insights with a real business return on investment that quantitative methods are not capable to discover. Such techniques allow:

  • discovery of patterns and trends within the multitude of “anecdotes”,
  • measurement of their relative importance to customers,
  • measurement of the sentiments associated with these patterns.

The cross-sectional representation of these findings may subsequently be validated via traditional quantitative methods.   The internet democratized many aspects in our lives and not everyone likes it. The selection of sampling strategies for research used to be the prerogative of professional researchers, who often act like high priests of the illusive cross-sectional representation probability standards. In reality, very few of them actually practice any probability sampling methods beyond relatively basic demographics. The proliferation of inexpensive online survey tools enable people, without special training, to conduct marketing research. Most marketing executives, the recipients of this research, have neither the background to venerate these methods nor have experienced a measurable advantage using them. On the other hand, customer reviews often can be subjected to sampling based on gender, geography, age, time published, etc. to improve probability of more full representation of the customer base.   Those who continue to belittle a value of untamed customer feedback to business will fall victim of their own elitism and become even less relevant than they are now. Change before you have to.

This entry was posted in Customer Experience CX, Customer Reviews Analytics, Market Intelligence, Social Media Research and tagged , , , , . Bookmark the permalink.

2 Responses to In Defense of Anecdotal Evidence

  1. Pingback: 3 Critical Shortfalls of Surveying Customers | Customer Experience IQ Blog

  2. Pingback: 3 Critical Shortfalls Of Surveying Customers | Business Intelligence Info

Leave a Reply

Your email address will not be published. Required fields are marked *