NAD Says Online Reviews Don’t Support ​“Most Recommended” Claim

Last week, in a case of first impression, the NAD determined that reviews collected from retailer websites were insufficiently representative or reliable to support a broad ​“America’s Most Recommended” claim. The claim was made by Euro-Pro, who advertised that its Shark vacuum was ​“America’s Most Recommended Vacuum.” The ads disclosed that the claim was based on the ​“percentage of consumer recommendations for upright vacuums on major national retailer websites.” Euro-Pro argued that its vacuum had a statistically significant higher percentage of ​“would recommend” responses than any of the other brands.

Even though the claim was based on over 10,000 reviews and the results were statistically significant, the NAD was concerned that Euro-Pro’s methodology lacked some of the controls that are typically present in surveys. For example, the NAD questioned whether the survey sample was representative of population as a whole. (Although the majority of vacuums are sold in brick-and-mortar stores, the majority of the reviews in Euro-Pro’s sample were from online-only stores.) Moreover, each website worded the ​“recommendation” question differently, making it difficult to make clear comparisons. And the questions were worded in a manner that cast further doubt on the reliability of the data.

The NAD noted that, in light of the wealth of consumer reviews available online, it is not surprising that advertisers want to use this data as a basis for ad claims. However, while the NAD ​“is open to advertisers using new technology and information to support their claims,” it stressed that ​“the standards of truthfulness, reliability, and representativeness to which advertiser’s substantiation is held remain the same.” It may not always be easy to meet those standards with crowd-sourced data.