NAD Doesn’t Trust Trustpilot Reviews

If a review site ranks your product as the top in a category, can you advertise that you’re “number 1” in that category? Not necessarily. A recent NAD decision explains why.

A competitor challenged TaxSlayer’s claim that it was “#1 Rated in the Tax Prep Software Category on Trustpilot.” NAD started its decision with a reminder that “a claim that is expressly truthful can still be misleading” if it conveys a message an advertiser can’t support. Here, NAD found that the claim suggested that the “rating was based on a reliable and representative survey of consumers using products across the entire tax preparation software category.”

NAD found that the survey did not meet this standard for a number of reasons. Here are some of the highlights:

• To support a claim that a product “number 1” within a category, an advertiser should generally compare itself to at least 85% of the relevant market. There was no evidence to suggest that happened here.
• Customers who are surveyed should be representative of the broad base of customers who use the product. NAD determined that the survey failed in this regard because Trustpilot collects reviews for programs sold by companies with whom it has a relationship at much higher rates than for those with whom it does not. Indeed, the market leader in tax preparation software – who does not have a relationship with Trustpilot – had only 15 reviews, compared to over 2,500 reviews for TaxSlayer.
• NAD was not convinced by TaxSlayer’s argument that consumers could simply visit the Trustpilot website to clarify any confusion about the ranking. “If a claim needs to be qualified to prevent it from being misleading, any disclosure should be clear and conspicuous and found within the four corners of the advertising in which the claim appears.” Consumers should not be forced to search for information.
• Trustpilot lacked various controls that are needed for a reliable survey. For example, Trustpilot could not verify that all reviews were submitted by consumers who actually purchased the products, and it did not have a mechanism to prevent consumers from submitting multiple reviews. Moreover, because Trustpilot used a proprietary system to weight reviews, NAD couldn’t fully evaluate the rankings.

After the challenge was filed, TaxSlayer changed its claim to read that it was “rated #1 in the tax prep software category on Trustpilot among companies with 3,500+ reviews.” Although this was arguably true – indeed, TaxSlayer was the only company with over 3,500 reviews – NAD found the claim to be misleading because consumers will reasonably assume that there is more than one company in that category.

Advertisers should be careful about making comparative claims based on review data from third-party sites. Even if a claim is literally true, it could be deemed misleading if an advertiser can’t prove that the site’s mechanisms for compiling and reporting reviews meets NAD’s high standards.