If you want to advertise that your product performs better than a competitor’s product, you’re likely going to have to run tests to substantiate that claim. In some cases, there may be industry standard tests that could help take the guess work out of designing a protocol. In absence of an industry standard test, though, you will need to come up with your own protocol. A recent NAD case involving teeth whitening claims demonstrates just how difficult that can be.

SmileDirectClub (or “SDC”) advertised that its Fast-Dissolving Whitening Strips could get teeth “2X whiter” than Crest Whitestrips. To support the claim, SDC conducted a test with 155 participants who were given one of the products (without knowing which one they had), instructions for use, and a “Shade Guide” with 24 different choices to evaluate improvements in tooth color. Participants were asked to comply with package instructions, measure changes with the Shade Guide at specified intervals, upload photos of their teeth at the same intervals, and complete a questionnaire regarding their experience.

Although the protocol seems fairly detailed, Crest argued that an in-home test like the one SDC designed is not sufficient to support a claim about whiter teeth. Instead, this type of claim should be supported by “a properly designed clinical study.” Among other things, Crest argued that “tooth color should be evaluated by a blinded-to-treatment expert or objectively measured with an instrument at the designated time points and the mean changes for each treatment must be statistically significant at the time points of interest.”

NAD sided with Crest. In its decision, NAD discussed the difference between “sensory performance claims” and “objective performance claims.” Sensory claims – such as “smells better” or “stronger taste” – relate to how consumers react to a product. “For such claims, sensory testing or consumer opinion testing is appropriate.” NAD noted, however, that “when a claim is about the tangible, objective results that a consumer can expect a product to provide, more objective testing is appropriate.” In this case, “2x whiter” conveys a claim about tangible, objective results.

NAD determined that “having participants record their opinion of what shade their teeth are does not equate to objective testing because asking untrained consumers to match their tooth to a shade guide with 24 different choices is a subjective practice and does not take into account the light in the room or that consumers may see and interpret shades of white and color differently.” (NAD also found some other issues with the protocol, including that SDC didn’t use a recognized industry standard tooth whitening shade guide, and questioned whether the size of the study was large enough.)

One key lesson in this case is that “claims that convey the message that they are objectively verifiable should be substantiated with objective testing that goes beyond simply asking consumers for their opinion.” That won’t be welcome news for many advertisers because this type of testing is likely to be more complicated and expensive to run, and it’s likely require a longer lead time before a claim can get to market. But it’s important for advertisers to have an idea about what is likely to be required before they develop a test protocol.

NAD Decision Holds Lessons for Developing Test Protocols