Questioning the reliability of a study supporting claims for Philosophy, Inc.’s “Time in a Bottle Age-Defying Serum,” the National Advertising Division recommended that the statements be discontinued.

Philosophy conducted an independent, blinded clinical study of its product. Based on the results, the advertiser claimed that “Women told us their skin looked 730 days younger, that’s 2 years on your side with our age-defying serum,” “82% showed improvement in signs of aging not yet visible on the surface after 4 weeks,” and “95% showed significant reduction in visible signs of aging after 8 weeks.”

The study was conducted over a six-month period in January through July of 2013 with subjects divided into three age groups (25 to 35, 36 to 45, and 46 to 55 years of age). Philosophy had subjects complete a self-assessment questionnaire at the two-month mark and conducted skin imaging analysis on a subset of the study’s participants.

Despite the advertiser’s position that the challenged claims were fully supported by the “robust and independent” clinical study, NAD identified several problems that led it to conclude that the claims were unsubstantiated.

The “Women told us” claim was based on the responses of the 46-55 age group to a fill-in-the-blank question asking “Skin appears ___ years younger.” While recognizing that self-assessment questionnaires are valid study tools to assess consumers’ subjective evaluation of a product, Philosophy’s version was flawed, the self-regulatory body said.

In addition to being an “inherently arbitrary” question and one of “multiple and similarly worded questions about each product attribute,” the advertiser featured it as question number 21 out of 53 in the questionnaire. The ASTM Standard Guide for Sensory Claim Substantiation advises that for single product tests, the key question should be positioned first, NAD explained, and “this question is not logically situated given the numerous anti-aging related questions that precede and follow it.”

As for the other claims, the decision found multiple problems inherent in the study. It failed to account for the two weather extremes from winter to spring to summer, and the excessive dryness and excessive moisture that results, and its skin imaging analysis relied on too small a subset of participants. Just 26 out of 117 subjects were analyzed and the subset – less than one quarter of the total subjects – was “insufficiently robust to constitute a representative sampling to support the challenged claims and visuals,” NAD said.

Philosophy also failed to use trained graders to conduct the visual assessments, NAD found, noting that prior cases reviewing anti-aging claims involved visual grading of anti-aging parameters by clinicians. The actual improvements in the identified anti-aging parameters were far more modest than the claims suggest, the self-regulatory body added.

Considering all the shortcomings of the study, NAD recommended that all of the claims be discontinued.

The self-regulatory body also considered a testimonial from a study subject who said, “Lines have disappeared and … I go makeup free.” Referencing the Federal Trade Commission’s Guides for Endorsements and Testimonials, NAD again recommended discontinuation of the claim.

“Regardless of how heartfelt the testimonial of the user of the product, it is not a substitute for reliable product testing demonstrating that consumers who use the product as instructed will no longer have facial lines,” according to the decision. “NAD has already determined that the results of the [imaging analysis], which show a near elimination of lines, were unreliable because it was based on a very small subset of subjects. In addition, the advertiser failed to control for environmental factors, which could undermine the reliability of the study’s objective and subjective measurements.”

To read NAD’s press release about the decision, click here.

Why it matters: After reviewing Philosophy’s study, NAD concluded that there was not a “good fit” between the advertiser’s substantiation and the scope and nature of the claims at issue. Philosophy disagreed and indicated in the advertiser’s statement that it will appeal NAD’s findings with regard to the performance claims (not the testimonial). The advertiser said the study was “one of the most significant and extensive studies conducted in the cosmetics industry,” and that “[l]eft uncorrected, the NAD’s decision would result in confusing guidance for the cosmetic industry and disincentivize manufacturers from conducting similar comprehensive studies.”