Agreeing with the National Advertising Division (NAD), a five-member panel of the National Advertising Review Board (NARB), the appellate unit of the advertising industry’s self-regulatory system, advised Euro-Pro Operating LLC to discontinue claims that its Shark-brand vacuum cleaners are “America’s Most Recommended Vacuum” and “America’s Most Recommended Vacuum Brand.”

Competitor Dyson had challenged the claims, which appeared in television and Web site advertising. In an earlier decision, the NAD said the data relied upon by Euro-Pro – essentially an aggregation of online consumer reviews – was not sufficiently reliable for such broad claims and recommended that the claims be discontinued. Euro-Pro appealed to the NARB on the grounds that the claims included a disclaimer that they were “Based on percentage of consumer recommendations for upright vacuums on major national retailer websites through August 2013, U.S. only.”

Euro-Pro based the claim on a quarterly survey of the Web sites of online retailers (Amazon, Bed Bath & Beyond, Best Buy, Costco, Home Depot, Kohl’s, Lowes’, Sam’s Club, Sears/Kmart, Target, and Walmart) that sell upright vacuum cleaners and also solicit customer reviews. Reviews on all but three of these sites (Amazon, Costco and Target) included a question asking whether the reviewer would recommend the product. After collecting and amalgamating the data – specifically the answers given by consumers regarding whether they would recommend the product under review – Euro-Pro then tallied and compared the percentage of recommendations for various brands.

But the panel agreed with the NAD that the analysis was not based on data from a representative sample of American vacuum cleaner consumers. The evidence showed that “the vast majority of vacuum cleaners – at least 84% – are purchased in brick and mortar stores” and the online reviews primarily reflected consumers who purchased vacuum cleaners online.

The sample was even further flawed because the three significant online retailers representing nearly 50% of all reviews considered by Euro-Pro – Amazon, Target, and Costco – were not included in Euro-Pro’s analysis of consumer recommendations, since those Web sites did not ask consumers whether they recommended the product being reviewed.

The panel also found that the advertiser’s analysis was not sufficiently reliable to support the claim since the questions were not the same on each of the sites, and did not provide consumers with a “don’t know” option, the questions all had “yes” as the first answer (i.e., the “yes” and “no” choices were not rotated), which increased the likelihood of bias, and the different sites had inconsistent policies as to how long consumer reviews were displayed, which raised the possibility that outdated models were reviewed.

To read the NARB’s press release about the decision, click here.

Why it matters: While Euro-Pro failed to substantiate the claims at issue, the NARB highlighted the fact that “traditional consumer surveys are not necessarily the only way to support ‘most recommended’ claims.” With more consumers turning to online reviews, the panel noted that its
“. . .decision is not intended to preclude the possibility that web-based consumer review data can be aggregated across websites in support of advertising claims.” In its response statement, Euro-Pro emphasized that the “prevalence of online reviews has changed the way consumers share and process information about products and services,” adding that the company intends to “develop an appropriate alternative way to communicate this important information to vacuum consumers.”