FTC: We Will Vet Your Peer-Reviewed Studies

In a letter written in defense of its finalized consent order with Carrot Neurotechnology, Inc. (Carrot), the FTC took the position that the peer-review process may not always catch the "misreporting of underlying data" and "improper data analysis" that is noticed by the FTC during its review of claim substantiation. When coupled with FDA's focus on data integrity issues, the FTC's statements in the letter suggest that companies may want to more closely scrutinize the data underlying even published, peer-reviewed study reports before relying on the reports as support for health-related product claims.

The FTC's settlement with Carrot came in response to an earlier complaint in which the FTC alleged that Carrot's ocular-improvement claims for its interactive game application called Ultimeyes were unsubstantiated. The finalized order included a provision that requires Carrot to "secure and preserve all underlying or supporting data and documents generally accepted by experts in the field as relevant to an assessment" of any test relied upon by Carrot as support for future health-benefit claims for any product, if the test was conducted, controlled, or sponsored, in part by Carrot or its agents. While this "data collection and retention" provision has become commonplace in FTC orders since 2014, statements by the agency in the letter reflect an intended use for the data that should not be taken lightly by any consumer product company considering whether a published, peer-reviewed study provides sufficient substantiation for a health-benefit claim.

Generally, before finalizing a proposed settlement order, the FTC publishes the order for public comment. In a letter responding to several comments submitted regarding the Ultimeyes order, the FTC conveyed a notable lack of faith in the accuracy of data included in published, peer-reviewed studies that may not always be recognized by consumer product companies. In pertinent part, the FTC remarked:

[I]n evaluating substantiation for advertising claims, the Commission assesses the quality and reliability of the scientific evidence underlying the claim. In this regard, peer review, while valuable and often indicative of a well-designed and executed study, is not a guarantor of high quality. During an investigation, the Commission may obtain additional information, not available to the peer reviewers, that is relevant to an assessment of the reliability and implications of the peer-reviewed study. For instance, this additional information may identify study design flaws, misreporting of the underlying data, or improper data analysis. Moreover, results from a peer-reviewed study may support some claims, but not the specific advertising claims the Commission has challenged. (emphasis added).

The references to "misreporting of the underlying data" and "improper data analysis," in particular, forewarn companies to think twice before blindly relying on data from published peer-reviewed sources. During an inquiry into the truthfulness and accuracy of a claim, the FTC may take it upon itself to double-check the accuracy of data and analysis reflected in a published peer-reviewed study offered in support of the claim. Further, if a discrepancy is discovered, the FTC may attempt to hold the company responsible for claims made in reliance on the inaccurate study report, especially if the study was conducted, controlled, or sponsored, in part by the company or its agent.

The FTC's interest in the accuracy of the underlying data and analysis reflected in study reports is in alignment with FDA's focus on similar data integrity issues. Over the past few years, FDA has issued a series of warning letters regarding alleged data integrity issues reflected in study reports that FDA has uncovered during facility inspections (see e.g., here, here, and here and, more generally, FDA's Bioresearch Monitoring Program). When reviewed in context, the FTC's focus on data integrity may reflect a convergence between what the FDA and FTC consider to be sufficient quality control measures for the data and analysis underlying study reports. With this in mind, companies may want to take advantage of the opportunity afforded by the agencies' publicized enforcement activity related to data integrity concerns to review (and strengthen) internal protocols for evaluating studies before they are relied upon to support health benefit-claims for consumer products.

© Arnold & Porter Kaye Scholer LLP 2016 All Rights Reserved. This blog post is intended to be a general summary of the law and does not constitute legal advice. You should consult with counsel to determine applicable legal requirements in a specific fact situation.