Mid-2010s to the present: Efforts to control how scientific analysis and peer review are conducted
Presently, the proposed reforms of regulatory science aim to change the nature of the scientific deliberations and underlying record itself. They target the agency’s initial scientific analysis and synthesis and prescribe substantial constraints on how this literature review and synthesis must be done. They also alter the composition of the science advisory boards that review these staff analyses. Yet, despite reaching deep down into how scientists assess the available literature, these proposed reforms do not emerge from the scientific community. They are proposed by congressmen and political appointees in the agencies and crafted largely without input or advice of science advisers and mainstream scientific organizations (3).
For example, proposed reforms in Congress and the agencies prohibit agency scientists from including studies in their synthesis of the literature unless the “dose response data and models…are publicly available in a manner sufficient for independent validation” (1). Under the EPA’s proposed version of this directive, any exceptions to this transparency requirement must be made by the EPA administrator. Exceptions are narrow and expressly limited to when the disclosure of data is infeasible because of “privacy, confidentiality, confidential business information, and… national and homeland security” (1).
H.R. 1430, legislation that passed the House without amendment in 2017, would establish even more far-reaching restrictions on the nature of information that the agency can consider. Under the terms of the bill, at least the following items underlying “all scientific and technical information” must be “publicly available online in a manner that is sufficient for independent analysis and substantial reproduction of research results” before that information can be used in the agency’s scientific analysis supporting a decision: “(i) materials, data, and associated protocols necessary to understand, assess, and extend conclusions; (ii) computer codes and models involved in the creation and analysis of such information; (iii) recorded factual materials; and (iv) detailed descriptions of how to access and use such information.”
These proposed legal standards would apply to all research used to inform a regulatory decision. They even extend to research conducted before the standards were put in place, apparently irrespective of whether compliance with the standards is feasible or even technically possible. The standards if passed as laws would inevitably be enforced by the courts. Lawyers and judges would use these legislated standards to determine what information is needed to allow a study to be capable of validation or replication. As laws, moreover, these legal pronouncements governing scientific deliberations would remain binding and enforceable until they are amended by Congress or formally revised by the appropriate agency.
Changes are also being made to how agency analyses are peer reviewed. Historically, when agencies establish peer-review panels, the composition of reviewers remained flexible and endeavored to enlist the nation’s top experts. At the same time, agencies required reviewers to disclose potential conflicts of interest to advance the goals of transparency and balance (4).
By contrast, a 2017 EPA directive by Administrator Pruitt (2) and H.R. 1431, a second bill passed by the House without amendment in 2017, both impose a flat prohibition on the nature of the experts who can serve as peer reviewers of the EPA’s analyses. EPA’s directive, for example, decrees that “no member of a federal advisory committee [may] currently receive EPA grants, either as principal investigator or co-investigator” (2). In this rule, industry experts with a stake in the proceeding are not excluded from serving on matters in which they or their employers have a financial interest; only researchers holding EPA grants (regardless of the size of the grant) are excluded (2). To our knowledge, there is no precedent for such a unilateral exclusion of federal grantees as peer reviewers in either existing regulatory practice or in the practices of scientific publishers or federal granting agencies. Since EPA issued the directive, at least a few respected scientists have been removed from EPA science advisory boards because they were not willing to abandon their EPA-funded research.
The impacts of the proposed reforms on scientific analysis and the newly enacted rules on peer review are not trivial. For example, if in the wake of EPA’s proposed transparency rule, EPA considers a study in its analysis for reasons the scientific community might generally view as meritorious, but the data are not available to the satisfaction of the law, the scientific analysis cannot be used unless the administrator explicitly exempts it (1). And if the top researcher in the country is tapped to help review a staff analysis, but that researcher has an EPA grant, he or she is legally prohibited from doing so.
These initiatives would, in a legally enforceable manner, constrain agencies in determining the best science to fulfill their statutory mandates. They would also limit the ability of scientific staff to use scientific judgment in individual cases; to adopt science innovations that conflict with these proposed legal rules; or to work more generally with the global community of scientists. As such, the proposed legal rules would substantially alter the terms of the open-ended scientific deliberations running through the history of regulatory science. It is important to remember that throughout this history, the agencies were required to explain and disclose the scientific basis of their action as a matter of law.
Moreover, the new reforms introduce legalized requirements for scientific terms, like transparency and reproducibility, that currently are at best in flux and at worst still largely undefined within the scientific community (3). Even proposals within science that sound similar to the proposed reforms are actually very different. In 2014 for example, a committee of academicians, scientific publishers, and funding agencies developed principles to encourage sharing of data, code, research methods and materials, and replication of studies (12). But these principles apply only to encourage better practices for research—they do not suggest that synthesis of the literature, including past studies, should be limited to research that meets these criteria (13). Moreover, even in the research context, this committee recognized that these principles were not universally applicable to all science investigations and described them as aspirational, providing flexibility on how they would be implemented.