Richard Denison, Ph.D., is a Senior Scientist.
This week I attended a workshop sponsored by the National Academy of Sciences’ Committee to Review the IRIS Process. This committee was established in response to a rider attached to an “omnibus” spending bill passed by Congress in late 2011. The committee’s charge is to “assess the scientific, technical, and process changes being implemented by the U.S. Environmental Protection Agency (EPA) for its Integrated Risk Information System (IRIS).”
EPA describes IRIS as “a human health assessment program that evaluates information on health effects that may result from exposure to environmental contaminants.” The key outputs of IRIS assessments are one or more so-called “risk values,” quantitative measures of an “acceptable” level of exposure to the chemical for each cancer and non-cancer health effect associated with the chemical. IRIS risk values are in turn used by regulators to set everything from cleanup standards at Superfund sites to limits in industrial facilities’ water discharge permits.
This week’s workshop – a detailed agenda is available here – was intended to provide expert input to the committee to inform its review of IRIS. It focused on the complex and controversial issue known as “weight of evidence” (WOE) evaluation. Here WOE refers to how EPA – in conducting an IRIS assessment of a particular chemical – selects studies, evaluates their quality, and assesses and integrates their findings, as well as how it communicates the results. At issue in particular in a WOE evaluation is how the assessor determines the relative importance – or weight – to be given to each study.
One of the many issues that came up in the discussion of WOE is how to identify and assess the “risk of bias” in individual studies – a concept borrowed from the evaluation of the reliability of clinical trials used in drug evaluations. (See this Powerpoint presentation by one of the committee’s members, Dr. Lisa Bero, which provides a nice overview of risk of bias in that setting). Evaluating a study’s risk of bias is critical for assessing its quality and in turn the weight it should be given, because bias in studies can result in significant under- or overestimates of the effects being observed.
One type of bias is so-called “funder bias.” Dr. Bero and other researchers have documented through extensive empirical research that there is a significantly increased likelihood that a study paid for by a drug manufacturer will overstate the efficacy or understate the side effects of a drug. As to studies of environmental chemicals, at the workshop and more generally, the chemical industry has pointed to adherence to Good Laboratory Practice (GLP) standards as a sufficient antidote to bias, including funder bias, a notion that has been heartily disputed by others.
But enough background. My intent here is not to fully describe the workshop discussions, but rather to provide the comments I presented during the public comment period at the end of the meeting. My comments addressed the issue of funder bias and also sought to urge the committee not to dive so deeply into the weeds in reviewing and proposing enhancements to EPA’s IRIS process that it loses sight of the need for a workable IRIS process that is able to provide in a timely manner information so critical to ensuring public health protection.
EDF comments to the National Academy of Sciences Committee to Review the IRIS Process” workshop on “weight of evidence” in chemical assessments
Good afternoon. I am Dr. Richard Denison, Senior Scientist at the Environmental Defense Fund.
I’d like to respond to a few things I’ve heard over the past two days.
First is this issue of risk of bias, which industry representatives appear not to believe is a problem – except when it comes to EPA-funded academic scientists. ACC has argued, including in its comments yesterday, that processes such as Good Laboratory Practice (GLP) have solved any possible problem and are sufficient to address any source of bias, including funder bias. It’s critical to note that GLP and related developments came about in direct response to misconduct and outright, large-scale fraud on the part of the chemical, pesticide and drug industries and their contract labs. [Wikipedia provides a good discussion of the history of GLP, the 1970s Industrial BioTest Labs scandal that led to its development, and a critique and contrast between industry-funded and academic studies.]
There is not empirical evidence to support ACC’s assertion that GLP and related measures minimize or eliminate risk of bias; contrast this with the strong empirical evidence of funder bias discussed yesterday by Drs. Robinson, MacLeod and Bero for pharmaceutical studies.
We’ve also heard calls over the last two days, especially from industry and its consultants, that we can afford to and should wait for more research, for the testing-out of all possible alternative hypotheses, for the collection of data on and a full understanding of mechanism of action, and for full consensus on how to identify, select, integrate and assess information on a chemical – all before completing an IRIS review and acting on such a review to address risk.
I would like to offer another perspective, as a public health and public interest scientist. All of these calls by industry would further delay a process that is already far too slow and inefficient. I am afraid it is a bit too easy for industry scientists to argue for such delays: They don’t live next to hazardous waste and Superfund sites; they don’t typically have to rely on contaminated groundwater for drinking water or live immediately downwind of facility smokestacks; they don’t work 8 hours a day on a factory floor; they aren’t generally subject to multiple exposures from multiple pollution sources, as are many poor people living in heavily impacted communities in this country. And they don’t typically suffer from other compounding factors such as poor nutrition, or higher disease rates due to more limited access to health care.
I could go on. My point is this: the IRIS program is not an intellectual exercise. The chemicals in line for assessment are there for a reason: people are being exposed to them even as we sit here and debate the finer points of systematic review and weight of evidence. I am not suggesting these points aren’t important, but it is essential that getting the science right is balanced with the need for timely assessments and decisions.
[UPDATE: I have modified this paragraph that could have been construed as discussing a confidential matter.] A useful charge to this committee would be, not just to critique the current IRIS process or offer recommendations for its improvement, but to design a process that would actually work, within current resource constraints and data and methodological limitations, to develop and finalize X number of credible assessments each year. This would ensure that the committee would grapple with the practicalities of effectively and efficiently carrying out an assessment process that not only is of sufficient scientific quality but also is timely and accountable to the public interest in having decisions made about the risks of chemicals to which people are actively being exposed.
My greatest fear about this committee’s IRIS review has been that it would run the risk of becoming a quest for the perfect science, yielding recommendations for enhancing IRIS’ process that in an ideal world would be great, but in practice would make it harder, not easier, for EPA to do its job of protecting human health.
The chemical industry can afford to wait; indeed, under our system where a pending assessment means no action can be taken, all of the rewards of delay fall to one side – the (un)regulated industry – and all of the risks fall on the public.
I’m not suggesting that you as scientists abandon the need to press EPA to get the science right – that’s critically important. But I urge that you not lose sight of the equally important need not to have the resulting process be so prescriptive and onerous that it invites delay, which will also delay or deny protection of public health.
I also ask that you recognize that the IRIS process can and should evolve and improve over time, incorporating further enhancements at a pace commensurate with resources and without slowing down progress toward completing ongoing assessments.
By all means, make your recommendations, but provide EPA with options that recognize that the IRIS process has to work in the real world and needs to provide for timely as well as scientifically credible decisions.
Thank you.