Rachel Shaffer is a research assistant.
Lately, much of the attention of the environmental health community has been focused on Capitol Hill and the Lautenberg-Vitter chemical safety reform bill that would amend the antiquated Toxic Substances Control Act (TSCA). Yet significant – if somewhat esoteric – developments are underway at EPA that will also have major impacts on how the safety of chemicals is assessed. EPA has been implementing improvements to its Integrated Risk Information System, commonly known as “IRIS.” The purpose of the IRIS program is to evaluate information on the effects of potential exposures to environmental substances and provide health hazard assessments, which are then used to support regulatory decisions across the agency. And while it isn’t directly affected by TSCA or its reform, IRIS provides both indirect and direct support to the office at EPA that does administer TSCA.
In other words, what happens in IRIS doesn’t stay in IRIS.
So… what’s IRIS up to?
As described on its webpage and factsheet, the IRIS program is in the process of implementing extensive enhancements to bolster its scientific integrity, productivity, and transparency. The details of some of these proposals, such as the new “stopping rules” and strengthened conflict of interest policies for peer review panels, have already been outlined, while others are still in development.
Yesterday, the program held a workshop to assist in its efforts to develop and implement “systematic review,” a structured process to assemble and integrate relevant evidence to answer a research question, into IRIS hazard assessments.
Systematic review is not new; it has been applied to evaluate studies in the area of clinical medicine for many years. What is new, however, are efforts to apply these same approaches to the evaluation of studies examining exposures to environmental chemicals. Several frameworks modeled on existing clinical tools have been proposed, and the purpose of EPA’s workshop was to more fully explore these methods as IRIS begins to develop and implement its own approach. Adopting a robust, empirically-based systematic review process will help significantly to improve the scientific integrity, transparency, and efficient development of IRIS assessments.
Because several members of the EDF Health team have been working on the subject of systematic review over the past few months (including reviewing and providing comments on the National Toxicology Program (NTP) Office of Health Assessment and Translation (OHAT)’s recently released draft protocols), EDF was invited to participate in the workshop. This post shares the central themes of the comments we provided.
Essential components for an IRIS systematic review process
A robust and credible process for systematic review must include several elements:
- A protocol developed up front, prior to initiating a particular review, is essential. The protocol would clearly delineate (1) the key question to be addressed by the IRIS assessment, (2) the strategy for conducting a literature search (including terms, database, and inclusion/exclusion criteria, stopping rules), (3) criteria to be used to evaluate study quality, (4) procedures to be used to integrate findings from multiple studies, and (5) strategies to resolve disagreements that may arise between reviewers. Such an a priori protocol is critical to ensure transparency and reproducibility of the systematic review.
- Transparent and consistent methods are needed for extracting key data from each study, including clear procedures to identify and address missing data.
- Appropriate and robust characterization of “risk of bias” (methodological approaches or choices used in a study that may lead to systematic error, such as selective reporting of results) needs to be part of the process for drawing final conclusions from these studies. Tools used to evaluate risk of bias need to be tailored to the specific type of data being evaluated (i.e., data from laboratory animal studies, human epidemiological studies, or in vitro studies that may elucidate the mechanisms by which chemicals are acting).
- Clear guidance needs to be provided for how to integrate all available evidence to develop a final hazard assessment.
EDF perspective on key issues in the debate over systematic review
Assessing study quality
- Studies should be evaluated for risk of bias, which is a critical measure of study quality because it assesses the degree to which the study methods generate accurate results.
- Alternative approaches to measuring quality that only entail reporting (clear documentation of procedures) or assessing reliability (whether or not results are reproducible) should not be viewed as adequate substitutes for robust analyses of risk of bias.
Use of mechanistic information
- As testing technology continues to progress, more mechanistic information on chemical toxicity will become available. (For additional information on new methods to obtain mechanistic data, visit EDF’s Chemical Testing in the 21st Century website.) However, until adequate frameworks are developed to evaluate the quality and relevance of these studies, they should not be considered in the same manner as human or laboratory animal data. OHAT recently announced an initiative to build these tools, and EDF supports such work that will hopefully help to better integrate in vitro and mechanistic data into systematic review in the near future.
- Because of limits in our current understanding of the mechanisms by which many, if not most, chemicals act, EDF does not support using these data as a central organizing principle for systematic review, as some have proposed. Similarly, having this type of information should not be a prerequisite for developing IRIS or other hazard assessments. However, we believe that, where available, mechanistic information can be used to increase confidence in conclusions reached by evaluating human and animal data and can be helpful in identifying potential susceptible populations.
Role of exposure information
- The IRIS program’s assessments encompass only the first two steps of the four-part process of risk assessment: (1) hazard identification, (2) dose-response assessment, (3) exposure assessment, and (4) risk characterization. Because exposure assessment is not part of an IRIS assessment, a systematic review should include all high-quality hazard information available and should not exclude studies based on assumptions about which routes or levels of exposure are “relevant.” Different regulators or other users of IRIS assessments around the globe may be dealing with different exposure scenarios, and IRIS assessments must remain relevant and able to be the basis for developing full risk assessments in all of these situations.
Public participation
- As EDF has articulated previously, providing opportunities for input from stakeholders must be balanced with assuring timeliness in the completion of these essential health assessments.
- While public input during the systematic review process is important, initiating public comment periods after each and every step of the process, as has been suggested by some industry parties, will undoubtedly introduce unnecessary delays that come at the risk of public health protection.