{"id":2762,"date":"2013-08-27T06:08:05","date_gmt":"2013-08-27T11:08:05","guid":{"rendered":"http:\/\/blogs.edf.org\/health\/?p=2762"},"modified":"2014-03-25T10:16:13","modified_gmt":"2014-03-25T15:16:13","slug":"edf-comments-at-epa-workshop-on-applying-systematic-review-methodology-to-iris-assessments","status":"publish","type":"post","link":"https:\/\/blogs.edf.org\/health\/2013\/08\/27\/edf-comments-at-epa-workshop-on-applying-systematic-review-methodology-to-iris-assessments\/","title":{"rendered":"EDF comments at EPA workshop on applying systematic review methodology to IRIS assessments"},"content":{"rendered":"<p><em>Rachel Shaffer<\/em> is a research assistant.<\/p>\n<p><span style=\"color: #000000;font-family: Calibri;font-size: medium\">Lately, much of the attention of the environmental health community has been focused on Capitol Hill and the <\/span><a href=\"https:\/\/blogs.edf.org\/health\/2013\/08\/01\/senate-hearing-builds-momentum-for-improving-and-moving-the-chemical-safety-improvement-act\/\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">Lautenberg-Vitter chemical safety reform bill<\/span><\/a><span style=\"font-family: Calibri;font-size: medium\"> that would amend the antiquated <\/span><a href=\"http:\/\/www.saferchemicals.org\/resources\/tsca.html\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">Toxic Substances Control Act (TSCA)<\/span><\/a><span style=\"color: #000000;font-family: Calibri;font-size: medium\">. Yet significant \u2013 if somewhat esoteric \u2013 developments are underway at EPA that will also have major impacts on how the safety of chemicals is assessed.\u00a0 EPA has been implementing improvements to its <\/span><a href=\"http:\/\/www.epa.gov\/iris\/intro.htm#whatiris\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">Integrated Risk Information System<\/span><\/a><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">, commonly known as \u201cIRIS.\u201d The purpose of the IRIS program is to evaluate information on the effects of potential exposures to environmental substances and provide health hazard assessments, which are then used to support regulatory decisions across the agency.\u00a0 And while it isn\u2019t directly affected by TSCA or its reform, IRIS provides both indirect and direct support to the office at EPA that does administer TSCA. \u00a0<\/span><\/span><\/span><\/p>\n<p><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">In other words, what happens in IRIS doesn\u2019t stay in IRIS. <\/span><\/span><\/span><\/p>\n<p><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">So\u2026 what\u2019s IRIS up to? <!--more--><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #000000;font-family: Calibri;font-size: medium\">As described on its <\/span><a href=\"http:\/\/www.epa.gov\/iris\/process.htm\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">webpage<\/span><\/a><span style=\"color: #000000;font-family: Calibri;font-size: medium\"> and <\/span><a href=\"http:\/\/www.epa.gov\/iris\/pdfs\/irisprocessfactsheet2013.pdf\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">factsheet<\/span><\/a><span style=\"color: #000000;font-family: Calibri;font-size: medium\">, the IRIS program is in the process of implementing extensive enhancements to bolster its scientific integrity, productivity, and transparency. The details of some of these proposals, such as the new \u201c<\/span><a href=\"http:\/\/www.epa.gov\/iris\/pdfs\/IRIS_stoppingrules.pdf\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">stopping rules<\/span><\/a><span style=\"color: #000000;font-family: Calibri;font-size: medium\">\u201d and strengthened <\/span><a href=\"http:\/\/www.epa.gov\/osa\/pdfs\/epa-process-for-contractor.pdf\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">conflict of interest policies<\/span><\/a><span style=\"font-family: Calibri\"><span style=\"font-size: medium\"><span style=\"color: #000000\"> for peer review panels, have already been outlined, while others are still in development. <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #000000;font-family: Calibri;font-size: medium\">Yesterday, the program held a <\/span><a href=\"http:\/\/www.epa.gov\/iris\/irisworkshops\/systematicreview\/index.htm\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">workshop<\/span><\/a><span style=\"color: #000000;font-family: Calibri;font-size: medium\"> to assist in its efforts to develop and implement \u201c<\/span><a href=\"http:\/\/en.wikipedia.org\/wiki\/Systematic_review\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">systematic review<\/span><\/a><span style=\"font-family: Calibri\"><span style=\"font-size: medium\"><span style=\"color: #0000ff\">,<\/span><span style=\"color: #000000\">\u201d a structured process to assemble and integrate relevant evidence to answer a research question, into IRIS hazard assessments. <\/span><\/span><\/span><\/p>\n<p><span style=\"font-family: Calibri\"><span style=\"font-size: medium\"><span style=\"color: #000000\">Systematic review is not new; it has been applied to evaluate studies in the area of clinical medicine for many years.\u00a0 What is new, however, are efforts to apply these same approaches to the evaluation of studies examining exposures to environmental chemicals.\u00a0 Several frameworks modeled on existing clinical tools have been proposed, and the purpose of EPA\u2019s workshop was to more fully explore these methods as IRIS begins to develop and implement its own approach.\u00a0 Adopting a robust, empirically-based systematic review process will help significantly to improve the scientific integrity, transparency, and efficient development of IRIS assessments.<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #000000;font-family: Calibri;font-size: medium\">Because several members of the EDF Health team have been working on the subject of systematic review over the past few months (including reviewing and providing <\/span><a href=\"http:\/\/ntp.niehs.nih.gov\/NTP\/OHAT\/EvaluationProcess\/PublicComm\/McPartland20130611_508.pdf\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">comments<\/span><\/a><span style=\"color: #000000;font-family: Calibri;font-size: medium\"> on the National Toxicology Program (NTP) <\/span><a href=\"http:\/\/ntp.niehs.nih.gov\/?objectid=960B6F03-A712-90CB-8856221E90EDA46E\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">Office of Health Assessment and Translation (OHAT)\u2019s recently released draft protocols<\/span><\/a><span style=\"font-family: Calibri\"><span style=\"font-size: medium\"><span style=\"color: #000000\">), EDF was invited to participate in the workshop. This post shares the central themes of the comments we provided. <\/span><\/span><\/span><\/p>\n<p><strong><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">Essential components for an IRIS systematic review process \u00a0<\/span><\/span><\/span><\/strong><\/p>\n<p><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">A robust and credible process for systematic review must include several elements:<\/span><\/span><\/span><\/p>\n<ul>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">A protocol developed up front, prior to initiating a particular review, is essential.\u00a0 The protocol would clearly delineate (1) the key question to be addressed by the IRIS assessment, (2) the strategy for conducting a literature search (including terms, database, and inclusion\/exclusion criteria, stopping rules), (3) criteria to be used to evaluate study quality, (4) procedures to be used to integrate findings from multiple studies, and (5) strategies to resolve disagreements that may arise between reviewers. \u00a0Such an <em>a priori<\/em> protocol is critical to ensure <span style=\"text-decoration: underline\">transparency<\/span> and <span style=\"text-decoration: underline\">reproducibility<\/span><\/span><span style=\"font-size: medium\"> of the systematic review. <\/span><span style=\"font-size: medium\">\u00a0<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">Transparent and consistent methods are needed for extracting key data from each study, including clear procedures to identify and address missing data. <\/span><\/span><\/span><\/li>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri;font-size: medium\">Appropriate and robust characterization of \u201c<\/span><\/span><a href=\"http:\/\/handbook.cochrane.org\/chapter_8\/8_2_1_bias_and_risk_of_bias.htm\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">risk of bias<\/span><\/a><span style=\"font-family: Calibri\"><span style=\"font-size: medium\"><span style=\"color: #000000\">\u201d (methodological approaches or choices used in a study that may lead to systematic error, such as selective reporting of results) needs to be part of the process for drawing final conclusions from these studies. Tools used to evaluate risk of bias need to be tailored to the specific type of data being evaluated (i.e., data from laboratory animal studies, human epidemiological studies, or in vitro studies that may elucidate the mechanisms by which chemicals are acting). <\/span><\/span><\/span><\/li>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">Clear guidance needs to be provided for how to integrate all available evidence to develop a final hazard assessment.<\/span><span style=\"font-size: medium\">\u00a0 \u00a0<\/span><\/span><\/span><\/li>\n<\/ul>\n<p><strong><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">EDF perspective on key issues in the debate over systematic review<\/span><\/span><\/span><\/strong><\/p>\n<p><span style=\"text-decoration: underline\"><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">Assessing study quality<\/span><\/span><\/span><\/span><\/p>\n<ul>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">Studies should be evaluated for risk of bias, which is a critical measure of study quality because it assesses the degree to which the study methods generate accurate results. <\/span><\/span><\/span><\/li>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">Alternative approaches to measuring quality that only entail reporting (clear documentation of procedures) or assessing reliability (whether or not results are reproducible) should not be viewed as adequate substitutes for robust analyses of risk of bias.<\/span><\/span><\/span><\/li>\n<\/ul>\n<p><span style=\"text-decoration: underline\"><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">Use of mechanistic information<\/span><\/span><\/span><\/span><\/p>\n<ul>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri;font-size: medium\">As testing technology continues to progress, more mechanistic information on chemical toxicity will become available. (For additional information on new methods to obtain mechanistic data, visit EDF\u2019s <\/span><\/span><a href=\"http:\/\/www.edf.org\/health\/chemical-testing-21st-century\"><span style=\"color: #0000ff\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">Chemical Testing in the 21<\/span><sup><span style=\"font-size: small\">st<\/span><\/sup><span style=\"font-size: medium\"> Century<\/span><\/span><\/span><\/a><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\"> website.) However, until adequate frameworks are developed to evaluate the quality and relevance of these studies, they should not be considered in the same manner as human or laboratory animal data. OHAT recently announced an initiative to build these tools, and EDF supports such work that will hopefully help to better integrate <em>in vitro<\/em> and mechanistic data into systematic review in the near future.\u00a0 \u00a0<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">Because of limits in our current understanding of the mechanisms by which many, if not most, chemicals act, EDF does not support using these data as a central organizing principle for systematic review, as some have proposed. Similarly, having this type of information should not be a prerequisite for developing IRIS or other hazard assessments. However, we believe that, where available, mechanistic information can be used to increase confidence in conclusions reached by evaluating human and animal data and can be helpful in identifying potential susceptible populations. <\/span><\/span><\/span><\/li>\n<\/ul>\n<p><span style=\"text-decoration: underline\"><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">Role of exposure information<\/span><\/span><\/span><\/span><\/p>\n<ul>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">The IRIS program\u2019s assessments encompass only the first two steps of the four-part process of risk assessment: \u00a0(1) hazard identification, (2) dose-response assessment, (3) exposure assessment, and (4) risk characterization. Because exposure assessment is <em>not<\/em><\/span><span style=\"font-size: medium\"> part of an IRIS assessment, a systematic review should include all high-quality hazard information available and should not exclude studies based on assumptions about which routes or levels of exposure are \u201crelevant.\u201d Different regulators or other users of IRIS assessments around the globe may be dealing with different exposure scenarios, and IRIS assessments must remain relevant and able to be the basis for developing full risk assessments in all of these situations. <\/span><span style=\"font-size: medium\">\u00a0\u00a0\u00a0\u00a0<\/span><\/span><\/span><\/li>\n<\/ul>\n<p><span style=\"text-decoration: underline\"><span style=\"font-size: medium\"><span style=\"color: #000000\"><span style=\"font-family: Calibri\">Public participation<\/span><\/span><\/span><\/span><\/p>\n<ul>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri;font-size: medium\">As EDF has articulated <\/span><\/span><a href=\"https:\/\/blogs.edf.org\/health\/2012\/11\/14\/edf-comments-at-epas-public-stakeholder-meeting-on-its-iris-program\/\"><span style=\"color: #0000ff;font-family: Calibri;font-size: medium\">previously<\/span><\/a><span style=\"font-family: Calibri\"><span style=\"font-size: medium\"><span style=\"color: #000000\">, providing opportunities for input from stakeholders must be balanced with assuring timeliness in the completion of these essential health assessments. <\/span><\/span><\/span><\/li>\n<li><span style=\"color: #000000\"><span style=\"font-family: Calibri\"><span style=\"font-size: medium\">While public input during the systematic review process is important, initiating public comment periods after each and every step of the process, as has been suggested by some industry parties, will undoubtedly introduce unnecessary delays that come at the risk of public health protection.<\/span><\/span><\/span><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Rachel Shaffer is a research assistant. Lately, much of the attention of the environmental health community has been focused on Capitol Hill and the Lautenberg-Vitter chemical safety reform bill that would amend the antiquated Toxic Substances Control Act (TSCA). Yet significant \u2013 if somewhat esoteric \u2013 developments are underway at EPA that will also have &#8230;<\/p>\n","protected":false},"author":11089,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[44,5009],"tags":[39167,68,39169,39170,39160,5017],"coauthors":[],"class_list":["post-2762","post","type-post","status-publish","format-standard","hentry","category-policy","category-health-science","tag-computational-toxicology","tag-epa","tag-in-vitro","tag-in-vivo","tag-national-academy-of-sciences-nas","tag-risk-assessment"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/posts\/2762","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/users\/11089"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/comments?post=2762"}],"version-history":[{"count":0,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/posts\/2762\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/media?parent=2762"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/categories?post=2762"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/tags?post=2762"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/coauthors?post=2762"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}