{"id":12136,"date":"2023-11-16T11:12:09","date_gmt":"2023-11-16T16:12:09","guid":{"rendered":"https:\/\/blogs.edf.org\/health\/?p=12136"},"modified":"2023-11-16T21:07:52","modified_gmt":"2023-11-17T02:07:52","slug":"fdas-approach-to-systematic-review-of-chemicals-got-off-on-the-wrong-foot","status":"publish","type":"post","link":"https:\/\/blogs.edf.org\/health\/2023\/11\/16\/fdas-approach-to-systematic-review-of-chemicals-got-off-on-the-wrong-foot\/","title":{"rendered":"FDA\u2019s approach to systematic review of chemicals got off on the wrong foot"},"content":{"rendered":"<h3><strong>What Happened<\/strong>?<\/h3>\n<p>Last month, FDA\u2019s scientists <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0273230023001551?via%3Dihub\">published<\/a> the toxicological reference value (TRV) for exposure to cadmium in the diet. This value is the amount of a chemical\u2014in this case cadmium\u2014a person can consume in their daily diet that would not be expected to cause adverse health effects and can be used for food safety decision-making. The TRV was based on a <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0273230022001301?via%3Dihub\">systematic review<\/a> FDA scientists published last year. We will turn to the TRV itself in an upcoming blog but are focusing on the systematic review here.<\/p>\n<p>In a May 2023 <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/S0273230023000582?via%3Dihub\">publication<\/a>, experts in systematic reviews from the University of California San Francisco (UCSF) raised concerns about FDA\u2019s \u201clack of compliance\u201d from <a href=\"https:\/\/ntp.niehs.nih.gov\/whatwestudy\/assessments\/noncancer\/handbook\">established procedures<\/a>.<\/p>\n<p>We discussed these concerns with FDA. They said:<\/p>\n<ul>\n<li>\u201cThe systematic review and the TRV\u201d publication \u201chave both undergone external peer review by a third-party and experts in the field.\u201d The agency expects to publish the reviews on its website, and<\/li>\n<li>FDA \u201cis working on developing a protocol for a systematic review of cardiovascular effects of cadmium exposure that will be published.\u201d<\/li>\n<\/ul>\n<h3><strong>Why It Matters<\/strong><\/h3>\n<p>Systematic review is a method designed to collect and synthesize scientific evidence on specific questions to increase <a href=\"https:\/\/ehp.niehs.nih.gov\/doi\/10.1289\/ehp.1307972\">transparency and objectivity<\/a> and provide conclusions that are more reliable and of higher confidence than traditional literature reviews. In particular, the National Academies of Sciences, Engineering, and Medicine have <a href=\"https:\/\/nap.nationalacademies.org\/catalog\/24758\/application-of-systematic-review-methods-in-an-overall-strategy-for-evaluating-low-dose-toxicity-from-endocrine-active-chemicals\">recommended<\/a> the use of systematic reviews to establish values such as the TRV that may be used to inform regulatory decisions.<\/p>\n<p>The National Toxicology Program (NTP) and <a href=\"https:\/\/prhe.ucsf.edu\/navigation-guide\">others<\/a> have developed specific methodologies to conduct systematic reviews. FDA\u2019s authors said they followed NTP\u2019s Office of Health Assessment and Translation (OHAT) <a href=\"https:\/\/ntp.niehs.nih.gov\/whatwestudy\/assessments\/noncancer\/handbook\">handbook<\/a>.<\/p>\n<p>Unfortunately, FDA\u2019s adherence to the methodology fell short on both transparency and objectivity grounds, undermining the credibility of its conclusions. Credibility is crucial because FDA\u2019s authors stated that \u201cthis systematic review ultimately supports regulatory decisions and FDA initiatives, such as <a href=\"https:\/\/www.fda.gov\/food\/environmental-contaminants-food\/closer-zero-reducing-childhood-exposure-contaminants-foods\"><em>Closer to Zero<\/em><\/a>, which identifies actions the agency will take to reduce exposures to contaminants like cadmium through foods.\u201d<\/p>\n<p><!--more--><\/p>\n<h3><strong>Our Take<\/strong><\/h3>\n<p>We are pleased to see FDA conducting systematic reviews of chemical safety assessments and using more rigorous methods for evaluating scientific evidence with increased transparency and trust in the outcomes.<\/p>\n<p>But while their efforts may be well intentioned, FDA\u2019s methodology has several flaws as identified by UCSF experts\u2014and the agency\u2019s unexplained variations from the <a href=\"https:\/\/ntp.niehs.nih.gov\/whatwestudy\/assessments\/noncancer\/handbook\">OHAT<\/a> handbook sets a troublesome precedent. Although most systematic review methods provide some flexibility, the study authors made significant changes without proper justification. The UCSF experts explained that FDA \u201cdeviations from the method threaten the validity\u201d of the outcome. They identified three major issues:<\/p>\n<ul>\n<li><strong>Failure to publish a protocol before starting the systematic review.<\/strong> A publicly available protocol is fundamental to trusting the outcome of a systematic review. The approaches to evidence selection, evaluation, synthesis, and integration must be stated <em>before<\/em> reviewing the evidence so that the results of studies do not bias the evaluation.<\/li>\n<li><strong>Inappropriate quality rating of animal and human studies to exclude outcomes from consideration.<\/strong> Contrary to OHAT recommendations, the FDA scientists conducted an initial rating of animal and human studies using their own expertise before conducting the risk of bias assessment that excluded outcomes from consideration to derive the TRV. The risk of bias assessment is a crucial component to rate the confidence in the evidence.<\/li>\n<li><strong>Inappropriate removal of a large number of epidemiological studies for consideration. <\/strong>The authors set up their own criteria to assess the risk of bias of human studies. They excluded 23 of 30 studies in a manner that was inconsistent with their own criteria.<\/li>\n<\/ul>\n<p>In a response to the critique of their study, <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/S0273230023000600\">FDA&#8217;s authors acknowledged<\/a> that they had a predefined protocol and a plan they followed, but that the protocol and plan were not published. They acknowledged that they could have \u201cbetter described\u201d the rating and exclusion of studies.<\/p>\n<p>FDA authors used a structured methodology designed to account for every decision made about the strength and confidence in the scientific evidence. Unfortunately, they had unexplained variations that diminished the credibility of the TRV for cadmium that would be protective of human health.<\/p>\n<h3><strong>What\u2019s Next?<\/strong><\/h3>\n<p>We will continue to engage with the agency and evaluate claims that they have conducted systematic reviews with the goal of improving the credibility of their scientific conclusions.<\/p>\n<h3><strong>Go Deeper<\/strong><\/h3>\n<p><a href=\"https:\/\/blogs.edf.org\/health\/?s=systematic+review&amp;searchsubmit=Search&amp;source=direct+%28blogs.edf.org%29&amp;sub_source=%28blank%29&amp;custom_string16=blogs.edf.org%2Fhealth%2F&amp;custom_string17=blogs.edf.org%2Fhealth%2F\">Read our blogs on systematic reviews<\/a>.<\/p>\n<p><a href=\"https:\/\/www.edf.org\/sites\/default\/files\/2023-11\/231116--EDF_Health_blog--FDA_systematic_review.pdf\"><strong>Download a PDF copy of this blog post<\/strong><\/a>. (PDF, 372KB)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>What Happened? Last month, FDA\u2019s scientists published the toxicological reference value (TRV) for exposure to cadmium in the diet. This value is the amount of a chemical\u2014in this case cadmium\u2014a person can consume in their daily diet that would not be expected to cause adverse health effects and can be used for food safety decision-making. &#8230;<\/p>\n","protected":false},"author":153009,"featured_media":12137,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[69,5009,39263],"tags":[107173,92366,91633,114127,39997,114126],"coauthors":[114046,114044],"class_list":["post-12136","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-fda","category-health-science","category-public-health","tag-cadmium","tag-chemicals-in-food","tag-fda","tag-regulatory-science","tag-systematic-review","tag-toxicological-reference-value-trv"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/posts\/12136","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/users\/153009"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/comments?post=12136"}],"version-history":[{"count":2,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/posts\/12136\/revisions"}],"predecessor-version":[{"id":12139,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/posts\/12136\/revisions\/12139"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/media\/12137"}],"wp:attachment":[{"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/media?parent=12136"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/categories?post=12136"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/tags?post=12136"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.edf.org\/health\/wp-json\/wp\/v2\/coauthors?post=12136"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}