Response to methane synthesis critiques

A new paper published in Science last week is the culmination of an extensive amount of research conducted over the past six years examining methane emissions from the U.S. oil and gas supply chain. The study finds that the U.S. oil and gas industry emits 13 million metric tons of methane per year – 60% higher than the EPA Greenhouse Gas Inventory (GHGI) estimates. Some industry sources have questioned the conclusions drawn in the paper and the methods employed to which I respond below.

The 25 co-authors of this Science paper represent 16 different research institutions. These experts were collectively responsible for generating most of the data analyzed and directly involved in writing the paper, which was subjected to a rigorous independent peer review as a condition of publication in one of the world’s top scientific journals.

Informed critiques are a valuable part of the scientific process. But the key issues raised by Energy in Depth (EID) and the American Gas Association seem to reflect a deep misunderstanding of the approaches used in the paper and the underlying data. Here, I explain the methods used to avoid bias and resolve differences among multiple data sets.

  1. “Exclusive Use of Facility-Scale Study Data Goes Against National Academy of Sciences’ Recommendations and Likely Exaggerates Emissions.”

The National Academy of Sciences recently published a report with recommendations on how to improve the accuracy of methane emission inventories. One of their key recommendation was that emission inventories should be verifiable – for example, validating estimates based on multiple kinds of empirical data. In fact, that is exactly what the new Science paper does.

The paper’s emissions estimate for production was based on site-level bottom-up data from over 400 well pads in six geologic basins, validated against “top-down” airborne measurements from nine basins. The paper's authors compared two totally independent data sets. The team also used component-level data (equipment counts and equipment-level emissions measurements) to generate a data set similar to that used in EPA’s GHGI. This alternative bottom-up inventory approach generated lower emission estimates than either site- or basin-level data.

  1. “Lack of Industry Collaboration Goes Against National Academy of Sciences’ Recommendations.”

EID asserts that the analysis in the paper goes against NAS recommendations by using site-level data collected without industry collaboration. But lack of industry collaboration does not impact the verifiability of emission estimates.

In fact the types of data used in the study were chosen specifically to avoid bias, whether inadvertent or otherwise. In the Science paper, well pad emissions were quantified using a mobile approach sampling methane downwind of a site. The advantage of this approach is that it eliminates potential bias that might result from giving companies advanced notice of measurements.

For example, the data collected during an EDF-coordinated study of transmission and storage facilities led by researchers at Colorado State University indicated that sites controlled by companies that were partners in the study had lower emissions on average than non-partner companies (based on EPA data). This was likely due to the “volunteer effect” – companies that raise their hands to participate in research on methane emissions often are ahead of the curve in reducing emissions. This potential source of bias is discussed in the paper.

Throughout the past six years of field research, EDF and our research partners valued industry’s collaboration on methane research. It is why dozens of oil and gas companies throughout the U.S. supply chain were involved in many studies encompassed in this culminating paper. It is also why EDF and our research partners are collaborating with producers on a set of international studies to better understand global oil and gas methane emissions.

  1. “‘Alternative’ Emissions Estimate That Is In Line With EPA Greenhouse Gas Inventory (And Past EDF Research) Is Not Included In Report.”

The body of research conducted over the past few years is very consistent and robust in showing that there are higher observed emissions than what is estimated by EPA at both the site and basin levels. Several hypotheses were evaluated to account for this discrepancy in the Science paper, and the measured differences are attributed largely to emissions occurring from abnormal operating conditions at the site level.

An alternative hypothesis is that EPA is underestimating emissions from specific operating components. In that case, a relatively simple update to equipment counts and emission factors (average emissions per component on a site), using best available data, would resolve the gap. EPA has made progress in recent years updating equipment counts, but some emission sources have not yet been revised based on recent empirical data. (This includes equipment leaks, which earlier studies suggest are likely underestimated by EPA, but others such as pneumatic controllers which are likely overestimated.)

The Science paper presents an alternative inventory that uses the most accurate equipment counts and emission factors available to estimate national emissions. The paper determined that an updated component-level inventory had some differences from the current GHGI for individual source categories, but yielded similar total emissions.

Based on all the available data, however, the evidence does not support the view that these equipment-based inventories are accurate, as both site- and basin-level emissions data closely align with each other and are higher than the inventory data. Rather, the data strongly suggest that there are inherent limitations to component-level data. This includes difficulty finding and quantifying all sources on a site, because they may be hidden, inaccessible, or unsafe to measure.

Finally, developing emission factors that are representative of the total population of sites requires deployment of stratified sampling strategies in order to effectively and accurately account for high-emitting sources. Quantifying total site-level emissions immediately downwind of individual sites can correct for these effects.

  1. “Attempts to Discredit Study That Finds Misrepresentation of Episodic Events Can Lead to Inflated Emissions Estimates Via Daytime Bias.”

This claim is based on a study conducted in the Fayetteville shale region, which showed that top-down measurements may not accurately represent average daily emissions because overflights occurred during the middle of the day when operators were manually unloading wells. This daytime bias that was observed in the Fayetteville does not necessarily apply to other regions.

To understand this issue the authors of the Science paper separated out emissions from manual unloadings when comparing top-down and site-level estimates in the Fayetteville. The authors also used data from the EPA Greenhouse Gas Reporting Program to assess if manual liquids unloading or blowdowns would cause higher daytime emissions in other basins.

It turns out that the Fayetteville is an outlier with a much higher contribution of these emissions. While there may be a small daytime bias in other basins, the effect would not account for a significant portion of the discrepancy between site-level and component-level emission estimates.

  1. Despite EDF’s Alarmist Characterizations, Natural Gas’ Climate Benefits Remain Clear

The Science paper clearly shows that gas use for power generation in the United States has, at best, marginal climate benefits over coal in the near term as things currently stand. Over the long-term gas beats coal, even with methane emissions because of the big CO2 difference between coal and gas.

Bottom line: If the oil and gas industry wants to maximize the climate benefits of using natural gas versus other fossil fuels, they will need to focus on understanding and mitigating the methane emissions associated with their activities. That’s precisely what the research underlying the Science paper was intended to do. And based on what we’ve learned, there are significant opportunities for the industry to dramatically and cost-effectively reduce methane emissions today.

What can be said about the climate impacts of using gas relative to coal for power generation in other parts of the world remains to be determined. We expect that the studies we’re currently undertaking – again in partnership with leading academic research institutions and industry – will shed light on this question.

This entry was posted in Methane, Natural Gas. Bookmark the permalink. Both comments and trackbacks are currently closed.