It seems daunting, but it is not if you follow a certain - well recognized - set of basic scientific principles when collecting samples and analyzing them. So lets compare and contrast these reports.
One of my main complaints centers on the identification and quantification of the carbon disulfide concentration Dr. Sattler bases the setback on as well as an indication of potential health concerns. And one of my main issues with the work performed by Dr. Sattler is the "backed in" data she uses to calculate her emission rate for the air dispersion modeling. I believe I have laid out a fairly well substantiated reasoning for why her model's predictions are invalid.
Dr. Sattler assumes that with one air sample taken at a distance from the source you can back in to the air dispersion model and calculate the emission rate of the contaminant detected from that source. With this emission rate, you can then plug it into the air dispersion model along with historical wind and weather information and predict the upper most concentration and distance from the source that could exceed a health based limit.
In my last post I attempted to show how that assumption - based on one single sample - assumes that one single sample accurately represents the air in and around the source. n = 1 makes the model's predictions invalid, let alone the assumption that you can back in data to obtain the emission rate. How confident is Dr. Sattler and her team on the representativeness of that one sample used to calculate carbon disulfide in Plot 1 and, most importantly, what do they base that confidence on?
In the ERG interim Report an emission rate was also calculated. So why should ERGs stand and Dr. Sattler's be discounted? Lets look at the assumption and the methodology used by ERG:
At 66 sites, preliminary quantitative emission estimates have been generated. Estimated annual emission rates were calculated from the short-term test data, assuming the measured emissions on the day of testing are representative of the emissions that occur over the year.This is an assumption - it states it as such. Scientist make assumptions all the time. Then we test, test again, test some more, and research other similar published work. Then when we look at all the data collected and we make a reasonable conclusion that our assumption is sound.
Even though ERG's emission rates were calculated from one SUMMA canister's results, there were 66 similar tests performed at 66 different locations. This gives a representative snapshot as to what is most likely - reasonably - taking place.
A researcher can never know the true value of the population being sampled. This is why we use statistics to calculate a mean, standard deviation, and variance. It is with those numbers that we look at our assumptions and make a judgement to its soundness.
With 66 different samples of the population (air within oil & gas sites) collected by ERG, we can look at the report's tables to get a pretty good idea of what is being emitted - provided some additional methodologies were also performed.
Because bias is always present, the sampling team and laboratory need to follow a strict protocol on how they will collect the sample, transport the sample, prepare the sample, test the sample, and report the results. This is all part of what is referred to as QA/QC. If you look at the ERG report you will see a section QA/QC and if you look at the TCEQ report you will see the same. If you look at the FWLN report and the reports by Alisa Rich for the Town of Dish, Texas, you will not see a QA/QC section. (The Dish Final Report does describe the sampling procedures and the analysis method performed.)
So the assumption is this; If the researchers follow their QA/QC plan, the impact of any bias will be reduced and the results can be considered representative and valid. Additionally, the laboratory follows a similar QA/QC plan and also may choose to follow a strict methodology and become NELAC accredited. (To be valid in Texas the laboratory must be NELAC accredited.)
For the laboratory that performs the analysis, it must be able to show that they are able to "accurately determine a compound and that they can acquire the same concentrations from different instruments or samples while they are sampling the same gas stream, with an acceptable level of uncertainty." This is known as measurement precision. ERG calculated analytical precision by collecting two sets of duplicate samples at two different sites and analyzed them in replicate (page 64). (1)
As well as precision, the laboratory is required to show that it has the "ability to acquire the correct concentration data from an instrument or sample analysis with an acceptable level of uncertainty while measuring a reference gas stream of a known concentration." This is known as accuracy. (1)
All of this data to show precision and accuracy is identified and report by both ERG and the TCEQ in their two separate reports on air quality in the Barnett Shale. This data is needed to support their assumption that the results they have put forth are representative and valid. Since we can never know the actual - or true - value, we need this information to support what we can reasonably ascertain.
This information is missing from Alisa Rich's reports as well as the FWNL report. With one exception...sort of...GD Air Testing, the laboratory used by FWLN and Alisa Rich - is NELAC Accredited.
However, when they perform analysis to identify carbon disulfide - a TIC - that analytical method is "not included in the Scope of NELAC Accreditation" (see note 'N' on the Lab report). Without the FWLN report's laboratory performing the analysis as per their NELAC Accreditation, the results for carbon disulfide is invalid. That's not the fault of GD Air Testing, its just the way we have to look at data that has not been generated under a strict protocol and methodology designed to minimize the impact of bias and substantiate validity.
To meet this, the samples had to be prepared and analyzed using a method that is designed specifically for the contaminant of concern.
So how did ERG and the TCEQ analyze for carbon disulfide? They used the correct method, TO-15.
 |
| ERG Report |
 |
| TCEQ Report |
And what did Alisa Rich (2) and Dr. Sattler (3) use for their reports?
 |
| Dish, Texas Report |
 |
| FWLN Report |
So the wrong method - TO-14 - was used by Alisa Rich and Dr. Sattler to identify the carbon disulfide. This creates uncertainty on the validity of all the assumptions that were based off the carbon disulfide quantification.
On the other hand, the use of the correct method - TO-15 - reduces the uncertainty and adds to the validity of the results and assumptions made by the TCEQ and ERG in their reports.
Because we cannot know the true value, assumptions are made. It then becomes necessary for the report writer to show how sound those assumptions are. Without representative sampling, without QA/QC, and without using the correct analytical method whereby precision and accuracy could be shown, the assumption presented by the FWLN are invalid.
Our assumptions are only as good as our data is sound.
Air Quality in the Barnett Shale - Part 26: That's a lot of n's!
.
No comments:
Post a Comment