Air Quality in the Barnett Shale - Part 28: Why Plot 2 used Carbonyl Disulfide

I was a bit curious as to why the FWLN Report sent to the FWISD included one model for carbon disulfide (Plot 1) and another model for carbonyl sulfide (Plot 2).  Everything up to this point had focused on the health concerns for carbon disulfide.

So I looked closer at the FWLN findings:

...and:

And then I looked at the statements printed on a green background in the report:

Okay...so it is all about the carbon disulfide.  So why produce Plot 2 for carbonyl sulfide?

Well duh!  Carbon disulfide was not detected by TO-15 in 46 out of 48 samples (see post) and ASTM Method D5504-08 detected carbon disulfide but had to qualify it as an artifact (see post).

But sample QLA5DC-1 did not have that inconvient qualifier:

So as I'm reading this and looking at the data, my mind is forming the question of; if there is no carbon disulfide detected in the BSEEC samples collected from different sites over different days, and carbon disulfide is the contaminant of concern, why spring carbonyl sulfide into the mix and continue to direct attention to carbon disulfide?

Doh!  Then it hit me.  FWLN is looking at carbonyl sulfide as a "byproduct" of carbon disulfide.  In other words, if you find carbonyl sulfide there must have been - or is - carbon disulfide!  Very astute FWLN!  Only you probably didn't figure on me checking into this, did you?

Yes, indeed, carbonyl sulfide is a "byproduct" of carbon disulfide.  It is produced when carbon disulfide is purposely manufactured.  Lets see what the EPA has to say on this:

You know what else the EPA has to say about carbonyl sulfide?

In the United States, the reported concentration of carbonyl sulfide in the air in rural areas was 0.27-0.80 microgram/m3.  Levels were 1.17 micrograms/m3 in Philadelphia, PA; 1.21 micrograms/m3 in Wallops Island, VA; and 1.37 micrograms/m3 in Lawton, OK.  Reported concentrations of carbonyl sulfide over salt marshes, which are major natural sources, range from about 60 to 180 micrograms/m3, and 14 to 19 micrograms/m3 over the ocean (HSDB 1994).

Well if I am going to be objective here, I need to follow this lead....

Okay...so finally we have something that indicates that we may have an issue with a contaminate in the Barnett Shale area above normally expected background.  And even if you subtract the maximum amount attributed to the artifact, in sample QLA5DC-1 you would still have 105 ug/m3.  That's more than comes off a salt marsh based on the EPA's research.

So lets work with this.  But first we need to make a few points clear.  First the presence of carbonyl sulfide in the ambient air samples collected has nothing to do with carbon disulfide.  And second, the non-detection of carbon disulfide in the BSEEC report using method TO-15 indicates that carbon disulfide is not present.

So the question I can reasonably ask now is carbonyl sulfide a concern? 

Now we are back to the one sample - low n's - issue.  However, carbonyl sulfide was detected above the artifact concentration of 25 ug/m3 in 6 of the 18 samples indicating a possible concentration above the levels found in an urban environment.  So lets assume that 130 ug/m3 (53 ppbv) is a valid number for carbonyl sulfide in the ambient air collected by BSEEC.

What does the BSEEC Report have to say about this:

Looks like the BSEEC folks got their units mixed up. This means that I need to check to see if the AMCV they report is in ppbv or ug/m3.  And guess what I find when I check?  There is no AMCV for carbonyl sulfide.  So where did "1800 ppbv" come from?

Well if I can find the acute ReV that would be the AMCV.  Search....nothing.  What about the RfC?  Checking the IRIS website....nothing, in fact they state:

The health effects data for carbonyl sulfide were reviewed by the U.S. EPA RfD/RfC Work Group and determined to be inadequate for the derivation of an inhalation RfC. The verification status for this chemical is currently not verifiable. No EPA documentation presently exists for this chemical.

Okay...so where did the BSEEC folks get a short-term AMCV of 1800 ppbv?  Well as of right now I have no idea.  I did find a short term ESL value that the TCEQ has derived in a 2008 memo:

So even at 130 ug/m3 (53 ppbv) the amount detected is well below the health-based ESL concentration of 540 ppbv which is 70% of the AMCV.....540 / 0.3...that equals 1800...hey...that's the AMCV the BSEEC folks reported!  

Okay, that gives us a value we can defend as health-based, assuming TCEQs POD and UFs are correct...they seem well supported.  And if the carbonyl sulfide ESL is listed for odor at 55 ppbm, it makes sense that the health-based threshold would be higher:

The short-term ESL is the lowest value of acute odor-, vegetation- and health-based ESLs. (Factsheet)

And at the highest carbonyl sulfide concentration detected - 53 ppbv - that amount is 10 times lower than the health-based ESL of 540 ppbv and more than 33 times lower than the ambient air concentration - the true health-based threshold - of 1800 ppbv.

Y'all double check my math (remember math makes my head hurt) to make sure I'm seeing this right....

Lab reports 53 ppbv...TCEQ states health-based ESL is 540 ppbv...at 540 ppbv the AMCV would be 1800 ppbv...okay...so the carbonyl sulfide in the Barnett Shale samples is well below health based levels.

So what does Dr. Sattler and the FWLN have to say about the carbonyl sulfide in their Report?

Huh?  How could the oil & gas source that produced sample QLA5DC-1 produce an emission rate that produces a model that that predicts concentrations one mile from the source that are "6 times greater than the health benchmark for carbonyl sulfide?"  At 1800 ppbv that would mean one mile out you would have concentrations at 10,800 ppbv?  Even if she incorrectly used the health-based ESL of 540 ppbv, that would show concentrations at 3,240 ppbv one mile from the source.

How is that physically even possible?

Maybe I'm missing something here, how can directly downwind from the source  - 50 -75 feet away - we find an actual concentration 130 ppbv but are able to get a concentration significantly higher further from the source?  How can the calculated emission rate from this actual value extend downwind one mile from the source at a concentration 6 times greater than what produced it at 50-75 feet away?  The same emission rate is being used, correct?  Doesn't Gaussian dispersion show the highest downwind concentrations nearest the source?

Wait a minute...Dr. Sattler is calculating an air concentration in and around the source (red area on her plot) of 655 ug/m3.  That's six times higher than the actual concentration detected within the red colored area around the source.  Now I'm really confused.  I mean the emission rate is staying constant, correct?  And the downwind concentration used to "back in" to obtain the emission rate is 130 ug/m3.  So with that "backed-in" emission rate, you would calculate 130 ug/m3 for the red area.  Yet in Plot 2, that area has a concentration of  655 ug/m3:

Okay...so lets assume Dr. Sattler's maximum 1 hour concentration for carbonyl sulfide is possible.  Would that maximum level pose a health-based concern?

• The highest concentration Dr. Sattler reports in Figure 1 is 655 ug/m3 or 267 ppbv.
• The maximum concentration of 267 ppbv is less than the ESL of  540 ppbv.
• 267 ppbv is less than one half the amount of the health based ESL
• 267 ppbv is less than six times the health based ambient air concentration of the AMCV.

So why would Dr. Sattler and the FWLN insist that the concentrations one mile from the source could be as high as "6 times the health benchmark for carbonyl sulfide?"  Well lets look at Dr. Sattler's Report on Plot 2 to see what's up about this:

The 1-hour Texas Commission on Environmental Quality (TCEQ) Effects Screening Level (ESL) for carbonyl sulfide is 135 µg/m3; one of the sample concentrations exceeded this level.

.... ah...that's why!  Dr. Sattler is using an odor ESL threshold for the "health benchmark" that is being exceeded:

...and if you will recall, "The short-term ESL is the lowest value of acute odor-, vegetation- and health-based ESLs."

So even if Dr. Sattler's model is correct, the carbonyl sulfide in Plot 2 does not pose and adverse health risk.

And if that is true...and the fact that 46 our of 48 BSEEC TO-15 samples showed non-detect for carbon disulfide, and the two samples that were positive were less than 0.3 ppbv... then there is no health based concern for carbon disulfide or for carbonyl sulfide and the requirement to have a one-mile setback based on the health concerns for these two contaminants is not warranted or substantiated by any of the data provided in the FWLN or BSEEC or ERG reports.

And just so we are clear on this.  The need to provide - or not provide - a setback for O&G operations is not being argued here.  The basis for a one mile setback recommendation due to health-based concerns regarding carbon disulfide and/or carbonyl sulfide is.

And that's all I can really say about this....

Next Post: Air Quality in the Barnett Shale - Part 29: Where should we go from here?

.

Air Quality in the Barnett Shale - Part 27: Plot 2; Artifact or the real deal?

Well apparently my attempt at humor and self-depreciation as a means to entertain and inform on the complex - and really dry - world of health, air monitoring, and air dispersion modeling made me look like a fool in the eyes of the FWLN.

So today I am writing my blog while dressed in a long sleeve shirt - with tie - slacks, and my recently polished dress shoes (black to match my belt according to my wife).  And to complete this outfit of visual competency and authority, I have put on my white lab coat, the one with my name embroidered over the pocket!

Hopefully this will get my street cred' back.  You know, nothing says "scientist" like a white lab coat!  In fact, look at what these two guys at Columbia Analytical Services are wearing:

You know what else you can find on their website?  This:

Do you know what "COS" and "CS2" is?  Well since I am wearing a white lab coat, I do.

• COS = Carbonyl sulfide
• CS2 - Carbon disulfide

Do those two chemicals sound familiar to you?  Well they do to me:

This post today is only going to focus on Plot 2 of the FWLN report.  Plot 2 is based on the concentration of carbonyl sulfide found in sample QLA5DC-1.  

According to the white lab coats at Columbia Analytical Services, there are two factors in play when a Tedlar bag is used to collect a gas sample.

1. Concentration range is from ppm to percent (1 percent = 10,000 ppm)
2. Artifacts may be produced which include carbonyl sulfide and carbon disulfide

If you will recall from my last post, artifact formation is a concern and was noted by the laboratory.  You can read more about issues with Tedlar bags at this website.  And when you look at the BSEEC report you will see that they used Tedlar bags:

...and artifacts were acknowledged:

...and the concentrations were reported as ppbv (which are 1000 times smaller than a ppm):

Now all of this give me pause - and when I am wearing my white lab coat - my scientist-senses start to tingle (analogous to Spider-Man's spider-sense telling him that something is wrong) - or it could just be my cell phone set on vibrate.  Anyway....

So what's the issue?  Seems that Plot 2 which was used in the FWLN shows "multiples [that] were 6 times the health benchmark for carbonyl sulfide."  

And how did they arrive at this "6 times" number?  Using a "backed in" emission rate calculated by Dr. Sattler as described in her 3/7/11 draft report titled "Dispersion Modeling of Carbonyl Sulfide Emissions from a Natural Gas Compressor Station near Lake Arlington."  (link not available).

In this report Dr. Sattler describes the sample concentrations for carbonyl sulfide she used as follows:

...which corresponds to these samples in the BEECE report:

...which are associated with these two laboratory reports:

Upwind Sample

Downwind Sample

Now there are two factors seen above that get my spider-senses....err my white lab coat senses buzzing.  The first is that there is no data qualifier "+" for the carbonyl sulfide quantification as there is for carbon disulfide.

Note 3/14/11: Spoke with the lab this morning.  the reason that there is no qualifier is that they set the highest number attributable to an artifact at 25 ppb.  This qualifier number is dependent on the Tedlar bag used.  So the quantity of 130 ppbv may be only as high as 105 ppbv if the bag did produce artifacts of carbonyl sulfide.  By the way, I just noticed that the laboratory that performed the analysis was the same lab with the white lab coat guys that produced the table I used above.  It's a small world.

Was this an oversight or were they able to exclude the possibility of an artifact in this sample?  The reason that I question this is that the possibility of the carbonyl sulfide being an artifact is noted in samples:

QA52DW-1, EMR3DE-1, EMR3DW-1, CAP4UC-1, CAP4DC2-1, CAP4DC2-DUP1, DLB6U-1, DLB6DE-1, DHH7U-1, CLP9UC-1, XRA4UE-1, and XRA4DE-1

In other words, out of 18 samples 12 identify the carbonyl sulfide as a "possible Tedlar bag artifact."  I will be contacting the laboratory next week to see if I can find out if this was in error.

The second reason I question the validity of these carbonyl sulfide results is that even though the laboratory may feel quite confident in the numbers they have reported for ASTM Method D5504-08 for sulfur compounds:

...while wearing my white lab coat, I went and read the ASTM method.  

...and here is what I found out:

...so we have to assume that both the sampler and the laboratory understood this and took the proper precautions.  Okay, so lets assume they did.  What next?

...well the test is for the gas itself...may not be appropriate for ambient air samples...still...rang is 0.01 ppm...what's that in ppb?....if 1000 ppb = 1 ppm, the range is down to 10 ppb...okay that means the 130 ppb is 130/1000 = 0.13 ppm....kind of close to the method's limit....okay....odd though, the lab's MRL states they can "confidently" go down to 0.0078 ppm.

So once again we are looking at the identification and quantification of a contaminant of concern that is questionable.  I am unsure why ASTM  D5504-08 was utilized in the first place.  Most likely it was to look for sulfur contaminants that are common to natural gas and are not picked up by EPA Method TO-15.  In any case, the use of a Tedlar bag was possibly the wrong choice if carbon disulfide and carbonyl sulfide were to be reported.

Hey....but wait a second....isn't TO-15 appropriate for carbon disulfide and carbonyl sulfide?

And didn't BSEEC run a TO-15 analysis on sample QLA5DC-1 from which Dr. Sattler obtained the quantity for her backed in emission rate calculation for her carbonyl sulfide model shown in plot 2?

...which means there should be a lab report for this sample showing the quantity of carbon disulfide and carbonyl sulfide in the same air on the same day.  And this sample is the correct method and does not have any footnotes calling into question its validity, such as a "possible artifact."  So looking at the lab results for sample QLA5DC-1 we see:

Doh!  They didn't analyze for carbonyl sulfide!  Had they tested for carbonyl sulfide with TO-15 it would have been much more definitive and I could have ended this post right here and now stating that the carbonyl sulfide reported was indeed an artifact. If only it would be that easy....

Still, there is good evidence to think it is - most likely - an artifact.  

The TO-15 method has a detection limit for carbon disulfide of 0.236 ppbv.  The TO-15 results for 46 one hour samples show non-detect and two samples show carbon disulfide at less than 0.61 ppbv.  Now compare this to the detection of carbon disulfide with ASTM Method D5504-08 which found carbon disulfide in 12 out of 21samples with a high concentration of 11 ppbv.

And with my white lab coat on - complete with embroidered name over the pocket - I am able to look at this and boldly state that Plot 2 in the FWLN report is based on a positive identification of carbonyl sulfide and a quantitative amount that is suspect.

And if Peter Parker (aka: Spider-Man) - who himself is a scientist - was looking at this, he too would see not a criminal number, but a number that is suspect.  Call it his spidey-sense, objectivity, or just plain-ol' scientific common knowledge, but that's what any good scientist would conclude - even one that uses self-depreciation and comic book characters to try and drive home his point.

The thing is, we scientists are only as good as the numbers we use and the place we get our data from.  And because of this, the air dispersion model that generated Plot 2 that was based on the results from sample QLA5DC-1, which was derived using ASTM Method D5504-08 from samples collected in a Tedlar bag which - according to other scientists wearing white lab coats - are designed for samples in the ppm range and are also known to produce artifacts of COS and CS2, which - while wearing my white lab coat - I have identified as the two culprits of concern in the FWLN report regarding the need for set-backs; carbonyl sulfide and carbon disulfide, makes the model's predictions of "multiples [that] were 6 times the health benchmark for carbonyl sulfide" suspect.

So what are we left with now?  

If the FWLN report is based on Plot 1 for carbon disulfide which was derived from one single sample collected on one single day (see post) and Plot 2 is based on carbonyl sulfide derived from using ASTM Method D5504-08, then nothing definitive, conclusive, alarming, substantiating, or realistic regarding set-back distance can be ascertained.

Next Post: Air Quality in the Barnett Shale - Part 28: Why Plot 2 used Carbonyl Disulfide

Air Quality in the Barnett Shale - Part 26: That's a lot of n's!

In my last post I looked at the quality of the data presented in the ERG report and the TCEQ report for air quality in the Barnett Shale.

Looking at how the data was obtained is important to understanding how valid it is.  Along with this is the number of individual data points - or n's - were collected.

Since we are making an assumption, such as the air "does not pose a health hazard" or the air is "2 times above a threshold where irreversible effects can occur" the quality of the data and the representation of the data to what is actually found becomes critical.

Alisa Rich in her report to the Town of Dish, Texas, modeled the air using only six samples - collected on one day - and analyzed with the wrong method, concluding

Dr. Sattler, using one sample - collected on one day - and analyzed with the wrong method, concluded in the FWLN report:

And with these two assumptions, FWLN had enough information to support their contention that the air quality in the Barnett Shale in the Fort Worth Area is significantly impacted:

I contend that because of the lack of representative sampling (too few n's) and the incorrect laboratory analysis (TO-14 instead of TO-15) the data Rich and Sattler generated is invalid and, therefore, their model's predictions are invalid, which makes these types of statements regarding carbon disulfide in the Barnett Shale area false.

So what do we know about the air in the Barnett Shale and the emissions from oil & gas production sites?

Quite a lot, actually.  Both the Barnett Shale Energy Education Council (BSEEC) TITAN Engineering report and the ERG report - referenced in the FWLN report - provide a considerable amount of data (lots of n's). And that data can be supported as valid because the reports can show precision and accuracy (which Alisa Rich and Dr. Sattler cannot).

Now you can choose to ignore the BSEEC and ERG's data, but the only grounds to do so is because you suspect some kind of conspiricy between O&G and them.  Possible, but not very plausible.  A conspiracy to distort, fake, or hide sample data implies both a lot of conspirators who will presumably attempt to conceal what they have done, and a lot of readers who can be successfully persuaded that the data in these two reports have been purposely distorted.

If you think there is a conspiracy by BSEEC and/or ERG to fake this information then you can stop reading now and go back to your land of implausibility.  Like I said earlier in my posts on the air in the Barnett Shale area, I assume that the data provided by Alisa Rich and Dr. Sattler has not been purposely manipulated.  It may be invalid, but I don't think it has been faked, tainted, or manipulated.  The same is true for BSEEC and ERG's data.  I assume it to be truthful and honest.

The BSEEC tested the air over four time periods in June, at nine (9) different locations.  EPA Method TO-15 found the following results for 93 samples (n's) of carbon disulfide:

The BSEEC report identified the following concentrations for carbon disulfide and carbonyl sulfide tested using ASTM Method D 5504-08 for sulfur compounds .  Please note that the number "4" by the quantity indicates:

Which means that it may have come from the bag itself and not from the air sampled:

A significant factor in the selection of filter media used for air sampling is the formation of artifacts due to the sorption of sulfur and nitrogen oxides on the filter. These artifacts can erroneously increase measured particulate concentrations. (1)

Here is what the BSEEC reported:

Thats over 21 samples (n's) for carbon disulfide, and of these samples, nine (9) were non-detectable.  Assuming that the concentration reported for this method were not artifacts, the highest concentration of  carbon disulfide was 11 ppbv.  

It should also be noted that only Tables 10, 14, 20, 23, and 26 showed a higher concentration of either contaminant downwind as compared to upwind ("D" = downwind in sample number)  The highest carbon disulfide concentration, 11 ppbv, was found in the upwind sample!

So the BSEEC report brings us a total of 114 n's for carbon disulfide.

Now lets look at the ERG Report, also referenced in the FWLN report.  ERG sampled nine (9) unique O&G sites plus one site designated as background:

At these sites a total of 92 detections for carbon disulfide were found with an average concentration of 0.243 ppbv and the highest concentration detected as 1.64 ppbv for a 24-hour sampling period:

So what do we know about the concentration of carbon disulfide in the air around oil & gas production facilities in the Barnett Shale?

Out of 206 samples (n's) the maximum concentration of carbon disulfide found in the air in the Barnett Shale area was no higher than 11 ppbv.

If Dr. Sattler's contention is true that from her one sample used in the FWLN was found to produce an emission rate that could produce up to 78 ppm (that's 78,000 ppb).  At the backed in calculated emission rate that she believes is emitting carbon disulfide, don't you think we would have seen concentrations above 11 ppbv in at least one of the 206 samples?

And the reason we are not seeing anything higher than 11 ppbv (or 10.8 if we assume the TIC she reported is accurate) is for one simple reason;  you cannot back in an ambient concentration into a Gaussian air dispersion model without placing directly downwind from the site and subtracting the upwind concentration from the result: (2)

The presence of carbon disulfide in the Barnett Shale area, regardless of source, is consistently way below the permitting ESL and the health based AMCV.

206 n's tell us this...and that's a lot of n's!

Next post: Air Quality in the Barnett Shale - Part 27:  Plot 2; Artifact or the Real Deal

.

Air Quality in the Barnett Shale - Part 25: What a statistically valid report looks like

So here we are with a bunch of reports...looking at a bunch of numbers...and making a bunch of different observations and conclusions.  Should we trust the TCEQ's findings?  Should we discount what the FWLN's report claims in terms of possible exposure to carbon disulfide?  What about ERG's report and all the work Alisa Rich did for the Town of Dish Texas?

It seems daunting, but it is not if you follow a certain - well recognized - set of basic scientific principles when collecting samples and analyzing them. So lets compare and contrast these reports.

One of my main complaints centers on the identification and quantification of the carbon disulfide concentration Dr. Sattler bases the setback on as well as an indication of potential health concerns.  And one of my main issues with the work performed by Dr. Sattler is the "backed in" data she uses to calculate her emission rate for the air dispersion modeling.  I believe I have laid out a fairly well substantiated reasoning for why her model's predictions are invalid.

Dr. Sattler assumes that with one air sample taken at a distance from the source you can back in to the air dispersion model and calculate the emission rate of the contaminant detected from that source.  With this emission rate, you can then plug it into the air dispersion model along with historical wind and weather information and predict the upper most concentration and distance from the source that could exceed a health based limit.

In my last post I attempted to show how that assumption - based on one single sample - assumes that one single sample accurately represents the air in and around the source.  n = 1 makes the model's predictions invalid, let alone the assumption that you can back in data to obtain the emission rate.  How confident is Dr. Sattler and her team on the representativeness of that one sample used to calculate carbon disulfide in Plot 1 and, most importantly, what do they base that confidence on?

In the ERG interim Report an emission rate was also calculated.  So why should ERGs stand and Dr. Sattler's be discounted?  Lets look at the assumption and the methodology used by ERG:

At 66 sites, preliminary quantitative emission estimates have been generated. Estimated annual emission rates were calculated from the short-term test data, assuming the measured emissions on the day of testing are representative of the emissions that occur over the year.

This is an assumption - it states it as such.  Scientist make assumptions all the time.  Then we test, test again, test some more, and research other similar published work.  Then when we look at all the data collected and we make a reasonable conclusion that our assumption is sound.

Even though ERG's emission rates were calculated from one SUMMA canister's results, there were 66 similar tests performed at 66 different locations.  This gives a representative snapshot as to what is most likely - reasonably - taking place.

A researcher can never know the true value of the population being sampled.  This is why we use statistics to calculate a mean, standard deviation, and variance.  It is with those numbers that we look at our assumptions and make a judgement to its soundness.

With 66 different samples of the population (air within oil & gas sites) collected by ERG, we can look at the report's tables to get a pretty good idea of what is being emitted - provided some additional methodologies were also performed.

Because bias is always present, the sampling team and laboratory need to follow a strict protocol on how they will collect the sample, transport the sample, prepare the sample, test the sample, and report the results.  This is all part of what is referred to as QA/QC.  If you look at the ERG report you will see a section QA/QC and if you look at the TCEQ report you will see the same.  If you look at the FWLN report and the reports by Alisa Rich for the Town of Dish, Texas, you will not see a QA/QC section. (The Dish Final Report does describe the sampling procedures and the analysis method performed.)

So the assumption is this; If the researchers follow their QA/QC plan, the impact of any bias will be reduced and the results can be considered representative and valid.  Additionally, the laboratory follows a similar QA/QC plan and also may choose to follow a strict methodology and become NELAC accredited.  (To be valid in Texas the laboratory must be NELAC accredited.)

For the laboratory that performs the analysis, it must be able to show that they are able to "accurately determine a compound and that they can acquire the same concentrations from different instruments or samples while they are sampling the same gas stream, with an acceptable level of uncertainty."  This is known as measurement precision.  ERG calculated analytical precision by collecting two sets of duplicate samples at two different sites and analyzed them in replicate (page 64). (1)

As well as precision, the laboratory is required to show that it has the "ability to acquire the correct concentration data from an instrument or sample analysis with an acceptable level of uncertainty while measuring a reference gas stream of a known concentration."  This is known as accuracy. (1)

All of this data to show precision and accuracy is identified and report by both ERG and the TCEQ in their two separate reports on air quality in the Barnett Shale.  This data is needed to support their assumption that the results they have put forth are representative and valid.  Since we can never know the actual - or true - value, we need this information to support what we can reasonably ascertain.

This information is missing from Alisa Rich's reports as well as the FWNL report.  With one exception...sort of...GD Air Testing, the laboratory used by FWLN and Alisa Rich - is NELAC  Accredited.

However, when they perform analysis to identify carbon disulfide - a TIC - that analytical method is "not included in the Scope of NELAC Accreditation" (see note 'N' on the Lab report).  Without the FWLN report's laboratory performing the analysis as per their NELAC Accreditation, the results for carbon disulfide is invalid.  That's not the fault of GD Air Testing, its just the way we have to look at data that has not been generated under a strict protocol and methodology designed to minimize the impact of bias and substantiate validity.

To meet this, the samples had to be prepared and analyzed using a method that is designed specifically for the contaminant of concern.

So how did ERG and the TCEQ analyze for carbon disulfide?  They used the correct method, TO-15.

ERG Report

TCEQ Report

And what did Alisa Rich (2) and Dr. Sattler (3) use for their reports?

Dish, Texas Report

FWLN Report

So the wrong method - TO-14 - was used by Alisa Rich and Dr. Sattler to identify the carbon disulfide.  This creates uncertainty on the validity of all the assumptions that were based off the carbon disulfide quantification.

On the other hand, the use of the correct method - TO-15 - reduces the uncertainty and adds to the validity of the results and assumptions made by the TCEQ and ERG in their reports.

Because we cannot know the true value, assumptions are made.  It then becomes necessary for the report writer to show how sound those assumptions are.  Without representative sampling, without QA/QC, and without using the correct analytical method whereby precision and accuracy could be shown, the assumption presented by the FWLN are invalid.

Our assumptions are only as good as our data is sound.

Air Quality in the Barnett Shale - Part 26: That's a lot of n's!


.

Air Quality in the Barnett Shale - Part 24: How confident in that 10.8 ppbv are you?

It has taken me a bit of time to get through the ERG "Interim Ambient Air Monitoring" Report as well as to re-read the Ft. Worth League of Neighborhoods (FWLN) "Recommendations for policy Changes for Gas Drilling Near Schools" report.

There is a lot of data to take in as well as looking at the methodology and conclusions.  I also try not to take things at face value and check the sources of the information (which is why I cite and link my sources).

So here is the issue;  Holy lots-o-dots Batman!  There's a bat-boat load of oil & gas related activity in the Ft. Worth area! (1)

So it is only natural that the public in and around these sites would question if their health and safety is being impacted.  And, when it comes to their children, as in "ensuring the safety of the 80,000 children who attend FWISD schools (2)" the need to be assured becomes elevated.  So what to do...what to do.

I am critical of bad science because it can cause people to do things that are more detrimental to their health then the thing they they were trying to avoid.  The issue in all of this should not be carbon disulfide.  Carbon disulfide is a non-issue and it is dominating the discussion and taking away from a real dialog which should be; What can we do to make oil & gas activity in and around homes and schools safer and reduce its impact on the environment?

The discussion should be; You want to extract here, then you need to adopt an "Environmentally Systems Drilling Friendly Program" I spoke about in a previous post.  Instead we have a "team of scientists and experts – Dr. Ramon Alvarez, Dr. Melanie Sattler, Dr. DavidSterling, and Carl Weimer – who donated their expertise and time to the League to produce this report (2)" describing a situation of concern that does not exist.

Whether a setback is a prudent idea is not what I am being critical of.  The FWLN report is basing this "one mile setback" on an air dispersion model's finding that carbon disulfide could be found at concentrations over three times the OSHA permissible exposure level (PEL) one mile from an O&G facility. (241 mg/m3 = 78 ppm)

Now reading this, and knowing that a "team of scientists and experts" - two of them Texas University professors - signed off on the findings, one might reasonably conclude; Holy exposure Batman!  The children!

But a closer look at how that number was derived would lead you to a different conclusion; Holy poor science Batman!  That data for carbon disulfide is invalid!

I have tried to present as objective and scientifically based argument as to why this is so in over 24 posts on this topic.  If I have not made my case then either you don't believe me, don't want to believe me, don't want to consider it, or have some other reason to ignore my conclusion as to why the methodology and findings in the Town of Dish Texas and the FWLN reports are wrong.

These blogs are open to anyone who wants to show where my premise, conclusion, or understanding of the science is incorrect, faulty, or wrong.  If I can dish it out (no pun intended) then I sure as better be able to take it.

So it comes down to this.  The FWLN report includes laboratory analysis for carbon disulfide for one single sample:

With the carbon disulfide concentration as:

So the air dispersion model generating Plot 1:

...was based on the analytical report shown above:

And if for no other reason than that - the results of the model and the conclusion for Plot 1 is invalid.  One sample cannot be used to test the null hypothesis that there is a relationship between the ambient air concentration at one location (SUMMA canister for 300 McNaughton Ln.)  and the expected ambient air concentration at another point (one mile out in Plot 1).

One sample is not representative of the air at that location.  Here is what EPA has to say about samples:

Representative: “a sample of a universe or whole (e.g., waste pile, lagoon, ground water) which can be expected to exhibit the average properties of the universe or whole."

Inferences about the population are made from samples selected from the population. For example, the sample mean (or average) is a consistent estimator of the population mean. In general, estimates made from samples tend to more closely approximate the true population parameter as the number of samples increases. The precision of these inferences depends on the theoretical sampling distribution of the statistic that would occur if the sampling process were repeated over and over using the same sampling design and number of samples.

This then leads to:

[a]fter a sample of a certain size, shape, and orientation is obtained in the field (as the primary sample), it is handled, transported, and prepared for analysis. At each stage, changes can occur in the sample (such as the gain or loss of constituents, changes in the particle size distribution, etc.). These changes accumulate as errors throughout the sampling process such that measurements made on relatively small analytical samples (often less than 1 gram) may no longer “represent” the population of interest.  Because sampling and analysis results may be relied upon to make decisions about a waste or media, it is important to understand the sources of the errors introduced at each stage of sampling samples and take steps to minimize or control those errors. In doing so, samples will be sufficiently “representative” of the population from which they are obtained.

When scientists make statements regrading their observations, the concept of precision and bias come into play:

• Precision is a measurement of the closeness of agreement between repeated measurements. 
• Bias is the systematic or consistent over- or underestimation of the true value

Precision is the ability to get the same - or very close - result each and every time you collect or analyze the sample.  Bias, on the other hand, results from a number of problems inherent in sampling and analysis. 

Sampling Bias: 

• Bias can be introduced in the field and the laboratory through the improper selection and use of devices for sampling and subsampling. Bias related to sampling tools can be minimized by ensuring all of the material of interest for the study is accessible by the sampling tool.

• Bias can be introduced through improper design of the sampling plan. Improper sampling design can cause parts of the population of interest to be over- or under-sampled, thereby causing the estimated values to be systematically shifted away from the true values. Bias related to sampling design can be minimized by ensuring the sampling protocol is impartial so there is an equal chance for each part of the waste to be included in the sample over both the spatial and temporal boundaries defined for the study. 

• Bias can be introduced in sampling due to the loss or addition of contaminants during sampling and sample handling. This bias can be controlled using sampling devices made of materials that do not sorb or leach constituents of concern, and by use of careful decontamination and sample handling procedures. For example, agitation or homogenization of samples can cause a loss of volatile constituents, thereby indicating a concentration of volatiles lower than the true value. Proper decontamination of sampling equipment between sample locations or the use of disposable devices, and the use of appropriate sample containers and 

Analytical Bias:

• Analytical (or measurement) bias is a systematic error caused by instrument contamination, calibration drift, or by numerous other causes, such as extraction inefficiency by the solvent, matrix effect, and losses during shipping and handling.

Statistical Bias:

• When the assumptions made about the sampling distribution are not consistent with the underlying population distribution, or
• When the statistical estimator itself is biased.

Because bias is always in play, the number of samples collected (replicates) and the dates/areas (representative) is increased to minimize the impact of these issues.

So how confident in the number 10.8 ppbv is the report's "team of scientists and experts?" How sure are they that 10.8 ppbv represents the true value of the air at that sample point?  I mean. look at all the potential errors that could have impacted it.  Shouldn't at least a duplicate sample have also been collected and analyzed?

It was that one single value that was used to determine the concentration of 78 ppm determined by the model to be in the plume:

Plume extends 1 mile from the source in this graphic. Full extent of plume was in excess of 2 miles. Plot 1 multiples were up to 1000 times the short term health benchmark for carbon disulfide.

How confident would (could) any reputable scientist be if one - and only one - sample - was used in their published research?

For that reason alone, Plot 1 is not valid.

Now, couple that with this:

 Notice that "N" in the very last column?  That's a note from the laboratory:

What the lab is saying is that all the statistical stuff they do - precision & accuracy - was not performed on this sample.  How confident are they in the number 10.8 ppbv?

So there you have it.  Plot 1 was developed using one sample and analyzed on an instrument that was not calibrated for carbon disulfide.

Holy bias Batman!  Plot 1 values are not statistically valid!


Yes Robin, that's what I have been trying to say all along.  The whole enchilada is an example of bad methodology, sampling, and analysis - and the results and conclusion are invalid.


Next post: Air Quality in the Barnett Shale - Part 25: What a statistically valid report looks like.


.