Content uploaded by Henry Christopher Frey
Author content
All content in this area was uploaded by Henry Christopher Frey
Content may be subject to copyright.
1
Uncertainty for Data with Non-Detects: Air Toxic Emissions 1
from Combustion
2
3
Pre-Print 4
5
Published As: 6
7
Zhao, Y. and H.C. Frey, “Uncertainty for Data with Non-Detects: 8
Air Toxic Emissions from Combustion,” Human and Ecological Risk 9
Assessment, 12(6):1171-1191 (Dec 2006). 10
11
2
ABSTRACT 1
2
Air toxic emission factor data sets often contain one or more points below a single or multiple 3
detection limits and such data sets are referred to as “censored”. Conventional methods used to 4
deal with censored data sets include removing non-detects, replacing the censored points with 5
zero, half of the detection limit or the detection limit. However, the estimated means of the 6
censored data set by conventional methods are usually biased. Maximum likelihood estimation 7
(MLE) and bootstrap simulation have been demonstrated as a statistically robust method to 8
quantify variability and uncertainty of censored data set and can provide asymptotically unbiased 9
mean estimates. The MLE/bootstrap method is applied to 16 cases of censored air toxic emission 10
factors, including benzene, formaldehyde, benzo(a)pyrene, mercury, arsenic, cadmium, total 11
chromium, chromium VI and lead from coal, fuel oil and/or wood waste external combustion 12
sources. The proportion of censored values in the emission factor data ranges from 4 to 80 13
percent. Key factors that influence the estimated uncertainty in the mean of censored data are 14
sample size and inter unit variability. The largest range of uncertainty in the mean was obtained 15
for the external coal combustion benzene emission factor, with 95 percent confidence interval of 16
the mean equal to minus 93 to plus 411 percent. 17
18
KEYWORDS: Urban air toxics, Emission factor, Censored data sets, Maximum likelihood 19
estimation, Bootstrap simulation 20
21
22
23
24
3
INTRODUCTION 1
2
The U.S. Environmental Protection Agency (EPA) has developed a list of 33 air toxics 3
that represent the priority for additional assessment of human exposure and health effects in 4
urban areas (Smith et al. 1999). In an exposure model to calculate average daily dose, emission 5
estimates are a key input and are used to calculate the contaminant concentration (EPA, 1997a). 6
Because exposure assessments should include explicit quantification of variability and 7
uncertainty (NAS 1994; EPA 1997b; Cullen and Frey 1999), there is a need for methods and 8
case studies for probabilistic emission estimates for air toxics. 9
Air toxics emissions are subject to both variability and uncertainty (Patrick 1994; Frey 10
and Rhodes 1996; Frey and Bharvirkar 2002). Variability and uncertainty in air toxics emissions 11
are a contributing factor to variability and uncertainty in estimates of exposure and risk. A key 12
step to develop probabilistic air toxic emission estimates is to quantify variability and uncertainty 13
in the emission factors. Uncertainty in a mean emission factor is attributable to random sampling 14
error, measurement error and non-representativeness (Frey and Bammi, 2002; Frey and Rhodes, 15
1996; Frey and Zheng, 2002a; Frey and Zheng, 2002b; Zheng and Frey, 2005). The uncertainty 16
in urban air toxic emission factor is typically large, on the order of a factor of two (Zhao and 17
Frey 2004b). An uncertainty analysis helps to answer key questions posed by decision makers 18
regarding how good numbers are, the key sources of uncertainty, and where resources should be 19
targeted in order to reduce uncertainty (Bloom et al., 1993; Thompson and Graham 1996). 20
Quantification of variability and uncertainty in air toxics emissions also helps to characterize the 21
quality of an emission inventory and to target data collection to reduce uncertainty. 22
A challenge to quantification of inter-unit variability in air toxic emissions is that, 23
because of inherent limitations of chemical/analytical measurement methods, emission factor 24
data sets often contain several observations reported as below a detection limit (DL), and are 25
referred to as “left - censored” (Rao et al. 1991). Multiple detection limits arise when individual 26
measurements are collected by different sampling and analytical procedures at different facilities 27
within a source category. If non-detected measurements are not properly accounted for, 28
variability and uncertainty in emissions, exposure and risk could be significantly misestimated 29
(Chrostowski 1994). 30
4
Four conventional methods are usually used for estimating means of censored data (Hass 1
and Scheff 1990; Helsel, 2005): 2
• Using values only above DL to calculate a mean value, and ignoring information 3
regarding non-detects, which leads to a biased estimate of the mean; 4
• Replacing values below DL with zero, which leads to an underestimate of the true mean; 5
• Replacing values below DL by DL/2, which leads to an approximate but biased estimate 6
of the true mean; and 7
• Replacing values below DL by DL, which leads to an overestimate of the true mean. 8
The conventional methods produce biased estimates of the mean and of other statistics, 9
such as the variance. The bias typically worsens as the amount of censoring increases. In 10
contrast, the maximum likelihood estimation (MLE) method, which is asymptotically unbiased, 11
has been used to fit parametric distributions to environmental censored data (Newman et al. 12
1989; Gilliom and Helsel 1986; Elvira et al. 1999; Burmaster and Wilson 2000). The method has 13
been found to be more accurate compared to the conventional methods (Newman et al. 1989; 14
Gilliom and Helsel 1986; Elvira et al. 1999; Burmaster and Wilson 2000). 15
Helsel (2005) summarize statistical methods used to deal with censored environmental 16
data, including MLE, Kaplan-Meier estimator and a so called “robust method” based on 17
regression on order statistics. The Kaplan-Meier estimator is a non-parametric approach. It 18
provides estimates of statistics (e.g. mean and variance) as well as confidence intervals of the 19
empirical population distribution for the data (Miller et al 1981). However, it does not lead to 20
estimates of the confidence intervals of the statistics. The so called “robust method” assumes a 21
distribution only for the censored portion of the distribution when estimating statistics, but 22
requires normality or lognormality when estimating confidence intervals for the mean. Due to the 23
limitations of the Kaplan-Meier estimator and the so called “robust method”, a method which is 24
able to quantify variability and uncertainty for censored environmental data in a broader domain 25
needs to be demonstrated. 26
Goodness of fit tests are used to determine if a candidate distribution type fit to a data set 27
should be rejected. The Shapiro-Francia test is used to test normality of uncensored data and has 28
been adjusted for censored data (Shapiro and Wilk 1965; Verrill and Johnson, 1988). It can also 29
be applied to log-transformed data. However, it is not directly applicable to other types of 30
5
distributions, such as gamma and Weibull, and thus is not applied in our study as a general 1
approach for goodness of fit test for censored data. 2
Censored data are different from missing data. Missing data refer to data that are not 3
recorded or measured, are unknown or lost, and no detection limits are reported. The methods for 4
dealing with missing data in data mining methods include ignoring the missing values, omitting 5
any records containing missing values, replacing missing values with the mode or mean, or 6
inferring missing values from existing values. Typical statistical methods other than MLE, such 7
as Gibbs sampler, can be used to analyze missing data (Albert and Chib 1993). 8
Besides left-censored data, there are also right-censored which contains nondetects larger 9
than the detection limit and interval-censored data which contains nondetects between two 10
detection limits. Right censored data occurs in many biomedical applications when the primary 11
endpoint of interest is time to a certain event, such as time to death (Miller et al 1981). 12
Interval-censored data sometimes appear in studies on emergence times of teeth (Bogaerts 2002). 13
Zhao and Frey (2004a) demonstrated an asymptotically unbiased method based upon 14
MLE and bootstrap simulation for quantifying inter-unit variability, and uncertainty in statistics 15
such as the mean, for censored data sets. In order to evaluate the method, it was applied to 16
synthetic data with various degrees of censoring (e.g. 0%, 30% and 60%), sample sizes (e.g. 20, 17
40 and 100), coefficients of variation (e.g. 0.5, 1 and 2) and number of detection limits (1, 2 and 18
3) as well as using different parametric distributions of lognormal, gamma and Weibull. The 19
method proved to be robust and reliable for quantifying variability and uncertainty for censored 20
data sets under these situations. The confidence intervals for distribution percentiles estimated 21
with MLE/bootstrap method compared favorably to results obtained with the nonparametric 22
Kaplan-Meier estimator. 23
In this study, the MLE/bootstrap method was applied to extensive empirical emission 24
factor data for combustion sources in order to quantify inter-unit variability and uncertainty in 25
mean emissions for selected air toxics. The proportion of censoring in these data ranges from 4 26
to 80 percent (EPA 1993a&b; EPA 2001). External combustion sources include steam-electric 27
generating plants, industrial boilers, and commercial and residential combustion systems, such as 28
for space heating. These sources, which use fuels including coal, fuel oil, and wood waste, are 29
significant emission sources of toxic air pollutants. For example, the coal-fired electric utility 30
6
generation industry has been identified as the largest anthropogenic source of mercury emissions 1
in the United States (Li et al, 2005). 2
The reliability of the emission factor data as reported by EPA is coded with data quality 3
ratings, ranging from A to D based on the test method, sample size and a judgment regarding the 4
representativeness of the data (EPA 1997c). However, such information can not be used to 5
quantitatively assess the uncertainty in the emission factor. In order to quantify the uncertainty in 6
the emission factor, statistical methods are applied here. 7
The objective of this paper is to demonstrate the use of the MLE/bootstrap method to 8
quantify the variability and uncertainty in censored air toxics emission factor data, based upon 9
selected case studies. The key questions addressed in this paper include: 10
• How should inter-unit variability and uncertainty in emission factors be quantified for 11
censored data sets? 12
• Is the mean estimate and the estimate of uncertainty in the mean sensitive to the choice of 13
parametric distribution for inter-unit variability? 14
• What characteristics of censored data sets are important determinants of uncertainty in 15
the mean? 16
• Can comparable results for uncertainty in the mean be obtained from a simplified 17
approach? 18
• What is the relative range of uncertainty in the mean estimates of selected air toxic 19
emission factors? 20
METHODOLOGY 21
The key elements of the MLE/bootstrap simulation, and the approaches used to assess 22
goodness-of-fit of parametric distributions compared to empirical data, are presented in this 23
section. 24
Method of Maximum Likelihood Estimation 25
In order to fit a parametric distribution representing inter-unit variability to censored data, 26
MLE is used to estimate the distribution parameters based upon the observed sample of data. 27
MLE methods are computed by solving a likelihood function (L), which defines the likelihood of 28
matching the observed distribution of data. L is a function of the parameters of the distribution. 29
L increases as the fit between the estimated distribution and the observed data improves. 30
Therefore the parameters of the distribution can be specified by optimization techniques, 31
7
choosing values to maximize L. For uncensored data, L is the product of the probability density 1
function for observed data, which describes the likelihood of observing individual values of xi 2
(Helsel, 2005). When applying MLE to left-censored data, the cumulative probability of the 3
detection limit, which defines the probability of being less than or equal to the detection limit, is 4
used in lieu of the likelihood for each non-detected measurement. The likelihood function for left 5
-censored data sets having multiple detection limits is (Cohen and Whitten 1988): 6
=∏ ∏∏
= ==
p
m
ND
jkmk
n
ii
mDLFxfL 1 1 212
11321 ),,,(),,,(),,(
θθθθθθθθθ
(1) 7
Where, 8
θ θ θ
, , ,
2
k
=
Parameters of the distribution 9
xi = Detected data point, where, i = 1, 2, …, n 10
NDm = Number of non-detects corresponding to detection limit DLm, 11
where, m = 1, 2, …, P. 12
P = Number of detection limits 13
f( ) = Probability density function 14
F( ) = Cumulative distribution function 15
Likelihood functions for right-censored data and interval-censored data are also available 16
(Cohen and Whitten, 1988). 17
Lognormal, Gamma and Weibull Distributions 18
For environmental data sets, such as concentrations or emission factors, lognormal, 19
gamma and Weibull distributions are often chosen as parametric distributions to represent 20
variability in data (Seinfeld 1986; Cullen and Frey 1999; Frey and Zheng 2002a&b; Frey and 21
Bharvirkar 2002; Frey and Bammi 2002). One of the most widely used distributional forms in 22
probabilistic assessment is the lognormal distribution. The lognormal distribution describes 23
random variables resulting from multiplicative processes (Ott 1990; Ott 1995). The gamma 24
distribution is non-negative, positively skewed, and similar to the lognormal distribution in many 25
cases but it is less “tail heavy” (Cullen and Frey 1999). The Weibull distribution is a flexible 26
distribution that can assume negatively skewed, symmetric, or positively skewed shapes (Cullen 27
and Frey 1999). It also may be used to represent non-negative quantities. 28
8
A comparison of the use of parametric distributions versus empirical distributions for 1
environmental data is described by EPA (1999). The choice between the two is usually a matter 2
of preference. Some prefer parametric distributions because empirical distribution functions are 3
restricted to the range of observed data while parametric distributions typically provide estimates 4
of “tails” of the distribution beyond the range of observed data, which may have intuitive or 5
theoretical appeal (EPA, 1999), and enable interpolation between observed data. The accuracy of 6
results obtained using the MLE/bootstrap method depend on the adequacy with which the 7
candidate parametric distributions provide a good representation of the data. The analyst should 8
appropriately select, evaluate, and justify the choice of parametric distribution based on 9
theoretical and empirical arguments (e.g., Cullen and Frey, 1999; Hahn and Shapiro, 1967). 10
Often, a graphical comparison of the fitted distribution versus the observed data is very helpful 11
and is recommended. 12
Bootstrap Simulation 13
Parametric bootstrap simulation is often used to estimate confidence intervals for 14
statistics of data sets or parameters of fitted distributions in cases without censoring (Efron and 15
Tibshirani 1993; Frey and Rhodes 1996; Frey and Bammi 2002; Sadiq et al 2002; Frey and 16
Bharvirkar 2002; Faraggi 2003). In conventional parametric bootstrap simulation, a parametric 17
probability distribution representing variability is fit to the observed data, which has a sample 18
size of n. To model random sampling error, the Monte Carlo approach is used to randomly 19
simulate B synthetic data sets, referred to as bootstrap samples, each of sample size n. Parametric 20
distributions are fit to each bootstrap sample and statistics of each distribution are calculated. B 21
replications of the statistics are obtained. The sampling distribution or confidence intervals of a 22
given statistic, such as the mean, variance, or distribution percentiles are further estimated based 23
upon B replications of the statistic. For example, the best estimate of the mean emission factor is 24
the average of the means from the replicates of the cumulative distribution function (CDF) in 25
bootstrap simulation. For censored data with only one detection limit that is smaller than all 26
observed values, parametric bootstrap simulation can also be used. In each bootstrap sample, 27
simulated values below the detection limit are identified as non-detects. Thus, each bootstrap 28
sample would be comprised of censored data with the same detection limit as the original data 29
set. 30
9
However, in the case of censored data with multiple detection limits, or with a single 1
detection limit that is larger than at least one detected point, the parametric bootstrap simulation 2
method discussed above can not be directly applied. In order to generate bootstrap samples with 3
random variation in the number of non-detects and for which some detection limits may be larger 4
than the sample values of some observations, empirical bootstrap simulation using bootstrap 5
pairs is used. Each data point in the original data set is paired with a binary indicator variable. 6
The binary indicator variable denotes whether the data point is an observation or a detection 7
limit. Both the data point and the indicator symbol are sampled together in an empirical bootstrap 8
simulation. Therefore, the status of each data point is known. The MLE method for fitting 9
distributions to censored data is applied to each bootstrap sample to get B replications of the 10
CDF and of statistics of interest (Zhao and Frey 2004a). The process is shown in Figure 1. In this 11
way, the variability and uncertainty of multiply censored data can be quantified. 12
Evaluation of Goodness-of-fit 13
The Kolmogorov-Smirnov (K-S) test and graphical comparison of the CDF of the fitted 14
distribution to the data are widely used to evaluate the goodness-of-fit of parametric distributions 15
fit to uncensored data (Morgan and Henrion 1990; Cullen and Frey 1999; Lu 2003). One 16
advantage of the tests is that they do not require a large sample sizes for the tests to be valid 17
(Morgan and Henrion 1990; Cullen and Frey 1999). For example, for the K-S test, the minimum 18
sample size is 5; for graphical comparison, the minimum sample size is 3. However, the K-S test 19
cannot be directly applied in the case of a censored data set. To gain insight regarding goodness-20
of-fit for a censored data set, an approximation procedure is used. For this purpose only, each 21
non-detected measurement was replaced with one half of its detection limit to create a modified 22
data set. The K-S test was applied to a distribution that was fit to the modified data set. If the 23
fitted distribution was not rejected by the K-S test at significance level of 0.05 for the modified 24
data, it was taken as a reasonable candidate parametric distribution for the original censored data. 25
The candidate parametric distribution type was then fit to the censored data taking into account 26
the presence of non-detects, resulting in different parameter estimates than for the modified data 27
set. This is a semi-quantitative approach that can be used to guide the selection of several 28
candidate probability distribution models to fit to censored data. A comparison of the bootstrap 29
confidence intervals for the CDF of the fitted distribution to the observed data was finally 30
applied to confirm the adequacy of the fit and to help guide the choice regarding a preferred 31
10
distribution. On average, it is expected that 95 percent of the data will fall inside of a 95 percent 1
confidence interval of CDF of a fitted distribution if the data are a random sample from the 2
assumed population distribution. The larger the proportion of data contained within the 3
confidence intervals, the stronger the preference toward a particular candidate distribution. 4
CASE STUDIES 5
In this section, the variability and uncertainty of 16 censored air toxic emission factor 6
data sets was quantified. The results are given regarding selection of a best fitting parametric 7
distribution for inter-unit variability, the MLE parameter estimates of the fitted distribution, and 8
the bootstrap confidence intervals for the mean which represent the uncertainty in the mean. The 9
mean estimates were compared with those from conventional methods. 10
Data 11
Empirical censored air toxic emission factor data from external combustion sources were 12
obtained from background documents of EPA’s report AP-42 that compiles emission factors for 13
stationary sources (EPA 1993a&b; EPA 2001). The fuels include coal, fuel oil and wood waste. 14
Urban air toxic pollutants for which data were available for one or more of the three fuels 15
considered include benzene, formaldehyde, benzo(a)pyrene (B(a)p), mercury, arsenic, cadmium, 16
chromium and lead. 17
Table 1 summarizes the available emission factor data with respect to the pollutant, fuel, 18
sample size, percentage of censoring, multiple detection limits, variability factor, relative 19
maximum detection limit, unit and EPA qualitative emission factor ratings. The variability factor 20
is the ratio of the largest detected value divided by the smallest detected value. The relative 21
maximum detection limit is defined as the largest detection limit divided by the largest detected 22
value. 23
Each data set in Table 1 is for a specific urban air toxic emitted from combustion of a 24
specific fuel type. The sample sizes vary from 8 to 29. The censored data of Case Nos. 1, 2, 13 25
and 14 have a single detection limit, while the others have multiple detection limits. There are 26
nine data sets in which less than 30% of the observations are censored. Four data sets have 27
between 30% and 60% censoring and three have greater than 60% censoring. The latter are 28
defined as highly censored data. The variability factors range from 2 to more than 30,000. For 29
Case Nos. 3, 7, 8 and 16, one or more detection limits are larger than the largest detected value. 30
Quantification of Inter-Unit Variability and Graphical Evaluation of Goodness-of- 31
11
Fit 1
The approximation procedure for identifying promising candidate distributions to fit to 2
the censored data was applied to 13 of the 16 emission factor data sets. Data sets for Case Nos. 3, 3
5 and 8 were not included because the procedure was deemed to be unreliable in cases with a 4
high degree of censoring. For these three cases, all three types of parametric distributions were 5
considered for evaluation in the later step involving graphical analysis. Of the remaining 13 6
cases, for 12 of them two or more of the three types of parametric distributions fit to the 7
modified data were not rejected by the K-S test at a 0.05 significance level. For Case 9, for 8
which all of the candidate distributions fit to the modified data were rejected, the difference 9
between the critical and test values of the K-S statistic was considered small for both the 10
lognormal and Weibull distributions; therefore, these two distributions were further evaluated 11
graphically. 12
The MLE method was used to fit each type of candidate parametric distribution identified 13
based upon “approximation procedure” to each original censored data set. As a second step for 14
evaluating goodness-of-fit, the confidence intervals for the fitted distribution were evaluated 15
using bootstrap simulation, as previously described. The inter-unit variability of the observed 16
data, the detection limits, the fitted parametric distribution, and the confidence intervals of the 17
fitted distribution were compared graphically. 18
The procedure is illustrated with Hg emissions from coal (Case 7) as shown in Figures 19
2(a), 2(b) and 2(c) for the lognormal, gamma and Weibull candidate distributions. The emission 20
factor data contain 29 data points, of which three are censored. Each of the three censored data 21
points has a different detection limit. The three detection limits are represented by dashed 22
vertical lines. In order to plot the data as a cumulative distribution function, it is necessary to 23
estimate the rank of each data point. In this case, all of the 26 detected data points are smaller 24
than the largest detection limit. For detected points which are smaller than one or more detection 25
limits, they have a range of possible ranks. For example, all three non-detected data points range 26
from zero to their detection limits and could have values either larger or smaller than the smallest 27
observed data point, hence, the smallest observed data point could have a rank ranging from 1 to 28
4. The largest observed data point could be larger or smaller than the true but unknown value 29
associated with the non detect that has the largest detection limit. Hence the rank of the largest 30
observed value could be 28 or 29. Therefore, there is ambiguity regarding what rank to assign to 31
12
each observed data point. The possible ranges of the cumulative probabilities of the detected 1
points are represented by vertical solid lines in the figures. 2
The 95 percent bootstrap confidence intervals enclose almost all of the detected points for 3
all three candidate distributions. The exceptions are that portions of the ranges of the possible 4
cumulative probabilities of six detected points for the lognormal distribution, three detected 5
points for the gamma distribution and two detected points for the Weibull distribution are partly 6
outside the 95 percent confidence interval. Thus the Weibull distribution is judged to provide the 7
preferred fit among the three distribution types. The lognormal distribution is “tail-heavy” and 8
gives a shorter lower tail but a longer upper tail than the gamma and Weibull distributions. 9
However, for this case, a mixture distribution seems more appropriate and should be evaluated in 10
future work. 11
For the other cases, a similar comparison of confidence intervals of the CDF to the 12
detected points was applied in order to determine the best fit from the candidate parametric 13
distributions. Generally, all of the candidate parametric distributions describe the data points 14
well. Since the lognormal distribution has a heavier upper tail than the gamma and Weibull 15
distributions, it is typically a better choice for more highly positively skewed data while the 16
Weibull distribution is typically more suitable for less positively skewed data. The recommended 17
distribution type for each case is given in Table 2. To illustrate typical results obtained, several 18
examples are shown graphically. Figure 3 displays results for Case 11 which has a relatively 19
large sample size of 28 and 5 censored data points. Case 10, as shown in Figure 4, has only one 20
detected point larger than the largest detection limit. Case 13 is a single detection limit case as 21
shown in Figure 5. As shown in Figure 6, Case 5 is an example with a high degree of censoring. 22
There is large amount of uncertainty in the non-detected region for this case. For these cases, 23
some data points are larger than the largest detection limit and their cumulative probabilities are 24
represented by circles. 25
Estimation of the Mean and Quantification of its Uncertainty 26
Based on the MLE/bootstrap simulation method, an asymptotically unbiased best 27
estimate of the mean and the uncertainty in the mean can be quantified (Zhao and Frey 2004a). 28
The MLE/bootstrap method was applied to all cases in Table 2 for each candidate parametric 29
distribution. The 95 percent confidence interval of the mean is shown in the table as a relative 30
percentage from the mean. For three of the emission factor data sets that had a large proportion 31
13
of censoring, Case Nos. 3, 5 and 8, all three candidate distributions were included for evaluation 1
using MLE/bootstrap simulation. However, because of numerical instabilities associated with 2
high proportions of censoring, ranging from 64 to 85 percent, it was not possible to obtain 3
reliable parameter estimates in some cases. For example, for a Weibull distribution applied to 4
Case 8, 10 out of 500 of the bootstrap parameter estimates were negative. Therefore, as a quality 5
control step, the results of such simulations were not included in the final analysis. Furthermore, 6
for each of these three cases, there was at least one parametric distribution for which reliable 7
results were obtained. 8
In some cases, all of the candidate distributions were good fits when evaluated 9
graphically, such as for Case Nos. 6, 7, 10, 14, and 16. In such cases, the differences in the best 10
estimated means were within 25% when comparing the results from alternative distributions to 11
that from the preferred distribution. The differences in the absolute upper and lower levels of the 12
95 percent confidence interval of the mean were within 10% when comparing the results from 13
alternative distributions to that from the preferred distribution. 14
For cases in which one of the candidate parametric distributions was clearly a better fit 15
than the others, the mean values and the 95 percent confidence intervals for the mean would 16
differ more substantially, such as for Case Nos. 1, 2, 5, 9, 11, 12, 13, and 15. In most of these 17
cases, the lognormal distribution was a better fit and was associated with a larger mean value and 18
a larger upper bound to the 95 percent confidence interval on the mean than the other candidate 19
distributions. 20
The estimated uncertainty is dependent upon sample size and inter-unit variability. Cases 21
with a 95 percent confidence interval wider than 200 percent of the mean based upon the 22
preferred distribution include Case Nos. 1, 4, 5, 9, 11 and 12. Cases Nos. 1, 9, and 11 have large 23
inter-unit variability. Case Nos. 4, 5 and 12 have relatively small sample sizes. The 24
recommended 95 percent confidence intervals of the means that are narrower than 100 percent 25
include Case Nos. 7, 8, 14 and 16. Case 7 has the largest sample size of 29. Case Nos. 8, 14 and 26
16 have small inter-unit variability. 27
Case Nos. 10 and 12 are similar in terms of sample size, number of detection limits, 28
proportion of censoring, and the estimated variability factor. However, they differ significantly 29
in the relative range of uncertainty for the mean. The data of Case 12 are more positively skewed 30
and are well-described by a lognormal distribution, whereas the data for Case 10 are well-31
14
described by a less skewed and less tail-heavy Weibull distribution; the heavy upper tail of the 1
lognormal distribution significantly increases the estimation of the upper level for the confidence 2
interval of the mean. Therefore, the 95 percent confidence interval for Case 12 is more than 3
twice as wide, on a relative basis, than for Case 10. 4
The uncertainty results, represented by the width of the 95% confidence intervals in the 5
estimated means in Table 2, are not comparable to the EPA data quality ratings in Table 1. For 6
example, the EPA data quality for Case No. 1 is A, but it has the largest uncertainty calculated 7
from the MLE/Bootstrap method. 8
Comparison of Conventional and MLE/Bootstrap Methods for Mean Estimates 9
A comparison of the estimated means from the conventional and MLE/Bootstrap 10
methods is shown in Table 3. The variation in mean estimates among the various methods is 11
typically small when the percentage of censoring is small, such as less than about 25 percent. 12
However, for cases in which the largest detection limit is comparable to or larger than the largest 13
observed data, there is more variation in the means estimated from the various methods. For 14
example, both Cases 7 and 9 have 10.3 percent of censoring, but the variation in the means 15
estimated from Case 7 is larger than that from Case 9 since the former has a large detection limit 16
while the latter has small detection limits. In contrast, for cases with a high percentage of 17
censoring, such as Case 8, there can be substantial variation in the mean estimates. 18
Replacing the non-detects with zero clearly produces estimates of the mean that are 19
biased low, while replacing the non-detects with the detection limit produces estimates of the 20
mean that are biased high. The true but unknown mean is generally expected to be enclosed 21
within the range of these two. This range is defined here as a “reference range.” The best 22
estimated mean from the MLE/bootstrap method is enclosed by the “reference range,” for Case 23
Nos. 3, 4, 6, 7, 8, 10, 15 and 16. However, since the MLE/bootstrap method extrapolates into the 24
censored range, taking into account observed data above the detection limit(s), it is possible at 25
times for the estimated means to be outside the “reference range” such as for Case Nos. 1, 2, 5, 9, 26
11, 12, 13 and 14. However, in all cases, the 95 percent confidence intervals of the 27
MLE/bootstrap means overlap with the “reference range.” 28
Comparison of Results for Censored Versus Modified Data 29
The purpose of this section is to compare the results of the MLE/Bootstrap method to a 30
more commonly used approach of replacing non-detects with one-half of the detection limit, in 31
15
order to illustrate key differences as well as the benefits of the MLE/Boostrap approach. For this 1
purpose, two data sets were considered for each emission factor: (1) the original censored data 2
set and (2) a modified data set in which each non-detect was replaced with one-half the detection 3
limit 4
To facilitate comparisons, ratios of results from the two data sets are shown in Table 4 for 5
the estimated means and for the width of the 95 percent confidence intervals for the mean. These 6
results are based upon the preferred distribution for each case as identified in Table 2. 7
Differences in estimates of means from the two data sets of less than 10 percent and differences 8
in the width of confidence intervals of less than approximately 25 percent were deemed not to be 9
substantial. Differences in estimated means of more than 10% were considered substantial, 10
which occurred for Case Nos. 3, 5, 7, 8, 9 and 12. Among these cases, Case Nos. 3, 5 and 8 have 11
greater than 60% censoring. Case Nos. 3, 7 and 8 have extreme detection limits which are much 12
larger than the largest detected value. The estimated means of the modified data are much larger 13
than those of the original censored data for Case Nos. 3, 7 and 8, which implies a tendency to 14
overestimate the mean for data with large proportions of censoring, large detection limits, or 15
both. In contrast, for Case Nos. 9, and 12, the means based upon the modified data are clearly 16
underestimated. For these two cases, the percentage of censoring is only 23 percent or less and 17
do not have extreme detection limits. However, these two cases have multiple detection limits 18
and involve substantial inter-unit variability in the detected values. 19
Based on these results, the key findings are that the MLE/bootstrap method is particularly 20
advantages under conditions of: (1) large percentage of censoring; and (2) large detection limits 21
relative to observed values; and (3) large proportion of variability and multiple detection limits. 22
The with of the 95 percent confidence interval for the mean estimated based upon the 23
modified data tends to be narrower than that based upon the original censored data. This implies 24
that failure to accurately account for censoring will typically lead to underestimation of 25
uncertainty in the mean. 26
However, when there are large DLs in the censored data, the uncertainty in the mean can 27
be more sensitive to a fixed value of DL/2 in the modified data than to an unfixed value ranging 28
from zero to DL in the censored data. For example, Case 8 has a relative maximum detection 29
limit larger than 75, and the width of the 95% confidence interval in the mean for the modified 30
data is much wider than that for the censored data. 31
16
1
CONCLUSIONS AND RECOMMENDATIONS 2
This study illustrates the application of MLE/Bootstrap method to quantify variability 3
and uncertainty in censored air toxic emission factor data from combustion sources. MLE was 4
used to fit parametric distributions to quantify the inter-unit variability. MLE is asymptotically 5
unbiased and takes into account the presence of non-detects. The uncertainty in selected statistics 6
and in the CDF was quantified using bootstrap pair simulation. 7
Simplified conventional methods for dealing with censored data provide biased estimates 8
of the mean. Results with conventional methods worsen when there are large detection limits 9
comparable to or larger than the largest observed data and for situations involving a large 10
percentage of censoring. The MLE/Bootstrap method provided consistent results for censored 11
data with single or multiple detection limits for 16 data sets evaluated. Even for data with 12
censoring as high as 80 percent, the MLE/Bootstrap method provided reasonable results when a 13
particular parametric distribution was an appropriate fit to the data. Although the MLE/bootstrap 14
method is more computationally intensive, it offers advantages of being asymptotically unbiased 15
and of providing uncertainty estimates. Thus, it is recommended for analyzing censored data 16
instead of using conventional methods. 17
The estimates of the mean and uncertainty in the mean are relatively insensitive to the 18
choice of parametric distribution when multiple types of distributions provide comparable 19
goodness-of-fit. However, as the degree of discrimination with regard to goodness-of-fit 20
becomes more pronounced among alternative distributions, the differences in estimates of the 21
mean and uncertainty in the mean become more substantial. In these latter cases, there is clearly 22
a difference, for example, in results obtained for a lognormal distribution compared to the less 23
tail-heavy gamma or Weibull distributions. 24
Sample size and variability in the censored data sets influence the estimated uncertainty. 25
For smaller sample sizes and larger variability in the data, the range of uncertainty in the mean or 26
in the CDF is typically larger. The nature of the censoring in a data set also has influence on the 27
uncertainty results. The ranges of uncertainty of the portions of the CDF below detection limit 28
become larger with more censoring. In particular, a large percentage of censoring, detection 29
limits large relative to the largest detected value, multiple detection limits, or some combination 30
of these three, typically leads to larger estimates of uncertainty in the mean than could be 31
17
obtained with simplified approximation procedures. Generally, the uncertainty in the mean is 1
enlarged by censoring compared to otherwise similar data sets that have no censoring. In 2
summary, the censoring issues become critical in the following situations: 3
• When there are some detection limits that are larger than the largest detected value; 4
• When there is a large amount of censoring; 5
• When the data are not very positively skewed and thus the influence by censoring on the 6
mean is more considerable; 7
• When a less tail heavy distribution, such as gamma and Weibull, is considered. 8
The width of the recommended 95 percent confidence interval of the mean ranges from 9
62 percent to 504 percent relative to the mean value among the 16 cases. When the range of 10
uncertainty is large, the confidence intervals are asymmetric because emission factors must be 11
non-negative. The large uncertainty in the censored emission factor data sets suggests that it is 12
important to quantify uncertainty. The quantified uncertainty should be taken into account when 13
reporting and using censored emission factors. 14
The quantified uncertainty here includes random sampling error and measurement error 15
in emission factors. Random measurement error is accounted for because the observed variability 16
in the data includes both the true variability and the random component of measurement error, 17
which in turn influences the range of the sampling distribution of the mean. However, since 18
information regarding the contribution of measurement error to each measurement is not 19
available, the uncertainties caused by the two types of error are not separately quantified. 20
Methods for separating measurement error from true variability are detailed elsewhere (Zheng 21
and Frey, 2005). 22
The EPA emission factor quality ratings are subjective and are not correlated with 23
quantitative confidence intervals obtained based on statistical analysis of the data. 24
The best estimated mean from MLE/Bootstrap method may be biased for a single case 25
when the sample size is small. However, for all of the 16 cases evaluated here, the estimated 26
uncertainty ranges of the mean were consistent with the bounding ranges of results obtained from 27
conventional methods in which either zero or the detection limit was assigned to each non-detect. 28
Thus, the MLE/bootstrap method is demonstrated to be a viable technique to estimate the mean 29
of censored data. 30
18
The MLE/bootstrap method developed here is a general method that can be applied to left 1
censored data in other fields. For example, it can be applied to bioassay data for which left 2
censoring is a characteristic. The MLE/bootstrap method can be easily extrapolated for right and 3
interval censored data. 4
ACKNOWLEGEMENTS 5
The work was supported by U.S. Environmental Protection Agency Science to Achieve 6
Results (STAR) Grant No. R826790 to the Department of Civil, Construction, and 7
Environmental Engineering at North Carolina State University. Yuchao Zhao conducted this 8
work as graduate research assistant at NC State University and is now a postdoctoral fellow at 9
CIIT Centers for Health Research. This paper has not been subject to any EPA review and 10
therefore does not necessarily reflect the views of the Agency, and no official endorsement 11
should be inferred. 12
13
REFERENCES 14
Albert, J. H. and Chib, S. (1993), “Bays Inference via Gibbs Sampling of Autoregressive Time 15
Series Subject to Markov Mean and Variance Shifts,” J. Bus. Econ. Statist, 11 (1), 1-15. 16
Bogaerts, K. (2002), “Modeling Tooth Emergency Data Based On Multivariate Interval- 17
Censored Data,” Statistics in Medicine, 21(24): 3775-3787. 18
Bloom, D. L., D. M. Byrne, and J. M. Anderson (1993), Communicating Risk to Senior EPA 19
Policy-Makers: A Focus Group Study, Prepared by Bloom Research and the Office of Air 20
Quality Planning and Standards, U.S. Environmental Protection Agency, Research Triangle 21
Park, NC. 22
Burmaster, D. and Wilson, A. (2000), “Fitting Second-Order Finite Mixture Models to Data with 23
Censored Values Using Maximum Likelihood Estimation,” Risk Anal, 20 (2): 261-271. 24
Chrostowski, P. C. (1994), “Exposure Assessment Principles,” in Toxic Air Pollution Handbook, 25
Chapter 8 in Patrick, D. Ed., Van Nostrand Reinhold: New York, pp 133-163. 26
Cohen, A. C. and Whitten, B. (1988), Parameter Estimation in Reliability and Life Span Models, 27
M. Dekker: New York. 28
Cullen, A.C. and Frey, H.C. (1999), Probabilistic Techniques in Exposure Assessment, Plenum 29
Press: New York and London. 30
19
Efron, B. and Tibshirani, R.J. (1993), An Introduction to Bootstrap, Monographs on Statistics 1
and Applied Probability, Chapman and Hall: New York. 2
Elvira, B., Rao, S., and Porter, P.S. (1999), “Identifying Pollution Source Regions using Multiple 3
Censored Data,” Environ Sci Technol, 33(13): 2273-2277. 4
EPA (1993a), “Emission Factor Documentation for AP-42 Section 1.1 Bituminous and 5
Subbituminous Coal Combustion,” Prepared by Acurex Environmental Corporation, Edward 6
Aul & Associates, Inc and E.H. Pechan and Associates, Inc, for Office of Air Quality 7
Planning and Standards, Office of Air and Radiation, U.S. Environmental Protection Agency, 8
Research Triangle Park, NC. http://www.epa.gov/ttn/chief/ap42/ch01/bgdocs/b01s01.pdf 9
(accessed on 5/ 2001) 10
EPA (1993b), “Emission Factor Documentation for AP-42 Section 1.3 Fuel Oil Combustion,” 11
Prepared by Acurex Environmental Corporation, Edward Aul & Associates, Inc and E.H. 12
Pechan and Associates, Inc, for Office of Air Quality Planning and Standards, Office of Air 13
and Radiation, U.S. Environmental Protection Agency, Research Triangle Park, NC. 14
http://www.epa.gov/ttn/chief/ap42/ch01/bgdocs/b01s03.pdf (accessed on 5/ 2001) 15
EPA (1997a), “Exposure Factors Handbook,” EPA/600/P-95/002Fa, Office of Research and 16
Development, National Center for Environmental Assessment, U.S. Environmental Protection 17
Agency, Washington DC, 20460. 18
EPA (1997b), “Guiding Principles for Monte Carlo Analysis,” EPA/630/R-97/001, Risk 19
Assessment Forum, U.S. Environmental Protection Agency, Washington DC. March 1997. 20
EPA (1997c), “Procedures for Preparing Emission Factor Documents,” EPA-454/R-95-015, 21
Office of Air Quality Planning and Standards, Office of Air and Radiation, U. S. 22
Environmental Protection Agency, Research Triangle Park, NC, 27711. November, 1997. 23
EPA (1999), “Report of the Workshop on Selecting Input Distributions for Probabilistic 24
Assessment,” EPA/630/R-98/004, U.S. Environmental Protection Agency, Washington, DC. 25
EPA (2001), “Background Document Report on Revisions to 5th Edition AP-42 Section 1.6 26
Wood Residue Combustion In Boilers,” Prepared by Eastern Research Group, for Office of 27
Air Quality Planning and Standards, Office of Air and Radiation, U.S. Environmental 28
20
Protection Agency, Research Triangle Park, NC. 1
http://www.epa.gov/ttn/chief/ap42/ch01/bgdocs/b01s04.pdf (accessed on 5/ 2001) 2
Faraggi, D., Izikson, P. and Reiser, B. (2003), “Confidence intervals for the 50 percent response 3
dose,” StatMed, 22 (12): 1977-1988. 4
Frey, H.C. and Bammi, S. (2002), “Quantification of Variability and Uncertainty in Lawn and 5
Garden Equipment NOx and Total Hydrocarbon Emission Factors,” J Air Waste Manag 6
Assoc, 52(4): 435-449. 7
Frey, H.C., and Bharvirkar, R. (2002), Quantification of Variability and Uncertainty. A Case 8
Study of Power Plant Hazardous Air Pollutant Emissions, Chapter - Human and Ecological 9
Risk Analysis, D. Paustenbach, Ed., John Wiley and Sons: New York. 10
Frey, H.C. and Rhodes, D.S. (1996), “Characterization and Simulation of Uncertainty Frequency 11
Distributions: Effects of Distribution Choice, Variability, Uncertainty and Parameter 12
Dependence,” Hum and Ecol Risk Assess, 4(2): 428-468. 13
Frey, H.C. and Zheng, J. (2002a), “Quantification of Variability and Uncertainty in Utility NOx 14
Emission Inventories,” J Air Waste Manag Assoc, 52(9): 1083-1096. 15
Frey, H.C. and Zheng, J. (2002b), “Probabilistic Analysis of Driving Cycle-Based Highway 16
Vehicle Emission Factors,” Environ Sci Technol, 36(23): 5184-5161. 17
Gilliom, R.J. and Helsel, D.R. (1986), “Estimation of Distributional Parameters for Censored 18
Trace Level Water Quality Data 1. Estimation Techniques,” Water Resour Res, 22(2): 135-19
146. 20
Haas, C.N. and Scheff, P.A. (1990), “Estimation of Averages in Truncated Samples,” Environ 21
Sci Technol, 24(6): 912-919. 22
Helsel, D.R. (2005), “Nondetects and Data Analysis: Statistics for Censored Environmental 23
Data,” Wiley: New Jersey. 24
Li, J., Gao, X.B., Goeckner, B., Kollakowsky, D., and Ramme, B. (2005), “A pilot study of 25
mercury liberation and capture from coal-fired power plant fly ash,” J Air Waste Manag 26
Assoc, 55 (3): 258-264. 27
Lu, H.C. (2003), “Comparisons of Statistical Characteristic of Air Pollutants in Taiwan by 28
Frequency Distribution,” J Air Waste Manag Assoc, 53(5): 608-616. 29
Miller, R. G., Gong, G., and Munoz, A. (1981) Survival Analysis, Wiley: New York. 30
21
Morgan, M.G. and Henrion, M. (1990), Uncertainty: A Guide to Dealing with Uncertainty in 1
Qualitative Risk and Policy Analysis, Cambridge University Press: New York, NY. 2
NAS (1994), Science and Judgment, National Academy of Sciences, National Academy Press: 3
Washington, DC. 4
Newman, M.C., Dixon, P.M., Looney, B.B., and Pinder, J.E. (1989), “Estimating Mean and 5
Variance for Environmental Samples with Below Detection Limit Observations,” Water Res 6
Bull, 25(4): 905-915. 7
Ott, W. (1990), “A Physical Explanation of the Lognormality of Pollutant Concentrations,” J Air 8
Waste Manag Assoc, 40 (10): 1378-1383. 9
Ott, W. (1995), Environmental Statistics and Data Analysis, Lewis Publishers, Boca Raton, FL. 10
Patrick, D. R. (1994), “Risk Assessment and Risk Management,” in Toxic Air Pollution 11
Handbook, Patrick, Ed., Van Nostrand Reinhold: New York. 12
Rao, S.T., Ku, J.Y., and Rao, K. S., (1991) “Analysis of Toxic Air Contaminant Data Containing 13
Concentrations Below the Limit of Detection,” J Air Waste Manag Assoc, 41(4): 442-448. 14
Sadiq, R., Husain, T. and Kar, S. (2002), “Chloroform Associated Health Risk Assessment Using 15
Bootstrapping: A Case Study for Limited Drinking Water Samples,” Water, Air and Soil 16
Pollut, 138 (1-4): 123-140. 17
Seinfeld, J.H. (1986), Atmospheric Chemistry and Physics of Air Pollution, John and Sons, New 18
York. 19
Shapiro, S.S. and Wilk, M.B. (1965), “An Analysis of Variance Test for Normality (Complete 20
Samples),” Biometrika, 52: 591-598. 21
Smith, R.L., French, C.L., Murphy, D.L., and Thompson, R. (1999), “Ranking and Selection of 22
Hazardous Air Pollutants for Listing under Section 112(k) of the Clean Air Act Amendments 23
of 1990,” Technical Support Document of Integrated Urban Air Toxics Strategy, office of Air 24
Quality Planning and Standards, Environmental Protection Agency (EPA). 25
http://www.epa.gov/ttn/atw/urban/urbanpg.html (accessed on 10/1/ 2000) 26
Thompson, K.M. and Graham, J.D. (1996), “Going beyond the single number: using 27
probabilistic risk assessment to improve risk management”, Hum Ecol Risk Assess, 2(4):1008-28
1034. 29
22
Verrill, S. and Hohnson, S.R. (1988), “Tables and Large-sample Distribution-Theory for 1
Censored-Data Correlations Statistics for Testing Normality,” J. Am. Stat. Assoc., 83 (404): 2
1192-1197. 3
Zhao, Y. and Frey, H.C. (2004a) “Quantification of Variability and Uncertainty for Censored 4
Data Sets and Application to Air Toxic Emission Factors,” Risk Anal 24 (4), 1019-1034. 5
Zhao, Y. and Frey, H. C. (2004b) “Development of Probabilistic Emission Inventory of Air 6
Toxics for Jacksonville, Florida,” J Air Waste Manag Assoc 54 (11), 1405-1421. 7
Zheng, J. (2002) “Quantification of Variability and Uncertainty in Emission Estimation: General 8
Methodology and Software Implementation,” Ph.D. Dissertation, Department of Civil 9
Engineering, North Carolina State University, Raleigh, 2002; www.lib.ncsu.edu. 10
Zheng, J., and H.C. Frey (2005), “Quantitative Analysis of Variability and Uncertainty with 11
Known Measurement Error: Methodology and Case Study,” Risk Analysis, 25(3):663-676. 12
13
14
15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34
23
Table 1. Summary of Emission Factor Data for Selected Urban Air Toxics from External 1 Combustion Emission Source Categories 2
Case
No. Air Toxic Fuela Nb PCc NDLsd VFe
Relative
Max.
DL
f
Unit Rg
1
Benzene
C
18
5.6
1
30459
0.07
g/tonne
A
2
Benzene
W
10
10
1
127
0.006
g/tonne
A
3 Benzene FO 14 78.6 10 5 4.1
10-5 g/
liter C
4 Formaldehyde C 14 35.7 5 90 0.09
10-2 g/
tonne A
5 Formaldehyde FO 14 64.3 8 118 0.27
10-3g/
liter C
6 B(a)p C 8 37.5 3 30 0.74
10-5g/
tonne
D
7 Mercury C 29 10.3 3 81 3.93
10-2g/
tonne
A
8 Mercury FO 13 84.6 11 2 75.5
10-6g/
liter
C
9 Arsenic C 29 10.3 3 3366 0.004
g/
tonne
A
10 Arsenic FO 13 23.1 3 48 0.8
10-3g/
liter
C
11 Cadmium C 28 17.9 5 984 0.09
10-2g/
tonne
A
12 Cadmium FO 13 23.1 3 43 0.1
10-4g/
liter
C
13 Chromium C 28 3.6 1 1307 0.03
g/
tonne A
14 Chromium FO 13 7.7 1 5 0.105
10-4g/
liter C
15 Chromium VI FO 10 50 5 3 0.5
10-4g/
liter
C
16 Lead FO 13 30.8 4 11 0.09
10-4g/
liter
A
a Fuel type; C = coal, W = wood waste, FO = fuel oil 3 b Sample size 4 c Percentage of data samples that are censored 5 d NDL = number of detection limits 6 e Variability Factor of the detected values, represented by the largest detected value divided by 7 the smallest detected value 8 f Relative maximum detection limit, represented by the largest detection limit divided by the 9 largest detected value 10 g R = EPA qualitative ratings representing the data quality of the emission factor, taking into 11 account test methods, sample size, and judgment regarding representativeness of the data. 12 13
24
Table 2. Results from MLE/Bootstrap Simulation for Candidate Parametric Distributions Fit to 1 Urban Air Toxic Emission Factor Data from External Combustion Sources 2
Case
No.
Air Toxic Fuela Unit
MLE/Bootstrap
Distributionb
Averagec
95% C.I.d
Widthe
1 Benzene C g/tonne
lognormal
0.95
(-93, 411)
504
Weibull
0.45
(-89, 228)
317
2 Benzene W g/tonne
lognormal
8.60
(-84, 259)
343
gamma
4.16
(-74, 87)
161
Weibull
4.36
(-72, 98)
170
3f
Benzene
FO
10-5 g/liter
gamma
1.56
(-60, 120)
180
4 Formaldehyde C
10-2
g/tonne
lognormal
0.74
(-77, 208)
285
Weibull
0.70
(-75, 161)
236
5f Formaldehyde FO 10-3g/liter
gamma
2.11
(-95, 118)
213
Weibull
7.05
(-94, 368)
462
6 B(a)p C 10-
5g/tonne
lognormal
1.40
(-72, 114)
186
gamma
1.28
(-70, 91)
161
Weibull
1.28
(-70, 94)
164
7 Mercury C 10-
2g/tonne
lognormal
3.83
(-28, 39)
67
gamma
3.33
(-27, 33)
60
Weibull
3.33
(-27, 35)
62
8f
Mercury
FO
10-6g/liter
lognormal
5.87
(-31, 32)
63
9 Arsenic C g/tonne
lognormal
0.34
(-91, 264)
355
Weibull
0.13
(-79, 161)
240
10 Arsenic FO 10-3g/liter
lognormal
0.16
(-45, 69)
114
gamma
0.13
(-47, 57)
104
Weibull
0.13
(-47, 59)
106
11 Cadmium C
10-
2
g/tonne
lognormal
2.47
(-62, 156)
218
Weibull
1.81
(-55, 105)
160
12 Cadmium FO 10-4g/liter
lognormal
0.56
(-69, 166)
235
gamma
0.43
(-64, 99)
163
Weibull
0.43
(-64, 104)
168
13 Chromium C g/tonne
lognormal
0.12
(-59, 123)
182
gamma
0.10
(-58, 85)
143
Weibull
0.10
(-56, 79)
135
14 Chromium FO 10-4g/liter
lognormal
1.03
(-32, 36)
68
gamma
0.98
(-33, 31)
64
Weibull
0.98
(-33, 31)
64
15 Chromium VI FO 10-4g/liter
lognormal
0.61
(-70, 235)
305
gamma
0.25
(-63, 63)
126
Weibull
0.28
(-49, 50)
99
16 Lead FO 10-4g/liter
gamma
0.17
(-52, 59)
111
Weibull
0.17
(-52, 53)
105
a Fuel type; C = coal, W = wood waste, FO = fuel oil 3 b Candidate parametric distributions, the preferred one is shown in bold. 4 c Best estimated mean based upon the average of the means of the 500 replicates of the CDF. 5 d Lower and upper levels of the 95% confidence interval relative to the mean value, as a percentage difference 6 compared to the mean value. 7 e Width of the 95% confidence interval, equal to the sum of the absolute value of the lower and 8 upper levels of the 95% confidence intervals. 9 f For Cases 3, 5 and 8, results are shown only for distributions for which parameter estimates 10 were within valid constraints. 11
25
1 Table 3. Comparison of Estimated Means from Conventional and MLE/Bootstrap Methods 2
Case
No. Air Toxic Fuela Unit
Means estimated from the
conventional methods
b
Mean from the
MLE/Bootstrap
Method
1
2
3
4
1
Benzene
C
g/tonne
0.53c
0.53c
0.53c
0.53c
0.95
2
Benzene
W
g/tonne
4.53
4.07
4.11
4.11
4.36
3 Benzene FO
10-5 g/
liter
2.52 0.60 3.00 5.50 1.56
4 Formaldehyde C
10-2 g/
tonne
1.11 0.70 0.82 0.91 0.74
5 Formaldehyde FO
10-3g/
liter
5.88 2.10 2.37 2.64 7.05
6 B(a)p C
10-5g/
tonne
1.56 0.99 1.36 1.77 1.40
7 Mercury C
10-2g/
tonne
3.54 3.17 4.20 5.18 3.33
8 Mercury FO
10-6g/
liter
6.47 1.20 115 229 5.87
9 Arsenic C
g/
tonne 0.17 0.16 0.16 0.16 0.34
10 Arsenic FO
10-3g/
liter 0.16 0.12 0.13 0.16 0.13
11 Cadmium C
10-2g/
tonne 2.18 1.81 1.85 1.93 2.47
12 Cadmium FO
10-4g/
liter 0.53 0.41 0.42 0.44 0.56
13 Chromium C
g/
tonne 0.11 0.10 0.10 0.10 0.12
14 Chromium FO
10-4g/
liter
1.05 0.97 0.98 0.98 1.03
15 Chromium VI FO
10-4g/
liter
0.46 0.23 0.28 0.31 0.28
16 Lead FO
10-4g/
liter
0.24 0.17 0.17 0.17 0.17
a Fuel type; C = coal, W = wood waste, FO = fuel oil 3 b Estimation of mean based upon conventional methods: 1 = removal of non-detects; 4 2 = replace non-detects with zero; 3 = replace nondetects with DL/2; 4 = replace non-detects 5 with DL 6 c The difference of the estimated means from different conventional methods lies in the third 7 significant digit and thus not shown with two significant digits. The third significant figure is not 8 statistically significant in all cases. 9 10 11 12 13 14
26
Table 4. Ratio of the Estimated Means and Width of the 95 Percent Confidence Intervals for the 1 Means for the MLE/Bootstrap Method Applied to Modified Versus Censored Data. 2
Case
No. Air Toxics Fuel Unit Ratio of Meana Ratio of Width
of 95% C.I.b
1
Benzene
Coal
g/tonne
1.09
1.19
2
Benzene
Wood waste
g/tonne
0.96
0.94
3
Benzene
Fuel Oil
10-5 g/liter
1.92
0.74
4
Formaldehyde
Coal
10-2 g/tonne
0.94
0.78
5
Formaldehyde
Fuel Oil
10-3g/liter
0.35
0.16
6
B(a)p
Coal
10-5g/tonne
1.09
0.88
7
Mercury
Coal
10-2g/tonne
1.25
1.69
8
Mercury
Fuel Oil
10-6g/liter
21.0
62.1
9
Arsenic
Coal
g/tonne
0.65
0.63
10
Arsenic
Fuel Oil
10-3g/liter
1.00
0.89
11
Cadmium
Coal
10-2g/tonne
0.92
0.76
12
Cadmium
Fuel Oil
10-4g/liter
0.89
0.81
13
Chromium
Coal
g/tonne
1.00
0.99
14
Chromium
Fuel Oil
10-4g/liter
1.01
1.03
15
Chromium VI
Fuel Oil
10-4g/liter
1.00
0.94
16
Lead
Fuel Oil
10-4g/liter
1.00
0.94
a. Ratio of the estimated mean of modified data to that of censored data based on the 3 MLE/Bootstrap method 4 b. Ratio of the width of the 95 percent confidence interval estimated based upon modified data to 5 that estimated based upon the MLE/Bootstrap method applied to censored data 6 7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
27
Figure 1. Flow Diagram for Quantification of Variability and Uncertainty for Censored Data 1 Sets 2 Figure 2. Variability and Uncertainty in Mercury Emission Factor from Coal Combustion (Case 3 7) Estimated Based Upon Three Distributions 4 Figure 3. Variability and Uncertainty in Cadmium Emission Factor from Coal Combustion 5 (Case 11) Estimated Based Upon a Lognormal Distribution 6 Figure 4. Variability and Uncertainty in Arsenic Emission Factor from Fuel Oil (Case 10) 7 Combustion Estimated Based Upon a Weibull Distribution 8 Figure 5. Variability and Uncertainty in Chromium Emission Factor from Coal Combustion 9 (Case 13) Estimated Based Upon a Lognormal Distribution 10 Figure 6. Variability and Uncertainty in Formaldehyde Emission Factor from Fuel Oil 11 Combustion (Case 5) Estimated Based Upon a Weibull Distribution 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
28
Figure 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14
15
16
17
18
19
20
21
22
23
24
25
26 27
28
29
30
31
32
33
34
35
36
Figure 2 37
Input original data sets, including values for detected data
and detection limits for censored data. For non-detects, a
data flag,
δ
, is set to 1, otherwise use 0.
Empirical bootstrap simulation to generate B replications of original data
sets together with its indicator symbol. For detected data, the data value is
sampled. For non-detected data, the detection limit is sampled.
Fit parametric distribution to each bootstrap sample using MLE
Monte Carlo simulation to generate p random samples from each
parametric distribution to represent its variability
Calculate statistics from p random samples from each distribution
Based upon B replications of selected statistics (e.g., mean), estimate
the sampling distribution of the statistics
29
Data Set
95 percent
90 percent
Fitted Distribution
Confidence Interval
50 percent
Detection Limt
DL1
0.0
0.2
0.4
0.6
0.8
1.0
Cumulative Probability
10
-1
10
0
10
1
10
2
10
3
(0.01 g Hg/tonne coal combusted)
DL2
Confidence Interval
90 percent
50 percent
95 percent
Detection Limit
Possible Plotting Position
Lognormal Distribution
DL3
1 (a) Lognormal Distribution 2 3
Data Set
95 percent
90 percent
Fitted Distribution
Confidence Interval
50 percent
Detection Limt
0.0
0.2
0.4
0.6
0.8
1.0
Cumulative Probability
10
-1
10
0
10
1
10
2
10
3
(0.01 g Hg/tonne coal combusted)
DL1
DL2
50 percent
90 percent
95 percent
Possible Plotting Position
Confidence Interval
Detection Limit
Gamma Distribution
DL3
4 (b) Gamma Distribution 5
30
Data Set
95 percent
90 percent
Fitted Distribution
Confidence Interval
50 percent
Detection Limt
0.0
0.2
0.4
0.6
0.8
1.0
Cumulative Probability
10
-1
10
0
10
1
10
2
10
3
(0.01 g Hg/tonne coal combusted)
50 percent
90 percent
95 percent
Possible Plotting Position
Confidence Interval
Detection Limit
Weibull Distribution
DL1
DL2 DL3
1 (c) Weibull Distribution 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
31
Figure 3 1
90 percent
Data Set
95 percent
90 percent
Fitted Distribution
Confidence Interval
50 percent
Detection Limt
10
-2
10
-1
10
0
10
1
10
2
(0.01 g Ca/tonne coal combusted)
0.0
0.2
0.4
0.6
0.8
1.0
Cumulative Probability
Data Set
Fitted Distribution
95 percent
Confidence Interval
Detection Limt
90 percent
Possible Probability
50 percent
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
(0.01 g Cd/tonne coal combusted)
32
Figure 4 1
90 percent
Data Set
95 percent
90 percent
Fitted Distribution
Confidence Interval
50 percent
Detection Limt
0.0
0.2
0.4
0.6
0.8
1.0
Cumulative Probability
0.00 0.20 0.40 0.60 0.80 1.00
(10 E-3 g As/liter fuel oil combusted)
Data Set
Fitted Distribution
95 percent
Confidence Interval
Detection Limt
90 percent
Possible Probability
50 percent
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
(0.001 g As/liter fuel oil combusted)
33
Figure 5 1
90 percent
Data Set
95 percent
90 percent
Fitted Distribution
Confidence Interval
50 percent
Detection Limt
10
-3
10
-2
10
-1
10
0
10
1
(g Cr/tonne coal combusted)
0.0
0.2
0.4
0.6
0.8
1.0
Cumulative Probability
Data Set
Fitted Distribution
95 percent
Confidence Interval
Detection Limt
90 percent
Possible Probability
50 percent
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
34
Figure 6 1
90 percent
Confidence Interval
Detection Limt
Possible Probability
50 percent
90 percent
95 percent
Fitted Distribution
Data Set
10
-7
10
-6
10
-5
10
-4
10
-3
10
-2
10
-1
10
0
10
1
(0.001 g Formaldehyde/liter fuel oil combusted)
10
2
0.0
0.2
0.4
0.6
0.8
1.0
Cumulative Probability
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
