Evaluating Limitations

Related Pages

This section explains how to evaluate potential limitations at your site. It shows you how to describe the limitations, determine if the limitations could affect your conclusions, and identify recommendations to address the limitations.

Throughout the PHA process, you must evaluate potential limitations in the available information and determine whether critical data are available and sufficient to support a public health conclusion. If critical data are missing, consider recommending actions to help fill those data gaps. You can use the Potential Limitations table [PDF – 174 KB] to help identify and navigate the potential limitations.

In some cases, you might need additional data to confirm or further support your decision. Carefully examine the importance of the missing data, whether you can obtain the needed data, and if you can obtain it in a timely manner. In some cases, the data might never be available (e.g., past exposure data), so you will need to use the best available data (e.g., more recent sampling data or in limited circumstances modeled data) to evaluate potential hazards and draw conclusions.

If you determine that available data are insufficient to draw a conclusion, clearly indicate this in your written document. Also, recommend additional actions when possible and state that a definitive conclusion cannot be drawn because of the absence of critical data.

Not all data gaps are data needs. Before recommending sampling or further investigation, carefully assess and distinguish what would be good to know versus what is needed to draw a public health conclusion. Also consider issues that the community needs to know or that it might reasonably expect ATSDR to address. Provide as much perspective as possible using available data.

You will need to clearly explain in your documents what is known and not known, and where and why there are limitations. Also, in your conclusions, state how you accounted for uncertainties related to these limitations. Whenever possible, ATSDR accounts for limitations by using protective, reasonable exposure estimates as the basis for determining whether harmful health effects are possible (see the Potential Types of Limitation Sources and Examples Table).

Potential Types of Limitation Sources and Examples

Who is Involved in the PHA Process
Types of Potential Limitation Sources
Examples of Potential Limitations by Source
Measurements/
sampling data
  • Limited number of samples
  • Lack of sampling data for the entire exposure period
  • Lack of laboratory analysis for the target contaminants
  • Reliance on laboratory results using surrogate contaminants
  • Qualified laboratory data, such as data flagged as estimated (e.g., “J” flags)
  • Questionable lab data, such as biased analytical results
Modeling*
  • Inherent limitations in the selected model (e.g., was the model designed as a screening tool to represent high end rather than site-specific exposures, was the model set up for a single-use house when the site is an apartment complex)
  • Assumptions made in a model that systematically over- or under-represent exposure (e.g., an air dispersion model that assumes receptors are continuously downwind of a source or remain stationary, model inputs that use “worst case” exposure assumptions for variabilities in ingestion rates, exposure frequencies, or inhalation rates)
Exposure units
  • Lack of site-specific information about how residents use or frequent the area for defining an exposure unit
  • No fate-and-transport information for determining whether contamination spread to other areas that should be defined as additional exposure units
  • Lack of sampling data in an exposure unit
Exposure point concentrations
  • Large number (close to 80%) of non-detect observations used to calculate 95% upper confidence limits of the arithmetic mean (95UCLs)
  • Data sets with large numbers of J-qualified results used to calculate 95UCLs
  • Data sets with fewer than eight samples, thus requiring use of the maximum concentration to estimate exposure instead of the 95UCL
Exposure doses
  • Using default parameters (e.g., exposure frequency, exposure duration, intake rate, body weight) when site-specific data are unavailable
  • Variation in exposure that occurs because of different racial and cultural groups
Site-specific population
  • No data on whether and to what extent residents were exposed to contaminants in the past, are in the present, or could be in the future
  • No data on site population characteristics to know if there are individuals more susceptible to environmental exposures
  • No data on population subsistence practices for sites with contaminated fish
  • No data on population gardening practices for sites with contaminated soil
Health guidelines and cancer risk values
  • No-observed-adverse-effect-level (NOAEL) based on an animal study (because of uncertainty in relative sensitivity of animals compared to humans)
  • Uncertainty in whether the lowest lowest-observed-adverse-effect-level (LOAEL) or highest NOAEL has been identified
  • Uncertainty in benchmark dose modeling or converting an animal dose to a human equivalent dose
  • Basis study not relevant to site exposure (e.g., different dosing method, exposure route, exposure duration, health endpoint)
  • Insufficient data on contaminant to develop health guideline or a cancer risk value for a known carcinogen
  • Unstudied toxic endpoints that could result in harmful effects lower than those used to derive health guideline or cancer risk value
Studies used to support analysis
  • Insufficient data regarding harmful health effects associated with environmental levels of exposure
  • Limitations of using health outcome data (e.g., long latency period for some outcomes like cancer, confounding risk factors that might be the real cause for an increased risk), which are rarely available or of sufficient quality to associate health outcomes with site-related exposures
  • No human or animal studies similar to site-specific exposures and health effects data
  • No human or animal studies on whether exposure to small concentrations of a contaminant might be harmful
  • No human or animal studies on the extent to which multiple contaminant exposures might result in harmful effects beyond those identified from individual contaminants
  • Only acute or sub-chronic studies are available to assess longer-term exposures

*Note: Modeling data can be used to supplement the available sampling data in limited circumstances. Health assessors need to discuss the appropriateness and feasibility of using modeled data beforehand with subject matter experts (SMEs).