Oak Ridge Reservation: Public Health Assessment Work Group
Public Health Assessment Work Group
August 28, 2001 - Meeting Minutes
ORRHES Members attending:
Bill Pardue (Work Group Chair), Al Brooks, Bob Craig, Don Creasia, Kowetha Davidson, Karen Galloway, David Johnson, Susan Kaplan, James Lewis, Pete Malmquist, and LC Manley.
Public Members attending:
Gina Broome, Romance Carrier, Al Chambles, Wayne Clark, Linda Gass, Susan
Gawarecki, Tim Joseph, Janet Michel, Bob Peele, Sandra Reid, and Janice
ATSDR staff attending:
Jack Hanley, Karl Markiewicz, Bill Murray, and Lucy Peipins
At the June 2001 meeting, Lucy Peipins delivered a presentation entitled Epidemiology 101 to help Oak Ridge Reservation Health Effects Subcommittee (ORRHES) better understand the basics of epidemiology. At previous meetings, the Public Health Assessment Work Group (PHAWG) had decided that it should receive further instruction from ATSDR about how to review epidemiological studies. The article that was chosen for review was: Cancer Mortality Near Oak Ridge, Tennessee. The author of the article is Joseph Mangano.
The purpose of this meeting was to help the subcommittee understand how epidemiologists evaluate studies. Specifically, the meeting sought to identify the standard criteria against which studies are evaluated. Mangano's study, instructions for conducting a study evaluation, and an evaluation worksheet were provided to the work group prior to the meeting. The work group was asked to review Mangano's study and complete the worksheet, following the instructions that were provided.
Peipins began with a brief introduction describing: (1) the purpose of scientific research (to present new findings); (2) some basic information that should be included in the research (e.g., methods used to test the hypotheses); and (3) the process for publishing research, including peer review. She then guided the work group's discussion to follow the questions within the worksheet. Each question is provided in italicized font on the following pages, followed by a list of the responses participants made to the question.
- Why was this study done? What question
is the author trying to answer (the hypothesis)? This
can usually be found in the title of the paper or in the
- Purpose: To examine cancer mortality from low-level
radiation among 94 communities within a 100-mile radius
of Oak Ridge Reservation.
1. The increase in the cancer mortality rate near Oak Ridge should exceed the national and regional increases.
2. Within the Oak Ridge region, the increase in the cancer mortality rate should be greatest in rural areas.
3. Within the Oak Ridge region, the increase in the cancer mortality rate should be greatest near the weapons plant.
4. Within the Oak Ridge region, the increase in cancer mortality rates should be greatest in mountainous areas.
5. Within the Oak Ridge region, the increases in the cancer mortality rate should be greatest in the area downwind of the weapons plant.
- Purpose: To examine cancer mortality from low-level radiation among 94 communities within a 100-mile radius of Oak Ridge Reservation.
- What type of study was this?
- PHAWG decided that Mangano's study is both descriptive
- PHAWG decided that Mangano's study is both descriptive and analytic.
- Keeping in mind that epidemiology
is the study of populations, who was the study population?
What groups of people were included in the study or the
- The study population was determined to be white
people who died from cancer and were living within
the 94 counties that are within in a 100-mile radius
of Oak Ridge during two time periods (19501952 and
- Each hypothesis also had its own population:
People living within the 94 counties within 100 miles
Rural compared to urban populations
People living within Anderson County compared to those living in the other 12 counties within 40 miles
Mountain compared to lowland populations
Upwind compared to downwind populations
- The study population was determined to be white people who died from cancer and were living within the 94 counties that are within in a 100-mile radius of Oak Ridge during two time periods (19501952 and 19871989).
- What was the exposure of interest?
This can be as general as "air pollution" or as specific
as blood lead levels.
- PHAWG decided that chronic ingestion of substantial
amounts of low-level radiation was the exposure of
- PHAWG decided that chronic ingestion of substantial amounts of low-level radiation was the exposure of interest.
- How was exposure defined and measured?
- The PHAWG decided exposure was not measured, but
it was defined as living in 94 counties within 100
miles from Oak Ridge, rural areas, close to Oak Ridge,
mountainous areas, or downwind.
- The PHAWG determined that the following types of
exposure classifications were used:
1) Residence in a defined geographical area (e.g.,
county) of a site
2) Residence in a geographical area in proximity to a site where exposure is assumed
3) Distance or duration of residence (duration was not measured)
- The PHAWG decided that these types of exposure classifications
used were generally considered poor in the hierarchy
provided in the worksheet.
What are some limitations or criticisms you might have about how exposure was measured in the study? Remember, the closer we can get to a measure of dose in the body, the more precise our exposure estimate will be.
- Exposure was not measured; Mangano cited other studies
that quantified releases, but not exposures.
- Just because you died of cancer according to your
death certificate, does not mean you were exposed
to low-level radiation.
- Vital statistics do not account for migration in
and out of the areas evaluated. PHAWG members stated
that there was a large influx of people to Oak Ridge
in 1943 until the time K-27 and K-31 were built. Then
there was a decline after the war when construction
was complete. The population in Anderson County were
said to follow roughly the same pattern of increase
- Some statements are made without supporting citations
(e.g., Mangano assumes that mountainous areas get
more rainfall, however, the Cumberland mountains near
Oak Ridge do not receive large amounts of precipitation).
- Streams and tributaries downstream from Oak Ridge
were not evaluated as a pathway, however, DOE reports
indicate radionuclides and chemicals were released
into the surface water.
- There was no consideration of underground water
- Specific to the fourth hypothesis, you would expect
that low-lying areas would have more food grown and
eaten than the mountainous areas because very little
food is grown in the mountainous region.
- There are a lot of other differences between the
1950 and 1980 populations that could account for the
increase in cancer rate.
- There are a lot of differences in the accuracy of
diagnosed and reported cancer cases between the 1950s
- Not all cancers are known to be caused by radiation.
- The PHAWG decided exposure was not measured, but it was defined as living in 94 counties within 100 miles from Oak Ridge, rural areas, close to Oak Ridge, mountainous areas, or downwind.
- What was the health outcome of
interest? Remember there is a wide spectrum of effects
such as wheezing, change in immune function, changes in
blood chemistry, DNA adducts, adverse birth outcomes,
developmental disabilities, clinical disease such as cancer
- Death from cancer
- Death from cancer
- How was the health outcome defined
and measured in this study? Remember, the goal of measuring
outcomes is to count all the cases in a particular exposed
group or population and compare it with cases in an unexposed
group or population. Was this done? Remember, also, all
of the possible sources of information on outcomes (death
certificates, registry data, questionnaires, medical examinations)
and their strengths and weaknesses.
- Vital statistics data from the National Cancer Institute
for the populations identified in the five hypotheses.
- Vital statistics data from the National Cancer Institute for the populations identified in the five hypotheses.
- Could there have been bias
(distortion or error) in how study subjects got selected
into the study? For instance, did all eligible subjects
participate? Why not? If the participation is low, the
study results may not represent the group and may not
- This question does not apply to this study because
subjects were not chosen.
- This question does not apply to this study because subjects were not chosen.
- Could there have been bias, (distortion or error) in how information
was collected from the study group and the comparison
group? For instance, was there a difference in how diseases
were measured or counted in the exposed and unexposed
study group? Or, in case-control studies, was exposure
measured differently for the diseased and non-diseased?
Remember, not only do we have to count or measure all cases in the study and comparison group,
we have to do it in the same way in order for there to
be a fair and valid comparison.
- The early years may actually have a higher rate
of death from cancer than reported because cancer
may not have been identified as the actual cause of
death in the 1950s and there was a better ability
to diagnose cancer in the 1980s. Therefore, the difference
observed between the 1950s to the 1980s may be exaggerated.
- Variation in doctor care between rural and urban
areas could have affected the ability of the doctor
to accurately diagnose the actual cause of death.
- In the 94 counties, and the Southeastern U.S. in
general, cancer rates may be lowered by the fact that
overall life expectancy is lowerthereby reducing
the number of people who will live long enough to
die of cancer.
- As life expectancy increases over time from the
1950s to the 1980s, you would expect deaths of cancer
to increase even without any underlying increase in
cancer-causing exposure because older people tend
to contract cancer.
- There were significant changes in population between
the 1950s and 1980s because of the war and plant construction
(e.g., there was a large influx of people into Anderson
County in the 1940s and a rapid decline after construction
was complete and the war ended). The result of these
changes was that there was a larger population of
young people, who are not likely to have cancer, in
the 1950s than there was in the 1980s.
- Because cancer rates fluctuate from year to year
around an average, Mangano may have inadvertently
chosen a time of low rate for the 1950s and a time
of high rate for the 1980s. The results may be different
if a few years before or after 19501952 and 19871989
were chosen. Knowing the annual increase in cancer
rates between 1950 and 1989 would have added a nice
- PHAWG questioned the ways in which the proximity
of counties to Oak Ridge and to each other were determined.
For example, three Oak Ridge plants are located in
Roane County, but Roane County is considered to be
further away from Oak Ridge than Anderson County is.
- The early years may actually have a higher rate of death from cancer than reported because cancer may not have been identified as the actual cause of death in the 1950s and there was a better ability to diagnose cancer in the 1980s. Therefore, the difference observed between the 1950s to the 1980s may be exaggerated.
- Could the results have been due
to confounding? Confounding means the possibility
that the observed association between exposure and disease
could have been due, totally or in part, to differences
between the study group and comparison group that affected
the risk of developing the disease. Another way of saying
this is to ask whether risk factors (such as smoking)
that could have caused the disease under investigation
and whether these were measured? For example, could an
association between cholesterol levels and heart disease
actually be due to obesity or smoking? These are also
risk factors for disease which should be taken into account
in a study and analysis.
- Mangano's study assumed that the comparison group
is not exposed to any natural or background sources
- The National Cancer Institute highlights a similar
trend of increase in cancer throughout the entire
Southeast and East, which would not be attributable
to Oak Ridge.
- In the 1980s, people were more exposed to carcinogenic
compounds (i.e., benzene) because of use of self-service
stations, rather than the full-service stations that
were used in the 1950s.
- Relatively high utilization of pesticides in rural
areas (as opposed to urban areas) might be confounded
with the hypothetically greater exposure to Oak Ridge
pollutants among rural residents.
- The smoking of unfiltered cigarettes and the use
of chewing tobacco is more prevalent in the southeastern
United States than elsewhere in that country.
- The use of chlorination processes to treat and supply
potable water was more prevalent in the 1980s than
it was the 1950s. An increase in stomach cancers was
attributed to the higher use of chlorination processes.
- Both coal mining and coal consumption were higher
in the 1950s than they were in the 1980s. This fact
may partially or completely explain the change in
cancer incidence reported for the northwest counties.
- People in the study area were exposed to chemicals
from plants in the area (e.g., Kingston Steam Plan,
Bull Run, Roane Electric, and Watt's Bar Plant) because
for years they did not have air pollution controls.
- Health care and diagnostic ability improved with
- Alcohol consumption might confound the results.
- Mangano's study assumed that the comparison group is not exposed to any natural or background sources of radiation.
- Describe the methods used to measure
the relationship between exposure and disease.
- Jack Hanley handed out an e-mail he received from
Ed Frome detailing his opinion about whether this
was the best analysis that could have been done to
answer the hypotheses. The e-mail from Frome disagreed
with Mangano's use of a direct method for age adjustment.
Another participant added that the age-adjusted rates
to the 1950 standard would be much less well-known
than the 19871989 rate. Looking at age-specific rates
might have been helpful.
- Jack Hanley handed out an e-mail he received from Ed Frome detailing his opinion about whether this was the best analysis that could have been done to answer the hypotheses. The e-mail from Frome disagreed with Mangano's use of a direct method for age adjustment. Another participant added that the age-adjusted rates to the 1950 standard would be much less well-known than the 19871989 rate. Looking at age-specific rates might have been helpful.
- What are the major results of the
- Mangano concluded that the cancer mortality rate
increase in the Oak Ridge area exceeded the national
and regional increases; the local urban areas experienced
less of an increase; the change was greatest in the
area closest to Oak Ridge; increases in mountain areas
surpass those in adjacent lowlands; and the increase
was greatest in the area downwind of Oak Ridge.
- Mangano concluded that the cancer mortality rate increase in the Oak Ridge area exceeded the national and regional increases; the local urban areas experienced less of an increase; the change was greatest in the area closest to Oak Ridge; increases in mountain areas surpass those in adjacent lowlands; and the increase was greatest in the area downwind of Oak Ridge.
- Do you believe the results of the
study? Were the results valid?
- PHAWG does not believe that Mangano's analyses and results support his conclusions.
Questions 13a-e and 14 were not discussed.
Janice Stokes wanted to know why ATSDR did not contact Dr. Mangano to have him defend his paper during the critique. Kowetha Davidson and Peipins explained that it is not customary or practical to contact authors of studies that are being scientifically critiqued. They explained that the author usually puts everything in the paper that he/she thought was important and that as a scientist you review what is available to the public, i.e., the article.
Stokes offered to bring a videotape of Dr. Gould and Dr. Mangano explaining the motivation and reasons for the hypotheses and conclusions made at Oak Ridge. PHAWG agreed to review the videotape before making any recommendations to ORRHES about the usefulness of Mangano's report. Tuesday morning before the ORRHES meeting a TV with VCR will be made available at the ATSDR Oak Ridge Field Office.
Two participants (Stokes and Gawarecki) questioned the use of Mangano's paper due to its controversial and emotional nature. Bill Pardue explained that at the prior meeting this report was specifically brought to the table. It was again emphasized that the purpose of this meeting was to learn how to objectively evaluate epidemiology studies; Mangano's report was just an example used to demonstrate the approach that is always taken for any epidemiological study. Lewis thought it was beneficial to evaluate Mangano's report because it allowed those who are knowledgeable about the site to be on familiar ground.
Gawarecki pointed out that the reputation of the journal in which a study is published and the quality of the study's citations are also important aspects to look into when evaluating a study. She asked if the International Journal of Health Sciences was a widely distributed, respected journal that conducted peer reviews. Pete Malmquist answered that the journal is known for promoting social causes rather than real science. Peipins replied that social issues are a legitimate epidemiologic concern.
Linda Gass remarked that it would be beneficial to see how the death certificates changed in the 1980s. She explained that an example of the death certificate is provided in the appendix of the dose reconstruction. Unfortunately, it is illegible. Hanley offered that he would try to provide legible examples of the old and new death certificates.
Sandra Reid asked how socioeconomic factors, such as the healthy worker effect, are considered in epidemiological studies. Peipins answered that the epidemiologist would compare similar populations as another hypothesis. She went on to explain that the healthy worker effect creates the illusion of a healthier than average population due to the fact that the people are working and are generally healthy. She also noted that the healthy worker effect is not affecting this particular study because it only exists in worker studies.
Al Brooks and Jack Hanley discussed wind direction with the use of an overhead. The Mangano study said that wind direction was from the southwest to the northeast. However, it was pointed out that the wind rose in the Iodine-131 dose reconstruction report indicated that at higher elevations (above 500 meters) the wind is primarily from the southwest to the northeast but in the valleys (below 250 to 500 meters) the wind blows approximately equally in both direction (southwest and northeast). This is why areas southwest of the Oak Ridge Reservation in Roane County were exposed to iodine 131. Lewis also pointed out that the largest increase of cancer deaths reportedly occurred in the Northwest, however, that area is not downwind of Oak Ridge.
Davidson pointed out that Mangano identified Anderson, Campbell, and Union Counties as the downwind counties with a percent rate change of 50.8%. She commented that since Anderson County has a rate change of 39.1%, Campbell and Union County are contributing a very high rate to that downwind change. She concluded that this disproves his proximity hypothesis.
The next ORRHES meeting is scheduled for September 11 and 12, 2001. Peipins
will present the main guidelines for evaluating an epidemiologic study
and will review the findings that resulted from this meeting. PHAWG agreed
that they needed to provide ORRHES with an opinion about the usefulness
of Mangano's report. However, they wish to wait until after viewing the
videotape that Stokes has offered to provide.
Peipins said that "there is no perfect study."