Federal money spent in the area of biodefense has increased significantly over the past four years. In fiscal year 2001 the Department of Health and Human Services (HHS) spent $294 million, and in fiscal year 2003, $3,000 million was spent.1 Has the federal money made a difference in the level of preparedness in the country and specifically in the public health system? Major reports from agencies such as the National Advisory Committee on Terrorism and Children and from the Association of State and Territorial Health Officials (ASTHO) have called for continual survey and assessment of the needs faced by local, state, and national organizations.

In fact, soon after 11 September, several prominent surveys were conducted at various levels of the public health and other government areas, including those sponsored by the Centers for Disease Control and Prevention (CDC). Particularly notable were two large surveys prepared voluntarily to assess local and state capacities as a baseline for future planning and appropriate use of federal funding:2, 3 Six months after $1 billion in federal funding was made available for public health preparedness, ASTHO conducted a survey to assess what differences this new stream of funding was making. In addition, a major survey of county preparedness was performed by the Department of Justice.

To our knowledge there has been no previous recording of some of the major surveys and evaluation instruments on local, state, and national levels and a brief comment on the results that are publicly releasable. We will particularly address the survey instruments or evaluation tools used since the ASTHO survey of all states in 2002 called the Bioterrorism Preparedness Accountability Indicators Project.4 The ASTHO report records the progress made from 1 June 2002 through 2 December 2002. While there have been advances since that report was published, it was a major step in beginning to chart the nation’s preparedness levels and identify major gaps that still need considerable effort.

The ASTHO report collected data from 44 states, Washington, DC, and Los Angeles County health agencies. It concluded that 80% of those interviewed reported that progress had been made in assessing public health systems’ capacities and that there has been significant progress in assuring 24/7 communication in the health agencies. The report stated, however, that there was a clear need to test plans, and only 25% had completed their written Strategic National Stockpile plans. The report also stated that one major challenge was to assemble the workforce required to carry out the preparedness planning. There were even larger problems finding people to fill leadership roles in planning. The major theme of the report was that preparedness is a process that will take long-term continued support and funding for it to become a reality. There is a need to continue monitoring and assessing the preparedness efforts made by different agencies to determine the country’s level of readiness.

An Overview of Some Major Assessments of Bioterrorism Preparedness Done by Professional Organizations and Foundations in the United States

Center for Infectious Disease Research and Policy:5 “Providing a Framework of Public Health Bioterrorism Preparedness: Public Health Workforce, Collaboration, and Infrastructure Issues.” May 2002. Data and information were collected through informal discussion of the adequacy of the public health workforce and the level of collaboration between agencies. Public health professionals (health departments, professional organizations, foundations, etc.) were surveyed. Some major conclusions:

• There is an overall shortage of qualified public health workers
• There are administrative barriers (noncompetitive pay, difficult hiring procedures, and lack of flexibility)
• There is a nursing shortage
• There is no collaboration between public health and law enforcement
• The goals of preparedness are undefined
• There is no local political support for public health

National Association of County and City Health Officials:6 “One Year Later: Improvements in Local Public Health Preparedness Since September 11, 2001.” November 2002. An open-ended questionnaire was administered online to local public health agency representatives from 44 states. Some major conclusions:

• Approximately one-third stated that response plans were complete
• There was an increased level of community collaboration
• Approximately 50% offered some form of preparedness training to staff
• Most stated that the staff size was insufficient

The State of Oregon’s Preparedness Advisory Committee and Emergency Management in Tualatin Valley:7 “Recurring Pitfalls in Hospital Preparedness and Response” (Jeffrey Rubin). January 2004. The survey used a literature review of available material on hospital preparedness, personal observances, and personal communications. Some major conclusions:

• Hospitals are operating at a financial loss
• Intrafacility communications are poor
• Security is poor, and most hospital security personnel are unarmed and lack arrest power
• Many hospitals find single-patient decontamination hard
• There is inadequate training, information, and personal protective equipment

National Association of School Resource Officers:8 “School Safety Threats Persist, Funding Decreasing: NASRO 2003 National School-Based Law Enforcement Survey.” August 2003. School resource officers attending the 13th annual NASRO conference were surveyed. Some major conclusions:

• Over 90% of schools are “soft targets” for terrorist attacks
• There are significant gaps in schools’ emergency preparedness planning
• There are no formal guidelines for what schools should do when there is a change in the national security alert level
• School crisis plans are not routinely exercised
• Funding is declining while increasing resources are needed to adequately prepare schools in the event of an attack

Drexel School of Public Health:9 “The Role of Federally Qualified Health Centers in the City of Philadelphia’s Response to Biological Terrorism” (Nilani Anne Jayatilaka). May 2003. Federally qualified health centers were surveyed (the medical director, if available, was surveyed). Some major conclusions: Federally qualified health centers

• Have extensive bilingual capacity and community ties
• Are limited in space, equipment, communication, training in bioterrorism, and supplies
• Could participate in a response by providing evaluation and triage, first aid, and some primary care
• Are limited in their ability to provide respiratory isolation
• Must undergo some reconstruction if they are to be able to handle mass causalities

Council of State and Territorial Epidemiologists:10 “National Assessment of Epidemiologic Capacity: Findings and Recommendations.” March 2003. The Epidemiology Capacity Assessment Questionnaire was given to state and territorial epidemiologists. Some major conclusions:

• Most epidemiologists believe that they have insufficient staff and resources
• 42% of the epidemiology workforce lacks formal epidemiology training
• Few states have surveillance systems that integrate emergency medical services and emergency departments
• Cultural competency training is rarely provided
• The epidemiologic capacity must increase
• There are large regional variations in per-person expenditures for epidemiology programs

Trust for America’s Health:11 “Ready or Not? Protecting the Public's Health in the Age of Bioterrorism” (Shelley Hearne et al.). December 2004. The Trust for America’s Health developed 10 indicators, and each of the 50 states received a score from 0 to 10 depending on the number of indicators achieved. Some major conclusions:

• Public health budgets are declining
• Approximately one-third of states cut funding to public health programs in 2003
• The public health system lacks funding and resources to offer salaries to compete with the private sector
• Approximately 50% of state health department epidemiologists lack formal training in their area of expertise
• 89% of the American population is covered by the CDC’s Health Alert Network

American Public Health Association:12 “One Year After the Terrorist Attacks: Is Public Health Prepared? A Report Card from the APHA.” September 2002. The report surveyed APHA members and compared the state of preparedness with the Principles for a Public Health Response to Terrorism developed by the APHA. Some major conclusions:

• Expansion of the laboratory response network
• No baseline performance goals and standards to assess preparedness
• Lack of regional preparedness
• Development of Medical Reserve Corps
• Severe shortage of epidemiologists, microbiologists, and public health nurses
• Lack of surge capacity
• Expanded communication systems

ANSER:13 “Drafted to Fight Terror” (Elin Gursky). August 2004. A literature review, self-administered survey, and focus groups of 54 professionals who hold organizational, academic, or other leadership positions, plus and a focus group at the NACCHO and ASTHO 2003 conference. Some major conclusions:

• Public health is late to develop data systems and communications equipment
• The public health workforce lacks a common standard basic level of training
• New talent needs to be recruited
• 42% of state and territorial epidemiologists lacked formal epidemiology training
• Epidemiologists make up only 0.5% of the public health workforce
• Nearly 50% of federal employees in biodefense will be eligible to retire in 2008
• Public health has expanded responsibilities with only limited resources
• Critical relationships do not exist between hospitals, health departments, the FBI, emergency medical services, etc.
• Public health is not accustomed to 24/7 operation and security roles

An Overview of Some Major Assessments of Bioterrorism Preparedness Done by Government Agencies in the United States

General Accounting Office:14 “HHS Bioterrorism Preparedness Programs: States Reported Progress but Fell Short of Program Goals for 2002.” February 2004. Interviews with state officials and self-administered progress reports required by CDC and/or the Health Resources and Services Administration for funding. Some major conclusions:

• No state had completed all CDC 2002 requirements
• Few states had a regional response plan across borders
• The smallpox vaccination program redirected many of the resources marked for completion of CDC’s requirements
• No state had a plan for its hospitals to respond to an epidemic affecting more than 500 patients

HHS:15 “Office of the Inspector General: State and Local Bioterrorism Preparedness.” December 2002. In-person interviews, document reviews, and self-administered questionnaires, plus inspection of 12 states and 36 local health departments. Some major conclusions:

• Surveillance systems are weak because they are not timely
• Epidemiologic capacity is low because of limited resources
• The laboratory response network is not fully functional
• States lack legal authority to take needed public health action

General Accounting Office:16 “Bioterrorism: Preparedness Varied Across State and Local Jurisdictions.” April 2003. In-person interviews, document reviews, and seven site visits to American cities. Some major conclusions:

• Lack of guidance from the federal government
• Desire to share best practices
• Regional planning between states is lacking
• Staffing shortages
• Inadequate surveillance systems
• Inadequate surge capacity

Discussion

There have been many advances in the area of preparedness, but we saw common threads regarding unmet training and preparedness needs in many sectors, including epidemiology and other public health infrastructure, laboratories, informatics and communications, hospital and workforce preparedness, and the general preparedness of communities broadly defined in many areas, especially in rural communities. For example a report sponsored by the Harvard School of Public Health and the University of Pittsburgh stated that rural health officials believe they are unprepared to respond to a terror attack, whether it be on food or nuclear power plants or another facility.17

We believe that these commonly repeated general results of some major surveys seem to have validity:

• There is a continuing deficit in the number of qualified professionals, especially epidemiologists and some laboratory workers, needed to fill the public health workforce.
• There is a need to systematically educate and train those workers already in the public health field about preparedness.
• There has been a marked advancement in the acquisition and use of communication systems by and within the public health sector.
• There have been some advances in programs regarding surveillance for diseases.
• Most of the emphasis, as far as funding, has centered on biological agents, with a special emphasis on smallpox.
• There was little emphasis on mental health or the needs of children and other vulnerable populations.
• Mass decontamination or isolation procedures would be difficult for many hospitals because of staffing and equipment.

Of note to local communities interested in public health assessments of readiness, the CDC is developing and preparing to beta test new evidence-based performance goals for public health preparedness in Pennsylvania. The goals are based on four preparedness principles:

• Systems are connected pieces (agencies should seek connections with other agencies to create an integrated response to an attack)
• Planning should be supported by evidence (goals should be based upon lessons learned and outcome studies)
• The focus is state support of local response (close collaboration between state and local public health agencies)
• Cost-effectiveness counts (planning that comes from an all-hazards approach will be more cost-effective than single-hazard planning)

Meanwhile, there seems to be a critical need to develop assessments that link local, state, regional, and national preparedness, and we trust that our nation’s public health and homeland security leaders will continue to aspire to this goal to use these and the information gained to continue to improve our public health readiness against terrorist threats. There also seems to be a continuing need to develop and report on surveys that address the needs of special populations, including those related to topics of mental illness and children and those that assess the continued progress of health agencies.

Notes on Our Methodology

We conducted a literature review using several search methods. We analyzed the surveys’ major findings and methods for content.

We used two search methods for web search engines (such as Yahoo and Google) and PubMed. We used these keywords:

• Evaluation of terrorism
• Assessments of terrorism
• Bioterrorism assessments
• Preparedness surveys

Web Search Engines

To collect all relevant information about each keyword, we considered the stems of the words and removed any preposition; Google considers word variations (stemming), so when using Google we just removed prepositions. For example, the first search keyword (evaluation of terrorism) was actually (evaluation OR evaluate) AND (terror OR terrorism). In the results from the web search engines, we considered the top 400 results as “candidates” and selected the results with titles that included all the words in the search keywords.

PubMed

Unlike web search engines, PubMed provides a sophisticated information retrieval method, indexing its documents using MeSH terms. To take advantage of this feature we used MeSH terms as search keywords; MeSH terms are manually assigned by the National Library of Medicine on indexing articles. However, MeSH terms are limited, so only MeSH terms for terrorism and bioterrorism are available. For other words in the search keywords Search Field Tag, [TI] as article title is used (for details refer to PubMed Help). This tag means that PubMed searches the words before the [TI] tag only in the titles of citations. Thus, the actual PubMed search keyword of the first search keyword is "Terrorism"[MeSH] evaluation[TI]. All the search results (the abstracts of citations) are considered “relevant” and are downloaded for further processing. For each search keyword, the top N nearest neighbors (articles) are selected based on our semantic ranking algorithm so that most irrelevant documents are excluded. This process is summarized in the following code.

For Each SearchKeyword (SK_i)

     WebSearchEngines (SK_i) ® WSE_SR_{SK_i}

     PubMed (SK_i) ® PM_SR_{SK_i}

     NearestNeighbor (WSE_SR_{SK_i} + PM_SR_{SK_i}, N) ® N_Doc

End For

Where SK_i is i^th search keyword, WSE_SR_{SK_i} is web search engine’s search result for i^th search keyword, N is the top N nearest neighbors, and N_Doc is N number of documents as output.

Notes on the Surveys We Reviewed

We selected surveys and assessments conducted by the government or professional organizations from 2002 through 1 January 2005 to be included in this review. We selected assessments and surveys that we believe had a major impact on the preparedness community in the previous two years at various levels.

We recognize several significant limitations of this approach. First, we did not have access to or clearance for all surveys or evaluations, including undoubtedly sensitive assessments and threat evaluations by the Departments of Homeland Security, Defense, and Health and Human Services, as well as other government departments or agencies. We recognize that it is not in the interest of national security for all such information to be disseminated.

Second, we undoubtedly missed many excellent surveys of disaster or terrorism preparedness at many levels, but we believe we captured most of the major ones during those two years. (We would welcome a summation of those being brought to our attention and/or reported to others in the peer-reviewed literature.)

Last, we realize that many local assessments of preparedness are not reported in the literature and may instead go to political leaders at the local or state level.

The assessments that have been done have used a wide variety of methods to obtain their results and formulate conclusions. Some studies used members of professional organizations, such as the Council of State and Territorial Epidemiologists, to fill out questionnaires and give opinions on the preparedness of their jurisdiction. Some used local and state health officials to fill out questionnaires to assess themselves and their jurisdiction’s level of prepared. Some sent out professionals to assess jurisdictions and interview those responsible for the preparedness efforts in the jurisdiction. Many methods relied on the impressions, thoughts, and suggestions of those working in the preparedness fields; therefore, much of the data were qualitative and difficult to apply to other areas. We also did not see evidence of evaluation tools’ being repeated commonly to assess trends toward preparedness—rather, new instruments are typically developed.

References

Click on an end note number to return to the article.

1. U.S. Department of Health and Human Services, “HHS Fact Sheet: Biodefense Preparedness—Record of Accomplishment.”

2. Centers for Disease Control and Prevention, “State Public Health Preparedness and Response Capacity Inventory,” December 2002; version 1.1:1-48.

3. Centers for Disease Control and Prevention, “Local Public Health Preparedness and Response Capacity Inventory.”

4. Association of State and Territorial Health Officials, “Public Health Preparedness: A Progress Report—The First Six Months.”

5. Center for Infectious Disease Research and Policy, “Providing a Framework for Public Health Bioterrorism Preparedness: Public Health Workforce, Collaboration, and Infrastructure Issues,” 17 May 2002.

6. The National Association of County and City Health Officials, “One Year Later: Improvements in Local Public Health Preparedness Since September 11, 2001,” Issue Brief Bioterrorism and Emergency Preparedness (serial online), November 2002.

7. Jeffrey N. Rubin, “Recurring Pitfalls in Hospital Preparedness and Response,” Journal of Homeland Security, January 2004.

8. National Association of School Resource Officers, “School Safety Threats Persist, Funding Decreasing: NASRO 2003 National School-Based Law Enforcement Survey.”

9. Nilani Anne Jayatilaka, E. Chernak, and R. Lum, “The Role of Federally Qualified Health Centers in the City of Philadelphia’s Response to Biological Terrorism.”

10. Council of State and Territorial Epidemiologists, “National Assessment of Epidemiologic Capacity in Public Health: Findings and Recommendations.”

11. Trust for America’s Health, “Ready or Not? Protecting the Public’s Health in the Age of Bioterrorism,” December 2003.

12. American Public Health Association, “One Year After the Terrorist Attacks: Is Public Health Prepared? A Report Card from the American Public Health Association.”

13. Elin Gursky, Drafted to Fight Terror (Arlington, VA: ANSER, August 2004).

14. U.S. General Accounting Office, “HHS Bioterrorism Preparedness Programs: States Reported Progress but Fell Short of Program Goals for 2002,” 10 February 2004.

15. HHS Office of the Inspector General, “State and Local Bioterrorism Preparedness” (OEI-02-01-00550), December 2002.

16. U.S. General Accounting Office, “Bioterrorism: Preparedness Varied Across State and Local Jurisdictions” (GAO-03-373), April 2003.

17. Lara Jakes Jordan, Associated Press, “Rural Areas Feel Unprepared for Attacks,” ABC News, 21 March 2005.