hypothesis generating questionnaire

• Search Menu
• Advance articles
• Editor's Choice
• 100 years of the AJE
• Collections
• Author Guidelines
• Submission Site
• Open Access Options
• About American Journal of Epidemiology
• About the Johns Hopkins Bloomberg School of Public Health
• Journals Career Network
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Abbreviations, a framework for hypothesis generation, acknowledgments.

• < Previous

Hypothesis Generation During Foodborne-Illness Outbreak Investigations

• Article contents
• Figures & tables
• Supplementary Data

Alice E White, Kirk E Smith, Hillary Booth, Carlota Medus, Robert V Tauxe, Laura Gieraltowski, Elaine Scallan Walter, Hypothesis Generation During Foodborne-Illness Outbreak Investigations, American Journal of Epidemiology , Volume 190, Issue 10, October 2021, Pages 2188–2197, https://doi.org/10.1093/aje/kwab118

• Permissions Icon Permissions

Hypothesis generation is a critical, but challenging, step in a foodborne outbreak investigation. The pathogens that contaminate food have many diverse reservoirs, resulting in seemingly limitless potential vehicles. Identifying a vehicle is particularly challenging for clusters detected through national pathogen-specific surveillance, because cases can be geographically dispersed and lack an obvious epidemiologic link. Moreover, state and local health departments could have limited resources to dedicate to cluster and outbreak investigations. These challenges underscore the importance of hypothesis generation during an outbreak investigation. In this review, we present a framework for hypothesis generation focusing on 3 primary sources of information, typically used in combination: 1) known sources of the pathogen causing illness; 2) person, place, and time characteristics of cases associated with the outbreak (descriptive data); and 3) case exposure assessment. Hypothesis generation can narrow the list of potential food vehicles and focus subsequent epidemiologic, laboratory, environmental, and traceback efforts, ensuring that time and resources are used more efficiently and increasing the likelihood of rapidly and conclusively implicating the contaminated food vehicle.

Shiga toxin-producing Escherichia coli

pulsed-field gel electrophoresis

whole-genome sequencing

hypothesis-generating questionnaire

Foodborne diseases are a continuing public health problem in the United States, where they cause an estimated 48 million illnesses, 128,000 hospitalizations, and 3,000 deaths annually ( 1 ). Public health and regulatory agencies rely on data from foodborne disease surveillance and outbreak investigations to prioritize food safety regulations, policies, and practices aimed at reducing the burden of disease ( 2 ). In particular, foodborne illness outbreaks provide critical information on the foods causing illness, common food-pathogen pairs, and high-risk production technologies and practices. However, only half of the foodborne outbreaks reported each year identify a pathogen, and less than half implicate a food vehicle, decreasing the utility of these data ( 3 ).

A model framework for hypothesis generation during a foodborne-illness outbreak investigation.

Foodborne disease outbreaks require rapid public health response to quickly identify potential sources and prevent future exposures; however, implicating a food vehicle in an outbreak can be challenging. The pathogens that contaminate food have many diverse reservoirs and can be transmitted in other ways (e.g., from one person to another or through contact with animals or contaminated water), resulting in seemingly limitless potential vehicles ( 2 ). Identifying a food vehicle is particularly challenging for clusters detected through national pathogen-specific surveillance: Cases can be geographically dispersed and lack an obvious epidemiologic link ( 4 ). Moreover, state and local health departments might have limited resources to dedicate to cluster and outbreak investigations ( 5 ). These challenges underscore the importance of hypothesis generation during an outbreak investigation. Hypothesis generation can narrow the list of potential food vehicles and focus subsequent epidemiologic, laboratory, environmental, and traceback efforts, ensuring that time and resources are used more efficiently and increasing the likelihood of timely identification of the vehicle. Timely investigations can prevent additional illnesses and increase the likelihood of identifying factors contributing to the outbreak.

The Integrated Food Safety Centers of Excellence were established in 2012 under the Food Safety Modernization Act to serve as resources for federal, state, and local public health professionals who detect and respond to foodborne illness outbreaks. The Integrated Food Safety Centers of Excellence aim to improve the quality of foodborne-illness outbreak investigations by providing public health professionals with training, tools, and model practices. In this paper, we provide a framework for generating hypotheses early during investigation of an outbreak or cluster detected through pathogen-specific surveillance; highlight tools to support rapid and effective hypothesis generation; and illustrate the practice of hypothesis generation using example outbreak case studies.

A hypothesis is “a supposition, arrived at from observation or reflection, that leads to refutable predictions; (or) any conjecture cast in a form that will allow it to be tested and refuted” ( 6 ). In a foodborne outbreak, the hypothesis states which food vehicle(s) could be the source of the outbreak and warrant further investigation. In practice, hypothesis generation is dynamic and iterative. It begins in the earliest stages of an investigation as investigators review available information and look for a pattern or “signal” that might emerge. As more information becomes available hypotheses are frequently evaluated and refined.

The framework presented here focuses on 3 primary sources of information for generating hypotheses, typically used in combination: 1) known sources of the pathogen causing illness; 2) person, place, and time characteristics of cases associated with the outbreak (descriptive data); and 3) case exposure assessment ( Figure 1 ). We discuss the approach for collecting, summarizing, and interpreting each of these sources of information and provide example outbreak case studies ( Table 1 ). We focus primarily on food exposures. However, at the onset of an investigation the transmission route is often unknown, and many pathogens commonly transmitted though food can also be transmitted through other routes (e.g., animal contact, person-to-person, waterborne). Thus, hypothesis generation should consider all potential transmission routes early in the investigation. Moreover, hypothesis generation should involve a multidisciplinary outbreak investigation team, including experienced colleagues who can provide information about past outbreaks and known sources of the pathogen causing illness.

Foodborne-Illness Outbreak Case Studies Highlighting Hypothesis-Generation Methods, United States, 2006–2018

Abbreviations: STEC: Shiga toxin-producing Escherichia coli , HG: hypothesis generation, HGQ: hypothesis-generating questionnaires, PFGE: pulsed-field gel electrophoresis.

Known pathogen sources

When generating a hypothesis, investigators should consider historical information about the causative pathogen, including known reservoirs; foods (and animals) implicated in past outbreaks; findings from case-control studies of sporadic illnesses (i.e., diagnosed cases investigated during routine surveillance not linked to other cases); and molecular subtyping information of the pathogen, including information about nonhuman isolates (i.e., food, animal, or environmental sources).

The reservoir of the infectious agent can indicate potential sources and contributing factors. Pathogens with a human reservoir (e.g., norovirus, hepatitis A virus, and Shigella ) are commonly associated with infected food handlers or ready-to-eat foods that have been contaminated with human feces. In contrast, pathogens with animal reservoirs (e.g., Shiga toxin-producing Escherichia coli (STEC), nontyphoidal Salmonella , and Campylobacter ) are often associated with food sources of animal origin or foods that have been contaminated by animal feces during production (e.g., fresh produce). Pathogens with environmental reservoirs (e.g., Vibrio spp., Listeria monocytogenes , Clostridium botulinum ) are commonly associated with foods that can become contaminated by soil or water. Tools that help identify known pathogen sources include the National Outbreak Reporting System Dashboard ( 7 ), the Food and Drug Administration Bad Bug Book ( 8 ), and An Atlas of Salmonella in the United States ( 9 ).

Food-pathogen pairs identified in past outbreaks and case-control studies of sporadic illnesses provide information on common food vehicles associated with a pathogen. Using data on reported outbreaks from 1998–2016, the Interagency Food Safety Analytics Collaboration estimated the proportion of illnesses attributable to 17 major food categories ( 10 ). The foods most commonly associated with Salmonella illnesses were seeded vegetables (e.g., tomatoes and cucumbers), chicken, pork, and fruit, whereas most STEC illnesses were attributed to leafy greens or beef, and most Listeria illnesses to dairy products or fruits. Similarly, case-control studies of sporadic illnesses have found associations between pathogens and specific foods; for example, Campylobacter and poultry ( 11 ) and Listeria monocytogenes and melons and hummus ( 12 ).

For pathogens with multiple reservoirs, information that distinguishes isolates of the same species by phenotypic or genotypic characteristics can provide increased specificity. For example, there are over 2,600 serotypes of Salmonella ; however, some serotypes have been associated with specific food vehicles, such as Salmonella enterica serotype Enteritidis (SE) and eggs and chicken; serotypes Uganda and Infantis and pork; and serotypes Litchfield, Poona, Oranienburg, and Javiana and fruit ( 13 ). Antimicrobial resistance has also proven useful in differentiating major sources of Salmonella serotypes found in both animal- and plant-derived food commodities. For example, antimicrobial-resistant Salmonella outbreaks were more likely to be associated with meat and poultry (e.g., beef, chicken, and turkey), whereas foods commonly associated with susceptible Salmonella outbreaks were eggs, tomatoes, and melons ( 14 ).

Molecular subtyping with pulsed-field gel electrophoresis (PFGE) has been an essential subtyping tool for outbreak detection, and PFGE patterns have been associated with specific foods . For example, SE isolates with PFGE PulseNet pattern JEGX01.0004 have commonly been associated with eggs (and more recently, chicken), pattern JEGX01.0005 with chicken, and pattern JEGX01.0002 with travel or exposure to the US Pacific Northwest region and Mexico. Similarly, the same PFGE pattern of STEC O157:H7 has been associated with recurrent romaine lettuce outbreaks ( 15 , 16 ). In July 2019, whole-genome sequencing (WGS) replaced PFGE as the standard molecular subtyping method for the national PulseNet network, providing greater discrimination and more reliable indication of genetically related groupings than PFGE. This change in molecular method might limit historical comparisons temporarily, particularly to isolates from before the transition, as PFGE patterns and WGS results are not readily comparable. However, WGS allele codes have been applied to sequenced historical isolates in PulseNet, and although this represents a small proportion of all isolates in PulseNet, the representativeness of the WGS database will increase with time. As historical isolates and regulatory isolates from the Food and Drug Administration and US Department of Agriculture Food Safety and Inspection Service are sequenced, information about recent findings in foods and animals will fill the national database maintained at the National Center for Biotechnology Information ( 17 ) and be readily comparable to sequenced human clinical isolates.

Subtyping of nonhuman isolates collected by regulatory agencies from foods and food chain environments through routine testing or special studies can lead to the identification of outbreaks of human illness by searching the PulseNet database for the same molecular subtypes in human infections, sometimes referred to as “backward” outbreaks. For example, in 2007 public health authorities were investigating a multistate outbreak of Salmonella serotype Wandsworth in which patients reported consuming a puffed vegetable-coated snack food. Food testing yielded the outbreak strain of Salmonella serotype Wandsworth, but it also yielded Salmonella serotype Typhimurium; a search in the PulseNet database identified matching isolates from human cases of Salmonella serotype Typhimurium infection, and these cases confirmed consumption of the same snack food upon re-interview ( 18 ). Importantly, identifying a close genetic match between strains from a product and an illness does not alone establish causation; epidemiologic investigation and traceback are needed to connect the product and patient.

Descriptive data

Descriptive epidemiology of cases, including person, place, or time characteristics, remains a powerful tool for hypothesis generation. Person characteristics can suggest foods that are more likely to be eaten by certain groups, whereas place and time characteristics can provide clues about the geographic distribution and shelf life of the food.

Person characteristics suggestive of certain foods include, but are not limited to, sex age, race, and ethnicity. For example, the median percentage of female cases in vegetable-associated STEC outbreaks was 64%, compared with 50% in beef STEC outbreaks ( 19 ). Likewise, there are differences in food consumption patterns by age, with the lowest median percent of children and adolescents in vegetable-associated STEC outbreaks and the highest in STEC dairy outbreaks ( 19 ). Similar trends are evident in the Centers for Disease Control and Prevention FoodNet Population Survey, a population-based survey to estimate the prevalence of risk factors for foodborne illness, which found that women reported consuming more fruits and vegetables than men, and men reported consuming more meat and poultry ( 20 ).

Time characteristics, displayed by the shape and pattern of an epidemic curve, can indicate the shelf life of a product or the harvest duration of a contaminated field. For example, cases spread over a longer time period might suggest a shelf-stable or frozen food item, ongoing harborage of the contaminating pathogen in a food processing plant, or other sustained mechanism of contamination. Conversely, cases with illness onset dates spread over a limited duration of time might suggest a perishable item, such as fresh produce. However, some fresh produce items have longer shelf lives than others and can cause more protracted outbreaks. Additionally, there are “special case” produce types. For example, outbreaks associated with sprouted seeds or beans, which have a short shelf life, are typically driven by a single contaminated seed lot, and un-sprouted seeds and beans can have a shelf life of months to years. Thus, single batches might be sprouted from the same contaminated lot of seeds at different times and in different places leading to a more sustained outbreak, or resulting in temporally and geographically distinct outbreaks ( 21 ). If an outbreak is detected early and exposure is ongoing, the temporal distribution of cases might be less clear early in an investigation. Thus, epidemic curves can provide supporting evidence that adds to the plausibility of a suspected food vehicle; however, depending on the outbreak, epidemic curves might provide more relevant information as the outbreak progresses.

Geographical mapping of cases can also help assess the plausibility of a suspected vehicle by comparing the distribution of cases with the distribution pattern of that food item, in consultation with regulatory and industry partners. For example, widespread outbreaks are caused by widely distributed commercial products, and some foods are more likely to be distributed nationally (e.g., bagged leafy greens, packaged cereal, national meat brands), whereas other are more likely to be distributed regionally (e.g., popular brands of ice cream) or locally (e.g., raw milk) ( 22 ). Likewise, if some outbreak-associated illnesses are clearly related to travel to a specific country, and others are in nontravelers, it suggests the latter might be associated with a product imported from that country. For example, a 2018 outbreak of Salmonella serotype Typhimurium infections in Canada occurred among persons traveling to Thailand, and among others who shopped at particular stores in Western Canada; the outbreak was ultimately traced to contaminated frozen profiteroles imported from Thailand ( 23 ). Similarly, in a 2011 multistate outbreak in the United States, a subset of cases traveled to Mexico and ate papaya there, and nontravel-associated cases ate papaya imported from Mexico ( 24 ).

Outbreak size and distribution can suggest certain food-pathogen pairs. For example, seafood toxins like ciguatoxin are typically produced or concentrated in an individual fish and therefore cause illness in a limited number of people in a single jurisdiction, whereas Salmonella and other bacterial pathogens can contaminate large amounts of a widely distributed product ( 22 ). The distribution of cases can be misleading or incomplete early in an outbreak, so investigators must use caution when using these parameters to rule out hypotheses and revisit as additional cases are identified. Moreover, an apparently local outbreak can be an early indicator of a larger problem. For example, in 2018, a large multistate outbreak of E. coli O157:H7 infections linked to romaine lettuce was initially detected in New Jersey in association with a single restaurant chain; within 8 days of detecting the cluster it had expanded to include many more cases with a variety of different exposure locations as far away as Nome, Alaska ( 15 ).

Case exposure assessment

Rapidly collecting detailed food histories from cases in an outbreak is the most critical step in identifying commonalities between these cases. Before a cluster is detected, local or state public health agencies typically attempt to interview each individual, reportable enteric-pathogen case using a standard pathogen-specific questionnaire. If a cluster is detected, a review of these routine interviews can provide information on obvious high-risk exposures. In most jurisdictions, detailed hypothesis-generating questionnaires (HGQs) historically have been used only if commonalities are not identified from the initial routine interviews or if the hypotheses identified from routine interviews collapse under further investigation. However, a growing number of state health jurisdictions are conducting hypothesis-generating interviews with all cases of laboratory-confirmed Salmonella and STEC infection, opting to gather this information during the initial interview. This method is considered a best practice to maximize exposure recall ( 25 ), shaving days or weeks off the delay between case exposure and hypothesis-generating interview.

There are 3 major types of HGQs used in the United States ( 26 ):

Oregon “shotgun” questionnaire: This questionnaire uses a “shotgun,” or “trawling” approach of asking mostly close-ended questions for a long list of individual food items. The section order is designed to prompt recall of specific food exposures through review of places where food was purchased or eaten out, and specific repetitive questions for high-risk exposures such as raw foods or sprouts.

Minnesota “long form” hypothesis-generating questionnaire: This questionnaire combines close-ended questions about fewer food items with open-ended questions that seek details on dining/purchase location and brand-variety details for all foods.

National Hypothesis Generating Questionnaire: This questionnaire is a hybridized approach developed by Centers for Disease Control and Prevention that contains elements of both the Oregon and Minnesota models. Close-ended questions are asked about an intermediate number of food items, and brand/variety details are obtained only for commonly eaten types of foods. During national cluster investigations, the National Hypothesis Generating Questionnaire is deployed across state and local health departments to improve standardization across jurisdictions.

In addition to these questionnaires, there are many modified state-specific versions and national pathogen-specific HGQs (e.g., Listeria Initiative questionnaire, Cyclospora ). The use of HGQs can be enhanced by adopting a dynamic or iterative cluster investigation approach. In this approach, if a suspected food item or branded product emerges during interviews, that food item can be added to questionnaires administered to subsequent cases, and individuals who have already been interviewed can be re-interviewed to systematically collect information about that exposure ( 27 ). Decisions about which exposures should be pursued through re-interviews can be informed by descriptive data, as well as incubation periods, which can help define the most likely exposure period ( 28 ).

The number of interviewers participating in hypothesis-generating interviews can depend on resources and the specifics of the outbreak. A single interviewer approach can be advantageous in that a single interviewer might more clearly remember what previously interviewed persons mentioned and pursue clues as they arise during a live interview. However, this approach could slow investigations, particularly in sizable multistate clusters. An alternative is the “lead investigator model,” in which a single person directs the interviewing team with a limited number of interviewers, reviews completed interviews, and decides which exposures to pursue. This approach can be faster and more efficient than the single interviewer approach. When interviews are done by multiple agencies, it is important that the completed interviews be forwarded to the lead investigator promptly and that the group meet regularly and review results of interviews as the investigation proceeds.

If interviews with HGQs do not yield an actionable hypothesis, investigators should consider alternative approaches, such as questionnaire modification or open-ended interviews. Deciding when to attempt an alternative approach depends on cluster size, velocity of incident cases, and investigation effort expended and time elapsed without identification of a solid hypothesis. Questionnaire modification could include adding questions, such as open-ended questions or supplemental questions about exposures that came up on previous interviews, or pruning questions. For example, after 8–10 interviews, items that no case reported “yes” or “maybe” to eating may be removed. Removal of questions should be done cautiously because certain foods (e.g., stealth ingredients such as cilantro and sprouts) might be reported by a low proportion of cases who ate them. Another approach is open-ended interviews of recent cases, which could be considered after 20–25 initial cases in a large multistate investigation have been interviewed without yielding solid hypotheses. Conducted by a single interviewer, if possible, open-ended interviews should cover everything that a case ate or drank in the exposure period of interest, as well as other exposures including animals, grocery stores, restaurants, travel, parties or events, and details about how they prepare their food at home, including recipes. After the first person is interviewed, objective questions about specific exposures can be added to the open-ended interviews of subsequent cases, creating a hybrid open-ended/iterative model. This requires cooperative patients and a persistent investigative approach but has yielded correct hypotheses with as few as 2 interviews ( 29 ).

Additional methods to ascertain exposures, such as obtaining consumer food purchase data, can be appropriate, particularly for outbreaks where obtaining a food history is challenging ( 30 ). For example, during a multistate Salmonella serotype Montevideo outbreak, initial hypothesis-generating interviews did not identify a clear signal beyond shopping at the same warehouse store. Investigators used shopper membership card purchase information to generate hypotheses, which ultimately helped identify red and black peppercorns coating a ready-to-eat salami as the vehicle ( 31 ). In addition, information from services for grocery home delivery, restaurant take-out delivery, and meal kits might help to clarify specific exposures. Other potential methods include focus-group interviews and household inspections, although these are used more rarely and in specific scenarios, with mixed results ( 32 ).

Binomial probability comparisons can further refine hypotheses by comparing the proportion of cases in an outbreak reporting a food exposure with the expected background proportion of the population reporting the food exposure ( 33 , 34 ). Binomial probability calculations in foodborne-disease outbreak investigations emerged in Oregon in 2003 as a complement to the pioneered “shotgun” questionnaire and use independent data sources on food exposure frequency from sporadic cases, past outbreak cases, or well persons sampled from the population. Such data sources include data from healthy people surveyed as part of the FoodNet Population Survey, standardized data collected in previous outbreaks, or sporadic cases as is done with the Listeria Initiative and Project Hg ( 33 , 35 , 36 ).

Hypothesis generation is a critical, but challenging, step in a foodborne outbreak investigation. A well-informed hypothesis can increase the likelihood of rapidly and conclusively implicating the contaminated food vehicle; conversely, the chances of implicating a food item are small if that item is not considered as part of the outbreak investigation. Inadequate hypothesis generation can delay investigation progress and limit investigators’ ability to rapidly identify the outbreak source, potentially leading to prolonged exposure and more illnesses. The 3 primary sources of information presented as part of this framework—known sources of the pathogen causing illness, descriptive data, and case exposure assessment—provide vital information for hypothesis generation, particularly when used in combination and revisited throughout the outbreak investigation.

Despite these sources of information, there are certain types of outbreaks for which hypothesis generation is inherently more challenging. These include outbreaks for which the vehicle has a high background rate of consumption (e.g., chicken) or outbreaks associated with a “stealth” food (e.g., garnishes, spices, chili peppers, or sprouts) that many cases could have consumed, but few remember eating. These challenges can sometimes be overcome by obtaining details on food exposures such as brand/variety and point of purchase. Obtaining this information is also critical to rapidly initiating a traceback investigation. An outbreak might also be caused by multiple contaminated food products when, for example, multiple foods have a single common ingredient or when poor sanitation or contaminated equipment leads to cross-contamination. Furthermore, the key exposure might not be a food at all, but rather an environmental or animal exposure, emphasizing that food should not be the default hypothesis.

There might be specific clues or “toe-holds” that help identify a hypothesis and accelerate an investigation. For example, cases with restricted diets, food diaries, or highly unusual or specific exposures can narrow the list of potential foods. This could include cases who traveled briefly to the outbreak location, and thus had a limited number of exposures. Smaller, localized clusters within a larger outbreak associated with restaurants, events, stores, or institutions, or “subclusters,” are often crucial to hypothesis generation, providing a finite list of foods. For example, in a multistate outbreak of Salmonella serotype Typhimurium infections associated with consumption of tomatoes, comparison of 4 restaurant-associated subclusters was instrumental in rapidly identifying a small set of potential vehicles ( 4 ). Subcluster investigations are precisely focused and as such can lead to much more rapid and efficient hypothesis generation and testing than attempts to assess all exposures among all cases in a large outbreak. Because of the immense value of subclusters, every effort should be made to quickly identify them through initial interviews and the iterative interviewing approach ( 25 ).

The majority of outbreaks are associated with common foods previously associated with that pathogen. In an investigation, it is important to both rule in and rule out common vehicles, while keeping an open mind about potential novel vehicles. If investigators suspect a novel vehicle, they should still rule out the most common vehicles when designing epidemiologic studies. For example, if an STEC outbreak investigation implicates cucumbers, regulatory partners will want to confirm that investigators have eliminated common STEC vehicles such as ground beef, leafy greens, and sprouts. That said, food vehicles change over time, reflecting changing food preferences and trends in food safety measures, and new vehicles continue to emerge (e.g., in recent years: SoyNut butter, raw flour, caramel apples, kratom, and chia seed powder). HGQs are biased toward previously implicated foods and a finite list of foods. If cases continue without a clear hypothesis emerging, it might be necessary to try open-ended hypothesis-generating interviews.

Hypothesis generation during foodborne outbreak investigation will evolve as laboratory techniques advance. Molecular sequencing techniques based on WGS might give investigators more conviction in devoting resources to following leads because there is more confidence that the cases have a common source for their illnesses ( 17 , 37 ). Concurrent or recent nonhuman isolates (e.g., food isolates) that match human case isolates by sequencing will be considered even more likely to be related to the human cases and become a priori hypotheses during investigations.

Foodborne-outbreak investigation methods are constantly evolving. Food production, processing, and distribution are changing to meet consumer demands. Outbreak investigations are more complex, given that laboratory methods for subtyping, strategies for epidemiologic investigation, and environmental assessments are also changing. Rapid investigation is essential, because with mass production and distribution, food safety errors can cause large and widespread outbreaks. Outbreak investigations balance the need for expediency to implement control measures with the need for accuracy. If hastily developed hypotheses are incorrect or insufficiently refined, analytical studies are unlikely to succeed and can waste time and resources. Alternatively, a refined hypothesis can lead directly to effective public health interventions, sometimes bypassing the need for an analytical study, if accompanied with other compelling evidence, such as laboratory evidence or traceback information.

Effectively and swiftly sharing data across jurisdictions increases an investigations team’s ability to quickly develop hypotheses and implicate food vehicles. Successful investigations depend on including the correct hypothesis, the result of a systematic approach to hypothesis generation. The exact path to identifying a hypothesis is rarely the same between outbreaks. Therefore, investigators should be familiar with different hypothesis-generating strategies and be flexible in deciding which strategies to employ.

Author affiliations: Department of Epidemiology, Colorado School of Public Health, Aurora, Colorado, United States (Alice E. White, Elaine Scallan Walter); Minnesota Department of Health, St. Paul, Minnesota, United States (Kirk E. Smith, Carlota Medus); Washington State Department of Health, Tumwater, Washington, United States (Hillary Booth); and Division of Foodborne, Waterborne, and Environmental Diseases, National Center for Emerging Zoonotic and Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, Georgia, United States (Robert V. Tauxe, Laura Gieraltowski).

This work was funded in part by the Colorado and Minnesota Integrated Food Safety Centers of Excellence, which are supported by the Epidemiology and Laboratory Capacity for Infectious Disease Cooperative Agreement through the Centers for Disease Control and Prevention.

Conflict of interest: none declared.

Scallan E , Hoekstra RM , Angulo FJ , et al.  Foodborne illness acquired in the United States—major pathogens . Emerg Infect Dis . 2011 ; 17 ( 1 ): 7 – 15 .

Google Scholar

Tauxe RV . Surveillance and investigation of foodborne diseases; roles for public health in meeting objectives for food safety . Food Control . 2002 ; 13 ( 6-7 ): 363 – 369 .

Dewey-Mattia D , Manikonda K , Hall AJ , et al.  Surveillance for foodborne disease outbreaks—United States, 2009–2015 . MMWR Morb Mortal Wkly Rep . 2018 ; 67 ( 10 ): 1 – 11 .

Behravesh CB , Blaney D , Medus C , et al.  Multistate outbreak of Salmonella serotype typhimurium infections associated with consumption of restaurant tomatoes, USA, 2006: hypothesis generation through case exposures in multiple restaurant clusters . Epidemiol Infect . 2012 ; 140 ( 11 ): 2053 – 2061 .

Boulton ML , Rosenberg LD . Food safety epidemiology capacity in state health departments—United States, 2010 . MMWR Morb Mortal Wkly Rep . 2011 ; 60 ( 50 ): 1701 – 1704 .

Porta MA A Dictionary of Epidemiology . 5th ed. New York, NY : Oxford University Press ; 2008 ( 4 ): 82 .

Centers for Disease Control and Prevention . National Outbreak Reporting System Dashboard. https://wwwn.cdc.gov/norsdashboard/ . Updated December 7, 2018 . Accessed April 9, 2021 .

Lampel KA , Al-Khaldi S , Cahill SM , eds. Bad Bug Book, Foodborne Pathogenic Microorganisms and Natural Toxins . 2nd ed. Washington, DC : Food and Drug Administration ; 2012 .

Google Preview

Centers for Disease Control and Prevention . An Atlas of Salmonella in the United States, 1968–2011: Laboratory-Based Enteric Disease Surveillance . Atlanta, GA : US Department of Health and Human Services, CDC ; 2013 . https://www.cdc.gov/salmonella/pdf/salmonella-atlas-508c.pdf . Accessed April 9, 2021 .

Interagency Food Safety Analytics Collaboration . Foodborne Illness Source Attribution Estimates for 2017 for Salmonella , Escherichia coli O157, Listeria monocytogenes , and Campylobacter Using Multi-Year Outbreak Surveillance Data, United States . Atlanta, GA and Washington DC : US Department of Health and Human Services ; 2019 . https://www.cdc.gov/foodsafety/ifsac/pdf/P19-2017-report-TriAgency-508-archived.pdf . Accessed April 9, 2021 .

Friedman CR , Hoekstra RM , Samuel M , et al.  Risk factors for sporadic Campylobacter infection in the United States: a case‐control study in FoodNet sites . Clin Infect Dis . 2004 ; 38 ( suppl 3 ): S285 – S296 .

Varma J , Samuel M , Marcus R , et al.  Listeria monocytogenes infection from foods prepared in a commercial establishment: a case-control study of potential sources of sporadic illness in the United States . Clin Infect Dis . 2007 ; 44 ( 4 ): 521 – 528 .

Jackson BR , Griffin PM , Cole D , et al.  Outbreak-associated Salmonella enterica serotypes and food commodities, United States, 1998--2008 . Emerg Infect Dis . 2013 ; 19 ( 8 ): 1239 – 1244 .

Brown AC , Grass JE , Richardson LC , et al.  Antimicrobial resistance in Salmonella that caused foodborne disease outbreaks: United States, 2003–2012 . Epidemiol Infect . 2017 ; 145 ( 4 ): 766 – 774 .

Centers for Disease Control and Prevention . Multistate outbreak of E. coli O157:H7 infections linked to romaine lettuce. https://www.cdc.gov/ecoli/2018/o157h7-04-18/index.html . Published June 28, 2018 . Accessed August 6, 2020 .

Centers for Disease Control and Prevention . Outbreak of E. coli infections linked to romaine lettuce. https://www.cdc.gov/ecoli/2019/o157h7-11-19/index.html . Published January 15, 2020 . Accessed August 6, 2020 .

Besser JM , Carleton HA , Trees E , et al.  Interpretation of whole-genome sequencing for enteric disease surveillance and outbreak investigation . Foodborne Pathog Dis . 2019 ; 16 ( 7 ): 504 – 512 .

Sotir MJ , Ewald G , Kimura AC , et al.  Outbreak of Salmonella Wandsworth and Typhimurium infections in infants and toddlers traced to a commercial vegetable-coated snack food . Pediatr Infect Dis J . 2009 ; 28 ( 12 ): 1041 – 1046 .

White A , Cronquist A , Bedrick E , et al.  Food source prediction of Shiga toxin-producing Escherichia coli outbreaks using demographic and outbreak characteristics, United States, 1998–2014 . Foodborne Pathog Dis . 2016 ; 13 ( 10 ): 527 – 534 .

Shiferaw B , Verrill L , Booth H , et al.  Sex-based differences in food consumption: Foodborne Diseases Active Surveillance Network (FoodNet) Population Survey, 2006–2007 . Clin Infect Dis . 2012 ; 54 ( suppl 5 ): S453 – S457 .

Ferguson DD , Scheftel J , Cronquist A , et al.  Temporally distinct Escherichia coli O157 outbreaks associated with alfalfa sprouts linked to a common seed source—Colorado and Minnesota, 2003 . Epidemiol Infect . 2005 ; 133 ( 3 ): 439 – 447 .

Tauxe RV . Emerging foodborne diseases: an evolving public health challenge . Emerg Infect Dis . 1997 ; 3 ( 4 ): 425 – 434 .

Public Health Agency of Canada . Public Health Notice—outbreak of Salmonella infections linked to Celebrate brand frozen classic/classical and egg nog flavoured profiteroles (cream puffs) and mini chocolate eclairs. https://www.canada.ca/en/public-health/services/public-health-notices/2019/outbreak-salmonella.html . Published June 27, 2019 . Accessed August 6, 2020 .

Mba-Jonas A , Culpepper W , Hill T , et al.  A multistate outbreak of human Salmonella Agona infections associated with consumption of fresh, whole papayas imported from Mexico—United States, 2011 . Clin Infect Dis . 2018 ; 66 ( 11 ): 1756 – 1761 .

Hedberg C . Guidelines for Foodborne Disease Outbreak Response . 3rd ed. Atlanta, GA : Council to Improve Foodborne Outbreak Response (CIFOR) ; 2020 .

Centers for Disease Control and Prevention . Foodborne disease outbreak investigation and surveillance tools. https://www.cdc.gov/foodsafety/outbreaks/surveillance-reporting/investigation-toolkit.html . Reviewed June 10, 2021 . Accessed July 2, 2021 .

Meyer SD , Kirk SE , Hedberg CH . Chapter 7.2—Surveillance for foodborne diseases, part 2: investigation of foodborne disease outbreaks. In: M'ikanatha NM , Lynfield R , Van Beneden CA , et al. eds. Infectious Disease Surveillance . 5th ed. West Sussex, UK : Wiley-Blackwell ; 2013 : 120 – 128 .

Chai S , Gu W , O'Connor KA , et al.  Incubation periods of enteric illnesses in foodborne outbreaks, United States, 1998-2013 . Epidemiol Infect . 2019 ; 147 :e285.

Angelo KM , Conrad AR , Saupe A , et al.  Multistate outbreak of Listeria monocytogenes infections linked to whole apples used in commercially produced, prepackaged caramel apples: United States, 2014-2015 . Epidemiol Infect . 2017 ; 145 ( 5 ): 848 – 856 .

Møller FT , Mølbak K , Ethelberg S . Analysis of consumer food purchase data used for outbreak investigations, a review . Euro Surveill . 2018 ; 23 ( 24 ):1700503.

Gieraltowski L , Julian E , Pringle J , et al.  Nationwide outbreak of Salmonella Montevideo infections associated with contaminated imported black and red pepper: warehouse membership cards provide critical clues to identify the source . Epidemiol Infect . 2013 ; 141 ( 6 ): 1244 – 1252 .

Ickert C , Cheng J , Reimer D , et al.  Methods for generating hypotheses in human enteric illness outbreak investigations: a scoping review of the evidence . Epidemiol Infect . 2019 ; 147 :e280.

Jervis RH , Booth H , Cronquist AB , et al.  Moving away from population-based case-control studies during outbreak investigations . J Food Prot . 2019 ; 82 ( 8 ): 1412 – 1416 .

Keene W . The use of binomial probabilities in outbreak investigations (abstract). In: Presented at the Annual OutbreakNet Conference, Long Beach . California ; September 22, 2011 .

McCollum JT , Cronquist AB , Silk BJ , et al.  Multistate outbreak of listeriosis associated with cantaloupe . N Engl J Med . 2013 ; 369 ( 10 ): 944 – 953 .

Centers for Disease Control and Prevention . National Listeria Surveillance: Listeria initiative. https://www.cdc.gov/nationalsurveillance/listeria-surveillance.html . Published September 13, 2018 . Accessed August 6, 2020

Jackson BR , Tarr C , Strain E , et al.  Implementation of nationwide real-time whole-genome sequencing to enhance listeriosis outbreak detection and investigation . Clin Infect Dis . 2016 ; 63 ( 3 ): 380 – 386 .

Sharapov UM , Wendel AM , Davis JP , et al.  Multistate outbreak of Escherichia coli O157:H7 infections associated with consumption of fresh spinach: United States, 2006 . J Food Prot . 2016 ; 79 ( 12 ): 2024 – 2030 .

Neil KP , Biggerstaff G , MacDonald JK , et al.  A novel vehicle for transmission of Escherichia coli O157:H7 to humans: multistate outbreak of E. coli O157:H7 infections associated with consumption of ready-to-bake commercial prepackaged cookie dough—United States, 2009 . Clin Infect Dis . 2012 ; 54 ( 4 ): 511 – 518 .

Miller BD , Rigdon CE , Ball J , et al.  Use of traceback methods to confirm the source of a multistate Escherichia coli O157:H7 outbreak due to in-shell hazelnuts . J Food Prot . 2012 ; 75 ( 2 ): 320 – 327 .

Medus C , Meyer S , Smith K , et al.  Multistate outbreak of Salmonella infections associated with peanut butter and peanut butter-containing products—United States, 2008–2009 . MMWR Morb Mortal Wkly Rep . 2009 ; 58 ( 4 ): 85 – 90 .

Gambino-Shirley KJ , Tesfai A , Schwensohn CA , et al.  Multistate outbreak of Salmonella Virchow infections linked to a powdered meal replacement product—United States, 2015–2016 . Clin Infect Dis . 2018 ; 67 ( 6 ): 890 – 896 .

Centers for Disease Control and Prevention . Multistate outbreak of Salmonella infections linked to kratom. https://www.cdc.gov/salmonella/kratom-02-18/index.html . 2018 . Published February 20, 2018 . Accessed September 14, 2020 .

Centers for Disease Control and Prevention . Multistate outbreak of Salmonella infections linked to kratom. https://www.cdc.gov/nationalsurveillance/listeria-surveillance.html . Last reviewed September 13, 2018 . Accessed July 2, 2021 .

• disease outbreaks
• pathogenic organism
• foodborne disease

Email alerts

Citing articles via, looking for your next opportunity.

• Recommend to your Library

Affiliations

• Online ISSN 1476-6256
• Print ISSN 0002-9262
• Copyright © 2024 Johns Hopkins Bloomberg School of Public Health
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Get Email Updates

To receive email updates about this page, enter your email address:

Related Links

• Foodborne Outbreak Tracking and Reporting
• Multistate Outbreaks Linked to Animals and Animal Products
• CDC and Food Safety
• Outbreak Response and Prevention Branch
• Division of Foodborne, Waterborne, and Environmental Diseases
• Salmonella Homepage
• E. coli Homepage
• Listeria Homepage
• Foodsafety.gov
• U.S. Food and Drug Administration (FDA)
• USDA Food Safety and Inspection Service

Foodborne Disease Outbreak Investigation and Surveillance Tools

Outbreak investigations require a lot of time and energy to find out what is making people sick. Public health officials have many tools available to them to help investigate and solve foodborne and other enteric (gastrointestinal) disease outbreaks. The information found on this page represents some of these important tools.

National Hypothesis Generating Questionnaire:

The National Hypothesis Generating Questionnaire is a set of questions used by public health officials to interview ill people in the early stages of a multistate foodborne or enteric (gastrointestinal) disease outbreak investigation.

The questionnaire collects a standard set of information about food and other exposures for all outbreak cases identified during a multistate investigation. By collecting the same information across many different areas, data analysis is more efficient. Also, the time it takes to pinpoint the source of an outbreak can be reduced.

De-identified data (data with all personal identifiers removed) collected by public health officials are usually combined and analyzed by CDC and the results are shared with federal, state, and local public health partners, including the U.S. Food and Drug Administration and the U.S. Department of Agriculture.

Visit the Council to Improve Foodborne Outbreak Response website for a downloadable PDF of the National Hypothesis Generating Questionnaire [PDF – 11 pages] .

Additional Standard Questionnaires:

• Minnesota Questionnaire [PDF – 23 pages]
• Minnesota Questionnaire [DOC – 219 KB]
• Oregon Questionnaire [PDF – 6 pages]
• Oregon Questionnaire [DOC – 903KB]

Additional Tools

• Listeria Case Report [PDF – 15 pages]
• Listeria Case Report – Spanish version [PDF – 15 pages]
• Cholera and Other Vibrio Illnesses Report Form [PDF – 4 pages]
• NORS Reporting Forms and Guidance
• FoodCORE Metrics
• Guide to Confirming a Diagnosis in Foodborne Disease
• Guidelines for Specimen Collection
• Public Health Training Network
• National Environmental Assessment Reporting System (NEARS)
• e-Learning on Environmental Assessment of Foodborne Illness Outbreaks

File Formats Help:

• Adobe PDF file
• Microsoft PowerPoint file
• Microsoft Word file
• Microsoft Excel file
• Audio/Video file
• Apple Quicktime file
• RealPlayer file
• Zip Archive file
• Page last reviewed: March 24, 2015
• Page last updated: March 24, 2015
• Centers for Disease Control and Prevention National Center for Emerging and Zoonotic Infectious Diseases (NCEZID) Division of Foodborne, Waterborne, and Environmental Diseases (DFWED)

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Epidemiol Infect
• v.148; 2020

The use of multiple hypothesis-generating methods in an outbreak investigation of Escherichia coli O121 infections associated with wheat flour, Canada 2016–2017

1 Public Health Agency of Canada, Guelph ON, Canada

2 Public Health Agency of Canada, Winnipeg MB, Canada

3 British Columbia Centre for Disease Control, Vancouver BC, Canada

4 Alberta Health, Edmonton AB, Canada

B. Adhikari

5 Saskatchewan Ministry of Health, Regina SK, Canada

Y. Whitfield

6 Public Health Ontario, Toronto, ON, Canada

C. Duchesne

7 Ministère de la Santé et des Services sociaux, Quebec City, QC, Canada

8 British Columbia Public Health Microbiology and Reference Laboratory, British Columbia Centre for Disease Control, Vancouver BC, Canada

9 Provincial Laboratory for Public Health: Alberta Public Laboratories, Edmonton AB, Canada

10 Canadian Food Inspection Agency, Ottawa ON, Canada

Associated Data

The data from this paper are not publicly available due to privacy concerns and legislative requirements. Please contact the corresponding author (VM) for additional information on data.

A Canadian outbreak investigation into a cluster of Escherichia coli O121 was initiated in late 2016. When initial interviews using a closed-ended hypothesis-generating questionnaire did not point to a common source, cases were centrally re-interviewed using an open-ended approach. The open-ended interviews led cases to describe exposures with greater specificity, as well as food preparation activities. Data collected supported hypothesis generation, particularly with respect to flour exposures. In March 2017, an open sample of Brand X flour from a case home, and a closed sample collected at retail of the same brand and production date, tested positive for the outbreak strain of E. coli O121. In total, 76% (16/21) of cases reported that they used or probably used Brand X flour or that it was used or probably was used in the home during their exposure period. Crucial hypothesis-generating techniques used during the course of the investigation included a centralised open-ended interviewing approach and product sampling from case homes. This was the first outbreak investigation in Canada to identify flour as the source of infection.

Introduction

The primary purpose of foodborne illness outbreak investigations is to identify the source of illness in order to implement control and prevention measures. Investigators collate evidence from three streams of investigation: microbiological, food safety and epidemiological. The epidemiological investigation involves collecting information to generate, refine and ultimately test hypotheses regarding the source of illness. The hypothesis generation process is a critical step, as the findings inform further investigative activities and the ability to take action [ 1 , 2 ]. Despite the importance of hypothesis generation, it is often not well described in published outbreak investigation reports, limiting the ability of investigators to learn from other experiences [ 3 ]. In some investigations, a hypothesis is easily identified, but in other investigations, particularly those involving novel or unusual food products, the hypothesis generation phase can be challenging and complex. Hypothesis generation methods vary by investigation and typically iterative and overlapping. Hypothesis generation methods that may be used in foodborne illness outbreaks include case interviewing techniques (e.g. close-ended hypothesis-generating questionnaires), food history supplementation (e.g. loyalty card data), food item investigation (e.g. sampling of food from case homes), expert consultations (e.g. discussions with commodity experts) and review of additional data sources (e.g. literature reviews).

Shiga-toxigenic E. coli (STEC) is an important pathogen that causes foodborne disease. In Canada, the number of reported cases of E. coli non-O157 infections has increased over the last several years with an incidence rate from 0.41/100 000 in 2012 to 0.99/100 000 in 2017, whereas the number of E. coli O157 infections has remained relatively constant [ 4 ]. The increase in E. coli non-O157 infections has been attributed to changes in testing and reporting practices [ 5 ]. Despite this change, the true burden of illness is likely underestimated as E. coli non-O157 strains are not routinely screened for by private laboratories in some provincial jurisdictions. Historically, E. coli non-O157 outbreaks have been associated with a wide range of products including leafy greens and beef, as well as a 2016 outbreak in the United States linked to wheat flour [ 6 – 9 ].

In December 2016, a cluster of six E. coli O121 cases were identified in Canada with matching pulsed-field gel electrophoresis (PFGE) pattern combinations. Cases were geographically dispersed and had symptom onset dates within 5 weeks of each other. This was the first national outbreak associated with a non-O157 strain of E. coli identified and the first outbreak linked to wheat flour. This paper describes the outbreak investigation, highlighting the various methods used for hypothesis generation during the investigation.

Case identification

A confirmed case was defined as a resident or visitor to Canada with E. coli non-O157 that had one of the outbreak PFGE pattern combinations or was closely related by whole-genome sequencing (WGS) with symptom onset on or after 1 November 2016. Closely related was defined as within 0–10 whole genome multi-locus sequencing typing (wgMLST) allele differences.

Laboratory investigation

To aid with case identification, PFGE was completed by provincial laboratories on all E. coli non-O157 isolates reported in Canada since 1 November 2016. WGS analysis was also completed by PulseNet Canada using wgMLST within Bionumerics v.7.6 (Applied Maths) [ 10 ]. Clinical and non-clinical isolates were considered related by WGS if they were within 0–10 wgMLST allele differences. PulseNet Canada compared the Canadian isolates to the 2016 US flour outbreak based on WGS data. In-silico prediction of virulence factors was done for Canadian isolates using the genotyping plug-in within BioNumerics v7.6 [ 11 ].

Hypothesis generation

A variety of hypothesis-generating methods were used during this outbreak investigation. The most successful methods are described in detail below, with additional methods listed in Table 1 .

Summary of hypothesis generation techniques used in the investigation of E. coli O121 infections, Canada, 2016–2017

Interview techniques

Initial case interviews were conducted by local public health investigators using routine provincial case questionnaires or the national E. coli hypothesis-generating questionnaire. Routine case questionnaires differ by the province in regard to the specific exposures included and the level of details collected on these exposures. This initial interview is often conducted before typing information is available to indicate the case is part of an outbreak investigation.

In order to collect more information on exposures, two initial cases were centrally re-interviewed by a single interviewer at the Public Health Agency of Canada (PHAC) using the E. coli national hypothesis-generating questionnaire, which is a close-ended questionnaire including an extensive list of possible sources of illness. When a suspect source was not identified, the interview strategy moved to open-ended interviewing by a single, centralised interviewer from PHAC. Open-ended interviews did not follow a questionnaire or script; these interviews were designed to elicit free-form responses. Cases were asked about food consumed the week prior to symptom onset as well as general food preferences, purchasing habits and cooking practices. Knowledge of the 2016 E. coli O121 flour-associated outbreak in the U.S. prompted interviewers to ask about baking when open-ended interviewing was initiated [ 9 ]. As new exposures were identified through case interviews, these exposures were included in subsequent open-ended case interviews as part of an iterative hypothesis generation process. Following a hypothesis generation session with the U.S. Centers for Disease Control and Prevention colleagues, who had investigated an outbreak of E. coli O121 that was associated with contaminated flour earlier in 2016, exposure to raw flour or dough was also asked during case re-interviews [ 9 ].

Food history supplementation

Cases were asked about the availability of grocery store loyalty card records to provide detailed information on grocery store purchases. In the early stages of the investigation, loyalty card information for purchase histories of 2 months prior to onset was requested. As the investigation progressed and lengthened in time, a longer period, up to a year, was used for loyalty card information. Information from loyalty card records was categorised, collated and analysed to identify common food products.

Food item investigation

Cases were asked if any leftover food items that were consumed in the week prior to their illness were available for sampling. The food items were tested for verotoxigenic E. coli (VTEC) if they were items of interest at that point in the investigation and were biologically plausible.

Food safety investigation

Once a positive finding of E. coli O121 was identified in an open sample of Brand X flour from a case home, two intact bags from the same production lot were collected and tested from a retail establishment. To determine if other flour lot codes could be affected, a range of flour products of different sizes and production dates manufactured at the originating mill between 1 September and 30 November 2016 were sampled and tested. An environmental investigation was conducted at the originating mill. Trace-back activities included reviewing raw grain input, production records and processing information to identify possible common inputs to the affected flour. Trace forward was done by following-up with food companies that had received recalled flour to determine if additional products had been made from the recalled flour that was for sale in a raw state (e.g. raw dough, uncooked pie crusts, etc.).

A total of 30 confirmed cases of E. coli O121 were identified in this outbreak investigation with symptom onset dates between 13 November 2016 and 3 April 2017. Cases ranged in age from 2 to 79 years (median 23.5 years) and 15 (50%) were female. Eight cases were hospitalised, one case developed haemolytic uremic syndrome and no deaths were reported.

Twenty-nine clinical outbreak isolates in this cluster had the same PFGE or highly similar PFGE pattern combination, all of which were new to the PulseNet Canada database. The one remaining clinical isolate had only the primary PFGE enzyme pattern available, but it was related to the other cases based on WGS. All clinical outbreak isolates were serotyped as E. coli O121. Of these, 28 were further typed as O121:H19 and two were unable to be typed further. All clinical and food isolates grouped together with 0–6 wgMLST allele differences and were considered related by WGS. None of the Canadian isolates were considered related by WGS to the 2016 US outbreak associated with flour or related to any other isolates in Canada. Based on in-silico testing, the Canadian clinical isolates carried the stx2 gene.

A timeline illustrating when key hypothesis generation methods were implemented and significant events in the outbreak investigation occurred can be seen in Figure 1 .

Timeline of significant events and key hypothesis generation methods used in the investigation of E. coli O121 infections, Canada, 2016–2017.

All 30 cases were interviewed by local public health investigators, 21 with routine provincial case questionnaires and nine with the national E. coli hypothesis-generating questionnaire. Two cases were re-interviewed by PHAC using the national E. coli hypothesis-generating questionnaire before the interview strategy moved to open-ended interviewing. Initial case interviews identified ground beef as a possible source of infections, as all six initial cases reported this exposure.

Three of these cases had consumed hamburgers at two different restaurant chains in the same province. Local public health investigators in that province determined that these restaurants obtained hamburgers from the same supplier. These initial cases were predominantly younger males and convenience foods and restaurant exposures were commonly reported by these cases.

As more cases were reported, cases continued to report ground beef consumption, but other hypotheses also emerged, including sausage style deli-meats (e.g. pepperoni, salami, bologna and sausage), bacon, pizza, pork pieces or parts and oats (e.g. raw oats, oatmeal and/or granola bars). As additional cases were reported, ground beef and sausage-style deli meats were no longer frequently reported and further data collected about these exposures did not converge on any specific products.

In total, 25 cases were re-interviewed centrally by PHAC using an open-ended approach. Just over half of these cases (13/25; 52%) were re-interviewed by PHAC on two separate occasions to ask additional questions and gather more specific information; two cases were re-interviewed three and four times, respectively.

During open-ended interviewing, the exploration of cooking and baking behaviours in the home revealed exposures to raw flour and consumption of raw batter/dough. Further questioning on brand names uncovered Brand X flour as a commonly used ingredient in baking and cooking.

Once flour was identified as the likely source, this information was strengthened by re-interviewing cases that had not been specifically asked about flour exposure earlier in the investigation to ask about their flour exposure. At the conclusion of the outbreak, 12 cases had reported using or probably using Brand X flour during the exposure period and 16 cases reported that Brand X flour was used or was probably used in the home during their exposure period ( Table 2 ). Twelve cases had both direct and indirect exposure to Brand X flour in the home and four cases had only indirect exposure to Brand X flour though baking done by others in the home. Information on brand was obtained for 7/12 (58%) cases that had direct exposure and 12/16 (75%) cases that had indirect exposure to Brand X flour only after flour was identified as the source of the outbreak. Of cases that did not report exposure to Brand X flour, one case reported exposure to pizza made at a restaurant that used flour produced by the implicated mill. An additional three cases may have had occupational exposure: one case was a baker in a restaurant (brand of flour used at work was unknown) and two cases were grocery store cashiers who may have had contact with flour through their work.

Proportion of cases reporting flour exposures in the investigation of E. coli O121, Canada, 2016–2017

Loyalty card purchase records were available for 12 cases and 28 stores. Analysis of these records did not identify potential hypotheses outside of those identified through case interviews. Purchase records of four cases verified the purchase of Brand X flour and purchase date. Three additional cases had purchased other brands of flour and the remaining five cases had no record of flour purchases.

Ten foods were collected from case homes during the investigation to aid hypothesis generation, including samples of flour ( N  = 2), oats/oatmeal ( N  = 3), ground beef ( N  = 2), baking soda ( N  = 1), vanilla ( N  = 1) and biscuits ( N  = 1) ( Table 3 ). On 23 March 2017, one of the open samples of Brand X flour collected from a case home tested positive for E. coli O121 with PFGE and WGS that matched the outbreak strain. This sample had complete lot code information available and the case had not used or handled the flour after illness onset. Results for all other samples were not detected for VTEC.

List of food items tested from confirmed case homes as part of hypothesis generation activities during the investigation of E. coli O121, Canada, 2016–2017

Both flour samples collected from a retail establishment tested positive for E. coli O121 and matched the clinical cases by PFGE and WGS. A national recall for the implicated lot of flour was conducted on 28 March 2017. A total of 109 samples from 257 bags of Brand X flour were sampled and tested (226 bags from retail establishments and 31 bags from consumer complaints). A total of 12% of samples tested positive for VTEC. The outbreak strain of E. coli O121 was isolated in several Brand X flour products of different sizes and production dates between 1 September and 30 November 2016. In addition to E. coli O121:H19, a number of different E. coli serotypes, including O8:H19, O8:H28, O15:H4, O88:H25 and O187:H52, were identified during testing of closed and open samples of flour, but no clinical cases were a match to these serotypes. Additional recalls for flour and flour-containing products were conducted between 28 March and 29 June 2017. No deviations were noted at the originating mill where the recalled product was manufactured. There was no kill step in flour processing and no testing for pathogens done at the mill. Investigation at the mill did not identify a specific source of E. coli contamination in the respective flour products.

This investigation utilised several hypothesis generation methods in order to identify Brand X flour as the source of the outbreak ( Table 1 ). Open-ended interviewing and product sampling from case homes proved to be successful hypothesis generation methods in this investigation. Other hypothesis-generating methods were not as useful in this investigation, included loyalty card data, recipe and pantry photo comparisons and food exposure reference values. However, outbreak investigations are unique and methods that may not have contributed information in this outbreak may prove to be more beneficial in other outbreak investigations. This outbreak also demonstrated the importance of triangulation of information from multiple hypothesis generation techniques to identify the source of infection.

The open-ended interview approach allowed the flexibility to explore sources of illness that were not initially considered and were not included in the hypothesis-generating questionnaires. The open-ended interviews were completed by two interviewers within one agency. However, the majority of interviews were completed by a single interviewer. Centralised re-interviewing cases from geographically dispersed jurisdictions enabled the interviewers to identify themes and food products in common among the cases. Although this method proved successful in this investigation, centralised open-ended interviewing can be resource-intensive and requires the availability of a highly trained professional. Also, the largely qualitative data obtained from these interviews can be challenging to analyse, there is not a consistent list of food items covered in each interview, and the data obtained may reflect food preferences or typical foods consumed rather than definitively consumed products. Open-ended interviewing with a single or central interviewer should be considered when investigating outbreaks where no source emerges through interviews with routine or hypothesis-generating questionnaires or when an ingredient, like flour, is suspected.

In this outbreak, the collection of food samples from case homes was conducted as part of hypothesis-generating activities. These items were collected if they represented products consumed by cases during the incubation period. As these were mostly open samples, the presence of bacteria would not automatically confirm the product as the source of the outbreak, but rather provide additional evidence to be considered in the investigation. In this investigation, the finding of the outbreak strain of E. coli O121 in an open sample resulted in a hypothesis that was tested by re-interviewing cases to ask specifically about flour as well as a food sampling plan. The open sample provided lot code and brand details which were necessary to inform the food safety investigation and take public health action (i.e. product recall).

Flour is a challenging food item to identify as it is a raw ingredient used both in cooking and baking and is not an easily recalled exposure by cases. Previous outbreaks have identified contaminated flour used in a specific product (i.e. cookie dough) as a risk factor or baking as a risk activity [ 9 , 12 , 13 ]. In this outbreak, flour was a possible exposure considered from the start because of the recent US outbreak. However, the majority of initial cases did not report baking or cooking and had other food exposures in common (i.e. ground beef and convenience foods). As additional cases were reported, no clear profile emerged: cases were varied in their age, gender, food preferences and baking or cooking behaviours. It was only after probing specifically about exposure to flour and licking the spoon while baking that flour became an item of interest. Many cases had to be contacted multiple times and asked multiple questions about flour, baking or cooking activities and consumption of raw dough or licking the spoon when there was baking in the home in order to get details on flour and Brand X exposure. There were also four cases who were exposed to Brand X flour though baking was done by others in the home, suggesting that cases should be asked about baking in the home as part of flour investigation.

Twenty-six per cent (6/23) of outbreak cases who were asked about flour exposure reported that they were not exposed to Brand X flour prior to their illness onset. This is not unusual in an outbreak investigation, as there are many reasons why a case may not recall exposure to a specific product (e.g. poor recall, ingredient in a food made by others, occupational exposure, cross-contamination). It is also possible that cases had exposure to the implicated flour through other products that were not identified during the food safety investigation or were not asked about during the re-interview (e.g. pizza made with Brand X flour).

This outbreak was the first national outbreak of non-O157 E. coli in Canada. Testing for non-O157 E. coli varies by province and territory. In some provinces, stool samples are tested for the Shiga-toxin-producing gene but in other provinces, this is not routine and only done by request. This may lead to under-diagnosis of non-O157 cases. In addition to these testing limitations, at the time of this outbreak, typing of E. coli non-O157 isolates using PFGE was not routinely done on all isolates. However, advancements in next-generation sequencing technologies as well as significant decreases in the cost of processing samples, coupled with the higher discriminatory power of WGS compared to traditional methods such as PFGE prompted Canada to transition to using WGS as the primary subtyping method for all non-O157 E. coli isolates in June 2018. This outbreak was the first in Canada to implicate flour as the source of illnesses and the second flour related outbreak in North America since 2015. Genetic analysis of E. coli O121 strains implicated in the Canadian and American outbreaks concluded that they were not related. In both these outbreaks, the root cause of contamination was not identified [ 9 ]. Flour is a raw agricultural product and is manufactured without the application of a kill step for pathogenic bacteria. Wheat is exposed to possible sources of contamination during growing and harvesting that can carry over to the end product. These recent E. coli O121 flour outbreaks, combined with the finding of other E. coli non-O157 serotypes in closed flour samples, suggest that flour is an emerging vehicle for non-O157 STEC infections and should be considered as a potential source in non-O157 STEC outbreak investigations.

Acknowledgements

The authors thank the following organisations for their contribution to the outbreak investigation: local public health colleagues and public health laboratories in British Columbia, Alberta, Newfoundland and Labrador, Saskatchewan, Ontario, Quebec and Washington State; British Columbia Centre for Disease Control; British Columbia Centre for Disease Control Public Health Laboratory; Alberta Health; Alberta Health Services; Public Health Laboratories; Newfoundland and Labrador Department of Health and Community Service; Saskatchewan Ministry of Health; Newfoundland & Labrador Public Health Laboratory; Roy Romanow Provincial Laboratory; Public Health Ontario; Public Health Ontario Laboratory; Ontario Ministry of Health and Long-Term Care; Ministère de la Santé et des Services sociaux du Québec; Laboratoire de santé publique du Québec; Canadian Food Inspection Agency; Health Canada; Public Health Agency of Canada and the Centres for Disease Control and Prevention.

Conflict of interest

Financial support.

This research received no specific grant from any funding agency, commercial or not-for-profit sectors.

Hypothesis Maker

Ai-powered research hypothesis generator.

• Scientific Research: Generate a hypothesis for your experimental or observational study based on your research question.
• Academic Studies: Formulate a hypothesis for your thesis, dissertation, or academic paper.
• Market Research: Develop a hypothesis for your market research study to understand consumer behavior or market trends.
• Social Science Research: Create a hypothesis for your social science research to explore societal or behavioral patterns.

New & Trending Tools

In-cite ai reference generator, legal text refiner, job search ai assistant.

• Video Idea Generator
• Video Script Generator
• Video Hook Generator
• YouTube Title
• YouTube Description
• YouTube Tags Generator
• YouTube Video SEO Generator
• Video Promotion Generator
• Facebook Caption
• Instagram Caption
• Twitter Caption
• Pinterest Description
• LinkedIn Caption
• TikTok Caption
• eBook Idea Generator
• eBook Title Generator
• eBook Outline Generator
• eBook Chapter Generator
• eBook Promotion Generator
• Blog Outline
• Google Ad Headline
• Google Ad Description
• Facebook Ad Copy
• LinkedIn Ad Copy
• Social Media Ideas
• Podcast ideas
• Email Subject Line
• Email Copy Generator
• Paraphrase Generator
• Text Extender
• AI Tone Changer
• Survey Questions
• Campaign Builder
• Start for free

AI Survey Questions Generator – Questionnaire Maker [Start For Free]

Come up with meaningful questions in seconds.

What Is StoryLab.ai’s Survey Questions Generator – Questionnaire Maker

StoryLab.ai is an online AI Tool that helps you improve your marketing. 

One of our AI-Powered Tools is a Suvey Question Generator. Some call it a Questionnaire Maker, Survey Maker or Random Question Generator. The tool helps you brainstorm questions to create meaningful surveys and get the data you need to improve your marketing campaigns, product, onboarding, you name it.

Simply describe the topic of your Survey and get ideas in seconds.

Don’t simply copy/paste the questions in your survey and send it out. You know your audience best. AI is here to help you broaden your creativity. By running the AI Questionnaire maker a couple of times, we’re sure you’ll be inspired to set up meaningful questions for your Survey.

Get started for free with the AI Survey Questions Generator.

StoryLab.ai – More than just an AI Survey Questions Generator

StoryLab.ai is an AI-Powered Content Creation Platform and the AI Questionnaire Maker is just one of our tools. Check out the video to see our Platform in action.

How the AI Survey Question Maker Works

It can be difficult to come up with the right questions for your Survey and you basically have one shot to get your audience to answer your questions and get the data you need to improve your marketing, product, onboarding, you name it.

Make sure the questions you’re asking are on point.

Spar with AI to come up with a long list of potential questions before selecting the best ones.

For this example, we wanted to come up with Survey Questions to learn more about how much our customers love our product.

Our input was: ‘How do you like our product’

Here’s what the AI came up with on our first run:

Outputs we’ve received from the AI Questionnaire maker on the first run:

1. On a scale of 1-10, how would you rate your overall satisfaction with our product?

2. Can you briefly describe your initial impression when you first started using our product?

3. What features of our product do you find the most useful, and why?

4. Have you encountered any difficulties while using our product? If yes, please provide details.

5. Compared to similar products in the market, how would you rate the quality of our product?

6. How well do you think our product meets your needs or solves your problems?

7. If you could change one thing about our product, what would it be and why?

8. Would you recommend our product to a colleague or friend? Why or why not?

9. How would you describe your customer service experience related to our product?

10. Are there any additional features or improvements you would like to see in future versions of our product? Please provide as much detail as you can.

As mentioned earlier. We don’t want to settle for the first couple of questions AI or we came up with. Here are the questions we got on the second run.

1. On a scale of 1 to 10, how satisfied are you with the overall performance of our product?

2. Which features of our product do you find the most useful, and why?

3. Can you identify any aspects of our product that could be improved? If yes, please elaborate.

4. How often do you use our product, and in what contexts?

5. Have you encountered any challenges or difficulties while using our product? If so, could you please describe them?

6. Compared to similar products on the market, how would you rate our product and why?

7. How well does our product meet your needs or solve the problems you face?

8. Have you utilized our customer support services? If yes, how would you rate your experience and what improvements, if any, would you suggest?

9. Would you recommend our product to others? Please explain your reasons.

10. How can we enhance our product or services to better meet your needs and expectations in the future?

We got 20 inspirational pieces for our Survey in just a couple of seconds. We can now fine-tune our inputs or take these and brainstorm further with the team to really nail our Survey. 

Create Great Survey Questions in half the time

Make sure the questions you’re asking in your Survey are meaningful. Here are the top benefits of using an AI Survey Question Maker at a glance.

Top benefits of the AI Survey Question Maker

Best practices for writing great survey questions.

Writing effective survey questions is essential for gathering accurate and meaningful data. Here are some best practices to consider:

Start with Clear Objectives:

Understand the purpose of your survey. What information do you want to gather? Having clear objectives will guide your question-writing process.

Keep it Simple:

Use simple and concise language. Avoid jargon, technical terms, or complex sentence structures that might confuse respondents.

Avoid Leading Questions:

Frame questions neutrally to avoid influencing respondents’ answers. For example, instead of asking “Don’t you agree that…”, ask “What is your opinion on…”.

Use Specific Language:

Be specific in your questions to ensure accurate responses. Avoid vague terms like “often” or “sometimes,” and use quantifiable terms like “weekly” or “monthly.”

One Question at a Time:

Each question should focus on a single topic. Avoid combining multiple questions into one, as it can lead to confusion.

Use Closed and Open-Ended Questions:

Closed-ended questions (multiple choice, yes/no) are easy to analyze, while open-ended questions provide valuable insights. Use a mix of both for a comprehensive view.

Provide Balanced Response Options:

In multiple-choice questions, offer a balanced range of response options that cover the spectrum of possible answers.

Include a “Prefer Not to Answer” Option:

Some respondents might prefer not to answer certain questions. Including this option respects their privacy and maintains data accuracy.

Consider Response Order:

Responses to the first few options in a list can be favored. Randomize the order of response options to avoid bias.

Avoid Double Negatives:

Refrain from using double negatives, as they can confuse respondents. Keep questions positively framed.

Avoid Assumptions:

Don’t assume respondents have knowledge they might not possess. If you need to ask about a complex topic, provide context or definitions.

Use Likert Scale Wisely:

If using a Likert scale (e.g., strongly agree to strongly disagree), keep the scale consistent across questions and avoid using too many scale points.

Pilot Test:

Before sending out the survey, test it with a small group to identify any issues with wording, question flow, or response options.

Avoid Sensitive or Personal Questions:

If possible, avoid asking respondents to divulge sensitive or personal information that they might be uncomfortable sharing.

Limit the Number of Questions:

Long surveys can lead to respondent fatigue and lower completion rates. Keep the survey concise and relevant.

Use Skip Logic:

If certain questions are only relevant to specific respondents, use skip logic to show or hide questions accordingly.

Prioritize Important Questions:

Place crucial questions at the beginning or middle of the survey to capture responses from more engaged participants.

Provide Clear Instructions:

If a question requires a specific format (like dates), provide clear instructions on how to answer.

Test for Clarity:

Ask colleagues or friends to review your survey for clarity and potential issues.

Thank and Provide Context:

Start with a thank-you message and a brief context for the survey. This can increase respondent engagement.

Remember that crafting effective survey questions requires a combination of writing skills and an understanding of your target audience. Following these best practices will help you create a survey that generates valuable insights.

AI Questionnaire Generator Pricing

Our pricing is set up as followed:

• Free – get 3 runs a month to try our tools out for free;
• Pro – $15 a month – 100 runs a month;
• Unlimited – $19 a month – unlimited runs a month;

You can also opt-in for the yearly membership and receive a 20% price reduction.

You can  check out our full pricing here .

What is an AI Survey Questions Generator? It’s a tool that uses artificial intelligence to create relevant and effective survey questions based on the desired topics, audience, and objectives of the survey.

How does an AI Survey Questions Generator work? The generator uses natural language processing and machine learning to understand the survey’s context and goals, then generates questions that are tailored to gather the needed information.

What are the benefits of using an AI Survey Questions Generator? Benefits include time-saving in question creation, ensuring question relevance and clarity, and potentially improving response rates and data quality due to well-crafted questions.

Can AI Survey Questions Generators customize questions for different industries? Yes, it can be tailored to generate questions specific to various industries by analyzing industry-related data and using relevant language and terminology.

How accurate are AI Survey Questions Generators? The accuracy can be high but you should still review to ensure the questions fully meet the survey’s objectives.

Are AI-generated survey questions biased? While AI strives to be unbiased, the quality and diversity of the data it was trained on can affect this. Human oversight is recommended to check for and correct any potential biases.

Can AI Survey Questions Generators handle open-ended questions? Yes, these generators are capable of creating open-ended questions, though the complexity and depth of such questions can vary based on the AI’s programming and training.

How do AI Survey Questions Generators improve survey engagement? By creating relevant, clear, and engaging questions, these generators can enhance respondent engagement, leading to more thoughtful responses and higher completion rates.

What should be considered when using an AI Survey Questions Generator? It’s important to consider the target audience, survey objectives, and the need for potential customization or human editing to ensure the questions fully align with the survey goals.

Can AI Survey Questions Generators create multilingual questions? Yes. Simply enter your inputs in the preferred language and you´ll get outputs in the same language.

How do AI Survey Questions Generators ensure question clarity? These generators use natural language processing techniques to create questions that are clear, concise, and free of ambiguity, enhancing the reliability of survey responses.

What types of surveys can benefit from AI Survey Questions Generators? They can be used for a wide range of surveys, including market research, customer satisfaction, employee feedback, and academic research.

Can these generators suggest question formats (e.g., multiple choice, rating scale)? Yes, The AI generator can suggest appropriate question formats based on the survey’s objectives and the type of data needed.

Are AI-generated questions customizable by the user? Yes. We always advise you to check the outputs before using the questions in your survey.

Can AI Survey Questions Generators adapt to different target audiences? Yes, these generators can adapt the style and complexity of questions to suit different target audiences, based on input parameters and training data.

How cost-effective are AI Survey Questions Generators for businesses? You can start for free and get 3 runs and upgrade if you need more. Check out our pricing to learn more.

What future advancements can be expected in AI Survey Questions Generators? Future advancements may include more nuanced understanding of complex topics, better integration with survey platforms, and enhanced customization options to cater to specific research needs.

Jump back to a section

• Que es StoryLab.aiGenerador de preguntas de encuestas - Creador de cuestionarios
• Cómo funciona el Creador de preguntas de encuestas de IA
• Cree excelentes preguntas para encuestas en la mitad del tiempo
• Prácticas recomendadas para redactar excelentes preguntas para encuestas
• Precios del generador de cuestionarios de IA
• Preguntas Frecuentes

YouTube Video Title Generator

Create catchy titles for your next videos.

YouTube Description Generator

Write great descriptions in half the time.

AI eBook Generator

Create full eBooks

Social Media Caption Generator

Get the most out of your Social Marketing.

Instagram Caption Generator

Stand out on Insta with great captions.

LinkedIn Caption Generator

Awesome captions in half the time

YouTube Video Idea Generator

Effective Video ideas

YouTube Video Script Generator

Engaging Videos

AI Video Hook Generator

Create Engaging Videos

All Marketing Copy Generator

See all our AI Geenrators