statement of findings in qualitative research

How To Write The Results/Findings Chapter

For qualitative studies (dissertations & theses).

By: Jenna Crossley (PhD). Expert Reviewed By: Dr. Eunice Rautenbach | August 2021

So, you’ve collected and analysed your qualitative data, and it’s time to write up your results chapter. But where do you start? In this post, we’ll guide you through the qualitative results chapter (also called the findings chapter), step by step. 

Overview: Qualitative Results Chapter

• What (exactly) the qualitative results chapter is
• What to include in your results chapter
• How to write up your results chapter
• A few tips and tricks to help you along the way
• Free results chapter template

What exactly is the results chapter?

The results chapter in a dissertation or thesis (or any formal academic research piece) is where you objectively and neutrally present the findings of your qualitative analysis (or analyses if you used multiple qualitative analysis methods ). This chapter can sometimes be combined with the discussion chapter (where you interpret the data and discuss its meaning), depending on your university’s preference.  We’ll treat the two chapters as separate, as that’s the most common approach.

In contrast to a quantitative results chapter that presents numbers and statistics, a qualitative results chapter presents data primarily in the form of words . But this doesn’t mean that a qualitative study can’t have quantitative elements – you could, for example, present the number of times a theme or topic pops up in your data, depending on the analysis method(s) you adopt.

Adding a quantitative element to your study can add some rigour, which strengthens your results by providing more evidence for your claims. This is particularly common when using qualitative content analysis. Keep in mind though that qualitative research aims to achieve depth, richness and identify nuances , so don’t get tunnel vision by focusing on the numbers. They’re just cream on top in a qualitative analysis.

So, to recap, the results chapter is where you objectively present the findings of your analysis, without interpreting them (you’ll save that for the discussion chapter). With that out the way, let’s take a look at what you should include in your results chapter.

What should you include in the results chapter?

As we’ve mentioned, your qualitative results chapter should purely present and describe your results , not interpret them in relation to the existing literature or your research questions . Any speculations or discussion about the implications of your findings should be reserved for your discussion chapter.

In your results chapter, you’ll want to talk about your analysis findings and whether or not they support your hypotheses (if you have any). Naturally, the exact contents of your results chapter will depend on which qualitative analysis method (or methods) you use. For example, if you were to use thematic analysis, you’d detail the themes identified in your analysis, using extracts from the transcripts or text to support your claims.

While you do need to present your analysis findings in some detail, you should avoid dumping large amounts of raw data in this chapter. Instead, focus on presenting the key findings and using a handful of select quotes or text extracts to support each finding . The reams of data and analysis can be relegated to your appendices.

While it’s tempting to include every last detail you found in your qualitative analysis, it is important to make sure that you report only that which is relevant to your research aims, objectives and research questions .  Always keep these three components, as well as your hypotheses (if you have any) front of mind when writing the chapter and use them as a filter to decide what’s relevant and what’s not.

Need a helping hand?

How do I write the results chapter?

Now that we’ve covered the basics, it’s time to look at how to structure your chapter. Broadly speaking, the results chapter needs to contain three core components – the introduction, the body and the concluding summary. Let’s take a look at each of these.

Section 1: Introduction

The first step is to craft a brief introduction to the chapter. This intro is vital as it provides some context for your findings. In your introduction, you should begin by reiterating your problem statement and research questions and highlight the purpose of your research . Make sure that you spell this out for the reader so that the rest of your chapter is well contextualised.

The next step is to briefly outline the structure of your results chapter. In other words, explain what’s included in the chapter and what the reader can expect. In the results chapter, you want to tell a story that is coherent, flows logically, and is easy to follow , so make sure that you plan your structure out well and convey that structure (at a high level), so that your reader is well oriented.

The introduction section shouldn’t be lengthy. Two or three short paragraphs should be more than adequate. It is merely an introduction and overview, not a summary of the chapter.

Pro Tip – To help you structure your chapter, it can be useful to set up an initial draft with (sub)section headings so that you’re able to easily (re)arrange parts of your chapter. This will also help your reader to follow your results and give your chapter some coherence.  Be sure to use level-based heading styles (e.g. Heading 1, 2, 3 styles) to help the reader differentiate between levels visually. You can find these options in Word (example below).

Section 2: Body

Before we get started on what to include in the body of your chapter, it’s vital to remember that a results section should be completely objective and descriptive, not interpretive . So, be careful not to use words such as, “suggests” or “implies”, as these usually accompany some form of interpretation – that’s reserved for your discussion chapter.

The structure of your body section is very important , so make sure that you plan it out well. When planning out your qualitative results chapter, create sections and subsections so that you can maintain the flow of the story you’re trying to tell. Be sure to systematically and consistently describe each portion of results. Try to adopt a standardised structure for each portion so that you achieve a high level of consistency throughout the chapter.

For qualitative studies, results chapters tend to be structured according to themes , which makes it easier for readers to follow. However, keep in mind that not all results chapters have to be structured in this manner. For example, if you’re conducting a longitudinal study, you may want to structure your chapter chronologically. Similarly, you might structure this chapter based on your theoretical framework . The exact structure of your chapter will depend on the nature of your study , especially your research questions.

As you work through the body of your chapter, make sure that you use quotes to substantiate every one of your claims . You can present these quotes in italics to differentiate them from your own words. A general rule of thumb is to use at least two pieces of evidence per claim, and these should be linked directly to your data. Also, remember that you need to include all relevant results , not just the ones that support your assumptions or initial leanings.

In addition to including quotes, you can also link your claims to the data by using appendices , which you should reference throughout your text. When you reference, make sure that you include both the name/number of the appendix , as well as the line(s) from which you drew your data.

As referencing styles can vary greatly, be sure to look up the appendix referencing conventions of your university’s prescribed style (e.g. APA , Harvard, etc) and keep this consistent throughout your chapter.

Section 3: Concluding summary

The concluding summary is very important because it summarises your key findings and lays the foundation for the discussion chapter . Keep in mind that some readers may skip directly to this section (from the introduction section), so make sure that it can be read and understood well in isolation.

In this section, you need to remind the reader of the key findings. That is, the results that directly relate to your research questions and that you will build upon in your discussion chapter. Remember, your reader has digested a lot of information in this chapter, so you need to use this section to remind them of the most important takeaways.

Importantly, the concluding summary should not present any new information and should only describe what you’ve already presented in your chapter. Keep it concise – you’re not summarising the whole chapter, just the essentials.

Tips for writing an A-grade results chapter

Now that you’ve got a clear picture of what the qualitative results chapter is all about, here are some quick tips and reminders to help you craft a high-quality chapter:

• Your results chapter should be written in the past tense . You’ve done the work already, so you want to tell the reader what you found , not what you are currently finding .
• Make sure that you review your work multiple times and check that every claim is adequately backed up by evidence . Aim for at least two examples per claim, and make use of an appendix to reference these.
• When writing up your results, make sure that you stick to only what is relevant . Don’t waste time on data that are not relevant to your research objectives and research questions.
• Use headings and subheadings to create an intuitive, easy to follow piece of writing. Make use of Microsoft Word’s “heading styles” and be sure to use them consistently.
• When referring to numerical data, tables and figures can provide a useful visual aid. When using these, make sure that they can be read and understood independent of your body text (i.e. that they can stand-alone). To this end, use clear, concise labels for each of your tables or figures and make use of colours to code indicate differences or hierarchy.
• Similarly, when you’re writing up your chapter, it can be useful to highlight topics and themes in different colours . This can help you to differentiate between your data if you get a bit overwhelmed and will also help you to ensure that your results flow logically and coherently.

If you have any questions, leave a comment below and we’ll do our best to help. If you’d like 1-on-1 help with your results chapter (or any chapter of your dissertation or thesis), check out our private dissertation coaching service here or book a free initial consultation to discuss how we can help you.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

22 Comments

This was extremely helpful. Thanks a lot guys

Hi, thanks for the great research support platform created by the gradcoach team!

I wanted to ask- While “suggests” or “implies” are interpretive terms, what terms could we use for the results chapter? Could you share some examples of descriptive terms?

I think that instead of saying, ‘The data suggested, or The data implied,’ you can say, ‘The Data showed or revealed, or illustrated or outlined’…If interview data, you may say Jane Doe illuminated or elaborated, or Jane Doe described… or Jane Doe expressed or stated.

I found this article very useful. Thank you very much for the outstanding work you are doing.

What if i have 3 different interviewees answering the same interview questions? Should i then present the results in form of the table with the division on the 3 perspectives or rather give a results in form of the text and highlight who said what?

I think this tabular representation of results is a great idea. I am doing it too along with the text. Thanks

That was helpful was struggling to separate the discussion from the findings

this was very useful, Thank you.

Very helpful, I am confident to write my results chapter now.

It is so helpful! It is a good job. Thank you very much!

Very useful, well explained. Many thanks.

Hello, I appreciate the way you provided a supportive comments about qualitative results presenting tips

I loved this! It explains everything needed, and it has helped me better organize my thoughts. What words should I not use while writing my results section, other than subjective ones.

Thanks a lot, it is really helpful

Thank you so much dear, i really appropriate your nice explanations about this.

Thank you so much for this! I was wondering if anyone could help with how to prproperly integrate quotations (Excerpts) from interviews in the finding chapter in a qualitative research. Please GradCoach, address this issue and provide examples.

what if I’m not doing any interviews myself and all the information is coming from case studies that have already done the research.

Very helpful thank you.

This was very helpful as I was wondering how to structure this part of my dissertation, to include the quotes… Thanks for this explanation

This is very helpful, thanks! I am required to write up my results chapters with the discussion in each of them – any tips and tricks for this strategy?

For qualitative studies, can the findings be structured according to the Research questions? Thank you.

Do I need to include literature/references in my findings chapter?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Qualitative Data Analysis

23 Presenting the Results of Qualitative Analysis

Mikaila Mariel Lemonik Arthur

Qualitative research is not finished just because you have determined the main findings or conclusions of your study. Indeed, disseminating the results is an essential part of the research process. By sharing your results with others, whether in written form as scholarly paper or an applied report or in some alternative format like an oral presentation, an infographic, or a video, you ensure that your findings become part of the ongoing conversation of scholarship in your field, forming part of the foundation for future researchers. This chapter provides an introduction to writing about qualitative research findings. It will outline how writing continues to contribute to the analysis process, what concerns researchers should keep in mind as they draft their presentations of findings, and how best to organize qualitative research writing

As you move through the research process, it is essential to keep yourself organized. Organizing your data, memos, and notes aids both the analytical and the writing processes. Whether you use electronic or physical, real-world filing and organizational systems, these systems help make sense of the mountains of data you have and assure you focus your attention on the themes and ideas you have determined are important (Warren and Karner 2015). Be sure that you have kept detailed notes on all of the decisions you have made and procedures you have followed in carrying out research design, data collection, and analysis, as these will guide your ultimate write-up.

First and foremost, researchers should keep in mind that writing is in fact a form of thinking. Writing is an excellent way to discover ideas and arguments and to further develop an analysis. As you write, more ideas will occur to you, things that were previously confusing will start to make sense, and arguments will take a clear shape rather than being amorphous and poorly-organized. However, writing-as-thinking cannot be the final version that you share with others. Good-quality writing does not display the workings of your thought process. It is reorganized and revised (more on that later) to present the data and arguments important in a particular piece. And revision is totally normal! No one expects the first draft of a piece of writing to be ready for prime time. So write rough drafts and memos and notes to yourself and use them to think, and then revise them until the piece is the way you want it to be for sharing.

Bergin (2018) lays out a set of key concerns for appropriate writing about research. First, present your results accurately, without exaggerating or misrepresenting. It is very easy to overstate your findings by accident if you are enthusiastic about what you have found, so it is important to take care and use appropriate cautions about the limitations of the research. You also need to work to ensure that you communicate your findings in a way people can understand, using clear and appropriate language that is adjusted to the level of those you are communicating with. And you must be clear and transparent about the methodological strategies employed in the research. Remember, the goal is, as much as possible, to describe your research in a way that would permit others to replicate the study. There are a variety of other concerns and decision points that qualitative researchers must keep in mind, including the extent to which to include quantification in their presentation of results, ethics, considerations of audience and voice, and how to bring the richness of qualitative data to life.

Quantification, as you have learned, refers to the process of turning data into numbers. It can indeed be very useful to count and tabulate quantitative data drawn from qualitative research. For instance, if you were doing a study of dual-earner households and wanted to know how many had an equal division of household labor and how many did not, you might want to count those numbers up and include them as part of the final write-up. However, researchers need to take care when they are writing about quantified qualitative data. Qualitative data is not as generalizable as quantitative data, so quantification can be very misleading. Thus, qualitative researchers should strive to use raw numbers instead of the percentages that are more appropriate for quantitative research. Writing, for instance, “15 of the 20 people I interviewed prefer pancakes to waffles” is a simple description of the data; writing “75% of people prefer pancakes” suggests a generalizable claim that is not likely supported by the data. Note that mixing numbers with qualitative data is really a type of mixed-methods approach. Mixed-methods approaches are good, but sometimes they seduce researchers into focusing on the persuasive power of numbers and tables rather than capitalizing on the inherent richness of their qualitative data.

A variety of issues of scholarly ethics and research integrity are raised by the writing process. Some of these are unique to qualitative research, while others are more universal concerns for all academic and professional writing. For example, it is essential to avoid plagiarism and misuse of sources. All quotations that appear in a text must be properly cited, whether with in-text and bibliographic citations to the source or with an attribution to the research participant (or the participant’s pseudonym or description in order to protect confidentiality) who said those words. Where writers will paraphrase a text or a participant’s words, they need to make sure that the paraphrase they develop accurately reflects the meaning of the original words. Thus, some scholars suggest that participants should have the opportunity to read (or to have read to them, if they cannot read the text themselves) all sections of the text in which they, their words, or their ideas are presented to ensure accuracy and enable participants to maintain control over their lives.

Audience and Voice

When writing, researchers must consider their audience(s) and the effects they want their writing to have on these audiences. The designated audience will dictate the voice used in the writing, or the individual style and personality of a piece of text. Keep in mind that the potential audience for qualitative research is often much more diverse than that for quantitative research because of the accessibility of the data and the extent to which the writing can be accessible and interesting. Yet individual pieces of writing are typically pitched to a more specific subset of the audience.

Let us consider one potential research study, an ethnography involving participant-observation of the same children both when they are at daycare facility and when they are at home with their families to try to understand how daycare might impact behavior and social development. The findings of this study might be of interest to a wide variety of potential audiences: academic peers, whether at your own academic institution, in your broader discipline, or multidisciplinary; people responsible for creating laws and policies; practitioners who run or teach at day care centers; and the general public, including both people who are interested in child development more generally and those who are themselves parents making decisions about child care for their own children. And the way you write for each of these audiences will be somewhat different. Take a moment and think through what some of these differences might look like.

If you are writing to academic audiences, using specialized academic language and working within the typical constraints of scholarly genres, as will be discussed below, can be an important part of convincing others that your work is legitimate and should be taken seriously. Your writing will be formal. Even if you are writing for students and faculty you already know—your classmates, for instance—you are often asked to imitate the style of academic writing that is used in publications, as this is part of learning to become part of the scholarly conversation. When speaking to academic audiences outside your discipline, you may need to be more careful about jargon and specialized language, as disciplines do not always share the same key terms. For instance, in sociology, scholars use the term diffusion to refer to the way new ideas or practices spread from organization to organization. In the field of international relations, scholars often used the term cascade to refer to the way ideas or practices spread from nation to nation. These terms are describing what is fundamentally the same concept, but they are different terms—and a scholar from one field might have no idea what a scholar from a different field is talking about! Therefore, while the formality and academic structure of the text would stay the same, a writer with a multidisciplinary audience might need to pay more attention to defining their terms in the body of the text.

It is not only other academic scholars who expect to see formal writing. Policymakers tend to expect formality when ideas are presented to them, as well. However, the content and style of the writing will be different. Much less academic jargon should be used, and the most important findings and policy implications should be emphasized right from the start rather than initially focusing on prior literature and theoretical models as you might for an academic audience. Long discussions of research methods should also be minimized. Similarly, when you write for practitioners, the findings and implications for practice should be highlighted. The reading level of the text will vary depending on the typical background of the practitioners to whom you are writing—you can make very different assumptions about the general knowledge and reading abilities of a group of hospital medical directors with MDs than you can about a group of case workers who have a post-high-school certificate. Consider the primary language of your audience as well. The fact that someone can get by in spoken English does not mean they have the vocabulary or English reading skills to digest a complex report. But the fact that someone’s vocabulary is limited says little about their intellectual abilities, so try your best to convey the important complexity of the ideas and findings from your research without dumbing them down—even if you must limit your vocabulary usage.

When writing for the general public, you will want to move even further towards emphasizing key findings and policy implications, but you also want to draw on the most interesting aspects of your data. General readers will read sociological texts that are rich with ethnographic or other kinds of detail—it is almost like reality television on a page! And this is a contrast to busy policymakers and practitioners, who probably want to learn the main findings as quickly as possible so they can go about their busy lives. But also keep in mind that there is a wide variation in reading levels. Journalists at publications pegged to the general public are often advised to write at about a tenth-grade reading level, which would leave most of the specialized terminology we develop in our research fields out of reach. If you want to be accessible to even more people, your vocabulary must be even more limited. The excellent exercise of trying to write using the 1,000 most common English words, available at the Up-Goer Five website ( https://www.splasho.com/upgoer5/ ) does a good job of illustrating this challenge (Sanderson n.d.).

Another element of voice is whether to write in the first person. While many students are instructed to avoid the use of the first person in academic writing, this advice needs to be taken with a grain of salt. There are indeed many contexts in which the first person is best avoided, at least as long as writers can find ways to build strong, comprehensible sentences without its use, including most quantitative research writing. However, if the alternative to using the first person is crafting a sentence like “it is proposed that the researcher will conduct interviews,” it is preferable to write “I propose to conduct interviews.” In qualitative research, in fact, the use of the first person is far more common. This is because the researcher is central to the research project. Qualitative researchers can themselves be understood as research instruments, and thus eliminating the use of the first person in writing is in a sense eliminating information about the conduct of the researchers themselves.

But the question really extends beyond the issue of first-person or third-person. Qualitative researchers have choices about how and whether to foreground themselves in their writing, not just in terms of using the first person, but also in terms of whether to emphasize their own subjectivity and reflexivity, their impressions and ideas, and their role in the setting. In contrast, conventional quantitative research in the positivist tradition really tries to eliminate the author from the study—which indeed is exactly why typical quantitative research avoids the use of the first person. Keep in mind that emphasizing researchers’ roles and reflexivity and using the first person does not mean crafting articles that provide overwhelming detail about the author’s thoughts and practices. Readers do not need to hear, and should not be told, which database you used to search for journal articles, how many hours you spent transcribing, or whether the research process was stressful—save these things for the memos you write to yourself. Rather, readers need to hear how you interacted with research participants, how your standpoint may have shaped the findings, and what analytical procedures you carried out.

Making Data Come Alive

One of the most important parts of writing about qualitative research is presenting the data in a way that makes its richness and value accessible to readers. As the discussion of analysis in the prior chapter suggests, there are a variety of ways to do this. Researchers may select key quotes or images to illustrate points, write up specific case studies that exemplify their argument, or develop vignettes (little stories) that illustrate ideas and themes, all drawing directly on the research data. Researchers can also write more lengthy summaries, narratives, and thick descriptions.

Nearly all qualitative work includes quotes from research participants or documents to some extent, though ethnographic work may focus more on thick description than on relaying participants’ own words. When quotes are presented, they must be explained and interpreted—they cannot stand on their own. This is one of the ways in which qualitative research can be distinguished from journalism. Journalism presents what happened, but social science needs to present the “why,” and the why is best explained by the researcher.

So how do authors go about integrating quotes into their written work? Julie Posselt (2017), a sociologist who studies graduate education, provides a set of instructions. First of all, authors need to remain focused on the core questions of their research, and avoid getting distracted by quotes that are interesting or attention-grabbing but not so relevant to the research question. Selecting the right quotes, those that illustrate the ideas and arguments of the paper, is an important part of the writing process. Second, not all quotes should be the same length (just like not all sentences or paragraphs in a paper should be the same length). Include some quotes that are just phrases, others that are a sentence or so, and others that are longer. We call longer quotes, generally those more than about three lines long, block quotes , and they are typically indented on both sides to set them off from the surrounding text. For all quotes, be sure to summarize what the quote should be telling or showing the reader, connect this quote to other quotes that are similar or different, and provide transitions in the discussion to move from quote to quote and from topic to topic. Especially for longer quotes, it is helpful to do some of this writing before the quote to preview what is coming and other writing after the quote to make clear what readers should have come to understand. Remember, it is always the author’s job to interpret the data. Presenting excerpts of the data, like quotes, in a form the reader can access does not minimize the importance of this job. Be sure that you are explaining the meaning of the data you present.

A few more notes about writing with quotes: avoid patchwriting, whether in your literature review or the section of your paper in which quotes from respondents are presented. Patchwriting is a writing practice wherein the author lightly paraphrases original texts but stays so close to those texts that there is little the author has added. Sometimes, this even takes the form of presenting a series of quotes, properly documented, with nothing much in the way of text generated by the author. A patchwriting approach does not build the scholarly conversation forward, as it does not represent any kind of new contribution on the part of the author. It is of course fine to paraphrase quotes, as long as the meaning is not changed. But if you use direct quotes, do not edit the text of the quotes unless how you edit them does not change the meaning and you have made clear through the use of ellipses (…) and brackets ([])what kinds of edits have been made. For example, consider this exchange from Matthew Desmond’s (2012:1317) research on evictions:

The thing was, I wasn’t never gonna let Crystal come and stay with me from the get go. I just told her that to throw her off. And she wasn’t fittin’ to come stay with me with no money…No. Nope. You might as well stay in that shelter.

A paraphrase of this exchange might read “She said that she was going to let Crystal stay with her if Crystal did not have any money.” Paraphrases like that are fine. What is not fine is rewording the statement but treating it like a quote, for instance writing:

The thing was, I was not going to let Crystal come and stay with me from beginning. I just told her that to throw her off. And it was not proper for her to come stay with me without any money…No. Nope. You might as well stay in that shelter.

But as you can see, the change in language and style removes some of the distinct meaning of the original quote. Instead, writers should leave as much of the original language as possible. If some text in the middle of the quote needs to be removed, as in this example, ellipses are used to show that this has occurred. And if a word needs to be added to clarify, it is placed in square brackets to show that it was not part of the original quote.

Data can also be presented through the use of data displays like tables, charts, graphs, diagrams, and infographics created for publication or presentation, as well as through the use of visual material collected during the research process. Note that if visuals are used, the author must have the legal right to use them. Photographs or diagrams created by the author themselves—or by research participants who have signed consent forms for their work to be used, are fine. But photographs, and sometimes even excerpts from archival documents, may be owned by others from whom researchers must get permission in order to use them.

A large percentage of qualitative research does not include any data displays or visualizations. Therefore, researchers should carefully consider whether the use of data displays will help the reader understand the data. One of the most common types of data displays used by qualitative researchers are simple tables. These might include tables summarizing key data about cases included in the study; tables laying out the characteristics of different taxonomic elements or types developed as part of the analysis; tables counting the incidence of various elements; and 2×2 tables (two columns and two rows) illuminating a theory. Basic network or process diagrams are also commonly included. If data displays are used, it is essential that researchers include context and analysis alongside data displays rather than letting them stand by themselves, and it is preferable to continue to present excerpts and examples from the data rather than just relying on summaries in the tables.

If you will be using graphs, infographics, or other data visualizations, it is important that you attend to making them useful and accurate (Bergin 2018). Think about the viewer or user as your audience and ensure the data visualizations will be comprehensible. You may need to include more detail or labels than you might think. Ensure that data visualizations are laid out and labeled clearly and that you make visual choices that enhance viewers’ ability to understand the points you intend to communicate using the visual in question. Finally, given the ease with which it is possible to design visuals that are deceptive or misleading, it is essential to make ethical and responsible choices in the construction of visualization so that viewers will interpret them in accurate ways.

The Genre of Research Writing

As discussed above, the style and format in which results are presented depends on the audience they are intended for. These differences in styles and format are part of the genre of writing. Genre is a term referring to the rules of a specific form of creative or productive work. Thus, the academic journal article—and student papers based on this form—is one genre. A report or policy paper is another. The discussion below will focus on the academic journal article, but note that reports and policy papers follow somewhat different formats. They might begin with an executive summary of one or a few pages, include minimal background, focus on key findings, and conclude with policy implications, shifting methods and details about the data to an appendix. But both academic journal articles and policy papers share some things in common, for instance the necessity for clear writing, a well-organized structure, and the use of headings.

So what factors make up the genre of the academic journal article in sociology? While there is some flexibility, particularly for ethnographic work, academic journal articles tend to follow a fairly standard format. They begin with a “title page” that includes the article title (often witty and involving scholarly inside jokes, but more importantly clearly describing the content of the article); the authors’ names and institutional affiliations, an abstract , and sometimes keywords designed to help others find the article in databases. An abstract is a short summary of the article that appears both at the very beginning of the article and in search databases. Abstracts are designed to aid readers by giving them the opportunity to learn enough about an article that they can determine whether it is worth their time to read the complete text. They are written about the article, and thus not in the first person, and clearly summarize the research question, methodological approach, main findings, and often the implications of the research.

After the abstract comes an “introduction” of a page or two that details the research question, why it matters, and what approach the paper will take. This is followed by a literature review of about a quarter to a third the length of the entire paper. The literature review is often divided, with headings, into topical subsections, and is designed to provide a clear, thorough overview of the prior research literature on which a paper has built—including prior literature the new paper contradicts. At the end of the literature review it should be made clear what researchers know about the research topic and question, what they do not know, and what this new paper aims to do to address what is not known.

The next major section of the paper is the section that describes research design, data collection, and data analysis, often referred to as “research methods” or “methodology.” This section is an essential part of any written or oral presentation of your research. Here, you tell your readers or listeners “how you collected and interpreted your data” (Taylor, Bogdan, and DeVault 2016:215). Taylor, Bogdan, and DeVault suggest that the discussion of your research methods include the following:

• The particular approach to data collection used in the study;
• Any theoretical perspective(s) that shaped your data collection and analytical approach;
• When the study occurred, over how long, and where (concealing identifiable details as needed);
• A description of the setting and participants, including sampling and selection criteria (if an interview-based study, the number of participants should be clearly stated);
• The researcher’s perspective in carrying out the study, including relevant elements of their identity and standpoint, as well as their role (if any) in research settings; and
• The approach to analyzing the data.

After the methods section comes a section, variously titled but often called “data,” that takes readers through the analysis. This section is where the thick description narrative; the quotes, broken up by theme or topic, with their interpretation; the discussions of case studies; most data displays (other than perhaps those outlining a theoretical model or summarizing descriptive data about cases); and other similar material appears. The idea of the data section is to give readers the ability to see the data for themselves and to understand how this data supports the ultimate conclusions. Note that all tables and figures included in formal publications should be titled and numbered.

At the end of the paper come one or two summary sections, often called “discussion” and/or “conclusion.” If there is a separate discussion section, it will focus on exploring the overall themes and findings of the paper. The conclusion clearly and succinctly summarizes the findings and conclusions of the paper, the limitations of the research and analysis, any suggestions for future research building on the paper or addressing these limitations, and implications, be they for scholarship and theory or policy and practice.

After the end of the textual material in the paper comes the bibliography, typically called “works cited” or “references.” The references should appear in a consistent citation style—in sociology, we often use the American Sociological Association format (American Sociological Association 2019), but other formats may be used depending on where the piece will eventually be published. Care should be taken to ensure that in-text citations also reflect the chosen citation style. In some papers, there may be an appendix containing supplemental information such as a list of interview questions or an additional data visualization.

Note that when researchers give presentations to scholarly audiences, the presentations typically follow a format similar to that of scholarly papers, though given time limitations they are compressed. Abstracts and works cited are often not part of the presentation, though in-text citations are still used. The literature review presented will be shortened to only focus on the most important aspects of the prior literature, and only key examples from the discussion of data will be included. For long or complex papers, sometimes only one of several findings is the focus of the presentation. Of course, presentations for other audiences may be constructed differently, with greater attention to interesting elements of the data and findings as well as implications and less to the literature review and methods.

Concluding Your Work

After you have written a complete draft of the paper, be sure you take the time to revise and edit your work. There are several important strategies for revision. First, put your work away for a little while. Even waiting a day to revise is better than nothing, but it is best, if possible, to take much more time away from the text. This helps you forget what your writing looks like and makes it easier to find errors, mistakes, and omissions. Second, show your work to others. Ask them to read your work and critique it, pointing out places where the argument is weak, where you may have overlooked alternative explanations, where the writing could be improved, and what else you need to work on. Finally, read your work out loud to yourself (or, if you really need an audience, try reading to some stuffed animals). Reading out loud helps you catch wrong words, tricky sentences, and many other issues. But as important as revision is, try to avoid perfectionism in writing (Warren and Karner 2015). Writing can always be improved, no matter how much time you spend on it. Those improvements, however, have diminishing returns, and at some point the writing process needs to conclude so the writing can be shared with the world.

Of course, the main goal of writing up the results of a research project is to share with others. Thus, researchers should be considering how they intend to disseminate their results. What conferences might be appropriate? Where can the paper be submitted? Note that if you are an undergraduate student, there are a wide variety of journals that accept and publish research conducted by undergraduates. Some publish across disciplines, while others are specific to disciplines. Other work, such as reports, may be best disseminated by publication online on relevant organizational websites.

After a project is completed, be sure to take some time to organize your research materials and archive them for longer-term storage. Some Institutional Review Board (IRB) protocols require that original data, such as interview recordings, transcripts, and field notes, be preserved for a specific number of years in a protected (locked for paper or password-protected for digital) form and then destroyed, so be sure that your plans adhere to the IRB requirements. Be sure you keep any materials that might be relevant for future related research or for answering questions people may ask later about your project.

And then what? Well, then it is time to move on to your next research project. Research is a long-term endeavor, not a one-time-only activity. We build our skills and our expertise as we continue to pursue research. So keep at it.

• Find a short article that uses qualitative methods. The sociological magazine Contexts is a good place to find such pieces. Write an abstract of the article.
• Choose a sociological journal article on a topic you are interested in that uses some form of qualitative methods and is at least 20 pages long. Rewrite the article as a five-page research summary accessible to non-scholarly audiences.
• Choose a concept or idea you have learned in this course and write an explanation of it using the Up-Goer Five Text Editor ( https://www.splasho.com/upgoer5/ ), a website that restricts your writing to the 1,000 most common English words. What was this experience like? What did it teach you about communicating with people who have a more limited English-language vocabulary—and what did it teach you about the utility of having access to complex academic language?
• Select five or more sociological journal articles that all use the same basic type of qualitative methods (interviewing, ethnography, documents, or visual sociology). Using what you have learned about coding, code the methods sections of each article, and use your coding to figure out what is common in how such articles discuss their research design, data collection, and analysis methods.
• Return to an exercise you completed earlier in this course and revise your work. What did you change? How did revising impact the final product?
• Find a quote from the transcript of an interview, a social media post, or elsewhere that has not yet been interpreted or explained. Write a paragraph that includes the quote along with an explanation of its sociological meaning or significance.

The style or personality of a piece of writing, including such elements as tone, word choice, syntax, and rhythm.

A quotation, usually one of some length, which is set off from the main text by being indented on both sides rather than being placed in quotation marks.

A classification of written or artistic work based on form, content, and style.

A short summary of a text written from the perspective of a reader rather than from the perspective of an author.

Social Data Analysis Copyright © 2021 by Mikaila Mariel Lemonik Arthur is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

• Dissertation Coaching
• Qualitative Data Analysis
• Statistical Consulting
• Dissertation Editing
• On-Demand Courses

How To Write the Findings Chapter for Qualitative Studies

Writing the findings chapter for qualitative studies is a critical step in the research process. This chapter allows researchers to present their findings, analyze the data collected, and draw conclusions based on the study’s objectives. In this blog post, I’lll explore the purpose, key elements, preparation, writing, and presentation of the findings chapter in qualitative studies.

Introduction: Contextualizing Your Findings

The introductory section serves as the gateway to your qualitative findings chapter. Begin by reiterating your problem statement and research questions, setting the stage for the data that will be presented. Highlighting the purpose of your research is crucial at this point to give context to the reader.

Example: The primary aim of this case study is to explore how educators perceive the integration of artificial intelligence (AI) tools in K-12 education settings. This research seeks to understand the perceptions and experiences of educators who have implemented AI technologies in their classrooms, framed within the context of Technological Pedagogical Content Knowledge (TPACK). In this chapter, data are systematically organized into three overarching themes, each comprising sub-themes that provide a nuanced understanding of the participants’ perspectives. The research questions guiding this investigation were: (1) What are educators’ perceptions of the role of AI tools in K-12 classrooms? (2) How do educators navigate the challenges of integrating AI tools into their teaching practices? The answers to these questions are integral to understanding the complex dynamics of AI implementation in educational contexts.  

Overview of Findings

Following this, offer a brief overview of your main findings. While you should not delve into the details here, giving the reader an idea of what to expect can be helpful. Explain the overall structure of your results chapter and how you’ve organized it to maintain coherence and logical flow.

The Heart of the Matter: The Body of Your Chapter

Presentation.

The body of the chapter is where you lay out your data for the reader. In qualitative research, this usually means dividing your data into themes or categories, which should be clearly described and substantiated with quotes and examples from your dataset. These themes provide the skeletal framework upon which your narrative is built.

Structure and Flow

When planning your qualitative findings chapter, carefully outline the sections and subsections to maintain the flow of the writing and improve readability. You can structure your chapter based on themes, which is often the case in qualitative research, but other formats like chronological or framework-based structures may be more appropriate depending on your specific research design.

✅ Consistency is Key: Make sure each portion of findings adheres to a standardized structure. This enhances consistency and enables the reader to follow your line of reasoning.

Objective and Descriptive Language

While your narrative might touch upon individual experiences and perspectives, remember to maintain an objective tone. Your task is to describe, not interpret—that comes later, in the Discussion chapter. Thus, avoid phrases that suggest interpretation, such as “suggests” or “implies.”

Visual Aids

Tables, figures, and other visual aids can add another layer of comprehension and break up the text, but make sure they can be understood independently of your body text. Label them clearly and use color coding judiciously to indicate differences or hierarchy.

Data Analysis and Interpretation

As you delve into the data, aim to narrate a coherent story. Interpret your findings in light of the literature in the field and your theoretical framework, but remember to clearly differentiate between your descriptions and your interpretations.

✅ References and Appendices: When using quotes or data excerpts, reference them appropriately. Use appendices to present additional data and ensure that you cite them according to the referencing style prescribed by your institution (e.g., APA, Harvard).

Bringing It All Together: The Concluding Summary

This is the section where you summarize your key findings in a concise manner, reiterating points that directly relate to your research questions. It serves as a stepping stone to the Discussion chapter, providing the reader with the essential takeaways. As a rule of thumb, this section should contain no new information.

Additional Tips and Tricks

– Write in the past tense, as you present findings that have already been gathered.

– Review your work multiple times, ensuring each theme or finding is backed by sufficient data.

– Use Microsoft Word’s “heading styles” for consistency.

Final Thoughts

With the right approach, writing the Findings chapter can be an enriching experience that showcases your research and prepares you for discussions and conclusions that follow. The tips and guidelines presented here are meant to make this crucial chapter as clear and impactful as possible, helping you make a valuable contribution to your field of study.

Need Further Assistance?

Dissertation by Design is here to support you every step of the way. Our team of experienced academic coaches and methodologists are dedicated to helping you confidently navigate through each stage of your dissertation journey.  Book a free, 30-minute consult today.

Happy writing!

Author:  Jessica Parker, EdD

Related posts.

Download our free guide on how to overcome the top 10 challenges common to doctoral candidates and graduate sooner.

Thank You 🙌

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

Jump to navigation

Cochrane Training

Chapter 15: interpreting results and drawing conclusions.

Holger J Schünemann, Gunn E Vist, Julian PT Higgins, Nancy Santesso, Jonathan J Deeks, Paul Glasziou, Elie A Akl, Gordon H Guyatt; on behalf of the Cochrane GRADEing Methods Group

Key Points:

• This chapter provides guidance on interpreting the results of synthesis in order to communicate the conclusions of the review effectively.
• Methods are presented for computing, presenting and interpreting relative and absolute effects for dichotomous outcome data, including the number needed to treat (NNT).
• For continuous outcome measures, review authors can present summary results for studies using natural units of measurement or as minimal important differences when all studies use the same scale. When studies measure the same construct but with different scales, review authors will need to find a way to interpret the standardized mean difference, or to use an alternative effect measure for the meta-analysis such as the ratio of means.
• Review authors should not describe results as ‘statistically significant’, ‘not statistically significant’ or ‘non-significant’ or unduly rely on thresholds for P values, but report the confidence interval together with the exact P value.
• Review authors should not make recommendations about healthcare decisions, but they can – after describing the certainty of evidence and the balance of benefits and harms – highlight different actions that might be consistent with particular patterns of values and preferences and other factors that determine a decision such as cost.

Cite this chapter as: Schünemann HJ, Vist GE, Higgins JPT, Santesso N, Deeks JJ, Glasziou P, Akl EA, Guyatt GH. Chapter 15: Interpreting results and drawing conclusions. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

15.1 Introduction

The purpose of Cochrane Reviews is to facilitate healthcare decisions by patients and the general public, clinicians, guideline developers, administrators and policy makers. They also inform future research. A clear statement of findings, a considered discussion and a clear presentation of the authors’ conclusions are, therefore, important parts of the review. In particular, the following issues can help people make better informed decisions and increase the usability of Cochrane Reviews:

• information on all important outcomes, including adverse outcomes;
• the certainty of the evidence for each of these outcomes, as it applies to specific populations and specific interventions; and
• clarification of the manner in which particular values and preferences may bear on the desirable and undesirable consequences of the intervention.

A ‘Summary of findings’ table, described in Chapter 14 , Section 14.1 , provides key pieces of information about health benefits and harms in a quick and accessible format. It is highly desirable that review authors include a ‘Summary of findings’ table in Cochrane Reviews alongside a sufficient description of the studies and meta-analyses to support its contents. This description includes the rating of the certainty of evidence, also called the quality of the evidence or confidence in the estimates of the effects, which is expected in all Cochrane Reviews.

‘Summary of findings’ tables are usually supported by full evidence profiles which include the detailed ratings of the evidence (Guyatt et al 2011a, Guyatt et al 2013a, Guyatt et al 2013b, Santesso et al 2016). The Discussion section of the text of the review provides space to reflect and consider the implications of these aspects of the review’s findings. Cochrane Reviews include five standard subheadings to ensure the Discussion section places the review in an appropriate context: ‘Summary of main results (benefits and harms)’; ‘Potential biases in the review process’; ‘Overall completeness and applicability of evidence’; ‘Certainty of the evidence’; and ‘Agreements and disagreements with other studies or reviews’. Following the Discussion, the Authors’ conclusions section is divided into two standard subsections: ‘Implications for practice’ and ‘Implications for research’. The assessment of the certainty of evidence facilitates a structured description of the implications for practice and research.

Because Cochrane Reviews have an international audience, the Discussion and Authors’ conclusions should, so far as possible, assume a broad international perspective and provide guidance for how the results could be applied in different settings, rather than being restricted to specific national or local circumstances. Cultural differences and economic differences may both play an important role in determining the best course of action based on the results of a Cochrane Review. Furthermore, individuals within societies have widely varying values and preferences regarding health states, and use of societal resources to achieve particular health states. For all these reasons, and because information that goes beyond that included in a Cochrane Review is required to make fully informed decisions, different people will often make different decisions based on the same evidence presented in a review.

Thus, review authors should avoid specific recommendations that inevitably depend on assumptions about available resources, values and preferences, and other factors such as equity considerations, feasibility and acceptability of an intervention. The purpose of the review should be to present information and aid interpretation rather than to offer recommendations. The discussion and conclusions should help people understand the implications of the evidence in relation to practical decisions and apply the results to their specific situation. Review authors can aid this understanding of the implications by laying out different scenarios that describe certain value structures.

In this chapter, we address first one of the key aspects of interpreting findings that is also fundamental in completing a ‘Summary of findings’ table: the certainty of evidence related to each of the outcomes. We then provide a more detailed consideration of issues around applicability and around interpretation of numerical results, and provide suggestions for presenting authors’ conclusions.

15.2 Issues of indirectness and applicability

15.2.1 the role of the review author.

“A leap of faith is always required when applying any study findings to the population at large” or to a specific person. “In making that jump, one must always strike a balance between making justifiable broad generalizations and being too conservative in one’s conclusions” (Friedman et al 1985). In addition to issues about risk of bias and other domains determining the certainty of evidence, this leap of faith is related to how well the identified body of evidence matches the posed PICO ( Population, Intervention, Comparator(s) and Outcome ) question. As to the population, no individual can be entirely matched to the population included in research studies. At the time of decision, there will always be differences between the study population and the person or population to whom the evidence is applied; sometimes these differences are slight, sometimes large.

The terms applicability, generalizability, external validity and transferability are related, sometimes used interchangeably and have in common that they lack a clear and consistent definition in the classic epidemiological literature (Schünemann et al 2013). However, all of the terms describe one overarching theme: whether or not available research evidence can be directly used to answer the health and healthcare question at hand, ideally supported by a judgement about the degree of confidence in this use (Schünemann et al 2013). GRADE’s certainty domains include a judgement about ‘indirectness’ to describe all of these aspects including the concept of direct versus indirect comparisons of different interventions (Atkins et al 2004, Guyatt et al 2008, Guyatt et al 2011b).

To address adequately the extent to which a review is relevant for the purpose to which it is being put, there are certain things the review author must do, and certain things the user of the review must do to assess the degree of indirectness. Cochrane and the GRADE Working Group suggest using a very structured framework to address indirectness. We discuss here and in Chapter 14 what the review author can do to help the user. Cochrane Review authors must be extremely clear on the population, intervention and outcomes that they intend to address. Chapter 14, Section 14.1.2 , also emphasizes a crucial step: the specification of all patient-important outcomes relevant to the intervention strategies under comparison.

In considering whether the effect of an intervention applies equally to all participants, and whether different variations on the intervention have similar effects, review authors need to make a priori hypotheses about possible effect modifiers, and then examine those hypotheses (see Chapter 10, Section 10.10 and Section 10.11 ). If they find apparent subgroup effects, they must ultimately decide whether or not these effects are credible (Sun et al 2012). Differences between subgroups, particularly those that correspond to differences between studies, should be interpreted cautiously. Some chance variation between subgroups is inevitable so, unless there is good reason to believe that there is an interaction, review authors should not assume that the subgroup effect exists. If, despite due caution, review authors judge subgroup effects in terms of relative effect estimates as credible (i.e. the effects differ credibly), they should conduct separate meta-analyses for the relevant subgroups, and produce separate ‘Summary of findings’ tables for those subgroups.

The user of the review will be challenged with ‘individualization’ of the findings, whether they seek to apply the findings to an individual patient or a policy decision in a specific context. For example, even if relative effects are similar across subgroups, absolute effects will differ according to baseline risk. Review authors can help provide this information by identifying identifiable groups of people with varying baseline risks in the ‘Summary of findings’ tables, as discussed in Chapter 14, Section 14.1.3 . Users can then identify their specific case or population as belonging to a particular risk group, if relevant, and assess their likely magnitude of benefit or harm accordingly. A description of the identifying prognostic or baseline risk factors in a brief scenario (e.g. age or gender) will help users of a review further.

Another decision users must make is whether their individual case or population of interest is so different from those included in the studies that they cannot use the results of the systematic review and meta-analysis at all. Rather than rigidly applying the inclusion and exclusion criteria of studies, it is better to ask whether or not there are compelling reasons why the evidence should not be applied to a particular patient. Review authors can sometimes help decision makers by identifying important variation where divergence might limit the applicability of results (Rothwell 2005, Schünemann et al 2006, Guyatt et al 2011b, Schünemann et al 2013), including biologic and cultural variation, and variation in adherence to an intervention.

In addressing these issues, review authors cannot be aware of, or address, the myriad of differences in circumstances around the world. They can, however, address differences of known importance to many people and, importantly, they should avoid assuming that other people’s circumstances are the same as their own in discussing the results and drawing conclusions.

15.2.2 Biological variation

Issues of biological variation that may affect the applicability of a result to a reader or population include divergence in pathophysiology (e.g. biological differences between women and men that may affect responsiveness to an intervention) and divergence in a causative agent (e.g. for infectious diseases such as malaria, which may be caused by several different parasites). The discussion of the results in the review should make clear whether the included studies addressed all or only some of these groups, and whether any important subgroup effects were found.

15.2.3 Variation in context

Some interventions, particularly non-pharmacological interventions, may work in some contexts but not in others; the situation has been described as program by context interaction (Hawe et al 2004). Contextual factors might pertain to the host organization in which an intervention is offered, such as the expertise, experience and morale of the staff expected to carry out the intervention, the competing priorities for the clinician’s or staff’s attention, the local resources such as service and facilities made available to the program and the status or importance given to the program by the host organization. Broader context issues might include aspects of the system within which the host organization operates, such as the fee or payment structure for healthcare providers and the local insurance system. Some interventions, in particular complex interventions (see Chapter 17 ), can be only partially implemented in some contexts, and this requires judgements about indirectness of the intervention and its components for readers in that context (Schünemann 2013).

Contextual factors may also pertain to the characteristics of the target group or population, such as cultural and linguistic diversity, socio-economic position, rural/urban setting. These factors may mean that a particular style of care or relationship evolves between service providers and consumers that may or may not match the values and technology of the program.

For many years these aspects have been acknowledged when decision makers have argued that results of evidence reviews from other countries do not apply in their own country or setting. Whilst some programmes/interventions have been successfully transferred from one context to another, others have not (Resnicow et al 1993, Lumley et al 2004, Coleman et al 2015). Review authors should be cautious when making generalizations from one context to another. They should report on the presence (or otherwise) of context-related information in intervention studies, where this information is available.

15.2.4 Variation in adherence

Variation in the adherence of the recipients and providers of care can limit the certainty in the applicability of results. Predictable differences in adherence can be due to divergence in how recipients of care perceive the intervention (e.g. the importance of side effects), economic conditions or attitudes that make some forms of care inaccessible in some settings, such as in low-income countries (Dans et al 2007). It should not be assumed that high levels of adherence in closely monitored randomized trials will translate into similar levels of adherence in normal practice.

15.2.5 Variation in values and preferences

Decisions about healthcare management strategies and options involve trading off health benefits and harms. The right choice may differ for people with different values and preferences (i.e. the importance people place on the outcomes and interventions), and it is important that decision makers ensure that decisions are consistent with a patient or population’s values and preferences. The importance placed on outcomes, together with other factors, will influence whether the recipients of care will or will not accept an option that is offered (Alonso-Coello et al 2016) and, thus, can be one factor influencing adherence. In Section 15.6 , we describe how the review author can help this process and the limits of supporting decision making based on intervention reviews.

15.3 Interpreting results of statistical analyses

15.3.1 confidence intervals.

Results for both individual studies and meta-analyses are reported with a point estimate together with an associated confidence interval. For example, ‘The odds ratio was 0.75 with a 95% confidence interval of 0.70 to 0.80’. The point estimate (0.75) is the best estimate of the magnitude and direction of the experimental intervention’s effect compared with the comparator intervention. The confidence interval describes the uncertainty inherent in any estimate, and describes a range of values within which we can be reasonably sure that the true effect actually lies. If the confidence interval is relatively narrow (e.g. 0.70 to 0.80), the effect size is known precisely. If the interval is wider (e.g. 0.60 to 0.93) the uncertainty is greater, although there may still be enough precision to make decisions about the utility of the intervention. Intervals that are very wide (e.g. 0.50 to 1.10) indicate that we have little knowledge about the effect and this imprecision affects our certainty in the evidence, and that further information would be needed before we could draw a more certain conclusion.

A 95% confidence interval is often interpreted as indicating a range within which we can be 95% certain that the true effect lies. This statement is a loose interpretation, but is useful as a rough guide. The strictly correct interpretation of a confidence interval is based on the hypothetical notion of considering the results that would be obtained if the study were repeated many times. If a study were repeated infinitely often, and on each occasion a 95% confidence interval calculated, then 95% of these intervals would contain the true effect (see Section 15.3.3 for further explanation).

The width of the confidence interval for an individual study depends to a large extent on the sample size. Larger studies tend to give more precise estimates of effects (and hence have narrower confidence intervals) than smaller studies. For continuous outcomes, precision depends also on the variability in the outcome measurements (i.e. how widely individual results vary between people in the study, measured as the standard deviation); for dichotomous outcomes it depends on the risk of the event (more frequent events allow more precision, and narrower confidence intervals), and for time-to-event outcomes it also depends on the number of events observed. All these quantities are used in computation of the standard errors of effect estimates from which the confidence interval is derived.

The width of a confidence interval for a meta-analysis depends on the precision of the individual study estimates and on the number of studies combined. In addition, for random-effects models, precision will decrease with increasing heterogeneity and confidence intervals will widen correspondingly (see Chapter 10, Section 10.10.4 ). As more studies are added to a meta-analysis the width of the confidence interval usually decreases. However, if the additional studies increase the heterogeneity in the meta-analysis and a random-effects model is used, it is possible that the confidence interval width will increase.

Confidence intervals and point estimates have different interpretations in fixed-effect and random-effects models. While the fixed-effect estimate and its confidence interval address the question ‘what is the best (single) estimate of the effect?’, the random-effects estimate assumes there to be a distribution of effects, and the estimate and its confidence interval address the question ‘what is the best estimate of the average effect?’ A confidence interval may be reported for any level of confidence (although they are most commonly reported for 95%, and sometimes 90% or 99%). For example, the odds ratio of 0.80 could be reported with an 80% confidence interval of 0.73 to 0.88; a 90% interval of 0.72 to 0.89; and a 95% interval of 0.70 to 0.92. As the confidence level increases, the confidence interval widens.

There is logical correspondence between the confidence interval and the P value (see Section 15.3.3 ). The 95% confidence interval for an effect will exclude the null value (such as an odds ratio of 1.0 or a risk difference of 0) if and only if the test of significance yields a P value of less than 0.05. If the P value is exactly 0.05, then either the upper or lower limit of the 95% confidence interval will be at the null value. Similarly, the 99% confidence interval will exclude the null if and only if the test of significance yields a P value of less than 0.01.

Together, the point estimate and confidence interval provide information to assess the effects of the intervention on the outcome. For example, suppose that we are evaluating an intervention that reduces the risk of an event and we decide that it would be useful only if it reduced the risk of an event from 30% by at least 5 percentage points to 25% (these values will depend on the specific clinical scenario and outcomes, including the anticipated harms). If the meta-analysis yielded an effect estimate of a reduction of 10 percentage points with a tight 95% confidence interval, say, from 7% to 13%, we would be able to conclude that the intervention was useful since both the point estimate and the entire range of the interval exceed our criterion of a reduction of 5% for net health benefit. However, if the meta-analysis reported the same risk reduction of 10% but with a wider interval, say, from 2% to 18%, although we would still conclude that our best estimate of the intervention effect is that it provides net benefit, we could not be so confident as we still entertain the possibility that the effect could be between 2% and 5%. If the confidence interval was wider still, and included the null value of a difference of 0%, we would still consider the possibility that the intervention has no effect on the outcome whatsoever, and would need to be even more sceptical in our conclusions.

Review authors may use the same general approach to conclude that an intervention is not useful. Continuing with the above example where the criterion for an important difference that should be achieved to provide more benefit than harm is a 5% risk difference, an effect estimate of 2% with a 95% confidence interval of 1% to 4% suggests that the intervention does not provide net health benefit.

15.3.2 P values and statistical significance

A P value is the standard result of a statistical test, and is the probability of obtaining the observed effect (or larger) under a ‘null hypothesis’. In the context of Cochrane Reviews there are two commonly used statistical tests. The first is a test of overall effect (a Z-test), and its null hypothesis is that there is no overall effect of the experimental intervention compared with the comparator on the outcome of interest. The second is the (Chi 2 ) test for heterogeneity, and its null hypothesis is that there are no differences in the intervention effects across studies.

A P value that is very small indicates that the observed effect is very unlikely to have arisen purely by chance, and therefore provides evidence against the null hypothesis. It has been common practice to interpret a P value by examining whether it is smaller than particular threshold values. In particular, P values less than 0.05 are often reported as ‘statistically significant’, and interpreted as being small enough to justify rejection of the null hypothesis. However, the 0.05 threshold is an arbitrary one that became commonly used in medical and psychological research largely because P values were determined by comparing the test statistic against tabulations of specific percentage points of statistical distributions. If review authors decide to present a P value with the results of a meta-analysis, they should report a precise P value (as calculated by most statistical software), together with the 95% confidence interval. Review authors should not describe results as ‘statistically significant’, ‘not statistically significant’ or ‘non-significant’ or unduly rely on thresholds for P values , but report the confidence interval together with the exact P value (see MECIR Box 15.3.a ).

We discuss interpretation of the test for heterogeneity in Chapter 10, Section 10.10.2 ; the remainder of this section refers mainly to tests for an overall effect. For tests of an overall effect, the computation of P involves both the effect estimate and precision of the effect estimate (driven largely by sample size). As precision increases, the range of plausible effects that could occur by chance is reduced. Correspondingly, the statistical significance of an effect of a particular magnitude will usually be greater (the P value will be smaller) in a larger study than in a smaller study.

P values are commonly misinterpreted in two ways. First, a moderate or large P value (e.g. greater than 0.05) may be misinterpreted as evidence that the intervention has no effect on the outcome. There is an important difference between this statement and the correct interpretation that there is a high probability that the observed effect on the outcome is due to chance alone. To avoid such a misinterpretation, review authors should always examine the effect estimate and its 95% confidence interval.

The second misinterpretation is to assume that a result with a small P value for the summary effect estimate implies that an experimental intervention has an important benefit. Such a misinterpretation is more likely to occur in large studies and meta-analyses that accumulate data over dozens of studies and thousands of participants. The P value addresses the question of whether the experimental intervention effect is precisely nil; it does not examine whether the effect is of a magnitude of importance to potential recipients of the intervention. In a large study, a small P value may represent the detection of a trivial effect that may not lead to net health benefit when compared with the potential harms (i.e. harmful effects on other important outcomes). Again, inspection of the point estimate and confidence interval helps correct interpretations (see Section 15.3.1 ).

MECIR Box 15.3.a Relevant expectations for conduct of intervention reviews

15.3.3 Relation between confidence intervals, statistical significance and certainty of evidence

The confidence interval (and imprecision) is only one domain that influences overall uncertainty about effect estimates. Uncertainty resulting from imprecision (i.e. statistical uncertainty) may be no less important than uncertainty from indirectness, or any other GRADE domain, in the context of decision making (Schünemann 2016). Thus, the extent to which interpretations of the confidence interval described in Sections 15.3.1 and 15.3.2 correspond to conclusions about overall certainty of the evidence for the outcome of interest depends on these other domains. If there are no concerns about other domains that determine the certainty of the evidence (i.e. risk of bias, inconsistency, indirectness or publication bias), then the interpretation in Sections 15.3.1 and 15.3.2 . about the relation of the confidence interval to the true effect may be carried forward to the overall certainty. However, if there are concerns about the other domains that affect the certainty of the evidence, the interpretation about the true effect needs to be seen in the context of further uncertainty resulting from those concerns.

For example, nine randomized controlled trials in almost 6000 cancer patients indicated that the administration of heparin reduces the risk of venous thromboembolism (VTE), with a risk ratio of 43% (95% CI 19% to 60%) (Akl et al 2011a). For patients with a plausible baseline risk of approximately 4.6% per year, this relative effect suggests that heparin leads to an absolute risk reduction of 20 fewer VTEs (95% CI 9 fewer to 27 fewer) per 1000 people per year (Akl et al 2011a). Now consider that the review authors or those applying the evidence in a guideline have lowered the certainty in the evidence as a result of indirectness. While the confidence intervals would remain unchanged, the certainty in that confidence interval and in the point estimate as reflecting the truth for the question of interest will be lowered. In fact, the certainty range will have unknown width so there will be unknown likelihood of a result within that range because of this indirectness. The lower the certainty in the evidence, the less we know about the width of the certainty range, although methods for quantifying risk of bias and understanding potential direction of bias may offer insight when lowered certainty is due to risk of bias. Nevertheless, decision makers must consider this uncertainty, and must do so in relation to the effect measure that is being evaluated (e.g. a relative or absolute measure). We will describe the impact on interpretations for dichotomous outcomes in Section 15.4 .

15.4 Interpreting results from dichotomous outcomes (including numbers needed to treat)

15.4.1 relative and absolute risk reductions.

Clinicians may be more inclined to prescribe an intervention that reduces the relative risk of death by 25% than one that reduces the risk of death by 1 percentage point, although both presentations of the evidence may relate to the same benefit (i.e. a reduction in risk from 4% to 3%). The former refers to the relative reduction in risk and the latter to the absolute reduction in risk. As described in Chapter 6, Section 6.4.1 , there are several measures for comparing dichotomous outcomes in two groups. Meta-analyses are usually undertaken using risk ratios (RR), odds ratios (OR) or risk differences (RD), but there are several alternative ways of expressing results.

Relative risk reduction (RRR) is a convenient way of re-expressing a risk ratio as a percentage reduction:

For example, a risk ratio of 0.75 translates to a relative risk reduction of 25%, as in the example above.

The risk difference is often referred to as the absolute risk reduction (ARR) or absolute risk increase (ARI), and may be presented as a percentage (e.g. 1%), as a decimal (e.g. 0.01), or as account (e.g. 10 out of 1000). We consider different choices for presenting absolute effects in Section 15.4.3 . We then describe computations for obtaining these numbers from the results of individual studies and of meta-analyses in Section 15.4.4 .

15.4.2 Number needed to treat (NNT)

The number needed to treat (NNT) is a common alternative way of presenting information on the effect of an intervention. The NNT is defined as the expected number of people who need to receive the experimental rather than the comparator intervention for one additional person to either incur or avoid an event (depending on the direction of the result) in a given time frame. Thus, for example, an NNT of 10 can be interpreted as ‘it is expected that one additional (or less) person will incur an event for every 10 participants receiving the experimental intervention rather than comparator over a given time frame’. It is important to be clear that:

• since the NNT is derived from the risk difference, it is still a comparative measure of effect (experimental versus a specific comparator) and not a general property of a single intervention; and
• the NNT gives an ‘expected value’. For example, NNT = 10 does not imply that one additional event will occur in each and every group of 10 people.

NNTs can be computed for both beneficial and detrimental events, and for interventions that cause both improvements and deteriorations in outcomes. In all instances NNTs are expressed as positive whole numbers. Some authors use the term ‘number needed to harm’ (NNH) when an intervention leads to an adverse outcome, or a decrease in a positive outcome, rather than improvement. However, this phrase can be misleading (most notably, it can easily be read to imply the number of people who will experience a harmful outcome if given the intervention), and it is strongly recommended that ‘number needed to harm’ and ‘NNH’ are avoided. The preferred alternative is to use phrases such as ‘number needed to treat for an additional beneficial outcome’ (NNTB) and ‘number needed to treat for an additional harmful outcome’ (NNTH) to indicate direction of effect.

As NNTs refer to events, their interpretation needs to be worded carefully when the binary outcome is a dichotomization of a scale-based outcome. For example, if the outcome is pain measured on a ‘none, mild, moderate or severe’ scale it may have been dichotomized as ‘none or mild’ versus ‘moderate or severe’. It would be inappropriate for an NNT from these data to be referred to as an ‘NNT for pain’. It is an ‘NNT for moderate or severe pain’.

We consider different choices for presenting absolute effects in Section 15.4.3 . We then describe computations for obtaining these numbers from the results of individual studies and of meta-analyses in Section 15.4.4 .

15.4.3 Expressing risk differences

Users of reviews are liable to be influenced by the choice of statistical presentations of the evidence. Hoffrage and colleagues suggest that physicians’ inferences about statistical outcomes are more appropriate when they deal with ‘natural frequencies’ – whole numbers of people, both treated and untreated (e.g. treatment results in a drop from 20 out of 1000 to 10 out of 1000 women having breast cancer) – than when effects are presented as percentages (e.g. 1% absolute reduction in breast cancer risk) (Hoffrage et al 2000). Probabilities may be more difficult to understand than frequencies, particularly when events are rare. While standardization may be important in improving the presentation of research evidence (and participation in healthcare decisions), current evidence suggests that the presentation of natural frequencies for expressing differences in absolute risk is best understood by consumers of healthcare information (Akl et al 2011b). This evidence provides the rationale for presenting absolute risks in ‘Summary of findings’ tables as numbers of people with events per 1000 people receiving the intervention (see Chapter 14 ).

RRs and RRRs remain crucial because relative effects tend to be substantially more stable across risk groups than absolute effects (see Chapter 10, Section 10.4.3 ). Review authors can use their own data to study this consistency (Cates 1999, Smeeth et al 1999). Risk differences from studies are least likely to be consistent across baseline event rates; thus, they are rarely appropriate for computing numbers needed to treat in systematic reviews. If a relative effect measure (OR or RR) is chosen for meta-analysis, then a comparator group risk needs to be specified as part of the calculation of an RD or NNT. In addition, if there are several different groups of participants with different levels of risk, it is crucial to express absolute benefit for each clinically identifiable risk group, clarifying the time period to which this applies. Studies in patients with differing severity of disease, or studies with different lengths of follow-up will almost certainly have different comparator group risks. In these cases, different comparator group risks lead to different RDs and NNTs (except when the intervention has no effect). A recommended approach is to re-express an odds ratio or a risk ratio as a variety of RD or NNTs across a range of assumed comparator risks (ACRs) (McQuay and Moore 1997, Smeeth et al 1999). Review authors should bear these considerations in mind not only when constructing their ‘Summary of findings’ table, but also in the text of their review.

For example, a review of oral anticoagulants to prevent stroke presented information to users by describing absolute benefits for various baseline risks (Aguilar and Hart 2005, Aguilar et al 2007). They presented their principal findings as “The inherent risk of stroke should be considered in the decision to use oral anticoagulants in atrial fibrillation patients, selecting those who stand to benefit most for this therapy” (Aguilar and Hart 2005). Among high-risk atrial fibrillation patients with prior stroke or transient ischaemic attack who have stroke rates of about 12% (120 per 1000) per year, warfarin prevents about 70 strokes yearly per 1000 patients, whereas for low-risk atrial fibrillation patients (with a stroke rate of about 2% per year or 20 per 1000), warfarin prevents only 12 strokes. This presentation helps users to understand the important impact that typical baseline risks have on the absolute benefit that they can expect.

15.4.4 Computations

Direct computation of risk difference (RD) or a number needed to treat (NNT) depends on the summary statistic (odds ratio, risk ratio or risk differences) available from the study or meta-analysis. When expressing results of meta-analyses, review authors should use, in the computations, whatever statistic they determined to be the most appropriate summary for meta-analysis (see Chapter 10, Section 10.4.3 ). Here we present calculations to obtain RD as a reduction in the number of participants per 1000. For example, a risk difference of –0.133 corresponds to 133 fewer participants with the event per 1000.

RDs and NNTs should not be computed from the aggregated total numbers of participants and events across the trials. This approach ignores the randomization within studies, and may produce seriously misleading results if there is unbalanced randomization in any of the studies. Using the pooled result of a meta-analysis is more appropriate. When computing NNTs, the values obtained are by convention always rounded up to the next whole number.

15.4.4.1 Computing NNT from a risk difference (RD)

A NNT may be computed from a risk difference as

where the vertical bars (‘absolute value of’) in the denominator indicate that any minus sign should be ignored. It is convention to round the NNT up to the nearest whole number. For example, if the risk difference is –0.12 the NNT is 9; if the risk difference is –0.22 the NNT is 5. Cochrane Review authors should qualify the NNT as referring to benefit (improvement) or harm by denoting the NNT as NNTB or NNTH. Note that this approach, although feasible, should be used only for the results of a meta-analysis of risk differences. In most cases meta-analyses will be undertaken using a relative measure of effect (RR or OR), and those statistics should be used to calculate the NNT (see Section 15.4.4.2 and 15.4.4.3 ).

15.4.4.2 Computing risk differences or NNT from a risk ratio

To aid interpretation of the results of a meta-analysis of risk ratios, review authors may compute an absolute risk reduction or NNT. In order to do this, an assumed comparator risk (ACR) (otherwise known as a baseline risk, or risk that the outcome of interest would occur with the comparator intervention) is required. It will usually be appropriate to do this for a range of different ACRs. The computation proceeds as follows:

As an example, suppose the risk ratio is RR = 0.92, and an ACR = 0.3 (300 per 1000) is assumed. Then the effect on risk is 24 fewer per 1000:

The NNT is 42:

15.4.4.3 Computing risk differences or NNT from an odds ratio

Review authors may wish to compute a risk difference or NNT from the results of a meta-analysis of odds ratios. In order to do this, an ACR is required. It will usually be appropriate to do this for a range of different ACRs. The computation proceeds as follows:

As an example, suppose the odds ratio is OR = 0.73, and a comparator risk of ACR = 0.3 is assumed. Then the effect on risk is 62 fewer per 1000:

The NNT is 17:

15.4.4.4 Computing risk ratio from an odds ratio

Because risk ratios are easier to interpret than odds ratios, but odds ratios have favourable mathematical properties, a review author may decide to undertake a meta-analysis based on odds ratios, but to express the result as a summary risk ratio (or relative risk reduction). This requires an ACR. Then

It will often be reasonable to perform this transformation using the median comparator group risk from the studies in the meta-analysis.

15.4.4.5 Computing confidence limits

Confidence limits for RDs and NNTs may be calculated by applying the above formulae to the upper and lower confidence limits for the summary statistic (RD, RR or OR) (Altman 1998). Note that this confidence interval does not incorporate uncertainty around the ACR.

If the 95% confidence interval of OR or RR includes the value 1, one of the confidence limits will indicate benefit and the other harm. Thus, appropriate use of the words ‘fewer’ and ‘more’ is required for each limit when presenting results in terms of events. For NNTs, the two confidence limits should be labelled as NNTB and NNTH to indicate the direction of effect in each case. The confidence interval for the NNT will include a ‘discontinuity’, because increasingly smaller risk differences that approach zero will lead to NNTs approaching infinity. Thus, the confidence interval will include both an infinitely large NNTB and an infinitely large NNTH.

15.5 Interpreting results from continuous outcomes (including standardized mean differences)

15.5.1 meta-analyses with continuous outcomes.

Review authors should describe in the study protocol how they plan to interpret results for continuous outcomes. When outcomes are continuous, review authors have a number of options to present summary results. These options differ if studies report the same measure that is familiar to the target audiences, studies report the same or very similar measures that are less familiar to the target audiences, or studies report different measures.

15.5.2 Meta-analyses with continuous outcomes using the same measure

If all studies have used the same familiar units, for instance, results are expressed as durations of events, such as symptoms for conditions including diarrhoea, sore throat, otitis media, influenza or duration of hospitalization, a meta-analysis may generate a summary estimate in those units, as a difference in mean response (see, for instance, the row summarizing results for duration of diarrhoea in Chapter 14, Figure 14.1.b and the row summarizing oedema in Chapter 14, Figure 14.1.a ). For such outcomes, the ‘Summary of findings’ table should include a difference of means between the two interventions. However, when units of such outcomes may be difficult to interpret, particularly when they relate to rating scales (again, see the oedema row of Chapter 14, Figure 14.1.a ). ‘Summary of findings’ tables should include the minimum and maximum of the scale of measurement, and the direction. Knowledge of the smallest change in instrument score that patients perceive is important – the minimal important difference (MID) – and can greatly facilitate the interpretation of results (Guyatt et al 1998, Schünemann and Guyatt 2005). Knowing the MID allows review authors and users to place results in context. Review authors should state the MID – if known – in the Comments column of their ‘Summary of findings’ table. For example, the chronic respiratory questionnaire has possible scores in health-related quality of life ranging from 1 to 7 and 0.5 represents a well-established MID (Jaeschke et al 1989, Schünemann et al 2005).

15.5.3 Meta-analyses with continuous outcomes using different measures

When studies have used different instruments to measure the same construct, a standardized mean difference (SMD) may be used in meta-analysis for combining continuous data. Without guidance, clinicians and patients may have little idea how to interpret results presented as SMDs. Review authors should therefore consider issues of interpretability when planning their analysis at the protocol stage and should consider whether there will be suitable ways to re-express the SMD or whether alternative effect measures, such as a ratio of means, or possibly as minimal important difference units (Guyatt et al 2013b) should be used. Table 15.5.a and the following sections describe these options.

Table 15.5.a Approaches and their implications to presenting results of continuous variables when primary studies have used different instruments to measure the same construct. Adapted from Guyatt et al (2013b)

15.5.3.1 Presenting and interpreting SMDs using generic effect size estimates

The SMD expresses the intervention effect in standard units rather than the original units of measurement. The SMD is the difference in mean effects between the experimental and comparator groups divided by the pooled standard deviation of participants’ outcomes, or external SDs when studies are very small (see Chapter 6, Section 6.5.1.2 ). The value of a SMD thus depends on both the size of the effect (the difference between means) and the standard deviation of the outcomes (the inherent variability among participants or based on an external SD).

If review authors use the SMD, they might choose to present the results directly as SMDs (row 1a, Table 15.5.a and Table 15.5.b ). However, absolute values of the intervention and comparison groups are typically not useful because studies have used different measurement instruments with different units. Guiding rules for interpreting SMDs (or ‘Cohen’s effect sizes’) exist, and have arisen mainly from researchers in the social sciences (Cohen 1988). One example is as follows: 0.2 represents a small effect, 0.5 a moderate effect and 0.8 a large effect (Cohen 1988). Variations exist (e.g. <0.40=small, 0.40 to 0.70=moderate, >0.70=large). Review authors might consider including such a guiding rule in interpreting the SMD in the text of the review, and in summary versions such as the Comments column of a ‘Summary of findings’ table. However, some methodologists believe that such interpretations are problematic because patient importance of a finding is context-dependent and not amenable to generic statements.

15.5.3.2 Re-expressing SMDs using a familiar instrument

The second possibility for interpreting the SMD is to express it in the units of one or more of the specific measurement instruments used by the included studies (row 1b, Table 15.5.a and Table 15.5.b ). The approach is to calculate an absolute difference in means by multiplying the SMD by an estimate of the SD associated with the most familiar instrument. To obtain this SD, a reasonable option is to calculate a weighted average across all intervention groups of all studies that used the selected instrument (preferably a pre-intervention or post-intervention SD as discussed in Chapter 10, Section 10.5.2 ). To better reflect among-person variation in practice, or to use an instrument not represented in the meta-analysis, it may be preferable to use a standard deviation from a representative observational study. The summary effect is thus re-expressed in the original units of that particular instrument and the clinical relevance and impact of the intervention effect can be interpreted using that familiar instrument.

The same approach of re-expressing the results for a familiar instrument can also be used for other standardized effect measures such as when standardizing by MIDs (Guyatt et al 2013b): see Section 15.5.3.5 .

Table 15.5.b Application of approaches when studies have used different measures: effects of dexamethasone for pain after laparoscopic cholecystectomy (Karanicolas et al 2008). Reproduced with permission of Wolters Kluwer

1 Certainty rated according to GRADE from very low to high certainty. 2 Substantial unexplained heterogeneity in study results. 3 Imprecision due to wide confidence intervals. 4 The 20% comes from the proportion in the control group requiring rescue analgesia. 5 Crude (arithmetic) means of the post-operative pain mean responses across all five trials when transformed to a 100-point scale.

15.5.3.3 Re-expressing SMDs through dichotomization and transformation to relative and absolute measures

A third approach (row 1c, Table 15.5.a and Table 15.5.b ) relies on converting the continuous measure into a dichotomy and thus allows calculation of relative and absolute effects on a binary scale. A transformation of a SMD to a (log) odds ratio is available, based on the assumption that an underlying continuous variable has a logistic distribution with equal standard deviation in the two intervention groups, as discussed in Chapter 10, Section 10.6  (Furukawa 1999, Guyatt et al 2013b). The assumption is unlikely to hold exactly and the results must be regarded as an approximation. The log odds ratio is estimated as

(or approximately 1.81✕SMD). The resulting odds ratio can then be presented as normal, and in a ‘Summary of findings’ table, combined with an assumed comparator group risk to be expressed as an absolute risk difference. The comparator group risk in this case would refer to the proportion of people who have achieved a specific value of the continuous outcome. In randomized trials this can be interpreted as the proportion who have improved by some (specified) amount (responders), for instance by 5 points on a 0 to 100 scale. Table 15.5.c shows some illustrative results from this method. The risk differences can then be converted to NNTs or to people per thousand using methods described in Section 15.4.4 .

Table 15.5.c Risk difference derived for specific SMDs for various given ‘proportions improved’ in the comparator group (Furukawa 1999, Guyatt et al 2013b). Reproduced with permission of Elsevier 

15.5.3.4 Ratio of means

A more frequently used approach is based on calculation of a ratio of means between the intervention and comparator groups (Friedrich et al 2008) as discussed in Chapter 6, Section 6.5.1.3 . Interpretational advantages of this approach include the ability to pool studies with outcomes expressed in different units directly, to avoid the vulnerability of heterogeneous populations that limits approaches that rely on SD units, and for ease of clinical interpretation (row 2, Table 15.5.a and Table 15.5.b ). This method is currently designed for post-intervention scores only. However, it is possible to calculate a ratio of change scores if both intervention and comparator groups change in the same direction in each relevant study, and this ratio may sometimes be informative.

Limitations to this approach include its limited applicability to change scores (since it is unlikely that both intervention and comparator group changes are in the same direction in all studies) and the possibility of misleading results if the comparator group mean is very small, in which case even a modest difference from the intervention group will yield a large and therefore misleading ratio of means. It also requires that separate ratios of means be calculated for each included study, and then entered into a generic inverse variance meta-analysis (see Chapter 10, Section 10.3 ).

The ratio of means approach illustrated in Table 15.5.b suggests a relative reduction in pain of only 13%, meaning that those receiving steroids have a pain severity 87% of those in the comparator group, an effect that might be considered modest.

15.5.3.5 Presenting continuous results as minimally important difference units

To express results in MID units, review authors have two options. First, they can be combined across studies in the same way as the SMD, but instead of dividing the mean difference of each study by its SD, review authors divide by the MID associated with that outcome (Johnston et al 2010, Guyatt et al 2013b). Instead of SD units, the pooled results represent MID units (row 3, Table 15.5.a and Table 15.5.b ), and may be more easily interpretable. This approach avoids the problem of varying SDs across studies that may distort estimates of effect in approaches that rely on the SMD. The approach, however, relies on having well-established MIDs. The approach is also risky in that a difference less than the MID may be interpreted as trivial when a substantial proportion of patients may have achieved an important benefit.

The other approach makes a simple conversion (not shown in Table 15.5.b ), before undertaking the meta-analysis, of the means and SDs from each study to means and SDs on the scale of a particular familiar instrument whose MID is known. For example, one can rescale the mean and SD of other chronic respiratory disease instruments (e.g. rescaling a 0 to 100 score of an instrument) to a the 1 to 7 score in Chronic Respiratory Disease Questionnaire (CRQ) units (by assuming 0 equals 1 and 100 equals 7 on the CRQ). Given the MID of the CRQ of 0.5, a mean difference in change of 0.71 after rescaling of all studies suggests a substantial effect of the intervention (Guyatt et al 2013b). This approach, presenting in units of the most familiar instrument, may be the most desirable when the target audiences have extensive experience with that instrument, particularly if the MID is well established.

15.6 Drawing conclusions

15.6.1 conclusions sections of a cochrane review.

Authors’ conclusions in a Cochrane Review are divided into implications for practice and implications for research. While Cochrane Reviews about interventions can provide meaningful information and guidance for practice, decisions about the desirable and undesirable consequences of healthcare options require evidence and judgements for criteria that most Cochrane Reviews do not provide (Alonso-Coello et al 2016). In describing the implications for practice and the development of recommendations, however, review authors may consider the certainty of the evidence, the balance of benefits and harms, and assumed values and preferences.

15.6.2 Implications for practice

Drawing conclusions about the practical usefulness of an intervention entails making trade-offs, either implicitly or explicitly, between the estimated benefits, harms and the values and preferences. Making such trade-offs, and thus making specific recommendations for an action in a specific context, goes beyond a Cochrane Review and requires additional evidence and informed judgements that most Cochrane Reviews do not provide (Alonso-Coello et al 2016). Such judgements are typically the domain of clinical practice guideline developers for which Cochrane Reviews will provide crucial information (Graham et al 2011, Schünemann et al 2014, Zhang et al 2018a). Thus, authors of Cochrane Reviews should not make recommendations.

If review authors feel compelled to lay out actions that clinicians and patients could take, they should – after describing the certainty of evidence and the balance of benefits and harms – highlight different actions that might be consistent with particular patterns of values and preferences. Other factors that might influence a decision should also be highlighted, including any known factors that would be expected to modify the effects of the intervention, the baseline risk or status of the patient, costs and who bears those costs, and the availability of resources. Review authors should ensure they consider all patient-important outcomes, including those for which limited data may be available. In the context of public health reviews the focus may be on population-important outcomes as the target may be an entire (non-diseased) population and include outcomes that are not measured in the population receiving an intervention (e.g. a reduction of transmission of infections from those receiving an intervention). This process implies a high level of explicitness in judgements about values or preferences attached to different outcomes and the certainty of the related evidence (Zhang et al 2018b, Zhang et al 2018c); this and a full cost-effectiveness analysis is beyond the scope of most Cochrane Reviews (although they might well be used for such analyses; see Chapter 20 ).

A review on the use of anticoagulation in cancer patients to increase survival (Akl et al 2011a) provides an example for laying out clinical implications for situations where there are important trade-offs between desirable and undesirable effects of the intervention: “The decision for a patient with cancer to start heparin therapy for survival benefit should balance the benefits and downsides and integrate the patient’s values and preferences. Patients with a high preference for a potential survival prolongation, limited aversion to potential bleeding, and who do not consider heparin (both UFH or LMWH) therapy a burden may opt to use heparin, while those with aversion to bleeding may not.”

15.6.3 Implications for research

The second category for authors’ conclusions in a Cochrane Review is implications for research. To help people make well-informed decisions about future healthcare research, the ‘Implications for research’ section should comment on the need for further research, and the nature of the further research that would be most desirable. It is helpful to consider the population, intervention, comparison and outcomes that could be addressed, or addressed more effectively in the future, in the context of the certainty of the evidence in the current review (Brown et al 2006):

• P (Population): diagnosis, disease stage, comorbidity, risk factor, sex, age, ethnic group, specific inclusion or exclusion criteria, clinical setting;
• I (Intervention): type, frequency, dose, duration, prognostic factor;
• C (Comparison): placebo, routine care, alternative treatment/management;
• O (Outcome): which clinical or patient-related outcomes will the researcher need to measure, improve, influence or accomplish? Which methods of measurement should be used?

While Cochrane Review authors will find the PICO domains helpful, the domains of the GRADE certainty framework further support understanding and describing what additional research will improve the certainty in the available evidence. Note that as the certainty of the evidence is likely to vary by outcome, these implications will be specific to certain outcomes in the review. Table 15.6.a shows how review authors may be aided in their interpretation of the body of evidence and drawing conclusions about future research and practice.

Table 15.6.a Implications for research and practice suggested by individual GRADE domains

The review of compression stockings for prevention of deep vein thrombosis (DVT) in airline passengers described in Chapter 14 provides an example where there is some convincing evidence of a benefit of the intervention: “This review shows that the question of the effects on symptomless DVT of wearing versus not wearing compression stockings in the types of people studied in these trials should now be regarded as answered. Further research may be justified to investigate the relative effects of different strengths of stockings or of stockings compared to other preventative strategies. Further randomised trials to address the remaining uncertainty about the effects of wearing versus not wearing compression stockings on outcomes such as death, pulmonary embolism and symptomatic DVT would need to be large.” (Clarke et al 2016).

A review of therapeutic touch for anxiety disorder provides an example of the implications for research when no eligible studies had been found: “This review highlights the need for randomized controlled trials to evaluate the effectiveness of therapeutic touch in reducing anxiety symptoms in people diagnosed with anxiety disorders. Future trials need to be rigorous in design and delivery, with subsequent reporting to include high quality descriptions of all aspects of methodology to enable appraisal and interpretation of results.” (Robinson et al 2007).

15.6.4 Reaching conclusions

A common mistake is to confuse ‘no evidence of an effect’ with ‘evidence of no effect’. When the confidence intervals are too wide (e.g. including no effect), it is wrong to claim that the experimental intervention has ‘no effect’ or is ‘no different’ from the comparator intervention. Review authors may also incorrectly ‘positively’ frame results for some effects but not others. For example, when the effect estimate is positive for a beneficial outcome but confidence intervals are wide, review authors may describe the effect as promising. However, when the effect estimate is negative for an outcome that is considered harmful but the confidence intervals include no effect, review authors report no effect. Another mistake is to frame the conclusion in wishful terms. For example, review authors might write, “there were too few people in the analysis to detect a reduction in mortality” when the included studies showed a reduction or even increase in mortality that was not ‘statistically significant’. One way of avoiding errors such as these is to consider the results blinded; that is, consider how the results would be presented and framed in the conclusions if the direction of the results was reversed. If the confidence interval for the estimate of the difference in the effects of the interventions overlaps with no effect, the analysis is compatible with both a true beneficial effect and a true harmful effect. If one of the possibilities is mentioned in the conclusion, the other possibility should be mentioned as well. Table 15.6.b suggests narrative statements for drawing conclusions based on the effect estimate from the meta-analysis and the certainty of the evidence.

Table 15.6.b Suggested narrative statements for phrasing conclusions

Another common mistake is to reach conclusions that go beyond the evidence. Often this is done implicitly, without referring to the additional information or judgements that are used in reaching conclusions about the implications of a review for practice. Even when additional information and explicit judgements support conclusions about the implications of a review for practice, review authors rarely conduct systematic reviews of the additional information. Furthermore, implications for practice are often dependent on specific circumstances and values that must be taken into consideration. As we have noted, review authors should always be cautious when drawing conclusions about implications for practice and they should not make recommendations.

15.7 Chapter information

Authors: Holger J Schünemann, Gunn E Vist, Julian PT Higgins, Nancy Santesso, Jonathan J Deeks, Paul Glasziou, Elie Akl, Gordon H Guyatt; on behalf of the Cochrane GRADEing Methods Group

Acknowledgements: Andrew Oxman, Jonathan Sterne, Michael Borenstein and Rob Scholten contributed text to earlier versions of this chapter.

Funding: This work was in part supported by funding from the Michael G DeGroote Cochrane Canada Centre and the Ontario Ministry of Health. JJD receives support from the National Institute for Health Research (NIHR) Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham. JPTH receives support from the NIHR Biomedical Research Centre at University Hospitals Bristol NHS Foundation Trust and the University of Bristol. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

15.8 References

Aguilar MI, Hart R. Oral anticoagulants for preventing stroke in patients with non-valvular atrial fibrillation and no previous history of stroke or transient ischemic attacks. Cochrane Database of Systematic Reviews 2005; 3 : CD001927.

Aguilar MI, Hart R, Pearce LA. Oral anticoagulants versus antiplatelet therapy for preventing stroke in patients with non-valvular atrial fibrillation and no history of stroke or transient ischemic attacks. Cochrane Database of Systematic Reviews 2007; 3 : CD006186.

Akl EA, Gunukula S, Barba M, Yosuico VE, van Doormaal FF, Kuipers S, Middeldorp S, Dickinson HO, Bryant A, Schünemann H. Parenteral anticoagulation in patients with cancer who have no therapeutic or prophylactic indication for anticoagulation. Cochrane Database of Systematic Reviews 2011a; 1 : CD006652.

Akl EA, Oxman AD, Herrin J, Vist GE, Terrenato I, Sperati F, Costiniuk C, Blank D, Schünemann H. Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database of Systematic Reviews 2011b; 3 : CD006776.

Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AD, Group GW. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ 2016; 353 : i2016.

Altman DG. Confidence intervals for the number needed to treat. BMJ 1998; 317 : 1309-1312.

Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, Guyatt GH, Harbour RT, Haugh MC, Henry D, Hill S, Jaeschke R, Leng G, Liberati A, Magrini N, Mason J, Middleton P, Mrukowicz J, O'Connell D, Oxman AD, Phillips B, Schünemann HJ, Edejer TT, Varonen H, Vist GE, Williams JW, Jr., Zaza S. Grading quality of evidence and strength of recommendations. BMJ 2004; 328 : 1490.

Brown P, Brunnhuber K, Chalkidou K, Chalmers I, Clarke M, Fenton M, Forbes C, Glanville J, Hicks NJ, Moody J, Twaddle S, Timimi H, Young P. How to formulate research recommendations. BMJ 2006; 333 : 804-806.

Cates C. Confidence intervals for the number needed to treat: Pooling numbers needed to treat may not be reliable. BMJ 1999; 318 : 1764-1765.

Clarke MJ, Broderick C, Hopewell S, Juszczak E, Eisinga A. Compression stockings for preventing deep vein thrombosis in airline passengers. Cochrane Database of Systematic Reviews 2016; 9 : CD004002.

Cohen J. Statistical Power Analysis in the Behavioral Sciences . 2nd edition ed. Hillsdale (NJ): Lawrence Erlbaum Associates, Inc.; 1988.

Coleman T, Chamberlain C, Davey MA, Cooper SE, Leonardi-Bee J. Pharmacological interventions for promoting smoking cessation during pregnancy. Cochrane Database of Systematic Reviews 2015; 12 : CD010078.

Dans AM, Dans L, Oxman AD, Robinson V, Acuin J, Tugwell P, Dennis R, Kang D. Assessing equity in clinical practice guidelines. Journal of Clinical Epidemiology 2007; 60 : 540-546.

Friedman LM, Furberg CD, DeMets DL. Fundamentals of Clinical Trials . 2nd edition ed. Littleton (MA): John Wright PSG, Inc.; 1985.

Friedrich JO, Adhikari NK, Beyene J. The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: a simulation study. BMC Medical Research Methodology 2008; 8 : 32.

Furukawa T. From effect size into number needed to treat. Lancet 1999; 353 : 1680.

Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services: Clinical Practice Guidelines We Can Trust. Washington, DC: National Academies Press; 2011.

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of Clinical Epidemiology 2011a; 64 : 383-394.

Guyatt GH, Juniper EF, Walter SD, Griffith LE, Goldstein RS. Interpreting treatment effects in randomised trials. BMJ 1998; 316 : 690-693.

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336 : 924-926.

Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, Alonso-Coello P, Falck-Ytter Y, Jaeschke R, Vist G, Akl EA, Post PN, Norris S, Meerpohl J, Shukla VK, Nasser M, Schünemann HJ. GRADE guidelines: 8. Rating the quality of evidence--indirectness. Journal of Clinical Epidemiology 2011b; 64 : 1303-1310.

Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, Brozek J, Norris S, Meerpohl J, Djulbegovic B, Alonso-Coello P, Post PN, Busse JW, Glasziou P, Christensen R, Schünemann HJ. GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes. Journal of Clinical Epidemiology 2013a; 66 : 158-172.

Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanicolas P, Akl EA, Vist G, Kunz R, Brozek J, Kupper LL, Martin SL, Meerpohl JJ, Alonso-Coello P, Christensen R, Schünemann HJ. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. Journal of Clinical Epidemiology 2013b; 66 : 173-183.

Hawe P, Shiell A, Riley T, Gold L. Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. Journal of Epidemiology and Community Health 2004; 58 : 788-793.

Hoffrage U, Lindsey S, Hertwig R, Gigerenzer G. Medicine. Communicating statistical information. Science 2000; 290 : 2261-2262.

Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Controlled Clinical Trials 1989; 10 : 407-415.

Johnston B, Thorlund K, Schünemann H, Xie F, Murad M, Montori V, Guyatt G. Improving the interpretation of health-related quality of life evidence in meta-analysis: The application of minimal important difference units. . Health Outcomes and Qualithy of Life 2010; 11 : 116.

Karanicolas PJ, Smith SE, Kanbur B, Davies E, Guyatt GH. The impact of prophylactic dexamethasone on nausea and vomiting after laparoscopic cholecystectomy: a systematic review and meta-analysis. Annals of Surgery 2008; 248 : 751-762.

Lumley J, Oliver SS, Chamberlain C, Oakley L. Interventions for promoting smoking cessation during pregnancy. Cochrane Database of Systematic Reviews 2004; 4 : CD001055.

McQuay HJ, Moore RA. Using numerical results from systematic reviews in clinical practice. Annals of Internal Medicine 1997; 126 : 712-720.

Resnicow K, Cross D, Wynder E. The Know Your Body program: a review of evaluation studies. Bulletin of the New York Academy of Medicine 1993; 70 : 188-207.

Robinson J, Biley FC, Dolk H. Therapeutic touch for anxiety disorders. Cochrane Database of Systematic Reviews 2007; 3 : CD006240.

Rothwell PM. External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet 2005; 365 : 82-93.

Santesso N, Carrasco-Labra A, Langendam M, Brignardello-Petersen R, Mustafa RA, Heus P, Lasserson T, Opiyo N, Kunnamo I, Sinclair D, Garner P, Treweek S, Tovey D, Akl EA, Tugwell P, Brozek JL, Guyatt G, Schünemann HJ. Improving GRADE evidence tables part 3: detailed guidance for explanatory footnotes supports creating and understanding GRADE certainty in the evidence judgments. Journal of Clinical Epidemiology 2016; 74 : 28-39.

Schünemann HJ, Puhan M, Goldstein R, Jaeschke R, Guyatt GH. Measurement properties and interpretability of the Chronic respiratory disease questionnaire (CRQ). COPD: Journal of Chronic Obstructive Pulmonary Disease 2005; 2 : 81-89.

Schünemann HJ, Guyatt GH. Commentary--goodbye M(C)ID! Hello MID, where do you come from? Health Services Research 2005; 40 : 593-597.

Schünemann HJ, Fretheim A, Oxman AD. Improving the use of research evidence in guideline development: 13. Applicability, transferability and adaptation. Health Research Policy and Systems 2006; 4 : 25.

Schünemann HJ. Methodological idiosyncracies, frameworks and challenges of non-pharmaceutical and non-technical treatment interventions. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen 2013; 107 : 214-220.

Schünemann HJ, Tugwell P, Reeves BC, Akl EA, Santesso N, Spencer FA, Shea B, Wells G, Helfand M. Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Research Synthesis Methods 2013; 4 : 49-62.

Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, Baldeh T, Zhang Y, Raid U, Neumann I, Norris SL, Thornton J, Harbour R, Treweek S, Guyatt G, Alonso-Coello P, Reinap M, Brozek J, Oxman A, Akl EA. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ: Canadian Medical Association Journal 2014; 186 : E123-142.

Schünemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision? Journal of Clinical Epidemiology 2016; 75 : 6-15.

Smeeth L, Haines A, Ebrahim S. Numbers needed to treat derived from meta-analyses--sometimes informative, usually misleading. BMJ 1999; 318 : 1548-1551.

Sun X, Briel M, Busse JW, You JJ, Akl EA, Mejza F, Bala MM, Bassler D, Mertz D, Diaz-Granados N, Vandvik PO, Malaga G, Srinathan SK, Dahm P, Johnston BC, Alonso-Coello P, Hassouneh B, Walter SD, Heels-Ansdell D, Bhatnagar N, Altman DG, Guyatt GH. Credibility of claims of subgroup effects in randomised controlled trials: systematic review. BMJ 2012; 344 : e1553.

Zhang Y, Akl EA, Schünemann HJ. Using systematic reviews in guideline development: the GRADE approach. Research Synthesis Methods 2018a: doi: 10.1002/jrsm.1313.

Zhang Y, Alonso-Coello P, Guyatt GH, Yepes-Nunez JJ, Akl EA, Hazlewood G, Pardo-Hernandez H, Etxeandia-Ikobaltzeta I, Qaseem A, Williams JW, Jr., Tugwell P, Flottorp S, Chang Y, Zhang Y, Mustafa RA, Rojas MX, Schünemann HJ. GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness. Journal of Clinical Epidemiology 2018b: doi: 10.1016/j.jclinepi.2018.1001.1013.

Zhang Y, Alonso Coello P, Guyatt G, Yepes-Nunez JJ, Akl EA, Hazlewood G, Pardo-Hernandez H, Etxeandia-Ikobaltzeta I, Qaseem A, Williams JW, Jr., Tugwell P, Flottorp S, Chang Y, Zhang Y, Mustafa RA, Rojas MX, Xie F, Schünemann HJ. GRADE Guidelines: 20. Assessing the certainty of evidence in the importance of outcomes or values and preferences - Inconsistency, Imprecision, and other Domains. Journal of Clinical Epidemiology 2018c: doi: 10.1016/j.jclinepi.2018.1005.1011.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Qualitative Research? | Methods & Examples

What Is Qualitative Research? | Methods & Examples

Published on June 19, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research.

Qualitative research is the opposite of quantitative research , which involves collecting and analyzing numerical data for statistical analysis.

Qualitative research is commonly used in the humanities and social sciences, in subjects such as anthropology, sociology, education, health sciences, history, etc.

• How does social media shape body image in teenagers?
• How do children and adults interpret healthy eating in the UK?
• What factors influence employee retention in a large organization?
• How is anxiety experienced around the world?
• How can teachers integrate social issues into science curriculums?

Table of contents

Approaches to qualitative research, qualitative research methods, qualitative data analysis, advantages of qualitative research, disadvantages of qualitative research, other interesting articles, frequently asked questions about qualitative research.

Qualitative research is used to understand how people experience the world. While there are many approaches to qualitative research, they tend to be flexible and focus on retaining rich meaning when interpreting data.

Common approaches include grounded theory, ethnography , action research , phenomenological research, and narrative research. They share some similarities, but emphasize different aims and perspectives.

Note that qualitative research is at risk for certain research biases including the Hawthorne effect , observer bias , recall bias , and social desirability bias . While not always totally avoidable, awareness of potential biases as you collect and analyze your data can prevent them from impacting your work too much.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Each of the research approaches involve using one or more data collection methods . These are some of the most common qualitative methods:

• Observations: recording what you have seen, heard, or encountered in detailed field notes.
• Interviews:  personally asking people questions in one-on-one conversations.
• Focus groups: asking questions and generating discussion among a group of people.
• Surveys : distributing questionnaires with open-ended questions.
• Secondary research: collecting existing data in the form of texts, images, audio or video recordings, etc.
• You take field notes with observations and reflect on your own experiences of the company culture.
• You distribute open-ended surveys to employees across all the company’s offices by email to find out if the culture varies across locations.
• You conduct in-depth interviews with employees in your office to learn about their experiences and perspectives in greater detail.

Qualitative researchers often consider themselves “instruments” in research because all observations, interpretations and analyses are filtered through their own personal lens.

For this reason, when writing up your methodology for qualitative research, it’s important to reflect on your approach and to thoroughly explain the choices you made in collecting and analyzing the data.

Qualitative data can take the form of texts, photos, videos and audio. For example, you might be working with interview transcripts, survey responses, fieldnotes, or recordings from natural settings.

Most types of qualitative data analysis share the same five steps:

• Prepare and organize your data. This may mean transcribing interviews or typing up fieldnotes.
• Review and explore your data. Examine the data for patterns or repeated ideas that emerge.
• Develop a data coding system. Based on your initial ideas, establish a set of codes that you can apply to categorize your data.
• Assign codes to the data. For example, in qualitative survey analysis, this may mean going through each participant’s responses and tagging them with codes in a spreadsheet. As you go through your data, you can create new codes to add to your system if necessary.
• Identify recurring themes. Link codes together into cohesive, overarching themes.

There are several specific approaches to analyzing qualitative data. Although these methods share similar processes, they emphasize different concepts.

Qualitative research often tries to preserve the voice and perspective of participants and can be adjusted as new research questions arise. Qualitative research is good for:

• Flexibility

The data collection and analysis process can be adapted as new ideas or patterns emerge. They are not rigidly decided beforehand.

• Natural settings

Data collection occurs in real-world contexts or in naturalistic ways.

• Meaningful insights

Detailed descriptions of people’s experiences, feelings and perceptions can be used in designing, testing or improving systems or products.

• Generation of new ideas

Open-ended responses mean that researchers can uncover novel problems or opportunities that they wouldn’t have thought of otherwise.

Researchers must consider practical and theoretical limitations in analyzing and interpreting their data. Qualitative research suffers from:

• Unreliability

The real-world setting often makes qualitative research unreliable because of uncontrolled factors that affect the data.

• Subjectivity

Due to the researcher’s primary role in analyzing and interpreting data, qualitative research cannot be replicated . The researcher decides what is important and what is irrelevant in data analysis, so interpretations of the same data can vary greatly.

• Limited generalizability

Small samples are often used to gather detailed data about specific contexts. Despite rigorous analysis procedures, it is difficult to draw generalizable conclusions because the data may be biased and unrepresentative of the wider population .

• Labor-intensive

Although software can be used to manage and record large amounts of text, data analysis often has to be checked or performed manually.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

There are five common approaches to qualitative research :

• Grounded theory involves collecting data in order to develop new theories.
• Ethnography involves immersing yourself in a group or organization to understand its culture.
• Narrative research involves interpreting stories to understand how people make sense of their experiences and perceptions.
• Phenomenological research involves investigating phenomena through people’s lived experiences.
• Action research links theory and practice in several cycles to drive innovative changes.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). What Is Qualitative Research? | Methods & Examples. Scribbr. Retrieved August 27, 2024, from https://www.scribbr.com/methodology/qualitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, how to do thematic analysis | step-by-step guide & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 22, Issue 1
• How to appraise qualitative research
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Calvin Moorley 1 ,
• Xabi Cathala 2
• 1 Nursing Research and Diversity in Care, School of Health and Social Care , London South Bank University , London , UK
• 2 Institute of Vocational Learning , School of Health and Social Care, London South Bank University , London , UK
• Correspondence to Dr Calvin Moorley, Nursing Research and Diversity in Care, School of Health and Social Care, London South Bank University, London SE1 0AA, UK; Moorleyc{at}lsbu.ac.uk

https://doi.org/10.1136/ebnurs-2018-103044

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

In order to make a decision about implementing evidence into practice, nurses need to be able to critically appraise research. Nurses also have a professional responsibility to maintain up-to-date practice. 1 This paper provides a guide on how to critically appraise a qualitative research paper.

What is qualitative research?

• View inline

Useful terms

Some of the qualitative approaches used in nursing research include grounded theory, phenomenology, ethnography, case study (can lend itself to mixed methods) and narrative analysis. The data collection methods used in qualitative research include in depth interviews, focus groups, observations and stories in the form of diaries or other documents. 3

Authenticity

Title, keywords, authors and abstract.

In a previous paper, we discussed how the title, keywords, authors’ positions and affiliations and abstract can influence the authenticity and readability of quantitative research papers, 4 the same applies to qualitative research. However, other areas such as the purpose of the study and the research question, theoretical and conceptual frameworks, sampling and methodology also need consideration when appraising a qualitative paper.

Purpose and question

The topic under investigation in the study should be guided by a clear research question or a statement of the problem or purpose. An example of a statement can be seen in table 2 . Unlike most quantitative studies, qualitative research does not seek to test a hypothesis. The research statement should be specific to the problem and should be reflected in the design. This will inform the reader of what will be studied and justify the purpose of the study. 5

Example of research question and problem statement

An appropriate literature review should have been conducted and summarised in the paper. It should be linked to the subject, using peer-reviewed primary research which is up to date. We suggest papers with a age limit of 5–8 years excluding original work. The literature review should give the reader a balanced view on what has been written on the subject. It is worth noting that for some qualitative approaches some literature reviews are conducted after the data collection to minimise bias, for example, in grounded theory studies. In phenomenological studies, the review sometimes occurs after the data analysis. If this is the case, the author(s) should make this clear.

Theoretical and conceptual frameworks

Most authors use the terms theoretical and conceptual frameworks interchangeably. Usually, a theoretical framework is used when research is underpinned by one theory that aims to help predict, explain and understand the topic investigated. A theoretical framework is the blueprint that can hold or scaffold a study’s theory. Conceptual frameworks are based on concepts from various theories and findings which help to guide the research. 6 It is the researcher’s understanding of how different variables are connected in the study, for example, the literature review and research question. Theoretical and conceptual frameworks connect the researcher to existing knowledge and these are used in a study to help to explain and understand what is being investigated. A framework is the design or map for a study. When you are appraising a qualitative paper, you should be able to see how the framework helped with (1) providing a rationale and (2) the development of research questions or statements. 7 You should be able to identify how the framework, research question, purpose and literature review all complement each other.

There remains an ongoing debate in relation to what an appropriate sample size should be for a qualitative study. We hold the view that qualitative research does not seek to power and a sample size can be as small as one (eg, a single case study) or any number above one (a grounded theory study) providing that it is appropriate and answers the research problem. Shorten and Moorley 8 explain that three main types of sampling exist in qualitative research: (1) convenience (2) judgement or (3) theoretical. In the paper , the sample size should be stated and a rationale for how it was decided should be clear.

Methodology

Qualitative research encompasses a variety of methods and designs. Based on the chosen method or design, the findings may be reported in a variety of different formats. Table 3 provides the main qualitative approaches used in nursing with a short description.

Different qualitative approaches

The authors should make it clear why they are using a qualitative methodology and the chosen theoretical approach or framework. The paper should provide details of participant inclusion and exclusion criteria as well as recruitment sites where the sample was drawn from, for example, urban, rural, hospital inpatient or community. Methods of data collection should be identified and be appropriate for the research statement/question.

Data collection

Overall there should be a clear trail of data collection. The paper should explain when and how the study was advertised, participants were recruited and consented. it should also state when and where the data collection took place. Data collection methods include interviews, this can be structured or unstructured and in depth one to one or group. 9 Group interviews are often referred to as focus group interviews these are often voice recorded and transcribed verbatim. It should be clear if these were conducted face to face, telephone or any other type of media used. Table 3 includes some data collection methods. Other collection methods not included in table 3 examples are observation, diaries, video recording, photographs, documents or objects (artefacts). The schedule of questions for interview or the protocol for non-interview data collection should be provided, available or discussed in the paper. Some authors may use the term ‘recruitment ended once data saturation was reached’. This simply mean that the researchers were not gaining any new information at subsequent interviews, so they stopped data collection.

The data collection section should include details of the ethical approval gained to carry out the study. For example, the strategies used to gain participants’ consent to take part in the study. The authors should make clear if any ethical issues arose and how these were resolved or managed.

The approach to data analysis (see ref  10 ) needs to be clearly articulated, for example, was there more than one person responsible for analysing the data? How were any discrepancies in findings resolved? An audit trail of how the data were analysed including its management should be documented. If member checking was used this should also be reported. This level of transparency contributes to the trustworthiness and credibility of qualitative research. Some researchers provide a diagram of how they approached data analysis to demonstrate the rigour applied ( figure 1 ).

• Download figure
• Open in new tab
• Download powerpoint

Example of data analysis diagram.

Validity and rigour

The study’s validity is reliant on the statement of the question/problem, theoretical/conceptual framework, design, method, sample and data analysis. When critiquing qualitative research, these elements will help you to determine the study’s reliability. Noble and Smith 11 explain that validity is the integrity of data methods applied and that findings should accurately reflect the data. Rigour should acknowledge the researcher’s role and involvement as well as any biases. Essentially it should focus on truth value, consistency and neutrality and applicability. 11 The authors should discuss if they used triangulation (see table 2 ) to develop the best possible understanding of the phenomena.

Themes and interpretations and implications for practice

In qualitative research no hypothesis is tested, therefore, there is no specific result. Instead, qualitative findings are often reported in themes based on the data analysed. The findings should be clearly linked to, and reflect, the data. This contributes to the soundness of the research. 11 The researchers should make it clear how they arrived at the interpretations of the findings. The theoretical or conceptual framework used should be discussed aiding the rigour of the study. The implications of the findings need to be made clear and where appropriate their applicability or transferability should be identified. 12

Discussions, recommendations and conclusions

The discussion should relate to the research findings as the authors seek to make connections with the literature reviewed earlier in the paper to contextualise their work. A strong discussion will connect the research aims and objectives to the findings and will be supported with literature if possible. A paper that seeks to influence nursing practice will have a recommendations section for clinical practice and research. A good conclusion will focus on the findings and discussion of the phenomena investigated.

Qualitative research has much to offer nursing and healthcare, in terms of understanding patients’ experience of illness, treatment and recovery, it can also help to understand better areas of healthcare practice. However, it must be done with rigour and this paper provides some guidance for appraising such research. To help you critique a qualitative research paper some guidance is provided in table 4 .

Some guidance for critiquing qualitative research

• ↵ Nursing and Midwifery Council . The code: Standard of conduct, performance and ethics for nurses and midwives . 2015 https://www.nmc.org.uk/globalassets/sitedocuments/nmc-publications/nmc-code.pdf ( accessed 21 Aug 18 ).
• Barrett D ,
• Cathala X ,
• Shorten A ,

Patient consent for publication Not required.

Competing interests None declared.

Provenance and peer review Commissioned; internally peer reviewed.

Read the full text or download the PDF:

• Privacy Policy

Home » Qualitative Research – Methods, Analysis Types and Guide

Qualitative Research – Methods, Analysis Types and Guide

Table of Contents

Qualitative Research

Qualitative research is a type of research methodology that focuses on exploring and understanding people’s beliefs, attitudes, behaviors, and experiences through the collection and analysis of non-numerical data. It seeks to answer research questions through the examination of subjective data, such as interviews, focus groups, observations, and textual analysis.

Qualitative research aims to uncover the meaning and significance of social phenomena, and it typically involves a more flexible and iterative approach to data collection and analysis compared to quantitative research. Qualitative research is often used in fields such as sociology, anthropology, psychology, and education.

Qualitative Research Methods

Qualitative Research Methods are as follows:

One-to-One Interview

This method involves conducting an interview with a single participant to gain a detailed understanding of their experiences, attitudes, and beliefs. One-to-one interviews can be conducted in-person, over the phone, or through video conferencing. The interviewer typically uses open-ended questions to encourage the participant to share their thoughts and feelings. One-to-one interviews are useful for gaining detailed insights into individual experiences.

Focus Groups

This method involves bringing together a group of people to discuss a specific topic in a structured setting. The focus group is led by a moderator who guides the discussion and encourages participants to share their thoughts and opinions. Focus groups are useful for generating ideas and insights, exploring social norms and attitudes, and understanding group dynamics.

Ethnographic Studies

This method involves immersing oneself in a culture or community to gain a deep understanding of its norms, beliefs, and practices. Ethnographic studies typically involve long-term fieldwork and observation, as well as interviews and document analysis. Ethnographic studies are useful for understanding the cultural context of social phenomena and for gaining a holistic understanding of complex social processes.

Text Analysis

This method involves analyzing written or spoken language to identify patterns and themes. Text analysis can be quantitative or qualitative. Qualitative text analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Text analysis is useful for understanding media messages, public discourse, and cultural trends.

This method involves an in-depth examination of a single person, group, or event to gain an understanding of complex phenomena. Case studies typically involve a combination of data collection methods, such as interviews, observations, and document analysis, to provide a comprehensive understanding of the case. Case studies are useful for exploring unique or rare cases, and for generating hypotheses for further research.

Process of Observation

This method involves systematically observing and recording behaviors and interactions in natural settings. The observer may take notes, use audio or video recordings, or use other methods to document what they see. Process of observation is useful for understanding social interactions, cultural practices, and the context in which behaviors occur.

Record Keeping

This method involves keeping detailed records of observations, interviews, and other data collected during the research process. Record keeping is essential for ensuring the accuracy and reliability of the data, and for providing a basis for analysis and interpretation.

This method involves collecting data from a large sample of participants through a structured questionnaire. Surveys can be conducted in person, over the phone, through mail, or online. Surveys are useful for collecting data on attitudes, beliefs, and behaviors, and for identifying patterns and trends in a population.

Qualitative data analysis is a process of turning unstructured data into meaningful insights. It involves extracting and organizing information from sources like interviews, focus groups, and surveys. The goal is to understand people’s attitudes, behaviors, and motivations

Qualitative Research Analysis Methods

Qualitative Research analysis methods involve a systematic approach to interpreting and making sense of the data collected in qualitative research. Here are some common qualitative data analysis methods:

Thematic Analysis

This method involves identifying patterns or themes in the data that are relevant to the research question. The researcher reviews the data, identifies keywords or phrases, and groups them into categories or themes. Thematic analysis is useful for identifying patterns across multiple data sources and for generating new insights into the research topic.

Content Analysis

This method involves analyzing the content of written or spoken language to identify key themes or concepts. Content analysis can be quantitative or qualitative. Qualitative content analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Content analysis is useful for identifying patterns in media messages, public discourse, and cultural trends.

Discourse Analysis

This method involves analyzing language to understand how it constructs meaning and shapes social interactions. Discourse analysis can involve a variety of methods, such as conversation analysis, critical discourse analysis, and narrative analysis. Discourse analysis is useful for understanding how language shapes social interactions, cultural norms, and power relationships.

Grounded Theory Analysis

This method involves developing a theory or explanation based on the data collected. Grounded theory analysis starts with the data and uses an iterative process of coding and analysis to identify patterns and themes in the data. The theory or explanation that emerges is grounded in the data, rather than preconceived hypotheses. Grounded theory analysis is useful for understanding complex social phenomena and for generating new theoretical insights.

Narrative Analysis

This method involves analyzing the stories or narratives that participants share to gain insights into their experiences, attitudes, and beliefs. Narrative analysis can involve a variety of methods, such as structural analysis, thematic analysis, and discourse analysis. Narrative analysis is useful for understanding how individuals construct their identities, make sense of their experiences, and communicate their values and beliefs.

Phenomenological Analysis

This method involves analyzing how individuals make sense of their experiences and the meanings they attach to them. Phenomenological analysis typically involves in-depth interviews with participants to explore their experiences in detail. Phenomenological analysis is useful for understanding subjective experiences and for developing a rich understanding of human consciousness.

Comparative Analysis

This method involves comparing and contrasting data across different cases or groups to identify similarities and differences. Comparative analysis can be used to identify patterns or themes that are common across multiple cases, as well as to identify unique or distinctive features of individual cases. Comparative analysis is useful for understanding how social phenomena vary across different contexts and groups.

Applications of Qualitative Research

Qualitative research has many applications across different fields and industries. Here are some examples of how qualitative research is used:

• Market Research: Qualitative research is often used in market research to understand consumer attitudes, behaviors, and preferences. Researchers conduct focus groups and one-on-one interviews with consumers to gather insights into their experiences and perceptions of products and services.
• Health Care: Qualitative research is used in health care to explore patient experiences and perspectives on health and illness. Researchers conduct in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
• Education: Qualitative research is used in education to understand student experiences and to develop effective teaching strategies. Researchers conduct classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
• Social Work : Qualitative research is used in social work to explore social problems and to develop interventions to address them. Researchers conduct in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
• Anthropology : Qualitative research is used in anthropology to understand different cultures and societies. Researchers conduct ethnographic studies and observe and interview members of different cultural groups to gain insights into their beliefs, practices, and social structures.
• Psychology : Qualitative research is used in psychology to understand human behavior and mental processes. Researchers conduct in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
• Public Policy : Qualitative research is used in public policy to explore public attitudes and to inform policy decisions. Researchers conduct focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

How to Conduct Qualitative Research

Here are some general steps for conducting qualitative research:

• Identify your research question: Qualitative research starts with a research question or set of questions that you want to explore. This question should be focused and specific, but also broad enough to allow for exploration and discovery.
• Select your research design: There are different types of qualitative research designs, including ethnography, case study, grounded theory, and phenomenology. You should select a design that aligns with your research question and that will allow you to gather the data you need to answer your research question.
• Recruit participants: Once you have your research question and design, you need to recruit participants. The number of participants you need will depend on your research design and the scope of your research. You can recruit participants through advertisements, social media, or through personal networks.
• Collect data: There are different methods for collecting qualitative data, including interviews, focus groups, observation, and document analysis. You should select the method or methods that align with your research design and that will allow you to gather the data you need to answer your research question.
• Analyze data: Once you have collected your data, you need to analyze it. This involves reviewing your data, identifying patterns and themes, and developing codes to organize your data. You can use different software programs to help you analyze your data, or you can do it manually.
• Interpret data: Once you have analyzed your data, you need to interpret it. This involves making sense of the patterns and themes you have identified, and developing insights and conclusions that answer your research question. You should be guided by your research question and use your data to support your conclusions.
• Communicate results: Once you have interpreted your data, you need to communicate your results. This can be done through academic papers, presentations, or reports. You should be clear and concise in your communication, and use examples and quotes from your data to support your findings.

Examples of Qualitative Research

Here are some real-time examples of qualitative research:

• Customer Feedback: A company may conduct qualitative research to understand the feedback and experiences of its customers. This may involve conducting focus groups or one-on-one interviews with customers to gather insights into their attitudes, behaviors, and preferences.
• Healthcare : A healthcare provider may conduct qualitative research to explore patient experiences and perspectives on health and illness. This may involve conducting in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
• Education : An educational institution may conduct qualitative research to understand student experiences and to develop effective teaching strategies. This may involve conducting classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
• Social Work: A social worker may conduct qualitative research to explore social problems and to develop interventions to address them. This may involve conducting in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
• Anthropology : An anthropologist may conduct qualitative research to understand different cultures and societies. This may involve conducting ethnographic studies and observing and interviewing members of different cultural groups to gain insights into their beliefs, practices, and social structures.
• Psychology : A psychologist may conduct qualitative research to understand human behavior and mental processes. This may involve conducting in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
• Public Policy: A government agency or non-profit organization may conduct qualitative research to explore public attitudes and to inform policy decisions. This may involve conducting focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

Purpose of Qualitative Research

The purpose of qualitative research is to explore and understand the subjective experiences, behaviors, and perspectives of individuals or groups in a particular context. Unlike quantitative research, which focuses on numerical data and statistical analysis, qualitative research aims to provide in-depth, descriptive information that can help researchers develop insights and theories about complex social phenomena.

Qualitative research can serve multiple purposes, including:

• Exploring new or emerging phenomena : Qualitative research can be useful for exploring new or emerging phenomena, such as new technologies or social trends. This type of research can help researchers develop a deeper understanding of these phenomena and identify potential areas for further study.
• Understanding complex social phenomena : Qualitative research can be useful for exploring complex social phenomena, such as cultural beliefs, social norms, or political processes. This type of research can help researchers develop a more nuanced understanding of these phenomena and identify factors that may influence them.
• Generating new theories or hypotheses: Qualitative research can be useful for generating new theories or hypotheses about social phenomena. By gathering rich, detailed data about individuals’ experiences and perspectives, researchers can develop insights that may challenge existing theories or lead to new lines of inquiry.
• Providing context for quantitative data: Qualitative research can be useful for providing context for quantitative data. By gathering qualitative data alongside quantitative data, researchers can develop a more complete understanding of complex social phenomena and identify potential explanations for quantitative findings.

When to use Qualitative Research

Here are some situations where qualitative research may be appropriate:

• Exploring a new area: If little is known about a particular topic, qualitative research can help to identify key issues, generate hypotheses, and develop new theories.
• Understanding complex phenomena: Qualitative research can be used to investigate complex social, cultural, or organizational phenomena that are difficult to measure quantitatively.
• Investigating subjective experiences: Qualitative research is particularly useful for investigating the subjective experiences of individuals or groups, such as their attitudes, beliefs, values, or emotions.
• Conducting formative research: Qualitative research can be used in the early stages of a research project to develop research questions, identify potential research participants, and refine research methods.
• Evaluating interventions or programs: Qualitative research can be used to evaluate the effectiveness of interventions or programs by collecting data on participants’ experiences, attitudes, and behaviors.

Characteristics of Qualitative Research

Qualitative research is characterized by several key features, including:

• Focus on subjective experience: Qualitative research is concerned with understanding the subjective experiences, beliefs, and perspectives of individuals or groups in a particular context. Researchers aim to explore the meanings that people attach to their experiences and to understand the social and cultural factors that shape these meanings.
• Use of open-ended questions: Qualitative research relies on open-ended questions that allow participants to provide detailed, in-depth responses. Researchers seek to elicit rich, descriptive data that can provide insights into participants’ experiences and perspectives.
• Sampling-based on purpose and diversity: Qualitative research often involves purposive sampling, in which participants are selected based on specific criteria related to the research question. Researchers may also seek to include participants with diverse experiences and perspectives to capture a range of viewpoints.
• Data collection through multiple methods: Qualitative research typically involves the use of multiple data collection methods, such as in-depth interviews, focus groups, and observation. This allows researchers to gather rich, detailed data from multiple sources, which can provide a more complete picture of participants’ experiences and perspectives.
• Inductive data analysis: Qualitative research relies on inductive data analysis, in which researchers develop theories and insights based on the data rather than testing pre-existing hypotheses. Researchers use coding and thematic analysis to identify patterns and themes in the data and to develop theories and explanations based on these patterns.
• Emphasis on researcher reflexivity: Qualitative research recognizes the importance of the researcher’s role in shaping the research process and outcomes. Researchers are encouraged to reflect on their own biases and assumptions and to be transparent about their role in the research process.

Advantages of Qualitative Research

Qualitative research offers several advantages over other research methods, including:

• Depth and detail: Qualitative research allows researchers to gather rich, detailed data that provides a deeper understanding of complex social phenomena. Through in-depth interviews, focus groups, and observation, researchers can gather detailed information about participants’ experiences and perspectives that may be missed by other research methods.
• Flexibility : Qualitative research is a flexible approach that allows researchers to adapt their methods to the research question and context. Researchers can adjust their research methods in real-time to gather more information or explore unexpected findings.
• Contextual understanding: Qualitative research is well-suited to exploring the social and cultural context in which individuals or groups are situated. Researchers can gather information about cultural norms, social structures, and historical events that may influence participants’ experiences and perspectives.
• Participant perspective : Qualitative research prioritizes the perspective of participants, allowing researchers to explore subjective experiences and understand the meanings that participants attach to their experiences.
• Theory development: Qualitative research can contribute to the development of new theories and insights about complex social phenomena. By gathering rich, detailed data and using inductive data analysis, researchers can develop new theories and explanations that may challenge existing understandings.
• Validity : Qualitative research can offer high validity by using multiple data collection methods, purposive and diverse sampling, and researcher reflexivity. This can help ensure that findings are credible and trustworthy.

Limitations of Qualitative Research

Qualitative research also has some limitations, including:

• Subjectivity : Qualitative research relies on the subjective interpretation of researchers, which can introduce bias into the research process. The researcher’s perspective, beliefs, and experiences can influence the way data is collected, analyzed, and interpreted.
• Limited generalizability: Qualitative research typically involves small, purposive samples that may not be representative of larger populations. This limits the generalizability of findings to other contexts or populations.
• Time-consuming: Qualitative research can be a time-consuming process, requiring significant resources for data collection, analysis, and interpretation.
• Resource-intensive: Qualitative research may require more resources than other research methods, including specialized training for researchers, specialized software for data analysis, and transcription services.
• Limited reliability: Qualitative research may be less reliable than quantitative research, as it relies on the subjective interpretation of researchers. This can make it difficult to replicate findings or compare results across different studies.
• Ethics and confidentiality: Qualitative research involves collecting sensitive information from participants, which raises ethical concerns about confidentiality and informed consent. Researchers must take care to protect the privacy and confidentiality of participants and obtain informed consent.

Also see Research Methods

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

One-to-One Interview – Methods and Guide

Questionnaire – Definition, Types, and Examples

Explanatory Research – Types, Methods, Guide

Triangulation in Research – Types, Methods and...

Mixed Methods Research – Types & Analysis

Correlational Research – Methods, Types and...

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Online First
• Strategies for adapting under pressure: an interview study in intensive care units
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-9937-6176 Bethan Page 1 , 2 ,
• Dulcie Irving 1 ,
• http://orcid.org/0000-0003-2042-7389 Jane Carthey 3 ,
• John Welch 4 ,
• http://orcid.org/0000-0001-5796-0377 Helen Higham 5 , 6 ,
• http://orcid.org/0000-0003-0270-0222 Charles Vincent 1
• 1 Department of Experimental Psychology , University of Oxford , Oxford , Oxfordshire , UK
• 2 Cicely Saunders Institute , King's College London , London , Greater London , UK
• 3 Human Factors and Patient Safety , Jane Carthey Consulting , Chiswick , UK
• 4 National Institute for Health and Care Research Central London Patient Safety Research Collaborative , University College London Hospitals NHS Foundation Trust , London , UK
• 5 Nuffield Department of Clinical Neurosciences , University of Oxford , Oxford , UK
• 6 Nuffield Department of Anaesthetics , Oxford University Hospitals NHS Foundation Trust , Oxford , UK
• Correspondence to Dr Bethan Page; bethan.page{at}kcl.ac.uk

Background Healthcare systems are operating under substantial pressures. Clinicians and managers are constantly having to make adaptations, which are typically improvised, highly variable and not coordinated across teams. This study aimed to identify and describe the types of everyday pressures in intensive care and the adaptive strategies staff use to respond, with the longer-term aim of developing practical and coordinated strategies for managing under pressure.

Methods We conducted qualitative semi-structured interviews with 20 senior multidisciplinary healthcare professionals from intensive care units (ICUs) in 4 major hospitals in the UK. The interviews explored the everyday pressures faced by intensive care staff and the strategies they use to adapt. A thematic template analysis approach was used to analyse the data based on our previously empirically developed taxonomy of pressures and strategies.

Results The principal source of pressure described was a shortage of staff with the necessary skills and experience to care for the increased numbers and complexity of patients which, in turn, increased staff workload and reduced patient flow. Strategies were categorised into anticipatory (in advance of anticipated pressures) and on the day. The dynamic and unpredictable demands on ICUs meant that strategies were mostly deployed on the day, most commonly by flexing staff, prioritisation of patients and tasks and increasing modes of communication and support.

Conclusions ICU staff use a wide variety of adaptive strategies at times of pressure to minimise risk and maintain a reasonable standard of care for patients. These findings provide the foundation for a portfolio of strategies, which can be flexibly employed when under pressure. There is considerable potential for training clinical leaders and teams in the effective use of adaptive strategies.

• Risk management
• Critical care
• Qualitative research
• Crisis management

Data availability statement

No data are available. The participants of this study did not give written consent for their interview transcripts to be shared beyond our research team. Due to the sensitive nature of the research, supporting data is not available.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:  https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/bmjqs-2024-017385

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Due to increasing pressures on the health system, clinical teams have to adapt everyday practice in order to manage risk and maintain patient flow, but there are currently very few developed strategies to manage these pressures.

WHAT THIS STUDY ADDS

This study empirically presents a menu of strategies used in intensive care for adapting under pressure gathered through interviews with different professionals involved in patient care and the running of the service.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

These findings could be used as the basis of training programmes for intensive care units to develop a set of coordinated strategies for adapting under pressure.

Introduction

Health services have been under increasing pressure for many years as the result of an ageing population, increasing complexity of illnesses and comorbidities, as well as a shortage of resources of all kinds (particularly skilled and experienced staff). The intensive care unit (ICU) is a particularly pressurised environment that requires frequent adaptation to maintain patient safety when placed under pressure. 1 2 The aftermath of the recent COVID-19 pandemic saw many nursing staff exiting the National Health Service (NHS), leaving ICUs chronically short of skilled, experienced staff. 3 4 Healthcare system pressures have tangible effects on the way work is done, staff experience and patient care. 5 6

Clinical teams adapt to these everyday pressures to minimise risk to patients, and in most cases, achieve good outcomes despite adverse circumstances. 6 While individual adaptations of care may be reasonable and necessary, the overall effect of multiple adaptations on the quality of care is variable. 7 We have previously argued that clinical leaders need to employ a portfolio of strategies when pressures are high and develop a coordinated approach for teams to deliver safe care and service efficiency within available resources. 5

Our previous review of empirical resilient healthcare studies explored the nature of adaptations to care at times of pressure and created a taxonomy of pressures and adaptations. 8 This review built on earlier work on resilient healthcare and adaptive capacity. 8–13 Our review found that the primary source of pressure is a mismatch between demand and capacity. Working conditions then become more difficult which in turn increases risk to patients and creates more pressure on staff. We found that adaptive strategies could be divided into actions taken in anticipation of pressures and actions taken on the day. The primary types of adaptive strategy, both in advance and on the day, concerned methods for increasing or flexing resources (e.g adapting staff skill mix), approaches to controlling demand and adaptations to the standard and manner in which care is delivered.

The studies included in the previous review contained useful descriptions of adaptations but are focused more on understanding and advancing resilience theory. This current paper seeks to describe strategies used by front-line clinicians so that they may be developed and used in a practical sense to help reduce risks to patient safety and burden on staff, rather than to further advance resilience theory. Only two studies included in our previous review examined intensive care. 14 15 In the current study, we explore in depth how different teams in ICUs are adapting to everyday pressures and describe the adaptive strategies they use to respond, using the previously published taxonomy as a framework for the analysis. The aim of the present study is therefore to explore the everyday pressures experienced in multiple adult ICUs and to identify ways in which intensive care teams adapt clinical practice to meet demand while managing risks to patients.

Study design

Semistructured interviews were conducted with senior healthcare professionals (defined as those with line-management responsibility for a team and/or the running of a service) working in adult ICUs in four hospitals across England to identify the types of everyday pressures experienced and the adaptive strategies used. Reporting of the methods follows the Consolidated Criteria for Reporting Qualitative Research guidelines checklist for interviews and focus groups.

Sampling and setting

Four acute hospitals from across England were purposively sampled to capture different geographical locations, size and type of hospital and population demographics. Three were large teaching hospitals with multiple ICUs split across different sites and one was a District General Hospital with a smaller ICU. As a collective, they provided adult general, surgery, cancer and haematology intensive care services. All ICUs had intensive care outreach services attached or a Medical Consultant assigned to outreach for each shift. Data collection was completed between September 2022 and October 2023.

We purposively sampled senior staff from across disciplines working in ICU (see definition above), as they have more autonomy and requirement to make system-level adaptations than more junior staff. We identified a lead contact known to the researchers in each hospital for consultation, suggestions of interviewees and communication of findings. We approached 25 members of ICU teams, with 20 agreeing to be interviewed and no response from five. After 20 interviews, it was agreed that data saturation had been reached. The final sample consisted of medical consultants (n=5, including one head of department), matron or nursing leads (n=9, including senior outreach staff and an ICU manager), senior nurses (n=4) and senior physiotherapists (n=2).

Data collection

A semistructured interview guide was developed, informed by the findings from our recent scoping review on resilient healthcare 8 and adapted in response to two pilot interviews ( online supplemental file 1 ). Participants were invited to take part via email and sent a full information sheet. Verbal consent was obtained at the beginning of each interview, which included permission to record the interview for the purpose of generating a transcript. Interviews were conducted by a combination of two of the research team: a human factors and patient safety consultant (JC) and researchers experienced in qualitative methods and healthcare research (BP and DI). Semistructured interviews were conducted over video call, audio recorded and transcribed verbatim. Field notes were taken during the interviews to follow-up on points of interest or for clarification. Each interview lasted approximately 1 hour. No repeat interviews were conducted.

Supplemental material

The data were analysed using a thematic template approach. 16 17 The qualitative data management tool NVivo was used to manage and code the interview data. The first stage of analysis was data familiarisation: BP and DI read the interview transcripts and shared initial reflections and preliminary coding strategies during a series of meetings with other members of the research team (CV and JC). In the second phase, we created and iteratively developed a framework for organising the data, drawing on the interview guide and our previously developed taxonomies of pressures and strategies. 8 The resulting coding framework provided the major theme headings for the analysis ( online supplemental file 2 ). The framework was piloted on a sample of four interviews and was found to capture the key pressures and strategies described. Data from each transcript were then indexed through systematically coding quotations and placing them in one (or more) of the framework categories. BP and DI led the data analysis, and CV and JC provided oversight. The interpretation of the data was sense-checked by clinicians HH and JW. The research team met regularly to discuss the analysis and coding framework.

Reflexivity and quality assurance

The researchers who conducted the interviews and led the data analysis were health services researchers with expertise and graduate degrees in the field of patient safety. The two coders independently coded the interviews and compared their results through discussion, providing assurance for the consistency of coding and interpretation. Emerging findings were discussed in regular research team meetings. The results were developed with input from two highly experienced clinicians (a consultant ICU nurse and a consultant anaesthetist) who supported the interpretation of the data, clarified clinical issues and which led to a deeper and more nuanced understanding of the data. The findings were presented and discussed with a wider group of front-line intensive care clinicians as a sense-check.

The pressures were broadly categorised into (1) demand exceeding capacity, (2) difficult working conditions and (3) problems with system functioning. 8

Those interviewed reported that the gap between demand and capacity has steadily increased in recent years, particularly post pandemic, because of the shortage of skilled, experienced ICU nurses. This has created severe skill-mix problems, leaving a very junior workforce expected to take on more responsibilities at an earlier stage of their careers and increasing the workload for the senior nurses who remain. Bed shortages were commonly cited, but the primary difficulty is the shortage of nurses to staff those beds rather than physical capacity. There are in addition multiple interactions and feedback loops between the different pressures and problems. For example, shortages of skilled staff and workload pressures often results in delayed assessment, missed or delayed care (for instance, missed medication) leading to longer patient stays which in turn reduces capacity to cope with new patients coming into the system. Figure 1 demonstrates this dynamic process and summarises the principal pressures described by intensive care teams.

• Download figure
• Open in new tab
• Download powerpoint

Interrelationships between pressures commonly experienced in intensive care.

Anticipatory strategies

In anticipation of increasing pressures, staff endeavoured to adjust resources to meet anticipated demand and make plans for managing the workload (see table 1 and box 1 for examples). Plans for controlling demand and adapting the delivery of care are typically made a few days in advance, whereas plans to increase resources tend to be longer term.

• View inline

Anticipatory strategies used in ICUs with illustrative examples

Anticipatory strategies in intensive care units

Increasing staff and opening services to increase capacity and reduce delays in care

“So did a workforce review last year and pushed forward a business case to increase by an additional practitioner, which allowed me to open up weekend working. That’s made a significant difference to our day-to-day workflow. It now means that when we come in on a Monday, we’re not coming in to upwards of nine to ten brand-new patients that have been sitting greater than 48 hours waiting for an assessment. We’re coming in to patients that have been seen over the weekend and risks are reduced” (Outreach Clinical Lead)

Creating or adapting protocols to help solve staffing skills and experience profile change post-pandemic

“We've really gone into overdrive in developing more visual and easier to see protocols, SOPs [Standard Operating Procedure], that kind of thing. We were moving towards that anyway. It's definitely gone up a gear, the view that people need to have that stuff to remind them of essentials of care…now we have a much bigger set of nicely coloured photographs and illustrated guides. They're either laminated or put onto a phone system. We have an app that you can call up some of these protocols and guides. We've also linked the same products to the electronic patient records that people can call up there too. There's now a whole set of ways of accessing that stuff.” [Consultant Nurse]

Planned meetings to monitor and coordinate contingency plans to prepare for sudden stresses on the system

“So we have our safety huddle in the morning. We will have a what-are-we-going-to-do-if conversation. We may have a day when we feel quite flush with staff, but we always have that awareness that even if you have two more members of staff, if the emergency goes much more than the system can cope with, you’ve got to know what you’re going to do, which is your plan B. How are you going to split out your resources? And that doesn’t come down to just numbers of staff. It comes down to which staff you’ve got, and which skill set they’ve got.” [Outreach Clinical Lead]

Increase resources

Efforts to increase resources in intensive care mostly focused on increasing staffing and improving skill mix rather than on beds, equipment or supplies. There were large recruitment drives underway for international nursing graduates in two of the hospitals to help fill the vacancies and address increased demand. New job roles had been created in some units. For instance, a new non-clinical team had been introduced in one hospital to manage the stores, allowing nurses to focus on patient care. All hospitals had invested in education and professional development teams, with special attention given to supporting and enhancing the skills of new international nurses.

Control demand

Forecasting was used by most units to anticipate and control demand. For instance, teams would assess how stable the staffing rota looked in the next 24–48 hours before agreeing that they could safely admit a new patient. In some cases, units employed their outreach teams to identify patients on wards who were at risk of deterioration and admit them earlier if the ICU had capacity or to support their care on the ward if there was no ICU capacity. Identifying patients ready for discharge was a daily priority, for which one hospital had developed a traffic light dashboard system to reduce the cognitive load of monitoring the status of the unit and making these decisions.

The criteria for entering and being discharged from intensive care were constantly flexed depending on the degree of pressure. Clinicians assessed patients for discharge home or transfer to another hospital, considering available support, time of day and day of week, and family factors such as distance to visit. At times of very short staffing, or on occasions such as Christmas, some units closed parts of the ICU in order to concentrate the available resources in one area. Some units, setting a completely new precedent, had now begun to discharge patients directly home from the ICU to be followed up by the outreach team at home over phone call.

Plans for managing the workload

Clinical protocols were standardised and simplified to make them more accessible and usable by junior nurses, so that more senior nurses could be used to supervise nurses with less experience of intensive care. Multiple fixed meetings were held throughout the day to monitor resources, communicate concerns and identify priorities. Technological aids such as walkie-talkies were introduced to make communication between side rooms or requesting blood from the lab more efficient. Education and training were delivered where possible by incorporating them alongside clinical work (eg, ‘tea trolley teaching’).

On-the-day adaptations

Forecasts and plans often changed on the day in intensive care, and so clinical teams had to be adaptable and amenable to change. Table 2 and box 2 illustrate strategies deployed to flex resources, prioritise demand and adapt ways of working on the day.

On-the-day adaptations used in ICUs with illustrative examples

On-the-day adaptations in intensive care

Flexing resources to provide safest care within the available resources

“We have a cardiac ICU, so at weekends they’re more helpful than in the week because in the week, if we’re transferring to cardiac ICU, it means cancelling a patient the next day. Whereas at weekend they’re not operating, we can utilise their beds occasionally, but we need to try and get them out by Monday so that they can carry on operating.” [Consultant]

Taking a step back to prioritise and reprioritise

“I guess taking five minutes to just reprioritise. Especially as a coordinator, you can get a bit swamped, but even at the bedside, just taking two minutes away is something I was taught when I started coordinating. Sometimes just go for a wee, regardless of whether you need to wee. Just go and lock yourself in a room, just shut the door, have a look at your handover sheet, what tasks still need doing, what haven’t I done, what haven’t I thought of? What still needs doing within the shift and how am I going to reprioritise that?” [Senior Nurse]

Regular contact with operations teams to work with the wider system

“We utilise the duty manager team, the operational team, who, if we are starting to feel pressure and we have patients who could be elsewhere, then when our pressures change, we’ll contact them again. And it may be that our risk has been increased because they’re looking at the hospital as a whole. We’ve admitted two patients in the last hour, which means we have no emergency space, whereas the surgical ward was full but actually now our risk is greater. So, they’ll take a patient back to surgery, which will reduce our risk down. So, we regularly contact. Throughout the day, we’re in contact with them as well to see whether they can create some flow back into the hospital again.” [Deputy Matron]

Providing additional support to staff and consolidating learning

“It’s about making sure that staff understand that I don’t like to put their head in the lion’s mouth, unless they’ve got a big helmet on. Have you got what you need? And some of it is emotional, but some of it is about making sure that they’re equipped adequately… It’s also celebrating when things are done really well. So if we come away going, that really went well, it’s saying to the team, why do you think that particular arrest went really well, and the other ones didn’t? So that we can also do that positive reporting.” [Outreach]

Flex resources

Decisions were often made on the day on how best to allocate the resources available to relieve pressures and maintain safety. During day shifts, last minute shortages might sometimes be remedied by borrowing staff from other areas of the hospital (eg, surgery or education teams). During night shifts, when the workforce is significantly reduced, staff relied more on task-switching strategies (eg, doctors helping nurses to turn patients). Adaptive reallocation of staff and adjustments to nurse–patient ratios were also deployed throughout the day according to staff experience and patient acuity. Participants spoke about continually assessing the safest place for patients at any given time, looking at the hospital as a whole and holding, transferring or relocating patients if pressures were high in the unit. There was a close collaboration within one group of hospitals who constantly monitored acuity and available resources across the network. However, it was understood that all possible avenues within a hospital would be exhausted before transferring a patient elsewhere.

Prioritise demand (patients and tasks)

Patient care was prioritised by clinical urgency based on clinical judgement and experience. This was especially difficult for more junior members of the team who were less able to judge which tasks were essential and which could potentially be delayed or missed. On busy days, some activities were deprioritised. For example, washing patients or education/training for staff would be postponed or cancelled so that necessary clinical care could be given.

Adapt ways of working

Clinical leadership and effective communication and teamwork are vital in intensive care at any time, but particularly important when care has to be constantly adapted to meet demand and manage risks. At times of high pressure, leaders spent more time on the unit to provide support at the bedside, identify emerging problems or tasks being missed. Leaders also ensured that staff took breaks and had food. Doctors who were new to the senior role on the unit were learning who in the hospital to call for support in a crisis.

Leaders of teams emphasised that careful listening and absolute clarity of communication was vital when under pressure—verbalise your own understanding of the patient and listen carefully to what others are saying. A senior leader explained that they would deliberately speak in a way that conveyed a sense of calm, speaking more slowly to reassure others that everything was under control. Methods of instant communication (eg, WhatsApp) were heavily relied on in these circumstances for quick updates or request urgent support.

Deploying a portfolio of strategies

We have described and classified a considerable number of strategies, which might imply that clinical teams select particular strategies to employ at any one time. However, clinical leaders most often employ a variety of strategies in combination to dynamically manage pressure ( box 3 ), though the approach varies considerably between individuals, teams and between hospitals. Furthermore, leaders are also having to decide which rules can be broken or temporarily suspended (eg, nurse–patient ratios) and which aspects of care can be sacrificed (eg, patient comfort) in order to manage risks and provide the best care possible within the available resources.

Case study. Dynamic use of strategies when demand exceeds resources (the sliding tile puzzle)

“You’ll get requests for admission, should we bring this patient to the ICU, and then you’ve got to talk to the nurse and see if you’ve got a physical bed available and, if there is, is that staffed? If it’s not staffed, can we rearrange the patients on the ICU to optimise the nurse utilisation?

And sometimes that might involve doing things like saying, look, can you phone infection control and see if any of these patients can be de-isolated, or maybe what we’ll do is we don’t really believe this isolation is terribly important, so even though it’s strictly by the rules to keep them, we’re just going to break that because that’s not important.

You may have people who you may discharge slightly earlier than you would otherwise. Some people, you say, well, maybe I’ll keep them an extra 12 hours just to be safe, and you’ll say, look, I’m going to forego that extra bit and I’ll just use the outreach team to follow them up on the ward and just deal with it that way.

Or, for instance, if it’s an admission from another hospital, of course you can just say, no, we can’t, there’s no room at the inn, try someone else. Or can we delay the admission? If it’s a ward patient, maybe we can keep them on the ward and use outreach to follow them up. It kind of depends case by case on how badly does, if you like, the incoming patient need the ICU compared to the patients already on the ICU, compared to can we stretch the nursing ratios? That’s the other thing.

Are there a couple of patients where, actually, strictly speaking, you need one to one, but actually they’re pretty well, and why don’t we do them one to two? And they are very specific to the patients, I would say. I guess, I haven’t really thought about it, there must be some sort of broadly, loosely held kind of idea of the order in which the rules might be broken.

For instance, if we really overflow, then of course you can say, well, we’ll keep them in the theatre recovery, for instance, and nurse them there, if physical beds were a problem.” [Consultant]

The principal source of pressure identified in ICUs is a shortage of staff with the necessary skills and experience to be able to cope with the increased numbers and complexity of patients they were receiving, which had a knock-on effect on numbers of open beds available, staff workload and patient flow. Staff deploy a portfolio of strategies to respond to these pressures, with some deployed in advance of anticipated pressures and some on the day to manage immediate pressures. Clinical staff described a wide variety of anticipatory strategies to increase resources, control demand and adapt the way in which care was delivered. Flexing resources on the day was widely used, with many examples of task shifting between different job roles and levels of seniority. Coordinating with other areas of the hospital and use of outreach teams were vital ways in which ICUs could discharge early or delay admissions to manage demand.

A particular challenge for clinical leaders was to minimise the burden on the reduced senior staff available to supervise an inexperienced junior workforce. Adapting to a less experienced workforce required more senior presence, increased directness and clarity of communication and the development of simpler standardised protocols to cover the basics of care. Clinical leaders needed to provide much more guidance on prioritisation of patients and be explicit about which tasks were safety critical and which could be delayed or missed. However, patients were nevertheless sometimes being cared for by nurses without the expected levels of skills and experience. Effective supervision in a variety of contexts has been shown to enhance safety when patients are cared for by less experienced staff. 18 19

Adaptive strategies almost always come at a cost, and some strategies cannot be used indefinitely without damaging the service in the longer-term. 7 There is a risk that temporary adjustments become long-term normalised deviations from best practice, 20 leading to a gradual but cumulative degradation of the system. For example, some ICUs deploy clinical skills practitioners, whose primary role is education and training, as a buffer when staff shortages occur. Clinical skills practitioners are experienced nurses who are highly capable at providing direct patient care, but the longer-term impact of this reallocation is a reduction in clinical skills training and a reduction in the overall skill level. 15 21 This highlights the need to clearly identify adaptations and set limits on the time that they can be employed. 5 14 20

Taking what works: evolving through adaptive responses to pressure

Although not the focus of the study, the timing of the interviews meant that many interviewees spoke about adaptive strategies deployed during the COVID-19 pandemic, which had been adopted into everyday practice. 7 Experience during the pandemic had highlighted the value of safety huddles and daily sitreps at times of high pressure. The pandemic had also led to new ways of monitoring patient status and staffing pressures, which enabled senior staff to control demand and flex resources more efficiently.

Considerable efforts had been made to simplify and standardise clinical protocols to make them more accessible to the junior staff with increased responsibility for very sick patients. The precise formulation of rules and standard operating procedures has been a longstanding concern in other high-risk industries. 22 Standardisation in the care of complex patients must be approached cautiously and an insensitive demand to follow guidelines in all circumstances can indeed be detrimental to patient safety. 23 However, articulating and explaining the routines used by expert clinicians is a powerful means of supporting less experienced staff and enhancing the safety of care. 24

Increased pressures in this environment has fostered a greater understanding of the need for support for intensive care staff due both to the inherently stressful nature of the work and to the increasingly frequent but inevitable departure from basic professional standards. 3 25 Some departures from standards are often accepted as necessary in the short-term, but in the longer-term, staff may experience distress and moral injury from not being able to meet personal and professional standards. 26 Psychologist and well-being teams have grown considerably in recent years with dedicated support for intensive care staff in all four hospitals. 26 27 There is a critical role for leaders, both executive and clinical, in discussing such compromises openly and supporting teams faced with unenviable decisions. The risk of moral injury will be less if such decisions are seen as a necessary collective decision rather than an individual personal failing. 26

Strengths and limitations

We purposely sampled four hospitals that were diverse in size and location, and across different professions, to provide a breadth of perspectives regarding the nature of pressures experienced and strategies used. Most participants were relatively senior, in that they were able to initiate system-level adaptations, 28 although of course more junior staff are adapting every day as well. Our choice of interviews for data collection meant that participants could discuss and reflect on adaptive strategies, which methodologies such as ethnography used in other similar studies are less suited to; conversely there may be some strategies that could be observed through ethnographic methods but are not easily describable by participants in interviews which we may have missed. Conducting the study relatively soon after the COVID-19 pandemic, which disproportionately affected intensive care staff more than other hospital settings, might have had some bearing on the types of pressures and strategies described. We were also not able to evaluate the effectiveness or impact and unintended consequences of the various strategies, which needs to be a priority for future research.

Future research on adaptive strategies

The taxonomies of pressures and strategies previously developed by the research team 8 provided an effective framework to categorise the results of this research: the framework was able to capture the key strategies and pressures described by participants. Future research is needed to test whether the framework works well in other clinical contexts such as primary care or mental health settings and also in other countries where healthcare team cultures and ways of working can be very different. 8 29 Research is also needed to understand how certain strategies can be linked with certain pressures, building on recent work by Sanford et al . 6

Within intensive care specifically, further research is needed to explore the effectiveness of different combinations of strategies. There may be certain strategies or combinations of strategies that are better than others or have differential trade-offs and impact on safety, staff well-being, patient flow and patient experience. 2 5 6 9 30 A strategy may for instance reduce risk for patients but increase burden on staff: this could be explored further through vignette-based studies, as has been explored by others. 2 Programmes could be established to test and evaluate different combinations of strategies in response to familiar pressures experienced in intensive care. 5

There is also a need to assess the benefits and risks of specific adaptive safety, such as discharging patients from ICU directly home. 31 32 Outreach services and acute hospital at home teams were increasingly being used to provide follow-up support for patients at home. Patients were carefully selected based on the complexity of their condition and the support from both family and professionals at home. However, if this becomes a more widely used approach, there is a risk of over-burdening families and exporting risk out of hospitals and into the home. 33 34

Training in adaptive strategies and working under pressure

The use of adaptive strategies is often learnt experientially on the job, highly individualised and not explicitly taught. 1 14 The strategies described here could help clinicians and managers respond to similar pressures, providing a portfolio of coordinated strategies that clinical teams could use or develop for their own contexts.

Many ICUs already have some form of simulation and skills training, which would naturally lend itself to training in team and system-level adaptations. For example, scenario-based teaching around managing competing demands could help to reduce stress and uphold safe practices when individuals have to make strategic decisions quickly in pressurised situations. 35 36 Wider, more formal training programmes will require organisations, and indeed regulators, to explicitly acknowledge the difficulties of maintaining standards of care when under pressure and see such training as a necessary form of proactive risk management. Future work will explore what this type of training might look like and how it could be organised, with attention given to efficacy, trade-offs and implications of a menu of strategies.

Intensive care is a highly pressurised environment which requires frequent adaptation to maintain patient safety. While these adaptive strategies are necessary and aimed at providing better care in the short-term, they also involve substantial risks to patients and potential longer-term degradation of standard of care. Our aim in this study however is not simply to point to the adaptations but to pave the way for a more open, transparent, coordinated and time-limited approach to adaptations. The strategies currently employed are highly variable, often improvised and often not shared within the clinical team. We believe that patients will be safer if we develop prepared and coordinated strategies, where the team has agreed an approach for managing risk under pressure. 5

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

This study involves human participants. This project was reviewed by the Oxford University Research Ethics Committee and classed as a service evaluation. As such, it was not subject to the Department of Health’s UK Policy Framework for Health and Social Care Research (2017) and a full ethics review. Participants gave informed consent to participate in the study before taking part.

• Hollnagel E
• Reader TW ,
• Simms-Ellis R ,
• Grange A , et al
• Endacott R ,
• Pattison N ,
• Dall’Ora C , et al
• Amalberti R ,
• Sanford N ,
• Lavelle M ,
• Markiewicz O , et al
• Knutsen Glette M ,
• Wiig S , et al
• Amalberti R , et al
• Rasmussen J
• Vincent C ,
• Anderson JE ,
• Back J , et al
• Hybinette K ,
• Pukk Härenstam K ,
• Saurin TA ,
• Bueno WP , et al
• Thorogood N
• Pistrang N ,
• Snowdon DA ,
• Leggat SG , et al
• Hennus MP ,
• Hennessy M , et al
• Martin D , et al
• Mannion R ,
• Exworthy M ,
• Wilkinson D
• Javed M , et al
• Carthey J , et al
• Spurgeon P ,
• Donnelly R ,
• Parvez S , et al
• Stelfox HT ,
• Niven DJ , et al
• Amalberti R
• Harrop E , et al
• Baxendale B

X @bethanpage21, @HFjane

Contributors BP, DI, JC and CV conceptualised the study and contributed to its design. JC, BP and DI conducted the interviews. BP and DI carried out the analyses, supervised by CV and with input from HH, JW and JC. BP and DI drafted the manuscript supervised by CV and with input from JC, HH and JW. All authors reviewed and agreed on the current version. BP is the guarantor.

Funding This study was funded by National Institute for Health Research Policy Research Programme (PR-PRU-1217-20702).

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Investigating the effectiveness of endogenous and exogenous drivers of the sustainability (re)orientation of family smes in slovenia: qualitative content analysis approach.

1. Introduction

2. literature review, 2.1. legal framework on sustainable corporate governance (with a focus on smes), 2.1.1. corporate sustainability reporting directive, 2.1.2. corporate sustainability due diligence directive, 2.1.3. scope of the csddd for smes, 2.2. drivers of the family businesses’ (re)orientation towards sustainability, 2.3. endogenous drivers, 2.3.1. the protection of sew, 2.3.2. ownership and management composition, 2.3.3. values, beliefs and attitudes of family owner-managers, 2.3.4. transgenerational continuity and long-term orientation, 2.3.5. knowledge of sustainability, 2.4. exogenous drivers, 2.4.1. stakeholders pressure, 2.4.2. the impact of institutional environment and local communities, 3. empirical research, 3.1. institutional context of slovenia, 3.2. research method, 3.3. sampling and data collection, 3.4. data analysis, 4.1. results of the final coding of the family businesses’ sustainability (re)orientation, 4.2. references to responsibility, preserving (natural) environment and sustainability/sustainable development in the analysed statements, 4.3. family businesses with a higher level of sustainability awareness and orientation, 5. discussion, 5.1. sustainability awareness and readiness of investigated family smes to comply with the new eu legal framework, 5.2. the effectiveness of endogenous and exogenous drivers of family businesses’ sustainability (re)orientation, 6. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of The Regions, An SME Strategy for a sustainable and digital Europe, COM/2020/103 final. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM%3A2020%3A103%3AFIN (accessed on 25 June 2024).
• Al Malki, M.A. Review of Sustainable Growth challenges faced by Small and Medium Enterprises. Int. J. Glob. Acad. Sci. Res. 2023 , 2 , 35–43. [ Google Scholar ] [ CrossRef ]
• European Parliament and the Council. Directive (EU) 2022/2464 of the European Parliament and of the Council of 14 December 2022 amending Regulation (EU) No 537/2014, Directive 2004/109/EC, Directive 2006/43/EC, and Directive 2013/34/EU. OJ L 322, 16.12.2022, 15–80. Available online: https://eur-lex.europa.eu/eli/dir/2022/2464 (accessed on 21 August 2024).
• Fogarty, V.; Flucker, S. Sustainability. In Data Centre Essentials: Design, Construction, and Operation of Data Centres for the Non-expert ; Wiley Online Library: Hoboken, NJ, USA, 2023. [ Google Scholar ] [ CrossRef ]
• Primec, A.; Belak, J. Sustainable CSR: Legal and Managerial Demands of the New EU Legislation (CSRD) for the Future Corporate Governance Practices. Sustainability 2022 , 14 , 16648. [ Google Scholar ] [ CrossRef ]
• Umantsiv, H. Reporting on sustainable development in the context of corporate social responsibility. Cientia Ructuosa 2023 , 149 , 59–71. [ Google Scholar ] [ CrossRef ]
• European Parliament and the Council. Directive of the European Parliament and of the Council on Corporate Sustainability Due Diligence and amending Directive (EU) 2019/1937 and Regulation (EU) 2023/2859. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CONSIL:PE_9_2024_REV_1 (accessed on 25 June 2024).
• Proposal for a Directive on Corporate Sustainability Due Diligence and amending Directive (EU) 2019/1937. COM/2022/71 final. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52022PC0071 (accessed on 29 June 2024).
• Sørensen, K.E. Corporate Sustainability Due Diligence in Groups of Companies. Eur. Co. Law 2022 , 19 , 119–130. [ Google Scholar ] [ CrossRef ]
• Arena, C.; Michelon, G. A matter of control or identity? Family firms’ environmental reporting decisions along the corporate life cycle. Bus. Strategy Environ. 2018 , 27 , 1596–1608. [ Google Scholar ] [ CrossRef ]
• Dekker, J.; Hasso, T. Environmental Performance Focus in Private Family Firms: The Role of Social Embeddedness. J. Bus. Ethics 2016 , 136 , 293–309. [ Google Scholar ] [ CrossRef ]
• Sharma, P.; Sharma, S. The Role of Family Firms in Corporate Sustainability. In The Oxford Handbook of Management Ideas ; Sturdy, A., Hueskinveld, S., Reay, T., Strang, D., Eds.; Oxford Handbooks Online: Oxford, UK, 2019; pp. 1–18. Available online: https://www.academia.edu/44598484/The_Role_of_Family_Firms_in_Corporate_Sustainability (accessed on 10 October 2023).
• European Family Business Barometer. EFB and KPMG Enterprise. Available online: http://www.europeanfamilybusinesses.eu/uploads/Modules/Publications/european-family-business-barometer7ed.pdf (accessed on 25 January 2023).
• Report on family businesses in Europe. Brussel: Committee on Industry, Research and Energy. Available online: https://www.europarl.europa.eu/doceo/document/TA-8-2015-0290_EN.pdf (accessed on 1 June 2024).
• Berrone, P.; Cruz, C.; Gomez-Mejia, L.R.; Larraza-Kintana, M. Socioemotional Wealth and Corporate Responses to Institutional Pressures: Do Family-Controlled Firms Pollute Less? Adm. Sci. Q. 2010 , 55 , 82–113. [ Google Scholar ] [ CrossRef ]
• Canavati, S. Corporate social performance in family firms: A meta-analysis. J. Fam. Bus. Manag. 2018 , 8 , 235–273. [ Google Scholar ] [ CrossRef ]
• Dyer, W.G.; Whetten, D.A. Family Firms and Social Responsibility: Preliminary Evidence from the S&P 500. Entrep. Theory Pract. 2006 , 30 , 785–802. [ Google Scholar ] [ CrossRef ]
• Cruz, C.; Larraza-Kintana, M.; Garcés-Galdeano, L.; Berrone, P. Are Family Firms Really More Socially Responsible? Entrep. Theory Pract. 2014 , 38 , 1295–1316. [ Google Scholar ] [ CrossRef ]
• Le Breton-Miller, I.; Miller, D. Family firms and practices of sustainability: A contingency view. J. Fam. Bus. Strategy 2016 , 7 , 26–33. [ Google Scholar ] [ CrossRef ]
• Samara, G.; Jamalia, D.; Sierrac, V.; Paradab, M.J. Who are the best performers? The environmental social performance of family firms. J. Fam. Bus. Strategy 2018 , 9 , 33–43. [ Google Scholar ] [ CrossRef ]
• Dal Maso, L.; Basco, R.; Bassetti, T.; Lattanzi, N. Family ownership and environmental performance: The mediation effect of human resource practices. Bus. Strategy Environ. 2020 , 29 , 1548–1562. [ Google Scholar ] [ CrossRef ]
• Dou, J.; Su, E.; Wang, S. When Does Family Ownership Promote Proactive Environmental Strategy? The Role of the Firm’s Long-Term Orientation. J. Bus. Ethics 2019 , 158 , 81–95. [ Google Scholar ] [ CrossRef ]
• Graafland, J. Family business ownership and cleaner production: Moderation by company size and family management. J. Clean. Prod. 2020 , 255 , 1–10. [ Google Scholar ] [ CrossRef ]
• Miroshnychenko, I.; De Massis, A.; Barontini, R.; Testa, F. Family Firms and Environmental Performance: A Meta-Analytic Review. Fam. Bus. Rev. 2022 , 35 , 68–90. [ Google Scholar ] [ CrossRef ]
• Van Gils, A.; Dibrell, C.; Neubaum, D.O.; Craig, J.B. Social Issues in the Family Enterprise. Fam. Bus. Rev. 2014 , 27 , 193–205. [ Google Scholar ] [ CrossRef ]
• Kim, J.; Fairclough, S.; Dibrell, C. Attention, Action, and Greenwash in Family-Influenced Firms? Evidence From Polluting Industries. Organ. Environ. 2017 , 30 , 304–323. [ Google Scholar ] [ CrossRef ]
• Sharma, P.; Sharma, S. Drivers of Proactive Environmental Strategy in Family Firms. Bus. Ethics Q. 2011 , 21 , 309–334. [ Google Scholar ] [ CrossRef ]
• Roxas, B.; Coetzer, A. Institutional Environment, Managerial Attitudes and Environmental Sustainability Orientation of Small Firms. J. Bus. Ethics 2012 , 111 , 461–476. [ Google Scholar ] [ CrossRef ]
• Vourvachis, P.; Woodward, T. Content analysis in social and environmental reporting research: Trends and challenges. J. Appl. Account. Res. 2015 , 16 , 166–195. [ Google Scholar ] [ CrossRef ]
• Eriksson, P.; Kovalainen, A. Qualitative Methods in Business Research , 2nd ed.; SAGE: Thousand Oak, CA, USA, 2016. [ Google Scholar ]
• Schreier, M. Qualitative Content Analysis in Practice ; SAGE: London, UK, 2012. [ Google Scholar ]
• Antončič, B.; Auer Antončič, J.; Juričič, D. Družinsko Podjetništvo: Značilnosti v Sloveniji (Family Entrepreneurship: Characteristics in Slovenia) ; EY: Ljubljana, Slovenia, 2015. [ Google Scholar ]
• Berrone, P.; Cruz, C.; Gomez-Mejia, L.R. Socioemotional Wealth in Family Firms: Theoretical Dimensions, Assessment Approaches, and Agenda for Future Research. Fam. Bus. Rev. 2012 , 25 , 258–279. [ Google Scholar ] [ CrossRef ]
• Gómez-Mejía, L.R.; Takács Haynes, K.; Núñez-Nickel, M.; Jacobson, K.J.L.; Moyano-Fuentes, J. Socioemotional Wealth and Business Risks in Family-controlled Firms: Evidence from Spanish Olive Oil Mills. Adm. Sci. Q. 2007 , 52 , 106–137. [ Google Scholar ] [ CrossRef ]
• Darnall, N.; Henriques, I.; Sadorsky, P. Adopting Proactive Environmental Strategy: The Influence of Stakeholders and Firm Size. J. Manag. Stud. 2010 , 47 , 1072–1094. [ Google Scholar ] [ CrossRef ]
• Gómez-Mejía, L.R.; Cruz, C.; Berrone, P.; De Castro, J. The Bind that Ties: Socioemotional Wealth Preservation in Family Firms. Acad. Manag. Ann. 2011 , 5 , 653–707. [ Google Scholar ] [ CrossRef ]
• Studdert, J.; Govender, V.; Spies, J.; Jarque Branguli, M.; Nievas, S.; Roca Silva, M.F.; Zhu, W.; Diebschlag, P.; Sassine, R.; Cilliers, W.; et al. United Nations Global Compact. Available online: https://unglobalcompact.org/what-is-gc/mission/principles (accessed on 17 March 2024).
• Primec, A.; Tičar, B. A new challenge in corporate governance—the new legal regulation of nonfinancial reporting. In Recent Challenges in Corporate Governance ; Bohinc, R., Ed.; Fakulteta za družbene vede, Založba FDV: Ljubljana, Slovenia, 2023; pp. 91–112. [ Google Scholar ] [ CrossRef ]
• Primec, A. Nefinančno poročanje kot pravni institut za vzpostavljanje korporativne družbene odgovornosti. Podjet. Delo 2023 , 6–7 , 1231–1240. Available online: https://www.dlib.si/details/URN:NBN:SI:doc-MF5DYSLE (accessed on 21 August 2024).
• Pacces, A.M. Civil Liability in the EU Corporate Sustainability Due Diligence Directive Proposal: A Law & Economics Analysis. European Corporate Governance Institute - Law Working Paper No. 691/2023, Amsterdam Law School Research Paper No. 2023-14, Amsterdam Center for Law & Economics Working Paper No. 2023-02. Forthcoming in Ondernemingsrecht. [ CrossRef ]
• Delmas, M.A.; Gergaud, O. Sustainable Certification for Future Generations: The Case of Family Business. Fam. Bus. Rev. 2014 , 27 , 228–243. [ Google Scholar ] [ CrossRef ]
• Bingham, J.B.; Dyer, W.G., Jr.; Smith, I.; Adams, G.L. A Stakeholder Identity Orientation Approach to Corporate Social Performance in Family Firms. J. Bus. Ethics 2010 , 99 , 565–585. [ Google Scholar ] [ CrossRef ]
• Cennamo, C.; Berrone, P.; Cruz, C.; Gomez-Mejia, L.R. Socioemotional Wealth and Proactive Stakeholder Engagement: Why Family-Controlled Firms Care More About Their Stakeholders. Entrep. Theory Pract. 2012 , 36 , 1153–1173. [ Google Scholar ] [ CrossRef ]
• Duh, M.; Tominc, P. Pomen, značilnosti in prihodnost družinskih podjetij (Importance, characteristics and future of family businesses). In Slovenski Podjetniški Observatorij 2004 , 2nd ed.; Rebernik, M., Tominc, P., Duh, M., Krošlin, T., Radonjič, G., Eds.; Inštitut za podjetništvo in management malih podjetij, Ekonomsko-poslovna fakulteta, Univerza v Mariboru: Maribor, Slovenia, 2005; pp. 19–31. [ Google Scholar ]
• Mandl, I. Overview of Family Business Relevant Issues. Final Report, Austrian Institute for SME Research: Vienna. Available online: https://ec.europa.eu/docsroom/documents/10389/attachments/1/translations/en/renditions/native (accessed on 1 February 2023).
• Sachs, J.D.; Kroll, C.; Lafortune, G.; Fuller, G.; Woelm, F. Sustainable Development Report 2021: Includes the SDG Index and Dashboards ; The Decade of Action for the Sustainable Development Goal; SDSN, Bertelsmann Stiftung, Cambridge University Press: Cambridge, UK, 2021. [ Google Scholar ] [ CrossRef ]
• Sachs, J.D.; Lafortune, G.; Fuller, G. The SDGs and the UN Summit of the Future. Sustainable Development Report 2024 ; SDSN: Paris, France; Dublin University Press: Dublin, Ireland, 2024. [ Google Scholar ] [ CrossRef ]
• UMAR. Poročilo o Razvoju 2023 (Report on Development 2023). Ljubljana: Urad RS za Makroekonomske Analize in Razvoj. Available online: https://www.umar.gov.si/fileadmin/user_upload/razvoj_slovenije/2023/slovenski/POR2023-splet.pdf (accessed on 10 April 2024).
• Creswell, J.W. Qualitative Inquiry and Research Design: Choosing among Five Approaches ; SAGE: Thousand Oaks, CA, USA, 2013. [ Google Scholar ]
• Myers, M.D. Qualitative Research in Business & Management ; SAGE: Thousand Oaks, CA, USA, 2013. [ Google Scholar ]
• De Massis, A.; Kotlar, A. The case study method in family business research: Guidelines for qualitative scholarship. J. Fam. Bus. Strategy 2014 , 5 , 15–29. [ Google Scholar ] [ CrossRef ]
• Campopiano, G.; De Massis, A. Corporate Social Responsibility Reporting: A Content Analysis in Family and Nonfamily. J. Bus. Ethics 2015 , 129 , 511–534. [ Google Scholar ] [ CrossRef ]
• Craig, J.; Dibrell, C. The Natural Environment, Innovation, and Firm Performance: A Comparative Study. Fam. Bus. Rev. 2006 , 19 , 275–288. [ Google Scholar ] [ CrossRef ]
• Sharma, P.; Chrisman, J.J.; Chua, J.H. Predictors of satisfaction with the succession process in family firms. J. Bus. Ventur. 2003 , 18 , 667–687. [ Google Scholar ] [ CrossRef ]
• Whittingham, K.L.; Earle, A.G.; Leyva-de la Hiz, D.I.; Argiolas, A. The impact of the United Nations Sustainable Development Goals on corporate sustainability reporting. Bus. Res. Q. 2023 , 26 , 45–61. [ Google Scholar ] [ CrossRef ]
• Cabrera-Suárez, K. Leadership transfer and the successor’s development in the family firm. Leadership Quarterly 2005 , 16 , 71–96. [ Google Scholar ] [ CrossRef ]
• Rus, D.; Močnik, D.; Crnogaj, K. Podjetniška Demografija in Značilnosti Startup in Scaleup Podjetij: Slovenski Podjetniški Observatorij 2022 (Business Demographics and Characteristics of Startup and Scaleup Companies: Slovenian Entrepreneurship Observatory 2022) ; niverza v Mariboru, Univerzitetna založba: Maribor, Slovenia, 2023. [ Google Scholar ]
• Data base of Slovenian exporters. Available online: https://www.sloexport.si/ (accessed on 30 August 2023).
• Bizi. Data base of financial and contact data of Slovenian companies. Available online: https://www.bizi.si/ (accessed on 30 August 2023).
• Family Business Slovenia 2019. Ljubljana: EY. Available online: https://www.ey.com/en_si/family-enterprise/familiy-business-book-slovenia-2019 (accessed on 30 November 2023).
• Family Business Slovenia 2020. Ljubljana: EY. Available online: https://www.ey.com/en_si/family-enterprise/family-business-book-slovenia-2021 (accessed on 30 November 2023).
• Family Business Slovenia 2021. Ljubljana: EY. Available online: https://www.ey.com/en_si/family-enterprise/family-business-book-slovenia-2020 (accessed on 30 November 2023).
• Payne, G.T.; Brigham, K.H.; Broberg, J.C.; Moss, T.W.; Short, J.C. Organizational Virtue Orientation and Family Firms. Bus. Ethics Q. 2011 , 21 , 257–285. [ Google Scholar ] [ CrossRef ]
• Schreier, M. Qualitative Content Analysis. In The SAGE Handbook of Qualitative Data Analysis ; Flick, U., Ed.; SAGE: Thousand Oaks, CA, USA, 2014; pp. 170–183. [ Google Scholar ]
• Müller-Stewens, G.; Lechner, C. Strategisches Management, 5. Auflage ; Schäffer-Poeschel Verlag: Stuttgart, Germany, 2016. [ Google Scholar ]
• Wheelen, T.L.; Hunger, J.D. Strategic Management and Business Policy. Toward Global Sustainability , 13th ed.; Prentice Hall, Pearson: New Jersey, NJ, USA, 2012. [ Google Scholar ]
• WCED. Our Common Future. Report of the World Commission on Environment and Development. Available online: https://sustainabledevelopment.un.org/content/documents/5987our-common-future.pdf (accessed on 1 October 2021).
• Sjåfjell, B. Reforming EU company law to secure the future of European business. Eur. Co. Financ. Law Rev. 2021 , 18 , 190–217. [ Google Scholar ] [ CrossRef ]

Share and Cite

Duh, M.; Primec, A. Investigating the Effectiveness of Endogenous and Exogenous Drivers of the Sustainability (Re)Orientation of Family SMEs in Slovenia: Qualitative Content Analysis Approach. Sustainability 2024 , 16 , 7285. https://doi.org/10.3390/su16177285

Duh M, Primec A. Investigating the Effectiveness of Endogenous and Exogenous Drivers of the Sustainability (Re)Orientation of Family SMEs in Slovenia: Qualitative Content Analysis Approach. Sustainability . 2024; 16(17):7285. https://doi.org/10.3390/su16177285

Duh, Mojca, and Andreja Primec. 2024. "Investigating the Effectiveness of Endogenous and Exogenous Drivers of the Sustainability (Re)Orientation of Family SMEs in Slovenia: Qualitative Content Analysis Approach" Sustainability 16, no. 17: 7285. https://doi.org/10.3390/su16177285

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• For authors
• New editors
• BMJ Journals

You are here

• Online First
• They are still children: a scoping review of conditions for positive engagement in elite youth sport
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-5244-6416 Stuart G. Wilson 1 ,
• http://orcid.org/0009-0000-1865-0915 Mia KurtzFavero 1 ,
• http://orcid.org/0000-0002-4616-2617 Haley H. Smith 1 ,
• http://orcid.org/0000-0003-1377-0234 Michael F Bergeron 2 ,
• http://orcid.org/0000-0002-3242-599X Jean Côté 1
• 1 School of Kinesiology & Health Studies , Queen's University , Kingston , Ontario , Canada
• 2 Performance Health , WTA Women’s Tennis Association , St. Petersburg , Florida , USA
• Correspondence to Dr Stuart G. Wilson; Stuart.wilson{at}queensu.ca

Objective The objective of this study is to characterise the key factors that influence positive engagement and desirable developmental outcomes in sport among elite youth athletes by summarising the methods, groups and pertinent topical areas examined in the extant published research.

Design Scoping review.

Data sources We searched the databases SPORTDiscus, APA PsycINFO, Web of Science and Sports Medicine & Education Index for peer-reviewed, published in English articles that considered the factors influencing positive developmental outcomes for athletes under 18 years competing at a national and/or international level.

Results The search returned 549 articles, of which 43 met the inclusion criteria. 16 studies used a qualitative approach, 14 collected quantitative data, 2 adopted mixed methods and 11 were reviews. Seven articles involved athletes competing in absolute skill contexts (ie, against the best athletes of any age) while the majority involved athletes competing in relative skill contexts (ie, against the best in a specific age or developmental group). The studies described the characteristics of the athletes, as well as their training, relationships with others, social and physical environments, and/or their overall developmental pathways.

Conclusion Existing research on positive engagement in elite youth sport aligned with and mapped onto established models of positive development in youth sport more generally. Our findings further support that, while certain youth athletes may demonstrate extraordinary performance capabilities, they are still children who benefit from positive engagement prompted and reinforced by developmentally appropriate and supportive activities, relationships and environments.

• Psychology, Sports
• Athletic Performance

Data availability statement

Data are available on reasonable request.

https://doi.org/10.1136/bjsports-2024-108200

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

X @SGWilsonQU, @mia_jkf, @hale_smith, @DrMBergeron_01, @jeancote46

Contributors SGW is the guarantor. JC and MFB conceived of the project. SGW, MK, HHS and JC designed and conducted the review. SGW, MK and JC drafted the paper, and all authors contributed to editing and revising the paper.

Funding This research was supported by a Research Grant from the International Olympic Committee (IOC) and an Insight Grant from the Social Sciences and Humanities Research Council of Canada (SSHRC Grant # 435-2020-0094).

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

Framing Collective Moral Responsibility for Climate Change: A Longitudinal Frame Analysis of Energy Company Climate Reporting

• Original Paper
• Open access
• Published: 26 August 2024

Cite this article

You have full access to this open access article

• Melanie Feeney 1 ,
• Jarrod Ormiston 2 ,
• Wim Gijselaers 1 ,
• Pim Martens 3 &
• Therese Grohnert 1  

109 Accesses

Explore all metrics

Responding to climate change and avoiding irreversible climate tipping points requires radical and drastic action by 2030. This urgency raises serious questions for energy companies, one of the world’s largest emitters of greenhouse gases (GHGs), in terms of how they frame, and reframe, their response to climate change. Despite the majority of energy companies releasing ambitious statements declaring net zero carbon ambitions, this ‘talk’ has not been matched with sufficient urgency or substantive climate action. To unpack the disconnect between talk and action, this paper draws on the literature on framing, organisational hypocrisy, and collective moral responsibility. We conduct a longitudinal qualitative content analysis of the framing of climate change used by the ten largest European investor-owned energy companies and the actions they have taken to shift their business practices. Our findings reveal three main categories of energy companies: (i) deflecting, (ii) stagnating, and (iii) evolving. We show key differences in the relationship between framing and action over time for each category, revealing how deflecting companies have larger and persistent gaps between green talk and concrete action and how stagnating companies are delaying action despite increased green talk, while evolving companies exhibit a closer link between talk and action that tends to be realised over time. Our analysis reveals how competing approaches to framing collective moral responsibility help understand the trajectories of talk and action across the different categories of energy companies. This research makes several contributions to the literature on organisational hypocrisy and collective moral responsibility in the context of climate change. Our analysis highlights the complex relationship between collective moral responsibility, organisational hypocrisy and climate action, revealing how different collective framings—diffuse, teleological, or agential—can both enable and offset substantive climate action. The study also enriches our understanding of the performative nature of collective moral responsibility by examining its temporal dimensions and showing how an agential, backward-looking focus is associated with more meaningful climate action.

Similar content being viewed by others

Accelerating Climate Action Beyond Company Gates

Drilling their Own Graves: How the European Oil and Gas Supermajors Avoid Sustainability Tensions Through Mythmaking

Explore related subjects.

• Artificial Intelligence

Avoid common mistakes on your manuscript.

Introduction

The energy sector remains one of the world’s largest emitters of greenhouse gases (GHGs), with electricity and heat production responsible for almost half of the world’s GHGs in 2014 (Ritchie & Roser, 2019 ; United Nations, 2019 ). In 2020, two of the world’s largest energy providers, Royal Dutch Shell and British Petroleum (BP), released statements declaring their net zero carbon ambitions by 2050 (Shell Ambrose, 2020 ; Global, 2020 ). However, recent studies suggest that responding to climate change and avoiding irreversible climate tipping points requires drastic action by 2030, not 2050 (Liu et al., 2019 ; Steffen et al., 2015a , 2015b ). Acknowledging this urgency, the European Union recently committed to fighting climate change through higher renewable energy targets by 2030, aiming to source 42.5% of its energy from renewable sources such as wind and solar (Reuters, 2023 ).

This increased urgency raises challenging questions for energy companies in terms of how they frame, and reframe, their response to climate change (Campbell et al., 2019 ; Cornelissen & Werner, 2014 ), and whether this framing can be matched by the radical transformation needed in their business models and actions. To understand this transformation, this paper explores how energy companies are framing their responses to climate change and their actions to shift their business practices. In doing so, we engage with the growing field of scholarship exploring the role of framing in justifying climate change responses and legitimising sustainability strategies (Hahn & Lulfs, 2014 ; Metze, 2018 ; Nyberg & Wright, 2006 ; Nyberg et al., 2018 ; Wright & Nyberg, 2017 ).

Despite the increase in talk about sustainable action and engagement with frames to discuss responses to climate change, we as a society are trending towards overstepping multiple environmental limits and planetary boundaries (O’Neill et al., 2018 ; Steffen et al., 2015a , 2015b ). This dissonance is matched in corporate sustainability reporting, where the expansion of sustainability talk has not been matched with sufficient sustainability action (Cho et al., 2015 ; Higgins et al., 2020 ; Milne & Gray, 2013 ). Understanding this disconnect between talk and action is crucial in ensuring the energy sector moves beyond discursive strategies to seek legitimacy, towards genuine climate action that will contribute to a just transition (Banerjee, 2012 ; Christensen et al., 2021 ). To unpack this potential disconnection between frames, decisions and action, we draw on the literature on organisational hypocrisy and collective moral responsibility.

Organisational hypocrisy aims to explain the discrepancies between the talk and actions of companies (Brunsson, 2002 ; Wagner et al., 2009 ). In recent years, there has been a growing body of literature that explores hypocrisy in corporate sustainability reporting by comparing symbolic approaches (talk) with substantive approaches (action) (Hyatt & Berente, 2017 ; Rodrigue et al., 2013 ; Schons & Steinmeier, 2016 ). Through a critical lens, the hypocritical gap between symbolic talk and substantive action can be viewed as a duplicitous attempt to conceal unsustainable practices or hide a lack of substantive action (Cho & Patton, 2007 ; Milne & Gray, 2013 ; Hyatt & Berente, 2017 ; Snelson-Powell et al., 2020 ). Alternatively, this hypocrisy may be viewed as inevitable as organisations attempt to juggle competing stakeholders' demands (Brunsson, 1986 , 1993 ; Higgins et al., 2020 ), and may be a signal for future substantive action (Clarkson, et al., 2008 ; Clune & O’Dwyer, 2020 ; Malsch, 2013 ).

To better understand the nature of organisational hypocrisy in energy company responses to climate change, we examine the role of collective moral responsibility in shaping the disconnect between talk and action. Moral responsibility refers to the blameworthiness or praiseworthiness for a particular situation (Bovens, 1998 ). We engage collectivist perspectives of moral responsibility, arguing that organisations may have a collective responsibility to respond to, or bring about, a particular state of affairs (Mellema, 1997 , 2003 ; Soares, 2003 ; Tamminga & Hindriks, 2020 ). In unpacking the role of collective moral responsibility in shaping climate action, we zoom in on the temporal nature of moral responsibility, differentiating between both backward-looking (reactive) and forward-looking (prospective) responsibility (Gilbert, 2006a , 2006b ; Sanbhu, 2012 ; Van de Poel, 2011 ). Backward-looking moral responsibility involves taking on blame for immoral past actions, while forward-looking moral responsibility refers to a sense of obligation to avoid future immoral actions (Sanbhu, 2012 ). We also draw on the work of Collins ( 2019 ) that explores a more nuanced understanding of the ‘collective’, differentiating between diffuse collectives that are loosely described groups of agents such as ‘society’, ‘the private sector’, teleological collectives that are responsive towards each other and act towards commons goals such as ‘the energy sector’, and agential collectives that have well-defined collective-level decision-making procedure such as a specific company, partnership or alliance.

To shed light on the disconnects between talk and action and the role of collective moral responsibility, we conduct a qualitative content analysis of the framing used by Europe’s ten largest investor-owned energy companies over a ten-year period. We review 111 sustainability reports from these energy companies between 2010 and 2019 to understand the evolution of their framing of climate change and the actions they have taken over time. The analysis is guided by the following overarching research questions: “How have energy companies framed their responses to climate change over time?”, “How does their framing relate to climate action?”, and “What is the relationships between different framings of collective moral responsibility and the nature of climate change talk and action?”.

Our analysis of framing and action over time reveals three main categories of energy companies: (i) deflecting, (ii) stagnating, (iii) evolving. Deflecting companies continue to engage in unsustainable business-as-usual practices despite offering some green rhetoric. Stagnating companies are making some progress but seem to be stalling and delaying more radical action despite increased sustainability talk. Evolving companies appear to be progressing towards a more sustainable future and questioning and rethinking their business models. We noticed key differences in the relationship between action and framing over time for each category, with evolving companies having a closer link between talk and action that tends to be realised over time, and deflecting companies having more significant and persistent gaps between green talk and concrete action.

The findings show how competing approaches to framing the nature of collective moral responsibility help to understand the trajectories of talk and action across the different categories of energy companies. As suggested by the data from our study of ten energy companies, deflecting firms seem to evoke a diffuse collective of society, deferring responsibility to other actors, including government and civil society and framing their own moral responsibility in a more forward-looking, prospective way. The companies we classified as stagnating seem to shift from a diffuse collective before framing their role as part of a broader teleological collective of the energy sector, yet remain somewhat vague in terms of their own responsibility for climate action. The companies we classified as evolving, seem to frame their role as an agential collective and acknowledge their own moral responsibility for causing or contributing to climate change. This backward-looking perspective on their moral responsibility appears to be shaping substantive action in the present.

By engaging with theories of collective moral responsibility, our paper contributes to the literature on business ethics and climate change in several ways. We contribute to the literature on business ethics, moral responsibility, and organisational hypocrisy by providing a nuanced understanding of the performative nature of collective moral responsibility (Soares, 2003 ; Tamminga & Hindriks, 2020 ). In doing so, we highlight the diverse ways in which conceptions of the collective as diffuse, teleological, or agential (Collins, 2019 ) are associated with different types of climate talk and action and different levels of organisational hypocrisy. Specifically, we show that agential collectives with a backward-looking sense of responsibility are more likely to engage in substantive action, while diffuse and teleological collectives tend to focus on symbolic talk. We contribute to the broader literature on framing (Cornelissen & Werner, 2014 ) and organisational hypocrisy (Brunsson, 2002 ) by unpacking the relationship between organisational framing of collective moral responsibility and the nature of organisational hypocrisy. We show how agential notions of the collective and backward-looking responsibility are associated with more substantive climate action. This insight extends prior research by highlighting the dynamic interplay between framing, moral responsibility and climate action. We also contribute to a temporal understanding of collective moral responsibility and organisational hypocrisy by adopting a temporal lens that reveals how the understanding of the collective and the direction of responsibility shift over time and how this relates to action and inaction on climate change (Brunson, 1986 , 1993 , 2002 ; Cho et al., 2015 ). Our longitudinal analysis shows the ways in which these shifts are critical for substantive action. Finally, we contribute to practice by highlighting the shifts in collective moral responsibility associated with energy companies becoming more sustainable and authentically engaging in climate action.

Theoretical Background

Frames and sustainability.

Corporate responses to climate change, particularly within the energy sector, require drastic changes to a company’s strategy, operations, and often even their identity (Boons et al., 2013 ; Frandsen & Johansen, 2011 ). Transitioning from a company that has operated as a leader in energy production sourced from fossil fuels to a company that prioritises a carbon–neutral energy mix is a complicated process (Mori, 2021 ). This shift requires companies to rethink what technologies they invest in, the speed at which they make these changes, and how to ensure their workforce is on-board and prepared for the change (Garavan & McGuire, 2010 ; Nisar et al., 2013 ). All of these require managers to make tough choices between long-term and short-term value (Slawinski & Bansal, 2015 ). This transformation requires companies to frame and reframe how they understand their role in terms of climate action. We thereby engage with literature on framing (Cornelissen & Werner, 2014 ) to explore how energy companies engage in meaning-making with regard to climate change and how this relates to their climate change responses and actions.

The construct of frame or framing was first introduced in the 1930s within the social sciences and has since gained popularity in a wide range of research traditions (Cornelissen & Werner, 2014 ), including cognitive psychology and behavioural economics (e.g., Kahneman et al., 1986 ), sociology and social movements (e.g., Fligstein & McAdam, 2011 ), political science (e.g., Barth & Bijsmans, 2018 ), and organisation and management studies (e.g., Gioia & Chittipeddi, 1991 ). According to Goffman ( 1974 ), no action or behaviour can be initiated without some form of framing, that is, making sense of what is going on. Frames are constructed based on past experiences and act as a point of reference for sense-making (Kahneman, 1984 ). Rather than viewing a frame as an isolated or static structure, framing is understood as an interactional and ongoing process of constructing meaning (Dewulf et al., 2009 ). Frames are, therefore, constantly updated and adjusted based on new experiences or information (Dewulf et al., 2009 ; Kahneman, 1984 ). In fact, Nyberg et al. ( 2016 ) argue that any theory of framing must contain time, which underpins our temporal analysis of energy company framing of climate change over a ten-year period.

Framing has been applied in research on cognition, sense-making and decision-making processes (Benner & Tripsas, 2012 ; Walsh, 1995 ; Weick, 1995 ), and in research on organised groups and organisations (Cornelissen & Werner, 2014 ). How an organisation frames its environment and where it sits within that environment is referred to as strategic framing (Gilbert, 2006a , 2006b , Kaplan, 2008 ). A strategic frame refers to “a set of cause-effect understandings about industry boundaries, competitive rules, and strategy-environment relationships available to a group of related firms in an industry” (Nadkarni & Narayanan, 2007 , p.689). Strategic frames, and subsequent decision-making processes, can therefore be influenced by a variety of actors, e.g., shareholders and other stakeholders, or external forces e.g., changing markets, industry trends or changing societal beliefs and values (Battilana et al., 2009 ; Gilbert, 2006a , 2006b ).

Traditionally the greatest forces of influence over corporate sustainability strategies have come from government legislation and regulation and changing market and industry trends (Boons et al., 2013 ; Brønn & Vidaver-Cohen, 2009 ; Christensen et al., 2021 ). Whilst these regulatory and market conditions still greatly influence corporate sustainability strategies, we are now also seeing increased pressure from social actors on companies to act ethically and responsibly (Banerjee, 2008 ; O’Brien et al., 2018 ; Porter & Kramer, 2011 ). As a result, the variety of stakeholder expectations that energy companies must consider, and the regulatory and market environments that they operate in, have become increasingly complex (Banerjee, 2008 ). With this growing complexity has come an increased interest from scholars in corporate sustainability framing and responses (Hahn et al., 2014 ).

Framing and sustainability responses in the energy sector have been a growing area of academic interest in recent years (Schlichting, 2013 , Hahn et al., 2014 ). Studies have sought to understand the frames adopted in political conversations around specific energy technologies like fracking (Metze, 2018 ; Nyberg et al., 2020 ), or the framing of intertemporal tensions in oil companies’ climate change responses (Slawninski & Bansal, 2015 ). These studies have found that how an organisation frames climate change has implications for the types of responses they enact in the short and long-term (Nyberg et al., 2020 ; Slawinski & Bansal, 2015 ). These studies further demonstrate the importance of unpacking energy company framing of climate change to understand current and future action and inaction.

Several theoretical and empirical articles that have contributed to our understanding of energy company framing of sustainability and climate change in recent years (Wright & Nyberg, 2017 , Hahn et al., 2014 , Shlichting, 2013 ). In 2013, Schlichting published an article that looked at the ways different industry actors (including energy sector actors) had framed climate change from 1990 to 2010, their reasoning for adopting each frame, and their strategies for communicating frames. The study revealed dominant frames at three moments of time across the two decades, starting with ‘scientific uncertainty’ from 1990 to the mid-1990s when industry actors questioned the science around climate change. From 1997 to the early 2000s, companies used ‘socioeconomic consequences' frames, where industry actors acknowledged the potential risks of climate change but drew attention to the costs to the company and consumers if they were to act in accordance with the Kyoto Protocol (that was passed in 1997). Finally, from the 2000s to 2010, companies adopted ‘industrial leadership’ frames, where industry actors acknowledged their role in climate change and saw technology as offering a win–win solution to remaining competitive while also responding to the threat of climate change. Whilst Schlichting ( 2013 ) contributes to our understanding of energy company framing of climate change, the article does not consider the specific actions or inactions that are related to each frame.

In a similar study, Wright and Nyberg ( 2017 ) looked at framing as one element of corporate responses to climate change from 2005 to 2015 and concluded that the dominant framing across all companies (including one oil and gas company) for climate change was ‘business case’ framing. Wright and Nyberg ( 2017 ) describe business case framing of climate change as when companies conformed to short-term market conditions and observed that over time, companies would regress toward traditional business concerns, i.e., profit maximisation. The authors offer some examples of how energy company framing aligns with actions in response to climate change, i.e., investment in renewable energy projects and greater attention given to potential regulatory changes, however, due to the diversity of companies included in the study, these examples are limited.

Finally, Hahn et al. ( 2014 ) identify the business case as a dominant frame in their review study of managers’ responses to sustainability. However, the authors positioned business case frames on a continuum with ‘paradoxical’ frames on the opposing end. Paradoxical frames capture a more developed understanding and appreciation for the tensions between social, environmental, and economic aspects of sustainability by managers and are more aligned with more radical, albeit slow, responses to sustainability issues (Hanh et al., 2014 ). Given that the Hahn et al. ( 2014 ) article is a review paper, the focus is largely theoretical and does not specifically observe the relationship between frames and actions.

Our study builds on previous research by focusing specifically on energy companies and attempting to understand the relationship between frames and actions. Previous research has paid limited attention to the relationship between climate change frames and actions adopted by energy companies (for example, Schlichting, 2013 , Hahn et al., 2014 ). Understanding the relationship between frames and action is important, as while frames are often viewed as causal mechanisms for shaping decisions and action, broader research on sustainability reporting highlights the pervasive disconnects between sustainability talk and action (Cho et al., 2015 , Higgins et al., 2020 , Hyatt & Berente, 2017 , Rodrigue et al., 2013 , Schons & Steinmeier, 2016 ). Our paper thereby aims to build on previous research by taking a more critical and nuanced stance in reviewing frames and sustainability reports that question the links between frames and action. In the following section, we introduce the literature on organisational hypocrisy (Brunsson, 2002 ; Wagner et al., 2009 ) to help unpack the potential for disconnects between symbolic talk and substantive action.

Symbolic Talk, Substantive Action, and Organisational Hypocrisy

Research on sustainability reporting suggests a persistent gap between talk and action in how corporations are responding to sustainability challenges (Cho et al., 2015 ; Higgins et al., 2020 ). Fassin and Buelens ( 2011 ) highlight this dissonance between sustainability rhetoric and actual business practices noting that the “idealism of corporate communication contrasts sharply with the reality of day-to-day business life” (pp. 586–587). To better understand the disconnect between sustainability talk and action in energy company responses to climate change, we engage with the literature on organisational hypocrisy. Organisational hypocrisy refers to the disconnect between talk and action (Brunsson, 2002 ; Wagner et al., 2009 ), as evidenced by “the distance between assertions and performance” (Fassin & Buelens, 2011 , p. 587). This literature on organisational hypocrisy is underpinned by early work in institutional theory that explores how organisations engage in myth and ceremony as they decouple talk from action in order to gain legitimacy (Bromley & Powell, 2012 ; Crilly et al., 2012 ; Meyer & Rowan, 1977 , Oliver, 1991 ). This research has tended to look at how talk is decoupled from action at particular moments in time, with insufficient exploration of the relationship between talk and action over time (Reinecke & Lawrence, 2023 ).

Significant literature on corporate sustainability and corporate social responsibility has explored organisational hypocrisy by comparing symbolic approaches (green talk) with substantive approaches (green action) (Hyatt & Berente, 2017 ; Rodrigue et al., 2013 ; Schons & Steinmeier, 2016 ). Substantive approaches involve meaningful ‘actions’ that shift practices to prioritise improved environmental performance (Hyatt & Berente, 2017 ; Sharma & Vredenburg, 1998 ). Ashforth and Gibbs ( 1990 ) define substantive approaches as those that involve “real, material changes in organizational goals, structures, and processes or socially institutionalized practices.” (p. 178). Substantive approaches thereby require tangible, observable shifts in organisational activities and resource use (Schons & Steinmeier, 2016 ).

Symbolic approaches refer to ‘talk’ that creates an appearance of commitment to sustainability without necessarily shifting organisational practices (Donia & Sirsly, 2016 ; Hyatt & Berente, 2017 ). Companies often engage in symbolic talk to enhance their reputation or to increase their legitimacy in the eyes of certain stakeholders (Ashforth & Gibbs, 1990 ; Elsbach & Sutton, 1992 ). Symbolic approaches can be viewed as ceremonial conformity to the demands of influential stakeholders without the actual changes to activities (Meyer & Rowan, 1977 ; Oliver, 1991 ). The goal of engaging in purely symbolic talk is often to deflect or conceal relatively poor environmental performance (Cho et al., 2010 ).

There are significant debates about the linkages between symbolic talk and substantive action and the implications of organisational hypocrisy for sustainability action over time (Rodrigue et al., 2013 ). From a critical perspective, the hypocritical gap between talk and action is viewed as an attempt to conceal continued poor environmental performance, recast unsustainable practices in a more positive light, or obscure a lack of substantive action (Cho & Patton, 2007 ; Milne & Gray, 2013 ; Hyatt & Berente, 2017 ). Some studies adopt a more positive lens on the disconnect between talk and action, suggesting that symbolic talk in the form of extensive environmental disclosures can be a signal for future substantive action on environmental issues (Clarkson, et al., 2008 ; Clune & O’Dwyer, 2020 ; Malsch, 2013 ). This stream of research suggests the potential for hypocrisy to play an aspirational role, as discrepancies between talk and action may serve to stimulate improvements in sustainability performance over time, even when companies do not meet their aspirations (Christensen et al., 2013 ).

Research on organisational hypocrisy also reveals competing perspectives on the nature of intentionality and duplicity associated with the disconnect between talk and action. Organisational hypocrisy is often used to describe situations in which companies have intentionally presented themselves in a way that does not reflect the underlying reality (Higgins et al., 2020 ; Laufer, 2003 ). This form of hypocrisy is viewed as duplicitous, where the intention is to deceive certain parties (Snelson-Powell et al., 2020 ). This duplicitous form of hypocrisy is echoed in the literature on greenwashing and other forms of unethical management practices (Delmas & Burbano, 2011 ; Laufer, 2003 ; Lyon & Montgomery, 2015 ). An alternative perspective views organisational hypocrisy as an inadvertent and inevitable response for organisations attempting to juggle competing demands and expectations in their broader environment (Higgins et al., 2020 ). Through this lens, organisations might construct conflicting ideologies and hypocritical talk and decisions in order to garner support and legitimacy in the face of incompatible demands (Brunsson, 1986 ) In this sense, organisations might engage in hypocrisy in an attempt to isolate competing stakeholder ideas and pressures from action, resulting in actions that are difficult to justify being compensated by talk in the opposite direction (Brunsson, 1993 ).

To unpack the evolution of talk and action over time, we draw on Dyllick and Muff’s ( 2016 ) article that introduces a typology of business sustainability actions and responses. The article provides examples of common sustainability-related strategies, the actions that support these strategies, and four levels of business sustainability, i.e., business-as-usual, sustainability 1.0, sustainability 2.0, and sustainability 3.0. We detail how the Dyllick and Muff ( 2016 ) framework was used as a starting point for analysing the frames and actions of energy company sustainability reports in the methods section.

To better understand the nature of, and reasons for, organisational hypocrisy in energy company responses to climate change, we now turn to the literature on collective moral responsibility as a lens to explore disconnects between talk and action.

Collective Moral Responsibility

To understand the relationship between climate change talk and action over time, we engage with the literature on collective moral responsibility. Moral responsibility refers to the blameworthiness or praiseworthiness for a certain state of affairs (Bovens, 1998 ). For moral responsibility (blame or praise) to be ascribed to an agent, they need to have autonomy, intentionality, and contextual knowledge, and there needs to be a direct or direct causal connection between the agent and the outcome (Constantinescu & Kaptein, 2015 ).

There are multiple philosophical and ethical debates between collectivist and individualist approaches of understanding moral responsibility (Miller & Makela, 2005 ; Soares, 2003 ). In this paper, we align with collectivist approaches to moral responsibility, which argue that a collective may have a responsibility to bring about a certain state of affairs and that while no individual might be individually responsible, they have an obligation as a member of the collective (Mellema, 1997 , 2003 ; Tamminga & Hindriks, 2020 ). This collective view has been adopted by business ethics scholars, who argue that this broader collective perspective on moral responsibility is needed to ensure corporations and organisations take into consideration the needs and interests of society (Soares, 2003 ). Through this lens, responsibility for a situation can be ascribed to the corporation and the individual members or both (Constantinescu & Kaptein, 2015 ). Corporations should thereby be viewed as intentional actors capable of responding to internal and external challenges (Soares, 2003 ).

Moral responsibility can be both backward and forward-looking (Gilbert, 2006a , 2006b ; Sanbhu, 2012 ; Van de Poel, 2011 ). Forward-looking moral responsibility is concerned with obligations to prevent future immoral actions, whereas backward-looking moral responsibility is concerned with the blameworthiness of immoral actions in the past (Sanbhu, 2012 ). Gilbert ( 2006a , 2006b ) describes the related yet distinct nature of backward-looking and forward-looking moral responsibility as follows: “Though we are not morally responsible for what happened, we are morally responsible for ameliorating its effects.” (p.94). Van de Poel ( 2011 ) outlines five normative notions of moral responsibility, three of which are backward-looking (accountability, blameworthiness and liability) and two which are forward-looking (responsibility as virtue and as moral obligation). In this sense, backward-looking responsibility involves seeing oneself as accountable or to blame for past actions. Whereas forward-looking responsibility is associated with future actions involved in seeing “to it that something is the case” rather than taking the blame for actions in the past.

Prior research is often ambiguous about the nature of the collective when exploring moral responsibility, and often uses backward-looking and forward-looking responsibility interchangeably. To provide a more nuanced perspective on the role of collective moral responsibility in shaping responses to climate change in the energy sector, we draw on the work of Collins ( 2019 ) which encourages a more nuanced understanding of the ‘collective’ and the temporal nature of moral responsibility. Collins ( 2019 ) suggests three forms of collective: diffuse, teleological and agential. As explained in the table below, diffuse collectives are loosely described groups of agents such as ‘society’, ‘humanity’, ‘the private sector’, teleological collectives that are responsive towards each other and act towards commons goals such as ‘the fossil fuel lobby’, ‘the energy sector’, and agential collectives that have well-defined collective-level decision-making procedures such as a specific company, partnership or alliance. Collins ( 2019 ) also differentiates between two forms of moral responsibility: backward-looking or reactive and forward-looking or prospective. These perspectives help to understand whether collectives are taking blame or praise for past actions or future obligations. Table 1 provides a description and example of each of these forms of collective and moral responsibility.

Methodology

We conducted a qualitative content analysis of ten European energy companies’ sustainability reports to determine how they are framing their responses to climate change and the related actions they have taken to shift their business practices. Qualitative content analysis is “a research method for the subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes or patterns” (Hsieh and Shannon, 2005 , p. 1278). Qualitative content analysis was chosen as it allows for a more contextual and circumstantial understanding of texts communicated by companies, rather than quantitative approaches that focus on the frequency of the texts or words used (Mayring, 2000 , 2010 ). Qualitative content analysis has been a widely used approach for analysing corporate sustainability reports (see for example, Boiral et al., 2019 ; Boiral, 2016 ; Hahn & Lulfs, 2014 ) and is viewed as an important method in business ethics research to understand talk and action (Cowton, 1998 ; Lock & Seele, 2015 ). In the following section, we detail the case selection, materials, and methods of content analysis that informed our findings.

Case Selection and Material

The sample consisted of sustainability reports (or Corporate social responsibility (CSR) reports or environment reports) from the ten Footnote 1 largest investor-owned European energy companies (see Table  2 ). The ten energy companies were selected based on the S&P Global Platts Top 250 companies based on their “asset worth, revenues, profits and return on invested capital” (S&P Global, 2020 , p. 3). We chose companies specifically within the European Union (at the time of reporting) to ensure the companies shared the same regulatory environment. Consequently, we excluded the Norwegian company Equinor ASA from the case selection. Additionally, we chose to focus on investor-owned energy companies as many of the largest state-owned energy companies did not publicly list their sustainability data and reports. We note that this lack of data on state-owned companies requires further study given their substantial environmental impact. Table 2 lists the ten companies included in the analysis in order of where they ranked in the S&P Global list of energy companies. The table also shows the country in which they are headquartered, and the number of reports included in the analysis.

For each of the ten selected companies, we then checked whether they were listed on relevant sustainability rankings. It was found that three of the selected companies had been listed on the Carbon Majors database, a global list of companies responsible for the largest amounts of carbon and methane emitted into the atmosphere (Climate Accountability Institute, 2017 ): Royal Dutch Shell, Total and BP. Three were listed on the Global Corporate Knights index, an independently organised ranking of companies based on their sustainability performance (Scott, 2020 ): Enel, Iberdrola and Ørsted. The remaining four companies were not listed on either ranking, including E.ON, Eni, OMV and Repsol. This range in sustainability performance across the ten companies ensured a rich and diverse case selection for exploring our research questions.

We collected PDF versions of each company’s publicly accessible sustainability reports from 2010 to 2019. We chose not to include reports from 2020 due to the potential impact of the COVID-19 pandemic on our analysis. In some instances, sustainability reports were not published for the full timeframe of interest. In these cases, company annual reports were analysed for climate change framing and actions. Similarly, several companies published multiple climate-related reports in the same year, for example, Eni published a ‘Decarbonization report’ and ‘Sustainability report’ in 2017. To ensure that an accurate interpretation of the company’s framing of climate change was captured, all available climate-related reports were included in the analysis. This resulted in a total of 111 reports. As the focus of the study was on climate change, a decision was also made to exclude sections of the sustainability reports not relevant to climate change, specifically some elements of the ‘social’ pillar of sustainability, e.g., ‘working with communities’, and ‘diversity’ as these issues were more closely related to employment and workforce matters rather than core operations.

Data Analysis

We applied a combination of Mayring’s ( 2014 ) step models of deductive and inductive approaches to analysing qualitative data. Our analysis has four main stages (i) coding climate talk and action; (ii) comparing talk and action over time (i.e. organisational hypocrisy); (iii) coding framing of collective moral responsibility; (iv) analysing linkages between organisational hypocrisy and collective moral responsibility.

Stage 1 – Coding talk and action

The first stage of analysis involved coding climate change talk and action in each report. A mix of climate talk and action was coded using Atlas.ti qualitative analysis software. We inductively coded talk and action regarding climate change which led to the emergence of the following dominant categories of codes: climate crisis, competitor mindset, external dialogue, governance, innovation and technology, policy and compliance, positioning, reporting, research and development, shared decision-making, strategy, sustainability goals, temporality, tension between actors, values prioritised.

We then separated and categorised talk from action in each report according to the following four categories that were derived from the Dyllick and Muff ( 2016 ) Business Sustainability Typology (BST). We thereby provided an overall rating for both action and framing for each report according to the following levels.

0.0 – Business as usual: : where companies prioritise financial outcomes and value creation for shareholders with limited focus on sustainability actions.

1.0 – Sustainability as risk management and compliance : where companies take some actions toward sustainability in response to pressure from external stakeholders, viewing sustainability actions as a form of risk management or compliance.

2.0 – Sustainability as multiple value creation: where companies begin attending to multiple forms of value creation (social/cultural, environmental, economic value) and develop defined goals and actions to address sustainability issues.

3.0 – Sustainable transformation: where companies aim to utilise their capabilities and expertise for the purpose of addressing pressing societal challenges such as climate change and enact actions to intentionally generate a positive impact on the world.

In some cases, we coded action or framing as between levels and thereby used 1.5 or 2.5 as the rating. Table 3 provides a description of each of those codes and representative quotes of both action and framing for each level of sustainability.

Stage 2 – Comparing talk and action over time (i.e. organisational hypocrisy)

Following our coding of climate talk and action according to the levels derived from the Dyllick and Muff ( 2016 ) typology, we then explored the relationship between talk and action for each company over the ten-year period. Plotting the shifts in climate talk and action over time allowed us to visualise the evolution of sustainable action from each energy company, as well as visualising the gap between talk and action (i.e. organisational hypocrisy). This analysis allowed us to ascertain different categories of energy companies.

Through this temporal analysis, we identified three categories of energy companies. The first category, Shell and BP, has the largest gaps between talk and action, especially at the beginning of the decade, and were the least progressed in terms of their level of sustainability, only reaching the Sustainability 1.0 level. The second category, Total, Eni, Enel, Repsol, OMV, has made some progress towards the Sustainability 2.0 level but at a relatively slow pace, with action remaining about half a step behind action throughout the decade. The third category, Ørsted, EON, Iberdrola, had made the most progress towards the Sustainability 3.0 level, with framing more closely linked to action, and with action eventually matching up to the talk. The figures presented in the findings section visualise the development of climate talk and action over the decade for each of the companies and the combined development for each category.

Stage 3 – Coding framing of collective moral responsibility

In the next phase of analysis, we sought to understand whether overarching approaches to framing climate change were shaping the nature of climate talk and action. Through a process of connecting, merging and subdivision of codes, we unpacked four overarching frames for conceptualising the role of energy companies in addressing climate change: ‘moral responsibility’, business case’, ‘technological’ and ‘disclosure’. We observed that energy companies did not simply adopt one frame but often adopted multiple frames to communicate and motivate their climate change responses. We observed that the ‘moral responsibility’ framing was the most relevant in differentiating between the three different categories. We thereby decided to recode our data to draw out a more nuanced understanding of how energy companies were framing the nature of moral responsibility.

In this phase of the analysis, we deductively coded the reports drawing on Collins’s ( 2019 ) theorisation of collective moral responsibility. We coded each report for three forms of the ‘collective’: diffuse, teleological, and agential. We also coded two forms of ‘moral responsibility’: backward-looking/reactive and forward-looking/prospective. The following table provides a description and representative quotes for each of these forms of collective and moral responsibility (Table  4 ).

Stage 4 – Linkages between organisational hypocrisy and collective moral responsibility

In the final stage of analysis, we explored the linkages between the framing of collective moral responsibility and the nature of talk and action by comparing the framing of the collective and the temporal nature of moral responsibility for each of the three categories. We observed that the first category tended to refer to a more diffuse collective and frame moral responsibility in a forward-looking manner. We named this category ‘Deflecting’ as they appear to shift blame and responsibility towards a diffuse collective of government, industry and broader societal actors which results in a larger gap between climate talk and action.

Alternatively, the third category tended to refer to a more agential collective over time as they began to take on blame for causing or contributing to climate change through a more backward-looking understanding of moral responsibility. We named this category ‘Evolving’ given that both their climate talk and action tended to improve over time as they adopted a more agential responsibility.

In analysing the remaining category, which sat between the deflecting and evolving groups, we noted that they had shifted more towards a teleological collective over time as they began to acknowledge the role of the energy sector in addressing climate change. This category was less specific about their own responsibility compared to the evolving group, and appeared to have stalled somewhat in their climate action over time. We named this category ‘Stagnating’, as a reflection of their slow movement towards more sustainable climate action.

Table 5 provides an overview of the three categories, the links between talk and action for each category, and the nature of collective moral responsibility for each category.

Our analysis of actions and framing over time revealed three main categories of energy companies: (i) Deflecting (ii) Stagnating, (iii) Evolving.

Deflecting companies largely maintain business as usual despite some green talk/rhetoric. These companies focus on staying compliant with regulations and ensuring awareness of changing societal standards and expectations. Deflecting companies are slow to adopt targets for emissions and energy intensity, choosing to focus more on targets for investment.

Stagnating companies seem to be stalling and delaying more radical action. While they tend to set clear emissions and energy intensity targets, progress in meeting these targets is slow. This is largely due to the fact that they focus on the easier wins that come by saving costs and waste through efficiencies. Stagnating companies also tend to invest in new renewable technologies, however, this is often framed as an opportunity to diversify their portfolio rather than radically transform their business models away from fossil fuels.

Evolving companies are progressing towards a more sustainable future and rethinking their business models. Evolving companies category bold emissions and energy intensity targets that they often meet and exceed, this is largely achieved by a combination of efficiency technologies and going beyond investment in renewables to diversify their portfolio and moving their entire business strategy away from fossil fuels.

We noticed meaningful differences in the relationship between action and framing over time for each category, with evolving companies having a closer link between talk and action and deflecting companies having larger gaps between green talk and concrete action.

We also observed competing approaches to framing the nature of collective moral responsibility. Deflecting firms seem to evoke a diffuse collective and frame moral responsibility in a more forward-looking, prospective way. Stagnating companies seem to engage in teleological moral responsibility over time but are somewhat vague. Evolving companies seem to frame their role as an agential collective and acknowledge a backward-looking moral responsibility for climate change.

Deflecting Companies

Deflecting firms like Shell and BP appear to maintain a largely business-as-usual approach to their activities despite increasing green talk and rhetoric. Figure  1 below illustrates how these companies enacted limited shifts in their climate actions over the decade despite increasing rhetoric. The trajectories highlight that these companies had the largest gaps between talk and action, especially earlier in the decade.

The relationship between talk and action for deflecting companies

As shown in the figures, deflecting companies exhibited the largest gaps between climate talk and action and evidenced the least progression in terms of substantive climate action. Deflecting companies tended to prioritise the growth of their existing fossil fuel-reliant business models over climate outcomes. Over the decade, they appear to justify their continued use of fossil fuels by the need to provide reliable energy to a growing global population of consumers. Using notions of ‘security of supply’ and ‘energy for all’ to position themselves as the solution to the growing demand for energy in developing and emerging economies and argue that renewable energies are not reliable or abundant enough to achieve this:

We have an important role to play in finding much needed resources of oil and gas to meet the growing energy demand. (BP, 2013 Report) The Arctic could be essential to meeting growing demand for energy in the future. It holds as much as 30% of the world’s undiscovered natural gas and around 13% of its yet-to-find oil, according to the U.S. Geological Survey. (Shell, 2010 Report) The world needs to produce enough energy to keep economies growing, while reducing the impact of energy use on a planet threatened by climate change. Shell works to help meet rising energy demand in a responsible way. That means operating safely, minimising our impact on the environment and building trust with the communities who are our neighbours (Shell, 2012 Report)

As illustrated in the following quotes, these energy companies will often prefix their climate commitments by first demonstrating the ways in which they will maximise shareholder returns in future energy scenarios, with climate change presented as an opportunity to increase profits or as a threat that must be dealt with to protect future profits. In these reports early in the decade, we see limited accountability or blame taken on by these companies.

BP’s objective is to create value for shareholders by helping to meet the world’s growing energy needs safely and responsibly (BP, 2011 Report) We are taking steps to prepare for the potential physical impacts of climate change on our existing and future operations. Projects implementing our environmental and social practices are required to assess the potential impacts to the project from the changing climate and manage any significant impacts identified. (BP, 2011 Report)

The actions of these deflecting companies highlight their support for a gas-led transition, offering gas as a cleaner fossil fuel to other alternatives like coal and oil. However, energy companies also use advancements in fossil fuel technologies to position other, more polluting, fossil fuels as ‘clean’.

Our approach to helping to tackle global CO2 emissions focuses on four main areas: producing more natural gas, helping to develop carbon capture and storage, producing low-carbon biofuel and working to improve energy efficiency in our operations (Shell, 2011 Report) We believe that, to meet global climate goals, the world should prioritize: Reducing emissions rather than promoting any one fuel as the answer. The world will need all forms of energy for a long time to come, so we need to make all fuels cleaner. (BP, 2017 Report)

Overall, it is clear from the way that these deflecting companies talk throughout the decade that they are not questioning their underlying business model, and not engaging in any forms of reactive or backward-looking responsibility.

We are producing almost as much cleaner-burning natural gas as oil, producing low-carbon biofuel, helping to develop carbon capture and storage (CCS) technologies, and putting in place steps to improve our energy efficiency. (Shell, 2012 report)

The disconnect between talk and action for deflecting companies was evidenced by the dissonance between their framing around meeting the needs of society while showing that their actions were still focused on business-as-usual practices. For example, both Shell and BP spoke of their desire to focus on the needs of society:

In 2017, we announced our ambition to cut the net carbon footprint of the energy products we provide by around half by 2050 in step with society’s drive to align with the goals of the Paris Agreement. (Shell, 2017 report) Today’s challenge is to manage and meet growing demand for secure, affordable energy while addressing climate change and other environmental and social issues. (BP, 2012 report)

Yet, both companies make clear that they will only act where it makes commercial sense. Or note that they continue with business as usual until climate inaction presents a financial risk:

Shell is a willing and able player in this transition. We will play our role where it makes commercial sense, in oil and gas, as well as in low-carbon technologies and renewable energy sources. (Shell, 2017 report) Even under the International Energy Agency’s most ambitious climate policy scenario (the 450 scenario a), oil and gas would still make up 50% of the energy mix in 2030...This is one reason why BP’s portfolio includes oil sands, shale gas, deepwater oil and gas production, biofuels and wind. (BP, 2012 report)

For deflecting companies, the distance between talk and actions is achieved by talking about what they want to do rather than substantiated action:

We want to help the world reach net zero and improve people’s lives and can only do this by being a safe, focused, responsible, well-governed and transparent organization. (BP, 2019 report)

Or by leading reports with ‘cherry-picked’ data that presents an incomplete story of their actions. For example, in the quote below, Shell draws attention to their success in improving energy intensity across their operations, despite the fact that their overall direct emissions increased in the same year. These deflections can be seen as attempts by Shell to avoid blame or accountability for past actions.

In 2014, we continued to improve our energy intensity (the amount of energy consumed for every unit of output). This is the result of work within our operations to improve the reliability of equipment and undertake energy efficiency projects. (Shell, 2014 report)

Our analysis suggests that deflecting companies seem to evoke a diffuse collective and frame moral responsibility in a more forward-looking, prospective way. The following quotes capture their framing of the diffuse collective of ‘businesses, governments and civil society’ and ‘society as a whole’ when discussing who is responsible for climate action:

Tackling climate change remains urgent and requires action by governments, industry and consumers. (Shell, 2010 report) Climate change is a major global challenge—one that will require the efforts of governments, industry and individuals (BP, 2010 report) Governments and civil society must work together to overcome the challenges of climate change and the energy-water-food stresses. We are encouraging this collaboration. (Shell, 2012 report)

These deflecting firms explicitly do not take responsibility as an agential collective, deferring to the broader diffuse notions of collective responsibility:

The scale of the global challenges that the world faces is too great for one company, or one sector, to resolve. (Shell, 2013 report) No one company or sector alone can deliver a low-carbon future. Everyone, from consumers to corporations to governments, needs to take responsibility. (BP, 2017 report)

The focus on the role of the diffuse collective remains at the end of the decade, despite the acknowledgement of the increased urgency:

In 2019, demands for urgent action on climate change grew ever louder. All of society, from consumers, to businesses, to governments, recognised the need to accelerate global efforts to reduce greenhouse gas emissions. (Shell, 2019 report) Of course, the task of tackling climate change is bigger than any single company. Everyone on the planet, from consumers, to businesses, to governments, must play their part in reducing greenhouse gas emissions. Everyone must work together. (Shell, 2019 report) A shared challenge. To meet the Paris goals, we believe the world must take strong action on a range of fronts (BP, 2018 Report)

Overall, we observed deflecting companies’ tendency to talk about the climate action they will take in the future, with a tendency to talk about what they want to do, rather than what they have been doing. Moral responsibility is thereby considered in a forward-looking, prospective sense:

In 2017, we announced our ambition to cut the net carbon footprint of the energy products we provide by around half by 2050 in step with society’s drive to align with the goals of the Paris Agreement. (Shell, 2017 report) We have set out our strategy for the coming decades, integrating our ambition to be a safe, strong, successful business with our aspiration to be a good corporate citizen and part of the solution to climate change. (BP, 2016 report) We want to help the world reach net zero and improve people’s lives and can only do this by being a safe, focused, responsible, well-governed and transparent organization. (BP, 2019 report)

Stagnating Companies

The category of stagnating companies, which in our study was found to be represented by Total, Eni, Enel, Repsol, and OMV, appear to be somewhat stalled in their attempts to enact more radical sustainability action. As shown in Fig.  2 below, despite early aspirations at the beginning of the decade, these stagnating companies were relatively slow in shifting their activities over the decade.

The relationship between talk and action for stagnating companies

While these companies are setting clear emissions and energy intensity targets, their progress toward meeting these targets is relatively slow. This is largely due to the fact that they focus on the easier wins that come by saving costs and waste through efficiencies. For example, the quote below from Total acknowledges the commitments made as part of the Paris Climate Agreement’s goal of remaining within 2 °C of global temperature increase from pre-industrial levels but aims to do this by making oil and gas more efficient rather than shifting their business model away from fossil fuels.

Under the 2 °C scenario, oil and gas will still make up almost 50% of the primary energy mix at that time. So yes, of course, we will still be an oil and gas major, meeting this demand. But our ambition is to put our talent to work to become the leader in responsible oil and gas, while also ramping up renewables. (Total, 2016 report)

Similarly, Eni draws attention to the reductions in GHG emissions they have achieved in their activities since 2010, while maintaining their conventional asset portfolio. Rather than signalling a shift away from a fossil fuel-based business model, they instead focus future reductions on increasing energy efficiencies. Similar to the deflecting companies, these stagnating companies appear to take limited blame or accountability for how their actions might have shaped the current state of the climate.

Our organic growth is based on a conventional asset portfolio. Since 2010, we have reduced our GHG emissions by 28%. In the future, we aim at a further reduction of 43% in our upstream emissions index, by decreasing flaring and fugitive methane emissions and increasing energy efficiency. (Eni, 2015 report)

Stagnating companies also tend to invest in new renewable technologies, however, this is often framed as an opportunity to diversify their portfolio rather than radically transform their business models away from fossil fuels.

Our ambition is summed up by the motto “20% in 20 years.” We want to make low-carbon businesses a genuine and profitable growth driver accounting for around 20% of our portfolio in 20 years’ time. (TOTAL, 2016 report)

The gap between talk and action for stagnating companies is less pronounced than for deflecting companies, but overall it tends to remain about half a step behind. For example, the below quote from Repsol shows that the company has clearly defined targets around reductions in C02 emissions, actions that could be aligned with sustainability 2.0 framing:

At Repsol, we are committed to the fight against climate change, which is reflected in the company’s new Strategic Plan 2016-2020. In this sense, we have set a goal to reduce CO2 emissions by 22% over the 2011- 2020 period when compared to 2010, and currently we have already reduced emissions by more than 15%. (Respol, 2015 report)

Whilst the company states that they are committed to fighting climate change and are on track toward meeting their defined emissions targets, stating they have already reduced 15% of their 22% target, they are also found to be taking actions that contradict these claims in the acquisition of Talisman Energy, a large oil and gas company, that increased their annual emissions by 50%. This provides one example of how their actions are not in line with their framing of climate change:

Direct emissions of CO2 equivalent during 2015 were 21 million tons, 50% greater than the previous year due to the inclusion of the emissions from new assets in exploration and production acquired from Talisman. All other business emissions remain at values comparable to 2014. (Repsol, 2015 report)

Similar disconnects were observed at OMV where the aspiration for reducing their carbon footprint is at odds with their actions focused on exploring new approaches to oil and gas. These future-focused targets highlight how moral responsibility is viewed in a prospective manner.

We have pledged to reduce the carbon emissions of our operations, as well as the carbon footprint of our product portfolio in order to make a significant contribution to climate protection. (OMV, 2019 report) To realize its mission of providing energy for a better life, OMV is committed to exploring the full potential of oil and gas at its best by following a responsible approach in producing, processing, and marketing oil and gas and petrochemical products. (OMV, 2019 report)

In analysing the relationship between talk, action, and framings of collective moral responsibility for stagnating companies, we noticed that they often engage in teleological moral responsibility, where they vaguely express responsibility at an industry or sector level for increasing GHG emissions. Despite adopting a less diffuse lens on the collective compared to deflecting companies, these stagnating companies tend to frame responsibility prospectively, where they focus on their role in the future of contributing to climate change solutions.

The quotes below from Eni illustrate their framing of responsibility to a teleological collective at an industry level. We note that they still situate this responsibility within the context of other large companies rather than fully taking responsibility for the industry’s role in contributing to climate change.

This is particularly significant given that the industry is responsible for 40% of all greenhouse gas emissions by companies listed in the Global 500 Index, which groups together the top 500 companies worldwide by revenue. (Eni, 2012 report) There is no doubt that much of the economic growth the world has seen over the past 100 years has been achieved thanks to the discovery and use of fossil fuels. For that, they deserve to be thanked. However, it is now abundantly clear that we can no longer continue to use fossil fuels. (Eni, 2017 report)

Similarly, Total assigns responsibility to a teleological collective of high-emitting industry actors, that includes power generation, and engages in prospective responsibility by claiming that they are charged with realising the energy transition. Rather than taking responsibility for contributing to climate change, Total instead focuses on the potential implications that climate change could have on their operations in the future:

The sectors most responsible for emissions in the EU (i.e., power generation, industry, transport, buildings and construction, as well as agriculture) are charged with making the transition to a low-carbon economy over the coming decades, and these issues could affect TOTAL’s operations in the future. (Total, 2014, report)

In another example, OMV below takes some vague accountability for the impacts their operations have on the environment and the broad areas where they attempt to minimise these impacts. These vague comments seem to fall short of an agential view on moral responsibility.

Due to the nature of our operations, we have an impact on the environment. We strive to minimize that impact at all times, particularly in the areas of spills, energy efficiency, greenhouse gas (GHG) emissions, water and waste management. (OMV, 2016 report)

Evolving Companies

The category of evolving companies, which in our study was found to be represented by Ørsted, EON, and Iberdrola, are not only investing in renewables to diversify their portfolios but are moving the entire business strategy away from fossil fuels. As illustrated in Fig.  3 , their framing still tends to be ahead of action, with actions eventually catching up.

The relationship between talk and action for evolving companies

Evolving companies often describe climate change as requiring radical transformation of business models and the energy sector and provide examples of how they are challenging, questioning and rethinking their business model on the path to more sustainable action. These actions include technological advancements to decarbonise the economy, reduce C0 2 emissions and combat climate change, for example, battery storage, localisation of the grid and electric vehicles. The following quotes provide evidence of the substantive actions evolving companies are undertaking as they transform their business activities, which demonstrate an underlying appreciation of their role as agents, and sense of accountability for past actions.

By the end of 2019, we had realised an 86% carbon reduction since 2006, and 86% of the energy we generated came from renewable sources. In just ten years, we met the transformation target we defined for 2040… We had installed 9.9GW renewable capacity, enough to power more than 15 million people. We had reduced our coal consumption by 91%, and 96% of the wooden biomass we sourced was certified sustainable biomass. (Ørsted, 2019 Report) Iberdrola has proposed the shut-down of all of its coal plants. – The company’s CO2 emissions are already 70% less than the average for the European electricity sector (Iberdola, 2017)

For these evolving companies, framing tends to eventually align with action. Evolving companies tend to go beyond what is required of them by law and set their own ambitions for achieving climate outcomes that exceed regulatory expectations. For example, in 2009 Ørsted set themselves the goal of transforming their energy mix from 85% fossil fuels and 15% renewables to 85% renewables and 15% fossil fuels by 2040. By setting bold emissions and energy intensity targets that they often meet and exceed, these energy companies provide insights on what transformation towards authentic and substantive climate action might look like.

We want sustainable energy to empower people, businesses and societies to unleash their potential without having to worry about harming the planet or reducing the opportunities for future generations…. We have now defined a new target of phasing out coal completely from our production by 2023, because coal is the type of fossil energy causing the highest amount of CO2 emissions. (Orsed, 2016 report) It has also set a goal of reducing greenhouse gas (GHG) emissions of absolute scope 1, 2 and 3, which has been approved by the Science-Based Target initiative…The company has committed to maintaining its position as one of the leading European companies with the lowest CO2 emissions per kWh produced, and to achieve this by focusing its efforts on reducing the intensity of greenhouse gases, promoting renewable technology and increasing efficiency. (Iberdola, 2019 report)

These evolving companies provide examples of how the gap in talk and action in evolving companies can be seen to be a positive sign of what is to come in terms of future action. For these companies, the aspirational talk in earlier years appears to have provided an authentic signal of more ambitious and meaningful climate actions rather than an attempt to hide poor sustainability performance.

Over time, evolving companies appear to be more focused on an agential view of the collective and their own responsibility for contributing to climate change in the past. Evolving companies draw attention to the ecological and societal stakes that are at risk by continuing down the path of fossil fuel-dependent energy systems and presenting themselves as being part of the transition toward a cleaner and more just energy future. They often frame the energy sector and their own company as being largely responsible for climate change and consider it their moral responsibility or obligation to reduce C0 2 emissions and respond to climate change.

Ørsted provides a great example of how evolving companies shift their framing of collective moral responsibility over time. At the beginning of the decade, Ørsted evokes a more diffuse collective with forward-looking prospective responsibility by speaking about the role of the energy sector in the future energy transition. Over time they shift towards a more agential collective and backward-looking moral responsibility where they take more ownership of both the blame and future solutions to climate change. The following quotes show how, earlier in the decade, Ørsted tended to evoke a more diffuse collective when discussing climate change:

The challenges facing the energy sector are part of a wider challenge concerning how we, as modern societies, use our resources. (Ørsted, 2011 report) the world is facing serious resource and climate challenges…With more people on the planet and a rapidly expanding consumer middle class, global resources and ecosystems are put under strain.(Ørsted, 2011 report)

In these early reports, Ørsted would frame their moral responsibility through a prospective lens:

As an energy company, we have a major responsibility to help steer the world in a more sustainable direction. We must develop and deploy low-carbon technologies that can meet the future energy demand of our customers, enabling people to live their lives and businesses to thrive. (Ørsted, 2014 report)

Towards the end of the decade, as Ørsted became more sustainable, there was a clear shift in the framing of collective moral responsibility towards agential collective and backward-looking responsibility whereby Ørsted acknowledged their contribution to the current situation.

We need to transform the global energy systems from black to green energy at a higher pace than the current trajectory. (Ørsted, 2017 report) At Ørsted, our vision directly addresses the challenge of climate change. We used to be one of the blackest energy companies in Europe. Today, we produce 64% green energy, and our target for 2023 takes us beyond 95%. (Ørsted, 2017 report)

By the end of the decade, the blameworthiness shifts to praiseworthiness as Ørsted begins to take credit in their own transition and leadership position in renewable energies. In this 2019 report, Ørsted has a strong framing on the role of transformational leadership and how they have transformed their entire business model. They also make frequent mention of their ambitious long-term targets that go beyond the expectations of the industry and underpin their view on their moral responsibility to combat climate change.

Over the past decade, we have been on a major decarbonisation journey to transform from one of Europe’s most carbon-intensive energy companies to a global leader in renewable energy. (Ørsted, 2019 report) In 2019, we adopted three new climate targets to guide our continued decarbonisation journey… Our biggest contribution is our actions to help fight climate change. (Ørsted, 2019 report)

Overall, these insights highlight how evolving companies combine an agential collective perspective with backward-looking responsibility to acknowledge their role in contributing to current climate situation, thereby taking ownership of past actions and future solutions.

A Typology of Energy Company Framing and Action in Response to Climate Change

Our findings illustrate how energy companies are framing their responses to climate change and the related actions they have taken to shift their business practices. Table 6 presents a typology of in energy company responses to climate change through the lens organisational hypocrisy and collective moral responsibility. The table summarises the relationship between talk, action and framing of collective moral responsibility, highlighting the implications for climate action.

Discussion and Concluding Comments

Our paper makes multiple contributions to the literature on business ethics and climate change. First, we contribute to the literature on business ethics, moral responsibility, and organisational hypocrisy by providing a nuanced understanding of the performative nature of collective moral responsibility (Soares, 2003 ; Tamminga & Hindriks, 2020 ). The performative nature of collective moral responsibility refers to how organisations’ talk and actions regarding moral obligations shape and are shaped by their sense of the collective and their relationship with broader stakeholders. As highlighted in our finding, this performativity suggest that collective moral responsibility is not static, but rather is actively constructed and reconstructed through organisational actions and discourses. Revealing this performativity highlights the diverse ways in which conceptions of the collective as diffuse, teleological, or agential (Collins, 2019 ) are associated with different types of climate talk and action and different levels of organisational hypocrisy. As highlighted in Table  6 , we show how organisations that frame their role as part of a more diffuse or teleological collective engage in forward-looking moral responsibility, which tends to promote symbolic talk rather than substantive action. For example, deflecting and stagnating companies made less sustainability progress over time and had larger gaps between talk and action than evolving companies. On the contrary, organisations that understand blameworthiness through a more agential collective, as was the case with Ørsted, E.ON and Iberdrola, seem to engage in substantive climate action as they view moral responsibility from a more backward-looking perspective. This more backward-looking perspective can create an obligation to authentically shift business practices. These findings highlight the importance of developing a more nuanced understanding of the ‘collective’ (Collins, 2019 ). Further, we reveal the value of differentiating between backward-looking (reactive) and forward-looking (prospective) moral responsibility (Gilbert, 2006a , 2006b ; Sanbhu, 2012 ; Poel, 2011 ) for understanding the connection between talk and action.

Our findings contribute to the broader literature on framing (Cornelissen & Werner, 2014 ) and organisational hypocrisy (Brunsson, 2002 ) by unpacking the relationship between organisational framing of collective moral responsibility and organisational hypocrisy. We show that organisations that view collective moral responsibility through the lens of diffuse collectives or teleological collectives (e.g., deflecting and stagnating companies) and forward-looking responsibility tend to have larger disconnects between talk and action and are less likely to engage in substantive action. Conversely, we show how companies that view moral responsibility through the lens of an agential collective (e.g., evolving companies) adopt a more backward-looking sense of responsibility that is associated with tighter linkages between talk and action and indicative of more substantive action over time. These insights extend prior research on framing and climate change by unpacking the relationship between frames and action (Campbell et al., 2019 ; Hahn & Lulfs, 2014 ; Metze, 2018 ; Nyberg & Wright, 2006 ; Nyberg et al., 2018 ; Wright & Nyberg, 2017 ) and showing how shifts in ethical frames relate to substantive shifts in action. Previous studies on framing and climate action have shown how specific political and social events can shape responses to climate change (Nyberg et al., 2018 ; Slawinski & Bansal, 2015 ). Our insights build on this work through a longitudinal analysis of how framing evolves over time and how different frames are correlated with different levels of organisational hypocrisy. In doing so we go beyond prior research by revealing the dynamic nature of these frames and their implications for action over time.

Through the use of a longitudinal study, we contribute to a temporal understanding of collective moral responsibility and organisational hypocrisy. By adopting a temporal lens, we reveal how the understanding of the collective and the direction of responsibility might shift over time and how this relates to action and inaction on climate change (Brunson, ; Cho et al., 2015 ). Evidence from the category of evolving companies, represented in this study by Ørsted, E.ON and Iberdrola, suggests that organisations that consider their own role as an agential collective with backward-looking responsibility seem to live up to aspirational talk over time. This finding extends insights that suggest that organisational hypocrisy can be beneficial to sustainability action when framing is eventually realised in future action (Christensen et al., 2013 ). The findings suggest that while these companies initially engaged in symbolic framing to signal their commitments to climate action, over time, they were able to align their actions with their talk and shift from the symbolic to the substantive. This evolution reveals how, in certain circumstances, organisational hypocrisy can lead to meaningful climate action.

Alternatively, the journey of the category of deflecting companies, represented in this study by BP and Shell, provides insights into the situations in which framing offsets action (Brunson, 1986 , 1993 , 2002 ) or creates facades (Cho et al., 2015 ) that draw attention away from poor performance or climate inaction. These findings align with the seminal work on organisational hypocrisy by Brunnson ( 1986 ), who theorised that talk, and subsequent decisions, often substitute for or postpone action, especially when organisations do not consider action important or desirable.

These temporal insights contribute to the broader literature on sustainability that highlights the need to adopt a process lens when understanding climate action and inaction (Mazutis et al., 2021 ; Schultz, 2022 ; Slawinski & Bansal, 2015 ; Slawinski et al., 2017 ) and respond to calls for a deeper understanding of the temporal elements involved in ethical considerations (Hockerts & Searcy, 2023 ). We show that as organisations shift from the notion of a diffuse collective to a more agential collective, they tend to move away from a forward-looking sense of moral responsibility towards a backward-looking sense that is associated with more substantive action. The impact of this shift in collective moral responsibility over time is best illustrated by the journey of Ørsted in radically transforming its business model. By revisiting institutional theory, we might understand these temporal shifts as being shaped by broader regulatory and normative institutional pressures (Bromley & Powell, 2012 ; Meyer & Rowan, 1977 ; Oliver, 1991 ). For example, shifts towards a stricter regulatory environment might expedite the shifts from symbolic talk to substantive action, while increased pressure from investors, community and NGOs and greater demands of transparency are also pushing energy companies towards a deeper sense of moral responsibility and more genuine climate action. Overall, we echo the call from Collins ( 2019 ) to consider the temporal horizon of moral responsibility in shaping climate action.

Building on these theoretical contributions, we see multiple fruitful avenues for future research. While this study explored talk and action over a ten-year period, future studies would benefit by investigating climate action and the framing of collective moral responsibility over longer time horizons through accessing historical data. Taking time seriously in studies of climate action would assist in developing more processual understanding of how green talk translates into action. Future research would also benefit from comparative studies that take a global lens and explore investor-owned energy companies along with publicly owned energy organisations. Increasing the heterogeneity of energy organisations would assist in understanding the influence of cultural and regulatory differences in shaping climate talk and action. Finally, extending research on framing and collective moral responsibility beyond the context of the climate crisis to human rights issues such as modern slavery and forced displacement would assist in unpacking the nature of organisational hypocrisy in social as compared to environmental crises.

Finally, we contribute to practice by highlighting the shifts in collective moral responsibility associated with energy companies becoming more sustainable and authentically engaging in climate action. As highlighted in the final row of Table  6 , the insights on the category of evolving companies, represented in this study by Ørsted, E.ON and Iberdrola, suggest that for symbolic talk to match substantive action, organisations need to actively question and reject current unsustainable practices. Evolving companies highlight how they are challenging, questioning, and rethinking their business model as they go beyond diversifying their portfolios towards moving their entire business strategy away from fossil fuels. Our findings suggest that aspirational talk is not sufficient to generate substantive climate action. We suggest that organisations that genuinely want to engage in climate action need to engage in a more agential view of the collective and reconsider their own responsibility for contributing to climate change in the past. Rather than deflecting and deferring responsibility to diffuse notions of society, government, civil society, and corporations, organisations that hope to genuinely contribute to climate action need to take ownership of both the blame for past action and their obligation to find future solutions.

Data Availability

This research is based on publicly available sustainability reports.

We note that E.ON separated into two companies in 2016 by setting up a separate entity Uniper to manage its fossil fuel assets. In order to gain a full picture of E.ON’s climate change responses we also reviewed Uniper’s sustainability reporting from 2016 to 2019. For consistency, we still refer to ten energy companies throughout the paper.

Ambrose, J. (2020). BP sets net zero carbon target for 2050 . Retrieved from: https://www.theguardian.com/business/2020/feb/12/bp-sets-net-zero-carbon-target-for-2050

Ashforth, B. E., & Gibbs, B. W. (1990). The double-edge of organizational legitimation. Organization Science, 1 (2), 177–194.

Article   Google Scholar  

Banerjee, S. B. (2008). Corporate social responsibility: The good, the bad and the ugly. Critical Sociology, 34 (1), 51–79.

Banerjee, S. B. (2012). A climate for change? Critical reflections on the Durban United Nations climate change conference. Organization Studies, 33 (12), 1761–1786.

Barth, C., & Bijsmans, P. (2018). The Maastricht Treaty and public debates about European integration: The emergence of a European public sphere? The Maastricht Treaty and public debates about European. Journal of Contemporary European Studies, 2804 , 1–17.

Google Scholar  

Battilana, J., Leca, B., & Boxenbaum, E. (2009). How actors change institutions: Towards a theory of institutional entrepreneurship. The Academy of Management Annals, 3 (1), 65–107.

Benner, M. J., & Tripsas, M. (2012). The influence of prior industry affiliation on framing in nascent industries: The evolution of digital cameras. Strategic Management Journal, 33 , 277–302.

Boiral, O. (2016). Accounting for the unaccountable: Biodiversity reporting and impression management. Journal of Business Ethics, 135 (4), 751–768.

Boiral, O., Heras-Saizarbitoria, I., & Brotherton, M. C. (2019). Assessing and improving the quality of sustainability reports: The auditors’ perspective. Journal of Business Ethics, 155 (3), 703–721.

Boons, F., Montalvo, C., Quist, J., & Wagner, M. (2013). Sustainable innovation, business models and economic performance: An overview. Journal of Cleaner Production, 45 , 1–8.

Bovens, M. (1998). The quest for responsibility: Accountability and citizenship in complex organisations. Cambridge University Press.

Bromley, P., & Powell, W. W. (2012). From smoke and mirrors to walking the talk: Decoupling in the contemporary world. Academy of Management Annals, 6 (1), 483–530.

Brønn, P. S., & Vidaver-Cohen, D. (2009). Corporate motives for social initiative: Legitimacy, sustainability, or the bottom line? Journal of Business Ethics, 87 , 91–109.

Brunsson, N. (1986). Organizing for inconsistencies: On organizational conflict, depression and hypocrisy as substitutes for action. Scandinavian Journal of Management Studies, 2 (3–4), 165–185.

Brunsson, N. (1993). Ideas and actions: Justification and hypocrisy as alternatives to control. Accounting, Organizations and Society, 18 (6), 489–506.

Brunsson, N. (2002). The organization of hypocrisy . Copenhagen Business School Press.

Campbell, N., McHugh, G., & Dylan-Ennis, P. (2019). Climate change is not a problem: Speculative realism at the end of organization. Organization Studies, 40 (5), 725–744.

Cho, C. H., Laine, M., Roberts, R. W., & Rodrigue, M. (2015). Organized hypocrisy, organizational façades, and sustainability reporting. Accounting, Organizations and Society, 40 , 78–94.

Cho, C. H., & Patten, D. M. (2007). The role of environmental disclosures as tools of legitimacy: A research note. Accounting Organizations and Society, 32 (7–8), 639–647.

Cho, C. H., Roberts, R. W., & Patten, D. M. (2010). The language of US corporate environmental disclosure. Accounting Organizations and Society, 35 (4), 431–443.

Christensen, H. B., Hail, L., & Leuz, C. (2021). Mandatory CSR and sustainability reporting: Economic analysis and literature review. Review of Accounting Studies, 26 (3), 1176–1248.

Christensen, L. T., Morsing, M., & Thyssen, O. (2013). CSR as aspirational talk. Organization, 20 (3), 372–393.

Clarkson, P. M., Li, Y., Richardson, G. D., & Vasvari, F. P. (2008). Revisiting the relation between environmental performance and environmental disclosure: An empirical analysis. Accounting, Organizations and Society, 33 (4–5), 303–327.

Climate Accountability Institute. (2017). CDP Carbon Majors Report 2017 . Retrieved from: https://cdn.cdp.net/cdp-production/cms/reports/documents/000/002/327/original/Carbon-Majors-Report-2017.pdf ?

Clune, C., & O’Dwyer, B. (2020). Organizing dissonance through institutional work: The embedding of social and environmental accountability in an investment field. Accounting, Organizations and Society, 85 , 101130.

Collins, S. (2019). Collective responsibility gaps. Journal of Business Ethics, 154 , 943–954.

Constantinescu, M., & Kaptein, M. (2015). Mutually enhancing responsibility: A theoretical exploration of the interaction mechanisms between individual and corporate moral responsibility. Journal of Business Ethics, 129 , 325–339.

Cornelissen, J. P., & Werner, M. D. (2014). The academy of management annals putting framing in perspective: A review of framing and frame analysis across the management and organizational literature. Academy of Management Annals, 8 (1), 181–235.

Cowton, C. J. (1998). The use of secondary data in business ethics research. Journal of Business Ethics, 17 (4), 423–434.

Crilly, D., Zollo, M., & Hansen, M. T. (2012). Faking it or muddling through? Understanding decoupling in response to stakeholder pressures. Academy of Management Journal, 55 (6), 1429–1448.

Delmas, M. A., & Burbano, V. C. (2011). The drivers of greenwashing. California Management Review, 54 (1), 64–87.

Dewulf, A., Gray, B., Putnam, L., & Lewicki, R. (2009). Disentangling approaches to framing in conflict and negotiation research: A meta-paradigmatic perspective. Human Relations, 62 (2), 155–193.

Donia, M. B., & Sirsly, C. A. T. (2016). Determinants and consequences of employee attributions of corporate social responsibility as substantive or symbolic. European Management Journal, 34 (3), 232–242.

Dyllick, T., & Muff, K. (2016). Clarifying the meaning of sustainable business: Introducing a typology from business-as-usual to true business sustainability. Organization & Environment, 29 (2), 156–174.

Elsbach, K. D., & Sutton, R. I. (1992). Acquiring organizational legitimacy through illegitimate actions: A marriage of institutional and impression management theories. Academy of Management Journal, 35 (4), 699–738.

Fassin, Y., & Buelens, M. (2011). The hypocrisy-sincerity continuum in corporate communication and decision making: A model of corporate social responsibility and business ethics practices. Management Decision, 49 (4), 586–600.

Fligstein, N., & McAdam, D. (2011). Toward a general theory of strategic action fields. Sociological Theory, 29 (1), 1–26.

Frandsen, F., & Johansen, W. (2011). Rhetoric, climate change, and corporate identity management. Management Communication Quarterly, 25 (3), 511–530.

Garavan, T. N., & McGuire, D. (2010). Human resource development and society: Human resource development’s role in embedding corporate social responsibility, sustainability, and ethics in organizations. Advances in Developing Human Resources, 12 (5), 487–507.

Gilbert, C. G. (2006a). Change in the presence of residual fit: Can competing frames coexist? Organization Science., 17 (1), 150–167.

Gilbert, M. (2006b). Who’s to blame? Collective moral responsibility and its implications for group members. Midwest Studies in Philosophy, 30 (1), 94–114.

Gioia, D. A., & Chittipeddi, K. (1991). Sensemaking and sensegiving in strategic change initiation. Strategic Management Journal, 12 (6), 433–448.

Goffman, E. (1974). Frame analysis: An essay on the organization of experience . North Eastern University Press.

Hahn, R., & Lulfs, R. (2014). Legitimizing negative aspects in GRI-oriented sustainability reporting: A qualitative analysis of corporate disclosure strategies. Journal of Business Ethics, 123 , 401–420.

Higgins, C., Tang, S., & Stubbs, W. (2020). On managing hypocrisy: The transparency of sustainability reports. Journal of Business Research, 114 , 395–407.

Hockerts, K., & Searcy, C. (2023). How to sharpen our discourse on corporate sustainability and business ethics—A view from the section editors. Journal of Business Ethics . https://doi.org/10.4324/9781315162935-11

Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15 (9), 1277–1288.

Hyatt, D. G., & Berente, N. (2017). Substantive or symbolic environmental strategies? Effects of external and internal normative stakeholder pressures. Business Strategy and the Environment, 26 (8), 1212–1234.

Kahneman, D. (1984). Choices, Values, and Frames. American Psychologist, 39 (4), 341–350.

Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1986). Fairness and the assumptions of economics. The Journal of Business Ethics, 59 (4), 5285–5300.

Kaplan, S. (2008). Framing contests: Strategy making under uncertainty. Organization Science, 19 (5), 729–752.

Laufer, W. S. (2003). Social accountability and corporate greenwashing. Journal of Business Ethics, 43 (3), 253–261.

Liu, Y., Kumar, M., Katul, G. G., & Porporato, A. (2019). Reduced resilience as an early warning signal of forest mortality. Nature Climate Change, 9 (11), 880–885.

Lock, I., & Seele, P. (2015). Quantitative content analysis as a method for business ethics research. Business Ethics: A European Review, 24 , S24–S40.

Lyon, T. P., & Montgomery, A. W. (2015). The means and end of greenwash. Organization & Environment, 28 (2), 223–249.

Malsch, B. (2013). Politicizing the expertise of the accounting industry in the realm of corporate social responsibility. Accounting, Organizations and Society, 38 (2), 149–168.

Mayring, P. (2014). Qualitative Content Analysis: A theoretical foundation, basic procedures and software solution .

Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative Social Research, 1 (2), 20.

Mayring, P. (2010). Qualitative content analysis: Basics and techniques (11th ed.). Beltz.

Mazutis, D., Slawinski, N., & Palazzo, G. (2021). A time and place for sustainability: A spatiotemporal perspective on organizational sustainability frame development. Business and Society, 60 (7), 1–42.

Mellema, G. (1997). Collective responsibility (Vol. 50). Rodopi Press.

Book   Google Scholar  

Mellema, G. (2003). Responsibility, taint, and ethical distance in business ethics. Journal of Business Ethics, 47 , 125–132.

Metze, T. (2018). Framing the future of fracking: Discursive lock-in or energy degrowth in the Netherlands? Journal of Cleaner Production, 197 , 1737–1745.

Meyer, J. W., & Rowan, B. (1977). Institutionalized organizations: Formal structure as myth and ceremony. American Journal of Sociology, 83 (2), 340–363.

Miller, S., & Makela, P. (2005). The collectivist approach to collective moral responsibility. Metaphilosophy, 36 (5), 634–651.

Milne, M. J., & Gray, R. (2013). W (h) ither ecology? The triple bottom line, the global reporting initiative, and corporate sustainability reporting. Journal of Business Ethics, 118 , 13–29.

Mori, A. (2021). How do incumbent companies’ heterogeneous responses affect sustainability transitions? Insights from China’s major incumbent power generators. Environmental Innovation and Societal Transitions, 39 , 55–72.

Nadkarni, S., & Narayanan, V. K. (2007). The evolution of collective strategy frames in. Organization Science, 18 (4), 688–710.

Nisar, A., Ruiz, F., & Palacios, M. (2013). Organisational learning, strategic rigidity and technology adoption: Implications for electric utilities and renewable energy firms. Renewable and Sustainable Energy Reviews, 22 , 438–445.

Nyberg, D., & Wright, C. (2006). Justifying business responses to climate change: Discursive strategies of similarity and difference. Environment and Planning a: Economy and Space, 44 (8), 1819–1835.

Nyberg, D., & Wright, C. (2016). Performative and political: Corporate constructions of climate change risk. Organization, 23 (5), 617–638.

Nyberg, D., Wright, C., & Kirk, J. (2018). Dash for gas: Climate change, hegemony and the scalar politics of fracking in the UK. British Journal of Management, 29 , 235–251.

Nyberg, D., Wright, C., & Kirk, J. (2020). Fracking the future: The temporal portability of frames in political contests. Organization Studies, 41 (2), 175–196.

O’Brien, K. O., Selboe, E., & Hayward, B. M. (2018). Exploring youth activism on climate change: dutiful, disruptive, and dangerous dissent. Ecology and Society . https://doi.org/10.5751/ES-10287-230342

Oliver, C. (1991). Strategic responses to institutional processes. Academy of Management Review, 16 (1), 145–179.

O’Neill, D. W., Fanning, A. L., Lamb, W. F., & Steinberger, J. K. (2018). A good life for all within planetary boundaries. Nature Sustainability, 1 (2), 88–95.

Porter, M. R., & Kramer, M. E. (2011). The big idea: Creating shared value. CFA Digest, 41 (1), 12–13.

Reinecke, J., & Lawrence, T. B. (2023). The role of temporality in institutional stabilization: A process view. Academy of Management Review, 48 (4), 639–658.

Reuters (2023, March 30) EU reaches deal on higher renewable energy share by 2030 . Retrieved from: https://www.reuters.com/business/sustainable-business/eu-reaches-deal-more-ambitious-renewable-energy-targets-2030-2023-03-30/

Ritchie, Hannah & Roser, Max. (2019). ‘C02 and Greenhouse Gas Emissions’. Published online at ourworldindata.org. Retrieved from: https://ourworldindata.org/co2-and-other-greenhouse-gas-emissions

Rodrigue, M., Magnan, M., & Cho, C. H. (2013). Is environmental governance substantive or symbolic? An empirical investigation. Journal of Business Ethics, 114 , 107–129.

Sandbu, M. E. (2012). Stakeholder duties: On the moral responsibility of corporate investors. Journal of Business Ethics, 109 , 97–107.

Schlichting, I. (2013). Strategic framing of climate change by industry actors: A meta-analysis strategic framing of climate change by industry actors: A meta-analysis. Environmental Communication, 7 (4), 493–511.

Schons, L., & Steinmeier, M. (2016). Walk the talk? How symbolic and substantive CSR actions affect firm performance depending on stakeholder proximity. Corporate Social Responsibility and Environmental Management, 23 (6), 358–372.

Schultz, M. (2022). The strategy–identity nexus: The relevance of their temporal interplay to climate change. Strategic Organization, 20 (4), 821–831.

Scott, M. (2020). Top company profile: Denmark’s Ørsted is 2020’s most sustainable corporation. Published online at corporateknights.com. Retrieved from: https://www.corporateknights.com/reports/2020-global-100/top-company-profile-Ørsted-15795648/

Sharma, S., & Vredenburg, H. (1998). Proactive corporate environmental strategy and the development of competitively valuable organizational capabilities. Strategic Management Journal, 19 (8), 729–753.

Shell Global. (2020). What is Shell’s net carbon footprint? Retrieved from: https://www.shell.com/energy-and-innovation/the-energy-future/what-is-shells-net-carbon-footprint-ambition.html

Slawinski, N., & Bansal, P. (2015). Short on time: Intertemporal tensions in business sustainability. Organization Science, 26 (2), 531–549.

Slawinski, N., Pinkse, J., Busch, T., & Banerjee, S. B. (2017). The role of short-termism and uncertainty avoidance in organizational inaction on climate change: A multi-level framework. Business & Society, 56 (2), 253–282.

Snelson-Powell, A. C., Grosvold, J., & Millington, A. I. (2020). Organizational hypocrisy in business schools with sustainability commitments: The drivers of talk-action inconsistency. Journal of Business Research, 114 , 408–420.

Soares, C. (2003). Corporate versus individual moral responsibility. Journal of Business Ethics, 46 , 143–150.

Steffen, W., Richardson, K., Rockström, J., Cornell, S. E., Fetzer, I., Bennett, E. M., & Sörlin, S. (2015a). Planetary boundaries: Guiding human development on a changing planet. Science . https://doi.org/10.1126/science.1259855

Steffen, W., Richardson, K., Rockström, J., Cornell, S. E., Fetzer, I., Bennett, E. M., & Sörlin, S. (2015b). Planetary boundaries: Guiding human development on a changing planet. Science, 347 (6223), 1259855.

Tamminga, A., & Hindriks, F. (2020). The irreducibility of collective obligations. Philosophical Studies, 177 , 1085–1109.

United Nations. (2019). The Sustainable Development Goals Report 2019 . Retrieved from:

van de Poel, I. (2011). The relation between forward-looking and backward-looking responsibility. Moral responsibility: Beyond free will and determinism (pp. 37–52). Springer Netherlands.

Chapter   Google Scholar  

Wagner, T., Lutz, R. J., & Weitz, B. A. (2009). Corporate hypocrisy: Overcoming the threat of inconsistent corporate social responsibility perceptions. Journal of Marketing, 73 (6), 77–91.

Walsh, J. P. (1995). Managerial and organizational cognition: Notes from a trip down memory lane. Organization Science, 6 (3), 280–321.

Weick, K. E. (1995). Sensemaking in organizations . Sage.

Wright, C., & Nyberg, D. (2017). An inconvenient truth: How organizations translate climate change into business as usual. Academy of Management Journal, 60 (5), 1633–1661.

Download references

Acknowledgements

We thank the Social Issues in Management (SIM) Division Sandbox community for their invaluable feedback on our paper.

Author information

Authors and affiliations.

The University of Maastricht, Tongersestraat 53, 6211 LM, Maastricht, The Netherlands

Melanie Feeney, Wim Gijselaers & Therese Grohnert

University of Technology Sydney, 15 Broadway, Ultimo, NSW, 2007, Australia

Jarrod Ormiston

The University of Maastricht, Kapoenstraat 2, 6211 KR, Maastricht, The Netherlands

Pim Martens

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Melanie Feeney .

Ethics declarations

Conflict of interest.

This research involved no conflicts of interest.

Consent to Participants

No human or animal participants were involved.

Informed Consent

No informed consent was required for this research.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Feeney, M., Ormiston, J., Gijselaers, W. et al. Framing Collective Moral Responsibility for Climate Change: A Longitudinal Frame Analysis of Energy Company Climate Reporting. J Bus Ethics (2024). https://doi.org/10.1007/s10551-024-05801-0

Download citation

Received : 17 June 2022

Accepted : 08 August 2024

Published : 26 August 2024

DOI : https://doi.org/10.1007/s10551-024-05801-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Climate change
• Moral responsibility
• Energy companies
• Sustainability reporting
• Content analysis
• Find a journal
• Publish with us
• Track your research