literature review analysis and synthesis

Literature Syntheis 101

How To Synthesise The Existing Research (With Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Eunice Rautenbach (DTech) | August 2023

One of the most common mistakes that students make when writing a literature review is that they err on the side of describing the existing literature rather than providing a critical synthesis of it. In this post, we’ll unpack what exactly synthesis means and show you how to craft a strong literature synthesis using practical examples.

This post is based on our popular online course, Literature Review Bootcamp . In the course, we walk you through the full process of developing a literature review, step by step. If it’s your first time writing a literature review, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Literature Synthesis

• What exactly does “synthesis” mean?
• Aspect 1: Agreement
• Aspect 2: Disagreement
• Aspect 3: Key theories
• Aspect 4: Contexts
• Aspect 5: Methodologies
• Bringing it all together

What does “synthesis” actually mean?

As a starting point, let’s quickly define what exactly we mean when we use the term “synthesis” within the context of a literature review.

Simply put, literature synthesis means going beyond just describing what everyone has said and found. Instead, synthesis is about bringing together all the information from various sources to present a cohesive assessment of the current state of knowledge in relation to your study’s research aims and questions .

Put another way, a good synthesis tells the reader exactly where the current research is “at” in terms of the topic you’re interested in – specifically, what’s known , what’s not , and where there’s a need for more research .

So, how do you go about doing this?

Well, there’s no “one right way” when it comes to literature synthesis, but we’ve found that it’s particularly useful to ask yourself five key questions when you’re working on your literature review. Having done so,  you can then address them more articulately within your actual write up. So, let’s take a look at each of these questions.

1. Points Of Agreement

The first question that you need to ask yourself is: “Overall, what things seem to be agreed upon by the vast majority of the literature?”

For example, if your research aim is to identify which factors contribute toward job satisfaction, you’ll need to identify which factors are broadly agreed upon and “settled” within the literature. Naturally, there may at times be some lone contrarian that has a radical viewpoint , but, provided that the vast majority of researchers are in agreement, you can put these random outliers to the side. That is, of course, unless your research aims to explore a contrarian viewpoint and there’s a clear justification for doing so. 

Identifying what’s broadly agreed upon is an essential starting point for synthesising the literature, because you generally don’t want (or need) to reinvent the wheel or run down a road investigating something that is already well established . So, addressing this question first lays a foundation of “settled” knowledge.

Need a helping hand?

2. Points Of Disagreement

Related to the previous point, but on the other end of the spectrum, is the equally important question: “Where do the disagreements lie?” .

In other words, which things are not well agreed upon by current researchers? It’s important to clarify here that by disagreement, we don’t mean that researchers are (necessarily) fighting over it – just that there are relatively mixed findings within the empirical research , with no firm consensus amongst researchers.

This is a really important question to address as these “disagreements” will often set the stage for the research gap(s). In other words, they provide clues regarding potential opportunities for further research, which your study can then (hopefully) contribute toward filling. If you’re not familiar with the concept of a research gap, be sure to check out our explainer video covering exactly that .

3. Key Theories

The next question you need to ask yourself is: “Which key theories seem to be coming up repeatedly?” .

Within most research spaces, you’ll find that you keep running into a handful of key theories that are referred to over and over again. Apart from identifying these theories, you’ll also need to think about how they’re connected to each other. Specifically, you need to ask yourself:

• Are they all covering the same ground or do they have different focal points  or underlying assumptions ?
• Do some of them feed into each other and if so, is there an opportunity to integrate them into a more cohesive theory?
• Do some of them pull in different directions ? If so, why might this be?
• Do all of the theories define the key concepts and variables in the same way, or is there some disconnect? If so, what’s the impact of this ?

Simply put, you’ll need to pay careful attention to the key theories in your research area, as they will need to feature within your theoretical framework , which will form a critical component within your final literature review. This will set the foundation for your entire study, so it’s essential that you be critical in this area of your literature synthesis.

If this sounds a bit fluffy, don’t worry. We deep dive into the theoretical framework (as well as the conceptual framework) and look at practical examples in Literature Review Bootcamp . If you’d like to learn more, take advantage of our limited-time offer to get 60% off the standard price.

4. Contexts

The next question that you need to address in your literature synthesis is an important one, and that is: “Which contexts have (and have not) been covered by the existing research?” .

For example, sticking with our earlier hypothetical topic (factors that impact job satisfaction), you may find that most of the research has focused on white-collar , management-level staff within a primarily Western context, but little has been done on blue-collar workers in an Eastern context. Given the significant socio-cultural differences between these two groups, this is an important observation, as it could present a contextual research gap .

In practical terms, this means that you’ll need to carefully assess the context of each piece of literature that you’re engaging with, especially the empirical research (i.e., studies that have collected and analysed real-world data). Ideally, you should keep notes regarding the context of each study in some sort of catalogue or sheet, so that you can easily make sense of this before you start the writing phase. If you’d like, our free literature catalogue worksheet is a great tool for this task.

5. Methodological Approaches

Last but certainly not least, you need to ask yourself the question: “What types of research methodologies have (and haven’t) been used?”

For example, you might find that most studies have approached the topic using qualitative methods such as interviews and thematic analysis. Alternatively, you might find that most studies have used quantitative methods such as online surveys and statistical analysis.

But why does this matter?

Well, it can run in one of two potential directions . If you find that the vast majority of studies use a specific methodological approach, this could provide you with a firm foundation on which to base your own study’s methodology . In other words, you can use the methodologies of similar studies to inform (and justify) your own study’s research design .

On the other hand, you might argue that the lack of diverse methodological approaches presents a research gap , and therefore your study could contribute toward filling that gap by taking a different approach. For example, taking a qualitative approach to a research area that is typically approached quantitatively. Of course, if you’re going to go against the methodological grain, you’ll need to provide a strong justification for why your proposed approach makes sense. Nevertheless, it is something worth at least considering.

Regardless of which route you opt for, you need to pay careful attention to the methodologies used in the relevant studies and provide at least some discussion about this in your write-up. Again, it’s useful to keep track of this on some sort of spreadsheet or catalogue as you digest each article, so consider grabbing a copy of our free literature catalogue if you don’t have anything in place.

Bringing It All Together

Alright, so we’ve looked at five important questions that you need to ask (and answer) to help you develop a strong synthesis within your literature review.  To recap, these are:

• Which things are broadly agreed upon within the current research?
• Which things are the subject of disagreement (or at least, present mixed findings)?
• Which theories seem to be central to your research topic and how do they relate or compare to each other?
• Which contexts have (and haven’t) been covered?
• Which methodological approaches are most common?

Importantly, you’re not just asking yourself these questions for the sake of asking them – they’re not just a reflection exercise. You need to weave your answers to them into your actual literature review when you write it up. How exactly you do this will vary from project to project depending on the structure you opt for, but you’ll still need to address them within your literature review, whichever route you go.

The best approach is to spend some time actually writing out your answers to these questions, as opposed to just thinking about them in your head. Putting your thoughts onto paper really helps you flesh out your thinking . As you do this, don’t just write down the answers – instead, think about what they mean in terms of the research gap you’ll present , as well as the methodological approach you’ll take . Your literature synthesis needs to lay the groundwork for these two things, so it’s essential that you link all of it together in your mind, and of course, on paper.

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

excellent , thank you

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

The Sheridan Libraries

• Write a Literature Review
• Sheridan Libraries
• Find This link opens in a new window
• Evaluate This link opens in a new window

Get Organized

• Lit Review Prep Use this template to help you evaluate your sources, create article summaries for an annotated bibliography, and a synthesis matrix for your lit review outline.

Synthesize your Information

Synthesize: combine separate elements to form a whole.

Synthesis Matrix

A synthesis matrix helps you record the main points of each source and document how sources relate to each other.

After summarizing and evaluating your sources, arrange them in a matrix or use a citation manager to help you see how they relate to each other and apply to each of your themes or variables.  

By arranging your sources by theme or variable, you can see how your sources relate to each other, and can start thinking about how you weave them together to create a narrative.

• Step-by-Step Approach
• Example Matrix from NSCU
• Matrix Template
• << Previous: Summarize
• Next: Integrate >>
• Last Updated: Sep 26, 2023 10:25 AM
• URL: https://guides.library.jhu.edu/lit-review

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 
• How to write a good literature review 
• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

• Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 
• Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 
• Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 
• Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 
• Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 
• Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

How to write a good literature review

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. 

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• Life Sciences Papers: 9 Tips for Authors Writing in Biological Sciences
• What is an Argumentative Essay? How to Write It (With Examples)

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, what is academic writing: tips for students, why traditional editorial process needs an upgrade, paperpal’s new ai research finder empowers authors to..., what is hedging in academic writing  , how to use ai to enhance your college..., ai + human expertise – a paradigm shift..., how to use paperpal to generate emails &..., ai in education: it’s time to change the..., is it ethical to use ai-generated abstracts without..., do plagiarism checkers detect ai content.

• University of Oregon Libraries
• Research Guides

How to Write a Literature Review

• 6. Synthesize
• Literature Reviews: A Recap
• Reading Journal Articles
• Does it Describe a Literature Review?
• 1. Identify the Question
• 2. Review Discipline Styles
• Searching Article Databases
• Finding Full-Text of an Article
• Citation Chaining
• When to Stop Searching
• 4. Manage Your References
• 5. Critically Analyze and Evaluate

Synthesis Visualization

Synthesis matrix example.

• 7. Write a Literature Review

• Synthesis Worksheet

About Synthesis

Approaches to synthesis.

You can sort the literature in various ways, for example:

How to Begin?

Read your sources carefully and find the main idea(s) of each source

Look for similarities in your sources – which sources are talking about the same main ideas? (for example, sources that discuss the historical background on your topic)

Use the worksheet (above) or synthesis matrix (below) to get organized

This work can be messy. Don't worry if you have to go through a few iterations of the worksheet or matrix as you work on your lit review!

Four Examples of Student Writing

In the four examples below, only ONE shows a good example of synthesis: the fourth column, or  Student D . For a web accessible version, click the link below the image.

Long description of "Four Examples of Student Writing" for web accessibility

• Download a copy of the "Four Examples of Student Writing" chart

Click on the example to view the pdf.

From Jennifer Lim

• << Previous: 5. Critically Analyze and Evaluate
• Next: 7. Write a Literature Review >>
• Last Updated: Jan 10, 2024 4:46 PM
• URL: https://researchguides.uoregon.edu/litreview

Contact Us Library Accessibility UO Libraries Privacy Notices and Procedures

1501 Kincaid Street Eugene, OR 97403 P: 541-346-3053 F: 541-346-3485

• Visit us on Facebook
• Visit us on Twitter
• Visit us on Youtube
• Visit us on Instagram
• Report a Concern
• Nondiscrimination and Title IX
• Accessibility
• Privacy Policy
• Find People

Literature Reviews

• Introduction
• Tutorials and resources
• Step 1: Literature search
• Step 2: Analysis, synthesis, critique
• Step 3: Writing the review

If you need any assistance, please contact the library staff at the Georgia Tech Library Help website . 

Analysis, synthesis, critique

Literature reviews build a story. You are telling the story about what you are researching. Therefore, a literature review is a handy way to show that you know what you are talking about. To do this, here are a few important skills you will need.

Skill #1: Analysis

Analysis means that you have carefully read a wide range of the literature on your topic and have understood the main themes, and identified how the literature relates to your own topic. Carefully read and analyze the articles you find in your search, and take notes. Notice the main point of the article, the methodologies used, what conclusions are reached, and what the main themes are. Most bibliographic management tools have capability to keep notes on each article you find, tag them with keywords, and organize into groups.

Skill #2: Synthesis

After you’ve read the literature, you will start to see some themes and categories emerge, some research trends to emerge, to see where scholars agree or disagree, and how works in your chosen field or discipline are related. One way to keep track of this is by using a Synthesis Matrix .

Skill #3: Critique

As you are writing your literature review, you will want to apply a critical eye to the literature you have evaluated and synthesized. Consider the strong arguments you will make contrasted with the potential gaps in previous research. The words that you choose to report your critiques of the literature will be non-neutral. For instance, using a word like “attempted” suggests that a researcher tried something but was not successful. For example: 

There were some attempts by Smith (2012) and Jones (2013) to integrate a new methodology in this process.

On the other hand, using a word like “proved” or a phrase like “produced results” evokes a more positive argument. For example:

The new methodologies employed by Blake (2014) produced results that provided further evidence of X.

In your critique, you can point out where you believe there is room for more coverage in a topic, or further exploration in in a sub-topic.

Need more help?

If you are looking for more detailed guidance about writing your dissertation, please contact the folks in the Georgia Tech Communication Center .

• << Previous: Step 1: Literature search
• Next: Step 3: Writing the review >>
• Last Updated: Apr 2, 2024 11:21 AM
• URL: https://libguides.library.gatech.edu/litreview

Library Guides

Literature reviews: synthesis.

• Criticality

Synthesise Information

So, how can you create paragraphs within your literature review that demonstrates your knowledge of the scholarship that has been done in your field of study?  

You will need to present a synthesis of the texts you read.  

Doug Specht, Senior Lecturer at the Westminster School of Media and Communication, explains synthesis for us in the following video:  

Synthesising Texts  

What is synthesis? 

Synthesis is an important element of academic writing, demonstrating comprehension, analysis, evaluation and original creation.  

With synthesis you extract content from different sources to create an original text. While paraphrase and summary maintain the structure of the given source(s), with synthesis you create a new structure.  

The sources will provide different perspectives and evidence on a topic. They will be put together when agreeing, contrasted when disagreeing. The sources must be referenced.  

Perfect your synthesis by showing the flow of your reasoning, expressing critical evaluation of the sources and drawing conclusions.  

When you synthesise think of "using strategic thinking to resolve a problem requiring the integration of diverse pieces of information around a structuring theme" (Mateos and Sole 2009, p448). 

Synthesis is a complex activity, which requires a high degree of comprehension and active engagement with the subject. As you progress in higher education, so increase the expectations on your abilities to synthesise. 

How to synthesise in a literature review: 

Identify themes/issues you'd like to discuss in the literature review. Think of an outline.  

Read the literature and identify these themes/issues.  

Critically analyse the texts asking: how does the text I'm reading relate to the other texts I've read on the same topic? Is it in agreement? Does it differ in its perspective? Is it stronger or weaker? How does it differ (could be scope, methods, year of publication etc.). Draw your conclusions on the state of the literature on the topic.  

Start writing your literature review, structuring it according to the outline you planned.  

Put together sources stating the same point; contrast sources presenting counter-arguments or different points.  

Present your critical analysis.  

Always provide the references. 

The best synthesis requires a "recursive process" whereby you read the source texts, identify relevant parts, take notes, produce drafts, re-read the source texts, revise your text, re-write... (Mateos and Sole, 2009). 

What is good synthesis?  

The quality of your synthesis can be assessed considering the following (Mateos and Sole, 2009, p439):  

Integration and connection of the information from the source texts around a structuring theme. 

Selection of ideas necessary for producing the synthesis. 

Appropriateness of the interpretation.  

Elaboration of the content.  

Example of Synthesis

Original texts (fictitious): 

Synthesis: 

Animal experimentation is a subject of heated debate. Some argue that painful experiments should be banned. Indeed it has been demonstrated that such experiments make animals suffer physically and psychologically (Chowdhury 2012; Panatta and Hudson 2016). On the other hand, it has been argued that animal experimentation can save human lives and reduce harm on humans (Smith 2008). This argument is only valid for toxicological testing, not for tests that, for example, merely improve the efficacy of a cosmetic (Turner 2015). It can be suggested that animal experimentation should be regulated to only allow toxicological risk assessment, and the suffering to the animals should be minimised.   

Bibliography

Mateos, M. and Sole, I. (2009). Synthesising Information from various texts: A Study of Procedures and Products at Different Educational Levels. European Journal of Psychology of Education,  24 (4), 435-451. Available from https://doi.org/10.1007/BF03178760 [Accessed 29 June 2021].

• << Previous: Structure
• Next: Criticality >>
• Last Updated: Nov 18, 2023 10:56 PM
• URL: https://libguides.westminster.ac.uk/literature-reviews

CONNECT WITH US

How to Synthesize Written Information from Multiple Sources

Shona McCombes

Content Manager

B.A., English Literature, University of Glasgow

Shona McCombes is the content manager at Scribbr, Netherlands.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

On This Page:

When you write a literature review or essay, you have to go beyond just summarizing the articles you’ve read – you need to synthesize the literature to show how it all fits together (and how your own research fits in).

Synthesizing simply means combining. Instead of summarizing the main points of each source in turn, you put together the ideas and findings of multiple sources in order to make an overall point.

At the most basic level, this involves looking for similarities and differences between your sources. Your synthesis should show the reader where the sources overlap and where they diverge.

Unsynthesized Example

Franz (2008) studied undergraduate online students. He looked at 17 females and 18 males and found that none of them liked APA. According to Franz, the evidence suggested that all students are reluctant to learn citations style. Perez (2010) also studies undergraduate students. She looked at 42 females and 50 males and found that males were significantly more inclined to use citation software ( p < .05). Findings suggest that females might graduate sooner. Goldstein (2012) looked at British undergraduates. Among a sample of 50, all females, all confident in their abilities to cite and were eager to write their dissertations.

Synthesized Example

Studies of undergraduate students reveal conflicting conclusions regarding relationships between advanced scholarly study and citation efficacy. Although Franz (2008) found that no participants enjoyed learning citation style, Goldstein (2012) determined in a larger study that all participants watched felt comfortable citing sources, suggesting that variables among participant and control group populations must be examined more closely. Although Perez (2010) expanded on Franz’s original study with a larger, more diverse sample…

Step 1: Organize your sources

After collecting the relevant literature, you’ve got a lot of information to work through, and no clear idea of how it all fits together.

Before you can start writing, you need to organize your notes in a way that allows you to see the relationships between sources.

One way to begin synthesizing the literature is to put your notes into a table. Depending on your topic and the type of literature you’re dealing with, there are a couple of different ways you can organize this.

Summary table

A summary table collates the key points of each source under consistent headings. This is a good approach if your sources tend to have a similar structure – for instance, if they’re all empirical papers.

Each row in the table lists one source, and each column identifies a specific part of the source. You can decide which headings to include based on what’s most relevant to the literature you’re dealing with.

For example, you might include columns for things like aims, methods, variables, population, sample size, and conclusion.

For each study, you briefly summarize each of these aspects. You can also include columns for your own evaluation and analysis.

The summary table gives you a quick overview of the key points of each source. This allows you to group sources by relevant similarities, as well as noticing important differences or contradictions in their findings.

Synthesis matrix

A synthesis matrix is useful when your sources are more varied in their purpose and structure – for example, when you’re dealing with books and essays making various different arguments about a topic.

Each column in the table lists one source. Each row is labeled with a specific concept, topic or theme that recurs across all or most of the sources.

Then, for each source, you summarize the main points or arguments related to the theme.

The purposes of the table is to identify the common points that connect the sources, as well as identifying points where they diverge or disagree.

Step 2: Outline your structure

Now you should have a clear overview of the main connections and differences between the sources you’ve read. Next, you need to decide how you’ll group them together and the order in which you’ll discuss them.

For shorter papers, your outline can just identify the focus of each paragraph; for longer papers, you might want to divide it into sections with headings.

There are a few different approaches you can take to help you structure your synthesis.

If your sources cover a broad time period, and you found patterns in how researchers approached the topic over time, you can organize your discussion chronologically .

That doesn’t mean you just summarize each paper in chronological order; instead, you should group articles into time periods and identify what they have in common, as well as signalling important turning points or developments in the literature.

If the literature covers various different topics, you can organize it thematically .

That means that each paragraph or section focuses on a specific theme and explains how that theme is approached in the literature.

Source Used with Permission: The Chicago School

If you’re drawing on literature from various different fields or they use a wide variety of research methods, you can organize your sources methodologically .

That means grouping together studies based on the type of research they did and discussing the findings that emerged from each method.

If your topic involves a debate between different schools of thought, you can organize it theoretically .

That means comparing the different theories that have been developed and grouping together papers based on the position or perspective they take on the topic, as well as evaluating which arguments are most convincing.

Step 3: Write paragraphs with topic sentences

What sets a synthesis apart from a summary is that it combines various sources. The easiest way to think about this is that each paragraph should discuss a few different sources, and you should be able to condense the overall point of the paragraph into one sentence.

This is called a topic sentence , and it usually appears at the start of the paragraph. The topic sentence signals what the whole paragraph is about; every sentence in the paragraph should be clearly related to it.

A topic sentence can be a simple summary of the paragraph’s content:

“Early research on [x] focused heavily on [y].”

For an effective synthesis, you can use topic sentences to link back to the previous paragraph, highlighting a point of debate or critique:

“Several scholars have pointed out the flaws in this approach.” “While recent research has attempted to address the problem, many of these studies have methodological flaws that limit their validity.”

By using topic sentences, you can ensure that your paragraphs are coherent and clearly show the connections between the articles you are discussing.

As you write your paragraphs, avoid quoting directly from sources: use your own words to explain the commonalities and differences that you found in the literature.

Don’t try to cover every single point from every single source – the key to synthesizing is to extract the most important and relevant information and combine it to give your reader an overall picture of the state of knowledge on your topic.

Step 4: Revise, edit and proofread

Like any other piece of academic writing, synthesizing literature doesn’t happen all in one go – it involves redrafting, revising, editing and proofreading your work.

Checklist for Synthesis

•   Do I introduce the paragraph with a clear, focused topic sentence?
•   Do I discuss more than one source in the paragraph?
•   Do I mention only the most relevant findings, rather than describing every part of the studies?
•   Do I discuss the similarities or differences between the sources, rather than summarizing each source in turn?
•   Do I put the findings or arguments of the sources in my own words?
•   Is the paragraph organized around a single idea?
•   Is the paragraph directly relevant to my research question or topic?
•   Is there a logical transition from this paragraph to the next one?

Further Information

How to Synthesise: a Step-by-Step Approach

Help…I”ve Been Asked to Synthesize!

Learn how to Synthesise (combine information from sources)

How to write a Psychology Essay

Writing in the Health and Social Sciences: Literature Reviews and Synthesis Tools

• Journal Publishing
• Style and Writing Guides
• Readings about Writing
• Citing in APA Style This link opens in a new window
• Resources for Dissertation Authors
• Citation Management and Formatting Tools
• What are Literature Reviews?
• Conducting & Reporting Systematic Reviews
• Finding Systematic Reviews
• Tutorials & Tools for Literature Reviews

Systematic Literature Reviews: Steps & Resources

These steps for conducting a systematic literature review are listed below . 

Also see subpages for more information about:

• The different types of literature reviews, including systematic reviews and other evidence synthesis methods
• Tools & Tutorials

Literature Review & Systematic Review Steps

• Develop a Focused Question
• Scope the Literature  (Initial Search)
• Refine & Expand the Search
• Limit the Results
• Download Citations
• Abstract & Analyze
• Create Flow Diagram
• Synthesize & Report Results

1. Develop a Focused   Question 

Consider the PICO Format: Population/Problem, Intervention, Comparison, Outcome

Focus on defining the Population or Problem and Intervention (don't narrow by Comparison or Outcome just yet!)

"What are the effects of the Pilates method for patients with low back pain?"

Tools & Additional Resources:

• PICO Question Help
• Stillwell, Susan B., DNP, RN, CNE; Fineout-Overholt, Ellen, PhD, RN, FNAP, FAAN; Melnyk, Bernadette Mazurek, PhD, RN, CPNP/PMHNP, FNAP, FAAN; Williamson, Kathleen M., PhD, RN Evidence-Based Practice, Step by Step: Asking the Clinical Question, AJN The American Journal of Nursing : March 2010 - Volume 110 - Issue 3 - p 58-61 doi: 10.1097/01.NAJ.0000368959.11129.79

2. Scope the Literature

A "scoping search" investigates the breadth and/or depth of the initial question or may identify a gap in the literature. 

Eligible studies may be located by searching in:

• Background sources (books, point-of-care tools)
• Article databases
• Trial registries
• Grey literature
• Cited references
• Reference lists

When searching, if possible, translate terms to controlled vocabulary of the database. Use text word searching when necessary.

Use Boolean operators to connect search terms:

• Combine separate concepts with AND  (resulting in a narrower search)
• Connecting synonyms with OR  (resulting in an expanded search)

Search:  pilates AND ("low back pain"  OR  backache )

Video Tutorials - Translating PICO Questions into Search Queries

• Translate Your PICO Into a Search in PubMed (YouTube, Carrie Price, 5:11) 
• Translate Your PICO Into a Search in CINAHL (YouTube, Carrie Price, 4:56)

3. Refine & Expand Your Search

Expand your search strategy with synonymous search terms harvested from:

• database thesauri
• reference lists
• relevant studies

Example: 

(pilates OR exercise movement techniques) AND ("low back pain" OR backache* OR sciatica OR lumbago OR spondylosis)

As you develop a final, reproducible strategy for each database, save your strategies in a:

• a personal database account (e.g., MyNCBI for PubMed)
• Log in with your NYU credentials
• Open and "Make a Copy" to create your own tracker for your literature search strategies

4. Limit Your Results

Use database filters to limit your results based on your defined inclusion/exclusion criteria.  In addition to relying on the databases' categorical filters, you may also need to manually screen results.  

• Limit to Article type, e.g.,:  "randomized controlled trial" OR multicenter study
• Limit by publication years, age groups, language, etc.

NOTE: Many databases allow you to filter to "Full Text Only".  This filter is  not recommended . It excludes articles if their full text is not available in that particular database (CINAHL, PubMed, etc), but if the article is relevant, it is important that you are able to read its title and abstract, regardless of 'full text' status. The full text is likely to be accessible through another source (a different database, or Interlibrary Loan).  

• Filters in PubMed
• CINAHL Advanced Searching Tutorial

5. Download Citations

Selected citations and/or entire sets of search results can be downloaded from the database into a citation management tool. If you are conducting a systematic review that will require reporting according to PRISMA standards, a citation manager can help you keep track of the number of articles that came from each database, as well as the number of duplicate records.

In Zotero, you can create a Collection for the combined results set, and sub-collections for the results from each database you search.  You can then use Zotero's 'Duplicate Items" function to find and merge duplicate records.

• Citation Managers - General Guide

6. Abstract and Analyze

• Migrate citations to data collection/extraction tool
• Screen Title/Abstracts for inclusion/exclusion
• Screen and appraise full text for relevance, methods, 
• Resolve disagreements by consensus

Covidence is a web-based tool that enables you to work with a team to screen titles/abstracts and full text for inclusion in your review, as well as extract data from the included studies.

• Covidence Support
• Critical Appraisal Tools
• Data Extraction Tools

7. Create Flow Diagram

The PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) flow diagram is a visual representation of the flow of records through different phases of a systematic review.  It depicts the number of records identified, included and excluded.  It is best used in conjunction with the PRISMA checklist .

Example from: Stotz, S. A., McNealy, K., Begay, R. L., DeSanto, K., Manson, S. M., & Moore, K. R. (2021). Multi-level diabetes prevention and treatment interventions for Native people in the USA and Canada: A scoping review. Current Diabetes Reports, 2 (11), 46. https://doi.org/10.1007/s11892-021-01414-3

• PRISMA Flow Diagram Generator (ShinyApp.io, Haddaway et al. )
• PRISMA Diagram Templates  (Word and PDF)
• Make a copy of the file to fill out the template
• Image can be downloaded as PDF, PNG, JPG, or SVG
• Covidence generates a PRISMA diagram that is automatically updated as records move through the review phases

8. Synthesize & Report Results

There are a number of reporting guideline available to guide the synthesis and reporting of results in systematic literature reviews.

It is common to organize findings in a matrix, also known as a Table of Evidence (ToE).

• Reporting Guidelines for Systematic Reviews
• Download a sample template of a health sciences review matrix  (GoogleSheets)

Steps modified from: 

Cook, D. A., & West, C. P. (2012). Conducting systematic reviews in medical education: a stepwise approach.   Medical Education , 46 (10), 943–952.

• << Previous: Citation Management and Formatting Tools
• Next: What are Literature Reviews? >>
• Last Updated: Apr 13, 2024 9:10 PM
• URL: https://guides.nyu.edu/healthwriting

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Chapter 5: Writing a Summary and Synthesizing

5.5 Synthesis and Literature Reviews

Literature reviews : synthesis and research.

Why do we seek to understand the ways that authors or sources “converse” with one another? So that we can synthesize various perspectives on a topic to more deeply understand it.

In academic writing, this understanding of the “conversation” may become the content of an explanatory  synthesis  paper – a paper in which you, the writer, point out various various themes or key points from a conversation on a particular topic. Notice that the example of synthesis in “What Synthesis Is” acknowledges that guns and gun control inspire passionate responses in Americans, that more than one kind of weapon is involved in gun violence, that guns in America are both legally and illegally owned, and that there are many constituencies whose experience with guns needs to considered if sound gun-control policy is to be achieved. The writer of this synthesis isn’t “pretending” to be objective (“Although gun violence is a problem in American today, people who want to increase gun control clearly don’t understand the Second Amendment”); nor is the writer arguing a point or attempting to persuade the audience to accept one perspective. The writer is making a claim about gun control that demonstrates his or her deepest understanding of the issue.

Another assignment that you may complete that also applies your synthesis skills is a l iterature review .  Literature reviews are often found in the beginning of scholarly journal articles to contextualize the author’s own research. Sometimes, literature reviews are done for their own sake; some scholarly articles are  just  Literature reviews.

Literature reviews (sometimes shortened to “lit reviews”) synthesize previous research that has been done on a particular topic, summarizing important works in the history of research on that topic. The literature review provides context for the author’s own new research. It is the basis and background out of which the author’s research grows. Context = credibility in academic writing. When writers are able to produce a literature review, they demonstrate the breadth of their knowledge about how others have already studied and discussed their topic.

• Literature reviews are most often  arranged by topic or theme , much like a traditional explanatory synthesis paper.
• If one is looking at a topic that has a long history of research and scholarship, one may conduct a chronological  literature review, one that looks at how the research topic has been studied and discussed in various time periods (i.e., what was published ten years ago, five years ago, and within the last year, for example).
• Finally, in some instances, one might seek to craft a literature review that is organized  by discipline or field. This type of literature review could offer information about how different academic fields have examined a particular topic (i.e., what is the current research being done by biologists on this topic? What is the current research being done by psychologists on this topic? What is the current research being done by [ insert academic discipline] on this topic?).

A Literature Review offers  only  a report on what others have already written about. The Literature Review does not reflect the author’s own argument or contributions to the field of research. Instead, it indicates that the author has read others’ important contributions and understands what has come before him or her. Sometimes, literature reviews are stand alone assignments or publications. Sometimes, they fit into a larger essay or article (especially in many of the scholarly articles that you will read throughout college. For more information on how literature reviews are a part of scholarly articles, see chapter 10.5 )

A Guide to Rhetoric, Genre, and Success in First-Year Writing by Melanie Gagich & Emilie Zickel is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Feedback/Errata

Comments are closed.

Literature Review Basics

• What is a Literature Review?
• Synthesizing Research
• Using Research & Synthesis Tables
• Additional Resources

About the Research and Synthesis Tables

Research Tables and Synthesis Tables are useful tools for organizing and analyzing your research as you assemble your literature review. They represent two different parts of the review process: assembling relevant information and synthesizing it. Use a Research table to compile the main info you need about the items you find in your research -- it's a great thing to have on hand as you take notes on what you read! Then, once you've assembled your research, use the Synthesis table to start charting the similarities/differences and major themes among your collected items.

We've included an Excel file with templates for you to use below; the examples pictured on this page are snapshots from that file.

• Research and Synthesis Table Templates This Excel workbook includes simple templates for creating research tables and synthesis tables. Feel free to download and use!

Using the Research Table

This is an example of a  research table,  in which you provide a basic description of the most important features of the studies, articles, and other items you discover in your research. The table identifies each item according to its author/date of publication, its purpose or thesis, what type of work it is (systematic review, clinical trial, etc.), the level of evidence it represents (which tells you a lot about its impact on the field of study), and its major findings. Your job, when you assemble this information, is to develop a snapshot of what the research shows about the topic of your research question and assess its value (both for the purpose of your work and for general knowledge in the field).

Think of your work on the research table as the foundational step for your analysis of the literature, in which you assemble the information you'll be analyzing and lay the groundwork for thinking about what it means and how it can be used.

Using the Synthesis Table

This is an example of a  synthesis table  or  synthesis matrix , in which you organize and analyze your research by listing each source and indicating whether a given finding or result occurred in a particular study or article ( each row lists an individual source, and each finding has its own column, in which X = yes, blank = no). You can also add or alter the columns to look for shared study populations, sort by level of evidence or source type, etc. The key here is to use the table to provide a simple representation of what the research has found (or not found, as the case may be). Think of a synthesis table as a tool for making comparisons, identifying trends, and locating gaps in the literature.

How do I know which findings to use, or how many to include?  Your research question tells you which findings are of interest in your research, so work from your research question to decide what needs to go in each Finding header, and how many findings are necessary. The number is up to you; again, you can alter this table by adding or deleting columns to match what you're actually looking for in your analysis. You should also, of course, be guided by what's actually present in the material your research turns up!

• << Previous: Synthesizing Research
• Next: Additional Resources >>
• Last Updated: Sep 26, 2023 12:06 PM
• URL: https://usi.libguides.com/literature-review-basics

Should I do a synthesis (i.e. literature review)?

• Questions & Quandaries
• Published: 18 April 2024

Cite this article

• H. Carrie Chen 1 ,
• Ayelet Kuper 2 , 3 , 4 ,
• Jennifer Cleland 5 &
• Patricia O’Sullivan 6  

60 Accesses

Explore all metrics

This column is intended to address the kinds of knotty problems and dilemmas with which many scholars grapple in studying health professions education. In this article, the authors address the question of whether one should conduct a literature review or knowledge synthesis, considering the why, when, and how, as well as its potential pitfalls. The goal is to guide supervisors and students who are considering whether to embark on a literature review in education research.

Avoid common mistakes on your manuscript.

Two junior colleagues come to you to ask your advice about carrying out a literature review on a particular topic. “Should they?” immediately pops into your mind, followed closely by, if yes, then what kind of literature review is appropriate? Our experience is that colleagues often come to suggest a literature review to “kick start” their research (in fact, some academic programs require them as part of degree requirements), without a full understanding of the work involved, the different types of literature review, and what type of literature review might be most suitable for their research question. In this Questions and Quandaries, we address the question of literature reviews in education research, considering the why, when, and how, as well as potential pitfalls.

First, what is meant by literature review? The term literature review has been used to refer to both a review of the literature and a knowledge synthesis (Maggio et al., 2018 ; Siddaway et al., 2019 ). For our purposes, we employ the term as commonly used to refer to a knowledge synthesis , which is a formal comprehensive review of the existing body of literature on a topic. It is a research approach that critically integrates and synthesizes available evidence from multiple studies to provide insight and allow the drawing of conclusions. It is an example of Boyer’s scholarship of integration (Boyer, 1990 ). In contrast, a review of the literature is a relatively casual and expedient method for attaining a general overview of the state of knowledge on a given topic to make the argument that a new study is needed. In this interpretation, a literature review serves as a key starting point for anyone conducting research by identifying gaps in the literature, informing the study question, and situating one’s study in the field.

Whether a formal knowledge synthesis should be done depends on if a review is needed and what the rationale is for the review. The first question to consider is whether a literature review already exists. If no, is there enough literature published on the topic to warrant a review? If yes, does the previous review need updating? How long has it been since the last review and has the literature expanded so much or are there important new studies that need integrating to justify an updated review? Or were there flaws in the previous review that one intends to address with a new review? Or does one intend to address a different question than the focus of the previous review?

If the knowledge synthesis is to be done, it should be driven by a research question. What is the research question? Can it be answered by a review? What is the purpose of the synthesis? There are two main purposes for knowledge synthesis– knowledge support and decision support. Knowledge support summarizes the evidence while decision support takes additional analytical steps to allow for decision-making in particular contexts (Mays et al., 2005 ).

If the purpose is to provide knowledge support, then the question is how or what will the knowledge synthesis add to the literature? Will it establish the state of knowledge in an area, identify gaps in the literature/knowledge base, and/or map opportunities for future research? Cornett et al., performed a scoping review of the literature on professional identity, focusing on how professional identity is described, why the studies where done, and what constructs of identity were used. Their findings advanced understanding of the state of knowledge by indicating that professional identity studies were driven primarily by the desire to examine the impact of political, social and healthcare reforms and advances, and that the various constructs of professional identity across the literature could be categorized into five themes (Cornett et al., 2023 ).

If, on the other hand, the purpose of the knowledge synthesis is to provide decision support, for whom will the synthesis be relevant and how will it improve practice? Will the synthesis result in tools such as guidelines or recommendations for practitioners and policymakers? An example of a knowledge synthesis for decision support is a systematic review conducted by Spencer and colleagues to examine the validity evidence for use of the Ottawa Surgical Competency Operating Room Evaluation (OSCORE) assessment tool. The authors summarized their findings with recommendations for educational practice– namely supporting the use of the OSCORE for in-the-moment entrustment decisions by frontline supervisors in surgical fields but cautioning about the limited evidence for support of its use in summative promotions decisions or non-surgical contexts (Spencer et al., 2022 ).

If a knowledge synthesis is indeed appropriate, its methodology should be informed by its research question and purpose. We do not have the space to discuss the various types of knowledge synthesis except to say that several types have been described in the literature. The five most common types in health professions education are narrative reviews, systematic reviews, umbrella reviews (meta-syntheses), scoping reviews, and realist reviews (Maggio et al., 2018 ). These represent different epistemologies, serve different review purposes, use different methods, and result in different review outcomes (Gordon, 2016 ).

Each type of review lends itself best to answering a certain type of research question. For instance, narrative reviews generally describe what is known about a topic without necessarily answering a specific empirical question (Maggio et al., 2018 ). A recent example of a narrative review focused on schoolwide wellbeing programs, describing what is known about the key characteristics and mediating factors that influence student support and identifying critical tensions around confidentiality that could make or break programs (Tan et al., 2023 ). Umbrella reviews, on the other hand, synthesize evidence from multiple reviews or meta-analyses and can illuminate agreement, inconsistencies, or evolution of evidence on a topic. For example, an umbrella review on problem-based learning highlighted the shift in research focus over time from does it work, to how does it work, to how does it work in different contexts, and pointed to directions for new research (Hung et al., 2019 ).

Practical questions for those considering a literature review include whether one has the time required and an appropriate team to conduct a high-quality knowledge synthesis. Regardless of the type of knowledge synthesis and use of quantitative or qualitative methods, all require rigorous and clear methods that allow for reproducibility. This can take time, up to 12–18 months. A high-quality knowledge synthesis also requires a team whose members have expertise not only in the content matter, but also in knowledge synthesis methodology and in literature searches (i.e. a librarian). A team with multiple reviewers with a variety of perspectives can also help manage the volume of large reviews, minimize potential biases, and strengthen the critical analysis.

Finally, a pitfall one should be careful to avoid is merely summarizing everything in the literature without critical evaluation and integration of the information. A knowledge synthesis that merely bean counts or presents a collection of unconnected information that has not been reflected upon or critically analyzed does not truly advance knowledge or decision-making. Rather, it leads us back to our original question of whether it should have been done in the first place.

Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate (pp. 18–21). Princeton University Press.

Cornett, M., Palermo, C., & Ash, S. (2023). Professional identity research in the health professions—a scoping review. Advances in Health Sciences Education , 28 (2), 589–642.

Article   Google Scholar  

Gordon, M. (2016). Are we talking the same paradigm? Considering methodological choices in health education systematic review. Medical Teacher , 38 (7), 746–750.

Hung, W., Dolmans, D. H. J. M., & van Merrienboer, J. J. G. (2019). A review to identify key perspectives in PBL meta-analyses and reviews: Trends, gaps and future research directions. Advances in Health Sciences Education , 24 , 943–957.

Maggio, L. A., Thomas, A., & Durning, S. J. (2018). Knowledge synthesis. In T. Swanwick, K. Forrest, & B. C. O’Brien (Eds.), Understanding Medical Education: Evidence, theory, and practice (pp. 457–469). Wiley.

Mays, N., Pope, C., & Popay, J. (2005). Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. Journal of Health Services Research & Policy , 10 (1_suppl), 6–20.

Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology , 70 , 747–770.

Spencer, M., Sherbino, J., & Hatala, R. (2022). Examining the validity argument for the Ottawa Surgical Competency operating room evaluation (OSCORE): A systematic review and narrative synthesis. Advances in Health Sciences Education , 27 , 659–689.

Tan, E., Frambach, J., Driessen, E., & Cleland, J. (2023). Opening the black box of school-wide student wellbeing programmes: A critical narrative review informed by activity theory. Advances in Health Sciences Education . https://doi.org/10.1007/s10459-023-10261-8 . Epub ahead of print 02 July 2023.

Download references

Author information

Authors and affiliations.

Georgetown University School of Medicine, Washington, DC, USA

H. Carrie Chen

Wilson Centre for Research in Education, University Health Network, University of Toronto, Toronto, Canada

Ayelet Kuper

Division of General Internal Medicine, Sunnybrook Health Sciences Center, Toronto, Canada

Department of Medicine, University of Toronto, Toronto, Canada

Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore

Jennifer Cleland

University of California San Francisco School of Medicine, San Francisco, CA, USA

Patricia O’Sullivan

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to H. Carrie Chen .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Chen, H.C., Kuper, A., Cleland, J. et al. Should I do a synthesis (i.e. literature review)?. Adv in Health Sci Educ (2024). https://doi.org/10.1007/s10459-024-10335-1

Download citation

Published : 18 April 2024

DOI : https://doi.org/10.1007/s10459-024-10335-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 13 December 2023

Arts and creativity interventions for improving health and wellbeing in older adults: a systematic literature review of economic evaluation studies

• Grainne Crealey 1 ,
• Laura McQuade 2 ,
• Roger O’Sullivan 2 &
• Ciaran O’Neill 3  

BMC Public Health volume  23 , Article number:  2496 ( 2023 ) Cite this article

1166 Accesses

22 Altmetric

Metrics details

As the population ages, older people account for a larger proportion of the health and social care budget. A significant body of evidence suggests that arts and creativity interventions can improve the physical, mental and social wellbeing of older adults, however the value and/or cost-effectiveness of such interventions remains unclear.

We systematically reviewed the economic evidence relating to such interventions, reporting our findings according to PRISMA guidelines. We searched bibliographic databases (MEDLINE, EMBASE, Econlit and Web of Science and NHSEED), trial registries and grey literature. No language or temporal restrictions were applied. Two screening rounds were conducted independently by health economists experienced in systematic literature review. Methodological quality was assessed, and key information extracted and tabulated to provide an overview of the published literature. A narrative synthesis without meta-analysis was conducted.

Only six studies were identified which provided evidence relating to the value or cost-effectiveness of arts and creativity interventions to improve health and wellbeing in older adults. The evidence which was identified was encouraging, with five out of the six studies reporting an acceptable probability of cost-effectiveness or positive return on investment (ranging from £1.20 to over £8 for every £1 of expenditure). However, considerable heterogeneity was observed with respect to study participants, design, and outcomes assessed. Of particular concern were potential biases inherent in social value analyses.

Conclusions

Despite many studies reporting positive health and wellbeing benefits of arts and creativity interventions in this population, we found meagre evidence on their value or cost-effectiveness. Such evidence is costly and time-consuming to generate, but essential if innovative non-pharmacological interventions are to be introduced to minimise the burden of illness in this population and ensure efficient use of public funds. The findings from this review suggests that capturing data on the value and/or cost-effectiveness of such interventions should be prioritised; furthermore, research effort should be directed to developing evaluative methods which move beyond the confines of current health technology assessment frameworks, to capture a broader picture of ‘value’ more applicable to arts and creativity interventions and public health interventions more generally.

PROSPERO registration

CRD42021267944 (14/07/2021).

Peer Review reports

The number and proportion of older adults in the population has increased in virtually every country in the world over past decades [ 1 ]. In 2015, there were around 901 million people aged 60 years and over worldwide, by 2030, this will have increased to 1.4 billion [ 2 ]. An ageing population is one of the greatest successes of public health but it has implications for economies in numerous ways: slower labour force growth; working-age people will have to make greater provisions in welfare payments for older people who are no longer economically active; provisions for increased long-term care; and, society must adjust to the changing needs, expectations and capabilities of an expanding group of its citizens.

The Covid-19 pandemic shone an uncompromising light on the health and social care sector, highlighting the seriousness of gaps in policies, systems and services. It also focused attention on the physical and mental health consequences of loneliness and social isolation. To foster healthy ageing and improve the lives of older people, their families and communities, sustained and equitable investment in health and wellbeing is required [ 3 ]. The prevailing model of health and social care which is based ostensibly on formal care provision is unlikely to be sustainable over the longer term. New models, which promote healthy ageing and recognise the need for increasing reliance on self-care are required, as will be evidence of their effectiveness, cost-effectiveness and scalability.

Arts and creativity interventions (ACIs) can have positive effects on health and well-being, as several reviews have shown [ 4 , 5 ]. For older people, ACI’s can enhance wellbeing [ 6 , 7 , 8 , 9 ], quality of life [ 10 , 11 ] and cognitive function [ 12 , 13 , 14 , 15 , 16 ]. They can also foster social cohesion [ 17 , 18 , 19 ] and reduce social disparities and injustices [ 20 ]; promote healthy behaviour; prevent ill health (including enhancing well-being and mental health) [ 21 , 22 , 23 , 24 , 25 ], reducing cognitive decline [ 26 , 27 ], frailty [ 28 , 29 , 30 , 31 , 32 , 33 ] and premature mortality [ 34 , 35 , 36 , 37 , 38 ]); support people with stroke [ 39 , 40 , 41 , 42 ]; degenerative neurological disorders and dementias and support end of life care [ 43 , 44 ]. Moreover, ACIs can benefit not only individuals, but also others, such as supporting the well-being of formal and informal carers, enriching our knowledge of health, and improving clinical skills [ 4 , 5 ].

The benefits of ACIs have also been acknowledged at a governmental level by those responsible for delivering health and care services: The UK All-Party Parliamentary Special Interest group on Arts, Health and Wellbeing produced a comprehensive review of creative intervention for health and wellbeing [ 45 ]. This report contained three key messages: that the arts can keep us well, aid recovery and support longer better lived lives; they can help meet major challenges facing health and social care; and that the arts can save money for the health service and social care.

Despite robust scientific evidence and governmental support, no systematic literature review has collated the evidence with respect to the value, cost or cost-effectiveness of such interventions. Our objective was to assess the economic impact of ACIs aimed at improving the health and wellbeing of older adults; to determine the range and quality of available studies; identify gaps in the evidence-base; and guide future research, practice and policy.

A protocol for this review was registered at PROSPERO, an international prospective register of systematic reviews (Registration ID CRD42021267944). We used pre-determined criteria for considering studies to include in the review, in terms of types of studies, participant and intervention characteristics.

The review followed the five-step approach on how to prepare a Systematic Review of Economic Evaluations (SR-EE) for informing evidence-based healthcare decisions [ 46 , 47 , 48 ]. Subsequent to developing and registering the protocol, the International Society for Pharmacoeconomic Outcomes and Research (ISPOR) published a good practice task force report for the critical appraisal of systematic reviews with costs and cost-effectiveness outcomes (SR-CCEOs) [ 49 ]. This was also used to inform the conduct of this review.

Eligibility criteria

Full economic evaluations are regarded as the optimal type of evidence for inclusion in a SR-EE [ 46 ], hence cost-minimisation analyses (CMA), cost-effectiveness analyses (CEA), cost-utility analyses (CUA) and cost–benefit analyses (CBA) were included. Social value analyses were also included as they are frequently used to inform decision-making and commissioning of services within local government. Additionally, they represent an important intermediate stage in our understanding of the costs and consequences of public health interventions, where significant challenges exist with regard to performing full evaluations [ 50 , 51 , 52 , 53 ].

Development of search strategies

The population (P), intervention (I), comparator (C) and outcomes (O) (PICO) tool provided a framework for development of the search strategy. Studies were included if participants were aged 50 years or older (or if the average age of the study population was 50 years or over). Interventions could relate to performance art (dance, singing, theatre, drama etc.), creative and visual arts (painting, sculpture, art making and design), or creative writing (writing narratives, poetry, storytelling). The intervention had to be active (for example, creating art as opposed to viewing art; playing an instrument as opposed to listening to music). The objective of the intervention had to be to improve health and wellbeing; it had to be delivered under the guidance of a professional; delivered in a group setting and delivered on more than one occasion. No restrictions were placed on the type of comparator(s) or the type of outcomes captured in the study. We deliberately limited the study to professionally led activities to provide a sharper distinction between social events where arts and creativity may occur and arts and creativity interventions per se. We set no language restriction nor a restriction on the date from which studies were reported.

Search methods

PRESS (peer-review electronic search strategies) guidelines informed the design our search strategy [ 54 , 55 ] and an information specialist adapted the search terms (outlined in Table S 1 ) for the following electronic bibliographic databases: MEDLINE, PubMed, EMBASE, Econlit and Web of Science and NHSEED. We also inspected references of all relevant studies; and searched trials registers (ClinicalTrials.gov). Search terms used included cost, return on investment, economic, arts, music, storytelling, dancing, writing and older adult as well as social return on investment (SROI). The last search was performed on 09/11/2022. As many economic evaluations of ACIs (especially SROIs) are commissioned by government bodies or charitable organisations, a search of the grey literature was undertaken.

Handling searches

A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow chart was used to document study selection, illustrating the numbers of records retrieved and selection flow through the screening rounds [ 56 , 57 , 58 ]; all excluded records (with rationale for exclusion) were documented.

Selection of studies

Two screening rounds were conducted independently by two health economists experienced in undertaking reviews (GC, CO’N). The first round screened the title and abstract of articles based on the eligibility criteria; those selected at this stage entered a second round of full text screening with eligibility based on the inclusion and exclusion criteria. Any disagreements were discussed among the two reviewers, with access to a third reviewer available to resolve disagreements, though this proved unnecessary.

Data extraction and management

Two reviewers extracted relevant information independently using an proforma developed specifically for the purposes of this study, which included all 35 items suggested by Wijnen et al. (2016) [ 48 ]. Information was extracted in relation to the following factors: (1) general information including study title, author, year, funding source, country, setting and study design; (2) recruitment details, sample size, demographic characteristics (age, gender) and baseline health data (diagnosis, comorbidities); (3) interventions, effectiveness and cost data; (4) type of economic evaluation, perspective, payer, beneficiary, time horizon, measure of benefit and scale of intervention; (5) quality assessment, strength of evidence, any other important information; (6) results; (7) analysis of uncertainty and (8) conclusions. The quality assessment/risk of bias checklists were included in the data extraction proforma, and picklists were used to enhance uniformity of responses. The data extraction form was piloted by two reviewers (GC and CON) on one paper and discussion used to ensure consistent application thereafter.

Assessment of study quality

Two reviewers (GC & CON) independently assessed study quality, with recourse to a third reviewer for resolution of differences though this proved unnecessary. Quality assessment was based on the type of economic evaluation undertaken. Full and partial trial-based economic evaluations were assessed using the CHEC-extended checklist [ 59 ]. SROI analyses were assessed using a SROI-specific quality framework developed for the purpose of systematic review [ 60 ].

Data analysis methods

Due to the small number of evaluations detected, possible sources of heterogeneity and a lack of consensus on appropriate methods for pooling cost-effectiveness estimates [ 61 ] a narrative synthesis analysis was undertaken.

Database searches returned 11,619 records; from this, 402 duplicates were removed leaving 11,214 reports. From these 113 reports were assessment against the inclusion and exclusion criteria resulting in 4 studies for inclusion in the review. Over 40 websites were searched for relevant content returning 2 further studies for inclusion. The PRISMA 2020 diagram is presented in Fig.  1 . A high sensitivity search strategy was adopted to ensure all relevant studies were identified, resulting in a large number of studies being excluded at the first stage of screening.

PRISMA 2020 flow diagram for new systematic reviews which include searches of databases, registers and other sources

A total of six studies were identified; key characteristics are presented in Table 1 . Identified studies were published between 2011 and 2020. Two studies used a health technology assessment (HTA) framework alongside clinical trials [ 62 , 63 ] to assess the cost-effectiveness of community singing interventions. Both evaluations scored highly on the CHEC-extended checklist (Table 2 ), with findings reported in line with the CHEERS (Consolidated Health Economic Estimation Reporting Standards) checklist 2022 [ 64 ].

Four further studies employed an SROI framework to assess art and/or craft interventions: two studies were published in the peer-reviewed literature [ 65 , 66 ] and a further two in the grey literature [ 67 , 68 ]. All four adhered closely to the suggested steps for performing an SROI and consequently secured high scores (Table 3 ). No quality differential was discerned between those studies published in the academic literature when compared with those from the grey literature.

Five of the studies were undertaken in the UK [ 63 , 66 , 67 , 68 , 69 ] and one in the US [ 63 ]. Four of the studies were designed for older adults with no cognitive impairment [ 62 , 63 , 67 , 68 ]; one was designed for participants with or without dementia [ 65 ], and another was specifically for older adults with dementia and their caregivers [ 66 ]. Three of the studies were delivered in a community setting [ 62 , 63 , 67 ], two in care homes [ 65 , 68 ] and one across a range of settings (hospital, community and residential) [ 66 ]. The length and duration of the ACIs varied; some lasted 1–2 h (with multiple classes available to participants) [ 65 ], whereas others were structured programmes with sessions lasting 90 min over a 14-week period [ 62 ]. The number of participants included in studies varied; the largest study contained data from 390 participants [ 63 ], whereas other studies measured engagement using numbers of care homes or housing associations included [ 67 , 68 ].

Costs were captured from a narrower perspective (i.e., the payer—health service) for those economic evaluations which followed a health technology assessment (HTA) framework [ 62 , 63 ]. Costs associated with providing the programme and health and social care utilisation costs were captured using cost diaries. Valuation of resource usage was in line with the reference case specified for each jurisdiction.

Social value analyses included in the review [ 65 , 66 , 67 , 68 ] captured a broader picture of cost; programme provision costs included were similar in nature to those identified using an HTA framework, however, the benefits captured went beyond the individual to capture costs to a wide range of stakeholders such as family members, activity co-ordinations and care home personnel. Costs were apportioned using financial proxies from a range of sources including HACT Social Value Bank [ 69 ] and market-based valuation methods.

The range of outcomes captured and valued across HTAs and SROIs was extensive: including, but not limited to, wellbeing, quality of life, physical health, cognitive functioning, communication, control over daily life choices, engagement and empowerment, social isolation, mobility, community inclusion, depressive symptoms, sadness, anxiety, loneliness, positive affect and interest in daily life. In the programmes assessed using an HTA framework, outcomes were captured using standardised and validated instruments, for both control and intervention groups across multiple time points. Statistical methods were used to assess changes in outcomes over time. Programmes assessed using SROI relied primarily on qualitative methods (such as reflective diaries and in-depth interviews) combined with routinely collected administrative data.

The evidence from the singing interventions was encouraging but not conclusive. The ‘Silver Song Club’ programme [ 62 ] reported a 64% probability of being cost-effective at a willingness-to-pay threshold of £30,000. This study was also included in the Public Health England (PHE) decision tool to support local commissioners in designing and implementing services to support older people’s healthy ageing, reporting a positive societal return on investment [ 70 ]. Evidence from the ‘Community of Voices’ trial [ 63 ] suggested that although intervention group members experienced statistically significant improvements in loneliness and interest in life compared to control participants, no significant group differences were observed for cognitive or physical outcomes or for healthcare costs.

A positive return on investment was reported by all social value analyses undertaken. The ‘Imagine Arts’ programme, reported a positive SROI of £1.20 for every £1 of expenditure [ 65 ]. A higher yield of between £3.20-£6.62 for each £1 invested was reported in the ‘Dementia and Imagination’ programme [ 66 ]. The ‘Craft Café’ programme, reported an SROI of £8.27 per £1 invested [ 68 ], and the ‘Creative Caring’ programme predicted a SROI of between £3 to £4 for every £1 spent [ 67 ]. The time period over which return on investment was calculated differed for each evaluation from less than one year to 4 years.

The primary finding from our review concerns the paucity of evidence relating to the value, cost and/or cost-effectiveness of ACIs aimed at improving health and wellbeing in this population. Despite few restrictions being applied to our search, only six studies were found which met our inclusion criteria. This is not indicative of research into ACIs in this population, as evidenced by the identification of ninety-three studies where arts and creativity interventions were found to support better health and wellbeing outcomes in another recent review [ 5 ]. An alternative explanation is that funders do not see the added value of undertaking such evaluations in this area. That is, for funders, the cost of evaluating an ACIs is likely to be deemed unjustified given the relatively small welfare loss a misallocation of resources to them might produce. While at first glance this may seem reasonable, it disadvantages ACIs in competing with other interventions for funding and arguably exposes an implicit prejudice in the treatment of interventions from which it may be difficult to extract profit in general. That is, the paucity of evidence, may reflect inherent biases within our political economy that favour the generation of marketable solutions to health issues from which value can be appropriated as profit. Pharmaceuticals are an obvious example of such solutions, where the literature is replete with examples of evaluations sponsored by pharmaceutical companies or where public funds are used to test the claims made by pharmaceutical companies in respect of the value of their products. If the potential of ACIs to improve health and well-being is to be robustly established, ACIs must effectively compete for funding with other interventions including those from pharma. This requires a larger, more robust evidence base than is currently available and investment in the creation of such an evidence base. As there is currently no ‘for-profit’ industry to generate such an evidence base, public funding of evaluations will be central to its creation.

Our second finding concerns the values reported in the meagre evidence we did find. In five of the six studies we identified, evidence indicated that ACIs targeted at older people offered value for money [ 62 , 65 , 66 , 67 , 68 ]. One study provided mixed evidence [ 63 ], however, in this study a ‘payer’ perspective was adopted when applying an HTA framework which, by virtue of the perspective adopted, excluded a range of benefits attributable to ACIs and public health interventions more generally. Among the four studies that adopted a SROI approach, estimated returns per £1 invested ranged from £1.20 to £8.27. Given the evident heterogeneity among studies in terms of context and methods, care is warranted in comparing estimates with each other or with other SROIs. Care is also required in accepting at face value the estimates reported given methodological issues that pertain to the current state of the art with respect to SROI. With these caveats in mind noted, the values reported for ACIs using the SROI approach are comparable with those from other SROI studies in other contexts including those as diverse as a first aid intervention [ 71 ], investment in urban greenways [ 72 ] and the provision of refuge services to those experiencing domestic violence [ 73 ] (a return on investment of £3.50-£4, £2.88-£5.81 and £4.94 respectively). Similarly, with respect to the study that adopted a cost-effectiveness approach, Coulton and colleagues (2015) reported a 64% probability of the intervention being cost-effective at a threshold of £30,000 [ 62 ]. Again, it is difficult to compare studies directly, but this is similar to that reported for interventions as diverse as a falls prevention initiative [ 74 ] and the treatment of depression using a collaborative approach [ 75 ] both in the UK. That the evidence base is meagre notwithstanding, there is, in other words, a prima facie case that ACIs are capable of offering value for money when targeted at older persons.

Our third finding relates to the state of the art with respect to SROIs in this area. Over the past 40 years, considerable time, effort and resources have been expended in the development of cost-effectiveness techniques in health and social care. While considerable heterogeneity can exist around their conduct, national guidance exists in many jurisdictions on the conduct of cost-effectiveness analyses (CEA) – such as the NICE reference case in the UK [ 76 ]– as well as in the reporting of these as set out in the CHEERS 2022 guidance [ 64 ]. This has helped raise the quality of published evaluations and the consistency with which they are reported. Despite the existence of a step-by-step guidance document on how to perform SROIs [ 77 ] which outlines how displacement effects, double counting, effect attribution and drop-off should be addressed, a significant body of work still remains to ensure that the methodology addresses a range of known biases in a robust manner. Where there is no comparator to the intervention being evaluated (as was the case in the SROIs reported here) it may be difficult to convince funders that the implicit incremental costs and benefits reported are indeed incremental and attributable to the intervention. Equally, where a comparator is present, greater consensus and standardisation is required regarding the identification, generation and application of, for example, financial proxies. Currently, SROI ratios combine value across a wide range of stakeholders, which is understandable if the objective is to capture all aspects of social benefit generated. This ratio, however, may not reflect the priorities and statutory responsibilities of healthcare funders. Whist all of the aforementioned issues can be addressed, investment is required to develop the SROI methodology further to more closely meet the needs of commissioning bodies.

Notwithstanding these challenges, social value analyses play a pivotal role within the procurement processes employed by government, local authorities and other non-departmental public bodies and should not be dismissed simply because the ‘burden of proof’ falls short of that required to secure remuneration within the health sector. As most SROIs are published in the grey literature, this means they often avoid peer scrutiny prior to publication and the potential quality assurance this can offer. It is noteworthy however that two of the SROIs included in this review [ 65 , 66 ] were published in the academic literature, suggesting that the academic community are engaging with this method which is to be applauded.

Moving forward, it is unlikely we will be able to meet all of the health and wellbeing needs of our ageing population solely in a primary or secondary care setting. New models of care are required, as are new models of funding to support interventions which can be delivered in non-healthcare settings. New hybrid models of evaluation will be required to provide robust economic evidence to assist in the allocation of scarce resources across health and non-healthcare settings; such evaluative frameworks must have robust theoretical underpinnings and be capable of delivering evidence from a non-clinical setting in a timely and cost-effective manner.

In the absence of a definitive evaluation framework for ACIs being currently available, we have a number of recommendations. First, and most importantly, all impact assessments should have a control group or credible counterfactual. This is currently not required when performing an SROI making it difficult to determine if all of the benefits ascribed to an intervention are in fact attributable. This recommendation is in line with the conclusion of a report by the London School of Economics [ 78 ] for the National Audit Office (NAO) which concluded that ‘any impact evaluation (and subsequent value for money calculation) requires construction of a counterfactual’. Second, a detailed technical appendix should accompany all impact assessments to allow independent review by a subject specialist. While this would assist peer review, it would allow providing greater transparency where peer review was not undertaken prior to publication. Furthermore, it would enable recalculation of SROI ratios to exclude ‘value’ attributable to stakeholders which are not relevant to a particular funder. Third, equity considerations should be addressed explicitly in all evaluations (this is currently not required in HTAs). Fourth, both costs and outcomes should be captured from a ‘broad’ perspective (adopting a ‘narrow’ healthcare perspective may underestimate the full economic impact), with non-healthcare sector costs being detailed as part of the analysis. Finally, data should be collected post-implementation to ensure that resources continue to be allocated efficiently.

As with any review, there are limitations which should be noted. A search of the grey literature was included as evaluations of applied public health interventions are not always reported in the academic literature. Systematically identifying grey literature and grey data can be problematic [ 79 , 80 , 81 , 82 , 83 ] as it is not collected, organised or stored in a consistent manner. Hence it is possible that we have not identified all relevant studies. Furthermore, as applied public health interventions can be performed in a non-healthcare setting we included SROIs in our review of economic evaluations. Current guidance on the systematic review of economic evaluations has been developed primarily for review of HTA as opposed to public health interventions and hence SROIs would be excluded, or if included would score poorly due to the inherent biases arising from no comparator or counterfactual being included.

This systematic review found that participation in group-based arts and creativity programmes was generally cost-effective and/or produced a positive return on investment whilst having a positive impact on older people’s physical, psychological, and social health and wellbeing outcomes. Unfortunately, the small number of studies identified, coupled with differences in methods used to assess economic impact hinders our ability to conclusively determine which types of art and creativity-based activities are more cost-effective or represent best value for money.

As well as the need for a greater focus on prevention of poor health as we age, new hybrid models of healthcare delivery are necessary to meet the needs of our ageing population. These models will integrate traditional medical care with other services such as home health aides (some of which may include artificial intelligence), telemedicine and social support networks. Alongside these, ACIs have the potential to provide a low cost, scalable, easily implementable and cost-effective solution to reduce the burden of illness in this age group and support healthy ageing.

Evidence on the cost-effectiveness of a range of ACIs is of utmost importance for policy and decision makers as it can both inform the development of policies that support the provision of ACIs in the context of ageing, but also identify the most cost-effective approaches for delivering such interventions. The development of hybrid models of evaluation, capable of capturing cost-effectiveness and social value, is becoming increasingly necessary as healthcare delivery for this age group moves beyond the realms of primary and secondary care and into the community. The development and refinement of such models will ensure a more comprehensive assessment of the impact of a diverse range of interventions providing a more nuanced understanding of the impact of an intervention. This will help inform decision making and ensure interventions are implemented in a cost-effective and socially beneficial manner.

Availability of data and materials

All data generated or analysed during this study are included in the published article and its supplementary information files.

United Nations, Department of Economic and Social Affairs, Population Division. World Population Prospects: The 2015 Revision, Key Findings and Advance Tables. Working Paper No. ESA/P/WP.241. 2015.

Office for National Statistics. Living longer: how our population is changing and why it matters. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/ageing/articles/livinglongerhowourpopulationischangingandwhyitmatters/2018-08-13#how-do-incomes-of-older-people-compare-with-younger-ages . 2018. Accessed 07/12/2022

Dyakova M, Hamelmann C, Bellis MA, Besnier E, Grey CNB, Ashton K, Schwappach A, Clar C. Investment for health and well-being: a review of the social return on investment from public health policies to support implementing the Sustainable Development Goals by building on Health 2020 [Internet]. Copenhagen: WHO Regional Office for Europe; 2017.

Google Scholar  

Fancourt D, Finn S. What is the evidence on the role of the arts in improving health and well-being? A scoping review. Copenhagen: WHO Regional Office for Europe; 2019.

McQuade L, O’Sullivan R. Examining arts and creativity in later life and its impact on older people’s health and wellbeing: a systematic review of the evidence. Perspect Publ Health. 2023;0(0). https://doi.org/10.1177/17579139231157533

Skingley A, De’Ath S, Napleton L. Evaluation of Edna: arts and dance for older people. Work Older People. 2016;20(1):46–56.

Article   Google Scholar  

Brustio PR, Liubicich ME, Chiabrero M, et al. Dancing in the golden age: a study on physical function, quality of life, and social engagement. Geriatr Nurs. 2018;39(6):635–9.

Article   PubMed   Google Scholar  

Beauchet O, Bastien T, Mittelman M, et al. Participatory art-based activity, community-dwelling older adults and changes in health condition: results from a pre-post intervention, single-arm, prospective and longitudinal study. Maturitas. 2020;134:8–14.

Article   CAS   PubMed   Google Scholar  

Roswiyani R, Hiew CH, Witteman CLM, et al. Art activities and qigong exercise for the well-being of older adults in nursing homes in Indonesia: a randomized controlled trial. Aging Ment Health. 2020;24(10):1569–78.

Shanahan J, Bhriain ON, Morris ME, et al. Irish set dancing classes for people with Parkinson’s disease: the needs of participants and dance teachers. Complement Ther Med. 2016;27:12–7.

Garcia Gouvêa JA, Antunes MD, Bortolozzi F, et al. Impact of senior dance on emotional and motor parameters and quality of life of the elderly. Rev Rene. 2017;18(1):51–8.

Sun J, Zhang N, Buys N, et al. The role of Tai Chi, cultural dancing, playing a musical instrument and singing in the prevention of chronic disease in Chinese older adults: a mind–body meditative approach. Int J Ment Health Pr. 2013;15:227–39.

Fu MC, Belza B, Nguyen H, et al. Impact of group-singing on older adult health in senior living communities: a pilot study. Arch Gerontol Geriatr. 2018;76:138–46.

Feng L, Romero-Garcia R, Suckling J, et al. Effects of choral singing versus health education on cognitive decline and aging: a randomized controlled trial. Aging-us. 2020;12(24):24798–816.

Seinfeld S, Figueroa H, Ortiz-Gil J, et al. Effects of music learning and piano practice on cognitive function, mood and quality of life in older adults. Front Psychol. 2013;4:810.

Article   PubMed   PubMed Central   Google Scholar  

MacRitchie J, Breaden M, Milne AJ, et al. Cognitive, motor and social factors of music instrument training programs for older adults’ improved wellbeing. Front Psychol. 2020;10:2868.

Freeman WJI. A neurobiological role of music in social bonding. In: Wallin N, Merkur B, Brown S, editors. The origins of music. Cambridge: MIT Press; 2000. http://escholarship.org/uc/item/9025x8rt .

Huron D. Is music an evolutionary adaptation? Ann N Y Acad Sci. 2001;930(1):43–61. https://doi.org/10.1111/j.1749-6632.2001.tb05724.x .

Tarr B, Launay J, Dunbar RIM. Music and social bonding: “self–other” merging and neurohormonal mechanisms. Front Psychol. 2014;5:1096. https://doi.org/10.3389/fpsyg.2014.01096 .

Cain M, Lakhani A, Istvandity L. Short and long term outcomes for culturally and linguistically diverse (cald) and at-risk communities in participatory music programs: a systematic review. Arts Health. 2016;8(2):105–24. https://doi.org/10.1080/17533015.2015.1027934 .

Martin L, Oepen R, Bauer K, Nottensteiner A, Mergheim K, Gruber H, et al. Creative arts interventions for stress management and prevention – a systematic review. Behav Sci (Basel). 2018;8(2):pii:E28. https://doi.org/10.3390/bs8020028 .

Linnemann A, Wenzel M, Grammes J, Kubiak T, Nater UM. Music listening and stress in daily life: a matter of timing. Int J Behav Med. 2018;25(2):223–30. https://doi.org/10.1007/s12529-017-9697-5 .

Linnemann A, Strahler J, Nater UM. The stress-reducing effect of music listening varies depending on the social context. Psychoneuroendocrinology. 2016;72:97–105. https://doi.org/10.1016/j.psyneuen.2016.06.003 .

Panteleeva Y, Ceschi G, Glowinski D, Courvoisier DS, Grandjean DM. Music for anxiety? meta-analysis of anxiety reduction in non-clinical samples. Psychol Music. 2017;46(4):473–87. https://doi.org/10.1177/0305735617712424 .

Fancourt D, Tymoszuk U. Cultural engagement and incident depression in older adults: evidence from the English longitudinal study of ageing. Br J Psychiatry. 2018;214(4):225–9. https://doi.org/10.1192/bjp.2018.267 .

Balbag MA, Pedersen NL, Gatz M. Playing a musical instrument as a protective factor against dementia and cognitive impairment: a population-based twin study. Int J Alzheimer’s Dis. 2014;2014:836748. https://doi.org/10.1155/2014/836748 .

Porat S, Goukasian N, Hwang KS, Zanto T, Do T, Pierce J, et al. Dance experience and associations with cortical gray matter thickness in the aging population. Dement Geriatr Cogn Dis Extra. 2016;6(3):508–17. https://doi.org/10.1159/000449130 .

Federici A, Bellagamba S, Rocchi MBL. Does dance-based training improve balance in adult and young old subjects? a pilot randomized controlled trial. Aging Clin Exp Res. 2005;17(5):385–9 PMID: 16392413.

Alpert PT, Miller SK, Wallmann H, Havey R, Cross C, Chevalia T, et al. The effect of modified jazz dance on balance, cognition, and mood in older adults. J Am Acad Nurse Pract. 2009;21(2):108–15. https://doi.org/10.1111/j.1745-7599.2008.00392.x .

Jeon MY, Bark ES, Lee EG, Im JS, Jeong BS, Choe ES. The effects of a Korean traditional dance movement program in elderly women. Taehan Kanho Hakhoe Chi. 2005;35(7):126876 (in Korean). PMID: 16418553.

Eyigor S, Karapolat H, Durmaz B, Ibisoglu U, Cakir S. A randomized controlled trial of Turkish folklore dance on the physical performance, balance, depression and quality of life in older women. Arch Gerontol Geriatr. 2009;48(1):84–8. https://doi.org/10.1016/j.archger.2007.10.008 .

Noopud P, Suputtitada A, Khongprasert S, Kanungsukkasem V. Effects of Thai traditional dance on balance performance in daily life among older women. Aging Clin Exp Res. 2018;31(7):961–7. https://doi.org/10.1007/s40520-018-1040-8 .

Trombetti A, Hars M, Herrmann FR, Kressig RW, Ferrari S, Rizzoli R. Effect of musicbased multitask training on gait, balance, and fall risk in elderly people: a randomized controlled trial. Arch Intern Med. 2011;171(6):525–33. https://doi.org/10.1001/archinternmed.2010.446 .

Hyyppä MT, Mäki J, Impivaara O, Aromaa A. Individual-level measures of social capital as predictors of all-cause and cardiovascular mortality: a population-based prospective study of men and women in Finland. Eur J Epidemiol. 2007;22(9):589–97. https://doi.org/10.1007/s10654-007-9153-y .

Hyyppä MT, Mäki J, Impivaara O, Aromaa A. Leisure participation predicts survival: a population-based study in Finland. Health Promot Int. 2006;21(1):5–12. https://doi.org/10.1093/heapro/dai027 .

Lennartsson C, Silverstein M. Does engagement with life enhance survival of elderly people in Sweden? the role of social and leisure activities. J Gerontol B Psychol Sci Soc Sci. 2001;56(6):S335–42. https://doi.org/10.1093/geronb/56.6.s335 .

Sundquist K, Lindström M, Malmström M, Johansson SE, Sundquist J. Social participation and coronary heart disease: a follow-up study of 6900 women and men in Sweden. Soc Sci Med. 1982;58(3):615–22. https://doi.org/10.1016/s0277-9536(03)00229-6 .

Väänänen A, Murray M, Koskinen A, Vahtera J, Kouvonen A, Kivimäki M. Engagement in cultural activities and cause-specific mortality: prospective cohort study. Prev Med. 2009;49(2–3):142–7. https://doi.org/10.1016/j.ypmed.2009.06.026 .

Särkämö T, Soto D. Music listening after stroke: beneficial effects and potential neural mechanisms. Ann N Y Acad Sci. 2012;1252(1):266–81. https://doi.org/10.1111/j.1749-6632.2011.06405.x .

Särkämö T, Pihko E, Laitinen S, Forsblom A, Soinila S, Mikkonen M, et al. Music and speech listening enhance the recovery of early sensory processing after stroke. J Cogn Neurosci. 2010;22(12):2716–27. https://doi.org/10.1162/jocn.2009.21376 .

Särkämö T, Ripollés P, Vepsäläinen H, Autti T, Silvenno HM, Salli E, et al. Structural changes induced by daily music listening in the recovering brain after middle cerebral artery stroke: a voxel-based morphometry study. Front Hum Neurosci. 2014;8:245. https://doi.org/10.3389/fnhum.2014.00245 .

Särkämö T, Tervaniemi M, Laitinen S, Forsblom A, Soinila S, Mikkonen M, et al. Music listening enhances cognitive recovery and mood after middle cerebral artery stroke. Brain. 2008;131(3):866–76. https://doi.org/10.1093/brain/awn013 .

Fancourt D, Steptoe A, Cadar D. Cultural engagement and cognitive reserve: museum attendance and dementia incidence over a 10-year period. Br J Psychiatry. 2018;213(5):661–3. https://doi.org/10.1192/bjp.2018.129 .

Fancourt D, Steptoe A, Cadar D. Cultural engagement predicts changes in cognitive function in older adults over a 10 year period: findings from the English longitudinal study of ageing. Sci Rep. 2018;8(1):10226. https://doi.org/10.1192/bjp.2018.129 .

All Party Parliamentary group on arts, health and wellbeing. Creative health: the arts for health and wellbeing. 2017.

van Mastrigt GA, Hiligsmann M, Arts JJ, Broos PH, Kleijnen J, Evers SM, Majoie MH. How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: a five-step approach (part 1/3). Expert Rev Pharmacoecon Outcomes Res. 2016;16(6):689–704. https://doi.org/10.1080/14737167.2016.1246960 . Epub 2016 Nov 2 PMID: 27805469.

Thielen FW, Van Mastrigt G, Burgers LT, Bramer WM, Majoie H, Evers S, Kleijnen J. How to prepare a systematic review of economic evaluations for clinical practice guidelines: database selection and search strategy development (part 2/3). Expert Rev Pharmacoecon Outcomes Res. 2016;16(6):705–21. https://doi.org/10.1080/14737167.2016.1246962 . Epub 2016 Nov 2 PMID: 27805466.

Wijnen B, Van Mastrigt G, Redekop WK, Majoie H, De Kinderen R, Evers S. How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: data extraction, risk of bias, and transferability (part 3/3). Expert Rev Pharmacoecon Outcomes Res. 2016;16(6):723–32. https://doi.org/10.1080/14737167.2016.1246961 . Epub 2016 Oct 21 PMID: 27762640.

Mandrik OL, Severens JLH, Bardach A, Ghabri S, Hamel C, Mathes T, Vale L, Wisløff T, Goldhaber-Fiebert JD. Critical appraisal of systematic reviews with costs and cost-effectiveness outcomes: an ISPOR good practices task force report. Value Health. 2021;24(4):463–72. https://doi.org/10.1016/j.jval.2021.01.002 . PMID: 33840423.

Kelly MP, McDaid D, Ludbrook A, Powell J: Economic appraisal of public health interventions. http://www.cawt.com/Site/11/Documents/Publications/Population%20Health/Economics%20of%20Health%20Improvement/Economic_appraisal_of_public_health_interventions.pdf

Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, Rice N, Sculpher M, Sowden A. Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations. Health Policy. 2009;93(2–3):85–92. https://doi.org/10.1016/j.healthpol.2009.07.012 . Epub 2009 Aug 25 PMID: 19709773.

Payne K, McAllister M, Davies LM. Valuing the economic benefits of complex interventions: when maximising health is not sufficient. Health Econ. 2012. https://doi.org/10.1002/hec.2795 .

Edwards RT, Charles JM, Lloyd-Williams H. Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues. BMC Public Health. 2013;24(13):1001. https://doi.org/10.1186/1471-2458-13-1001.PMID:24153037;PMCID:PMC4015185 .

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26. https://doi.org/10.1016/j.jclinepi.2014.11.025 . Epub 2015 Feb 7 PMID: 25766056.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6. https://doi.org/10.1016/j.jclinepi.2016.01.021 . Epub 2016 Mar 19 PMID: 27005575.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement. PLoS Med. 2008;6:e1000097. https://doi.org/10.1371/journal.pmed.1000097 .

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;21(339):b2700. https://doi.org/10.1136/bmj.b2700.PMID:19622552;PMCID:PMC2714672 .

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group. PLoS Med. 2009;6(7):e1000097 Evers S, Goossens M, De Vet H, et al. Criteria list for assessment of methodological quality of economic evaluations: consensus on health economic criteria. Int J Technol Assess Health Care. 2005;21(02):240–245.

Evers S, Goossens M, de Vet H, van Tulder M, Ament A. Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. Int J Technol Assess Health Care. 2005;21(2):240–5 PMID: 15921065.

Hutchinson CL, Berndt A, Gilbert-Hunt S, George S, Ratcliffe J. Valuing the impact of health and social care programmes using social return on investment analysis: how have academics advanced the methodology? A protocol for a systematic review of peer-reviewed literature. BMJ Open. 2018;8(12):e022534. https://doi.org/10.1136/bmjopen-2018-022534 . PMID:30530579;PMCID:PMC6303612.

Higgins J, Green S. Cochrane handbook for systematic reviews of interventions version 5.1. 0. Chichester: The Cochrane Collaboration; 2013.

Coulton S, Clift S, Skingley A, Rodriguez J. Effectiveness and cost-effectiveness of community singing on mental health-related quality of life of older people: randomised controlled trial. Br J Psychiatry. 2015;207(3):250–5. https://doi.org/10.1192/bjp.bp.113.129908 . Epub 2015 Jun 18 PMID: 26089304.

Johnson JK, Stewart AL, Acree M, Nápoles AM, Flatt JD, Max WB, Gregorich SE. A community choir intervention to promote well-being among diverse older adults: results from the community of voices trial. J Gerontol B Psychol Sci Soc Sci. 2020;75(3):549–59. https://doi.org/10.1093/geronb/gby132 . PMID:30412233;PMCID:PMC7328053.

Husereau D, Drummond M, Augustovski F, de Bekker-Grob E, Briggs AH, Carswell C, Caulley L, Chaiyakunapruk N, Greenberg D, Loder E, Mauskopf J, Mullins CD, Petrou S, Pwu RF, Staniszewska S, CHEERS 2022 ISPOR Good Research Practices Task Force. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. Value Health. 2022;25(1):3–9. https://doi.org/10.1016/j.jval.2021.11.1351 . PMID: 35031096.

Bosco A, Schneider J, Broome E. The social value of the arts for care home residents in England: a social return on investment (SROI) analysis of the imagine arts programme. Maturitas. 2019;124:15–24. https://doi.org/10.1016/j.maturitas.2019.02.005 . Epub 2019 Mar 13 PMID: 31097173.

Jones C, Windle G, Edwards RT. Dementia and imagination: a social return on investment analysis framework for art activities for people living with dementia. Gerontologist. 2020;60(1):112–23. https://doi.org/10.1093/geront/gny147 . PMID: 30476114.

Social Value Lab and Impact Arts Craft Café: creative solutions to isolation and loneliness; Social return on investment. 2011. http://www.socialvaluelab.org.uk/wp-content/uploads/2013/05/CraftCafeSROI.pdf

MB associates. Make my day: the impact of Creative Caring in older people’s care homes. 2013. https://www.suffolkartlink.org.uk/wp-content/uploads/2014/10/CreativeCarersSROIReport_Nov2013.pdf

HACT. n.d. UK Social Value Bank. Retrieved December 11, 2023. from https://hact.org.uk/tools-and-services/uk-social-value-bank/ .

The Older Adults’ NHS and social care return on investment tool. Project report. Public health England. December 2019. Last accessed 27/03/2023.

British Red Cross – Valuing First Aid Education. 2018. https://socialvalueuk.org/wp-content/uploads/2018/12/Valuing-First-Aid-Education-Social-Return-on-Investment-Report-on-the-value-of-First-Aid-Education-Assured-Report.pdf . Accessed 17/02/2023

Hunter R, Dallat M, Tully M, O’Neill C, Heron L, Kee F. Social return on investment analysis of an urban greenway. Cities and Health. 2020. https://doi.org/10.1080/23748834.2020.1766783 .

NEF Consulting. Refuge: A social return on investment evaluation. 2016. https://socialvalueuk.org/wp-content/uploads/2017/04/Refuge-SROI-2016.pdf Accessed 17/02/2022

Corbacho B, Cockayne S, Fairhurst C, Hewitt CE, Hicks K, Kenan AM, Lamb SE, MacIntosh C, Menz HB, Redmond AC, Rodgers S, Scantlebury A, Watson J, Torgerson DJ, on behalf of the REFORM study. Cost-Effectiveness of a Multifaceted Podiatry Intervention for the Prevention of Falls in Older People: The REducing Falls with Orthoses and a Multifaceted Podiatry Intervention Trial Findings. Gerontology. 2018;64(5):503–12. https://doi.org/10.1159/000489171 . Epub 2018 Jun 26 PMID: 29945150.

Green C, Richards DA, Hill JJ, Gask L, Lovell K, Chew-Graham C, Bower P, Cape J, Pilling S, Araya R, Kessler D, Bland JM, Gilbody S, Lewis G, Manning C, Hughes-Morley A, Barkham M. Cost-effectiveness of collaborative care for depression in UK primary care: economic evaluation of a randomised controlled trial (CADET). PLoS ONE. 2014;9(8):e104225. https://doi.org/10.1371/journal.pone.0104225.PMID:25121991;PMCID:PMC4133193 .

National Institute for Health and Care Excellence (NICE). NICE health technology evaluations: the manual. 2022. Retrieved 27 March, 2023 from  https://www.nice.org.uk/process/pmg36/chapter/introduction-to-health-technology-evaluation

NEF Consulting. SSE – Beatrice SROI framework – guidance document. https://www.sse.com/media/svnn5jpk/sroi-methodology-guidance-nef-consulting.pdf . Accessed 17/02/2022

Gibbons S, McNally S, Overman H. Review of Government Evaluations: A report for the NAO. London: National Audit Office; 2013.

Turner AM, Liddy ED, Bradley J, Wheatley JA. Modeling public health interventions for improved access to the gray literature. J Med Libr Assoc. 2005;93(4):487–94 PMID: 16239945; PMCID: PMC1250325.

PubMed   PubMed Central   Google Scholar  

Benzies KM, Premji S, Hayden KA, Serrett K. State-of-the-evidence reviews: advantages and challenges of including grey literature. Worldviews Evid Based Nurs. 2006;3(2):55–61. https://doi.org/10.1111/j.1741-6787.2006.00051.x . PMID: 17040510.

Franks H, Hardiker NR, McGrath M, McQuarrie C. Public health interventions and behaviour change: reviewing the grey literature. Public Health. 2012;126(1):12–7. https://doi.org/10.1016/j.puhe.2011.09.023 . Epub 2011 Nov 29 PMID: 22130477.

Mahood Q, Van Eerd D, Irvin E. Searching for grey literature for systematic reviews: challenges and benefits. Res Synth Methods. 2014;5(3):221–34. https://doi.org/10.1002/jrsm.1106 . Epub 2013 Dec 6 PMID: 26052848.

Godin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst Rev. 2015;22(4):138. https://doi.org/10.1186/s13643-015-0125-0 . PMID:26494010;PMCID:PMC4619264.

Download references

Acknowledgements

We would like to thank Ms. Louise Bradley (Information Resource Officer, Institute of Public Health) for her assistance in refining search strategies and literature search.

This study was supported by the Institute of Public Health (IPH), 200 South Circular Road, Dublin 8, Ireland, D08 NH90. This study was a collaboration between two health economists (GC, CO’N) and two members of staff from the funding organisation (LM, RO’S). Input from IPH staff was fundamental in defining the scope of work and research question, refining search terms and review and editing of the manuscript. Staff from IPH were not involved in quality assurance or review of papers included in the manuscript.

Author information

Authors and affiliations.

Clinical Costing Solutions, Belfast, BT15 4EB, UK

Grainne Crealey

Institute of Public Health, 200 South Circular Road, Dublin 8, D08 NH90, Ireland

Laura McQuade & Roger O’Sullivan

Centre for Public Health, Institute of Clinical Sciences, Royal Victoria Hospital, Belfast, BT12 6BA, UK

Ciaran O’Neill

You can also search for this author in PubMed   Google Scholar

Contributions

LMcQ and ROS were involved in defining the scope of work, refining the research question, provision of subject specific (public health) context, review of search strategy, review & editing of manuscript. CON and GC were involved in refining the research question and search strategy, provision of health economics and systematic reviewing expertise, review of returned reports, original draft preparation, review, editing and submission of manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ciaran O’Neill .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

: Table S1. Search strategy for electronic databases and grey literature.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Crealey, G., McQuade, L., O’Sullivan, R. et al. Arts and creativity interventions for improving health and wellbeing in older adults: a systematic literature review of economic evaluation studies. BMC Public Health 23 , 2496 (2023). https://doi.org/10.1186/s12889-023-17369-x

Download citation

Received : 23 April 2023

Accepted : 28 November 2023

Published : 13 December 2023

DOI : https://doi.org/10.1186/s12889-023-17369-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Economic evaluation
• Older adults

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]