a literature review and implications for nursing practice

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 19, Issue 1
• Reviewing the literature
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Joanna Smith 1 ,
• Helen Noble 2
• 1 School of Healthcare, University of Leeds , Leeds , UK
• 2 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• Correspondence to Dr Joanna Smith , School of Healthcare, University of Leeds, Leeds LS2 9JT, UK; j.e.smith1{at}leeds.ac.uk

https://doi.org/10.1136/eb-2015-102252

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Implementing evidence into practice requires nurses to identify, critically appraise and synthesise research. This may require a comprehensive literature review: this article aims to outline the approaches and stages required and provides a working example of a published review.

Are there different approaches to undertaking a literature review?

What stages are required to undertake a literature review.

The rationale for the review should be established; consider why the review is important and relevant to patient care/safety or service delivery. For example, Noble et al 's 4 review sought to understand and make recommendations for practice and research in relation to dialysis refusal and withdrawal in patients with end-stage renal disease, an area of care previously poorly described. If appropriate, highlight relevant policies and theoretical perspectives that might guide the review. Once the key issues related to the topic, including the challenges encountered in clinical practice, have been identified formulate a clear question, and/or develop an aim and specific objectives. The type of review undertaken is influenced by the purpose of the review and resources available. However, the stages or methods used to undertake a review are similar across approaches and include:

Formulating clear inclusion and exclusion criteria, for example, patient groups, ages, conditions/treatments, sources of evidence/research designs;

Justifying data bases and years searched, and whether strategies including hand searching of journals, conference proceedings and research not indexed in data bases (grey literature) will be undertaken;

Developing search terms, the PICU (P: patient, problem or population; I: intervention; C: comparison; O: outcome) framework is a useful guide when developing search terms;

Developing search skills (eg, understanding Boolean Operators, in particular the use of AND/OR) and knowledge of how data bases index topics (eg, MeSH headings). Working with a librarian experienced in undertaking health searches is invaluable when developing a search.

Once studies are selected, the quality of the research/evidence requires evaluation. Using a quality appraisal tool, such as the Critical Appraisal Skills Programme (CASP) tools, 5 results in a structured approach to assessing the rigour of studies being reviewed. 3 Approaches to data synthesis for quantitative studies may include a meta-analysis (statistical analysis of data from multiple studies of similar designs that have addressed the same question), or findings can be reported descriptively. 6 Methods applicable for synthesising qualitative studies include meta-ethnography (themes and concepts from different studies are explored and brought together using approaches similar to qualitative data analysis methods), narrative summary, thematic analysis and content analysis. 7 Table 1 outlines the stages undertaken for a published review that summarised research about parents’ experiences of living with a child with a long-term condition. 8

• View inline

An example of rapid evidence assessment review

In summary, the type of literature review depends on the review purpose. For the novice reviewer undertaking a review can be a daunting and complex process; by following the stages outlined and being systematic a robust review is achievable. The importance of literature reviews should not be underestimated—they help summarise and make sense of an increasingly vast body of research promoting best evidence-based practice.

• ↵ Centre for Reviews and Dissemination . Guidance for undertaking reviews in health care . 3rd edn . York : CRD, York University , 2009 .
• ↵ Canadian Best Practices Portal. http://cbpp-pcpe.phac-aspc.gc.ca/interventions/selected-systematic-review-sites / ( accessed 7.8.2015 ).
• Bridges J , et al
• ↵ Critical Appraisal Skills Programme (CASP). http://www.casp-uk.net / ( accessed 7.8.2015 ).
• Dixon-Woods M ,
• Shaw R , et al
• Agarwal S ,
• Jones D , et al
• Cheater F ,

Twitter Follow Joanna Smith at @josmith175

Competing interests None declared.

Read the full text or download the PDF:

The Power of Mass Media and Feminism in the Evolution of Nursing’s Image: A Critical Review of the Literature and Implications for Nursing Practice

• Published: 12 November 2019
• Volume 42 , pages 371–386, ( 2021 )

Cite this article

• Jasmine Gill   ORCID: orcid.org/0000-0002-7534-1091 1 &
• Charley Baker 1  

7505 Accesses

18 Citations

6 Altmetric

Explore all metrics

Nursing has evolved, yet media representation has arguably failed to keep up. This work explores why representation has been slow in accurately depicting nurses' responsibilities, impacts on public perceptions and professional identity. A critical realist review was employed as this method enables in-depth exploration into why something exists. A multidisciplinary approach was adopted, drawing from feminist, psychological and sociological theories to provide insightful understanding and recommendations. One main feminist lens has been implemented, using Laura Mulvey’s ‘Male-Gaze’ framework for content analysis of three nurse-related advertisements to explore how the profession's female status influences representation, public perception and how this might impact nursing. Nurse representation has important real-world consequences. It is essential to improve unnecessary negative portrayals and contest ingrained stereotypes as there are costs to public opinion and nursing's self-identity. Nursing's female status has an impact within a male-dominated media industry, with a leisurely approach adopted toward changing representation. Media images become societally ingrained, this reiterates the significance of accurate/positive depictions. Social media is an instant method of communication with the public to combat stereotypes and maintain engagement to provide better understanding of what nurses do.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

The Role of Communication in Stereotypes, Prejudices and Professional Identity: The Case of Nurses

Professionalism in a Pandemic: Shifting Perceptions of Nursing Through Social Media

Religious orientation, endorser credibility, and the portrayal of female nurses by the media.

Chyong-Ling Lin, Jin-Tsann Yeh, … Wei-Chung Lee

24 Hours in A&E . 2011. Channel 4. 11 May. London: The Garden Productions Ltd.

Archer, Margaret. 1995. Realist Social Theory: The Morphogenetic Approach . Cambridge: Cambridge University Press.

Book   Google Scholar  

Bandura, Albert. 2001. “Social Cognitive Theory of Mass Communication.” Media Psychology 3 (3): 265-299.

Article   Google Scholar  

BBC News – British Broadcasting Corporation. 2015. “#ImInWorkJeremy: NHS Staff Post Weekend Working Photos.” Accessed 1 September 2017. http://www.bbc.co.uk/news/uk-33578990 .

Bhaskar, Roy. 1998. The Possibility of Naturalism . 3rd edition. London: Routledge.

Google Scholar  

_____. 2008. A Realist Theory of Science . London: Verso.

Bright, Jim. 1997. “Carry On Remembering.” Classic Television 1: 25-33.

Broadcasters’ Audience Research Board. 2017. Viewing Data . Accessed 4 June 2017. http://www.barb.co.uk/viewing-data/ .

Brodie, David A., Gavin J. Andrews, Justin P. Andrews, Gail B. Thomas, Josephine Wong, and Lorna Rixon. 2004. “Perceptions of Nursing: Confirmation, Change and the Student Experience.” International Journal of Nursing Studies 41 (7): 721-733.

Bronner, Gila, Chava Peretz and Mally Ehrenfeld. 2003. “Sexual Harassment of Nurses and Nursing Students.” Journal of Advanced Nursing 42 (6): 637-644.

Bryman, A. 2016. Social Research Methods . 5th edition. Oxford: Oxford University Press.

Byerly, Carolyn M. 2013. “Women and the Concentration of Media Ownership.” In Seeking Equity for Women in Journalism and Mass Communication Education , edited by Ramona R. Rush, Carol E. Oukrop and Pamela J. Creedon, 245. Mahwah: Taylor and Francis.

Cabaniss, Reitha. 2011. “Educating Nurses to Impact Change in Nursing's Image.” Teaching and Learning in Nursing 6 (3): 112-118.

Carroll, Stacey M. and Katherine C. Rosa. 2016. “Role and Image of Nursing in Children's Literature: A Qualitative Media Analysis.” Journal of Pediatric Nursing 31 (2): 141-151.

Carry On Doctor . 1967. Directed by Gerald Thomas. United Kingdom. Rank Organisation.

Carry On Matron . 1972. Directed by Gerald Thomas. United Kingdom. The Rank Organisation.

Carry On Nurse . 1959. Directed by Gerald Thomas. United Kingdom. Anglo-Amalgamated.

Carter, Cynthia., Linda Steiner, and Lisa McLaughlin. 2014. The Routledge Companion to Media and Gender. London: Routledge.

Ceulemans, Mieke and Guido Fauconnier. 1979. “Mass Media: The Image, Role and Social Conditions of Women: A Collection and Analysis of Research Materials.” Reports and Papers on Mass Communication 84:26-27.

Cohen, Shelley and Kathleen Bartholomew. 2009. The Image of Nursing: Perspectives on Shaping, Empowering, and Elevating the Nursing Profession . Danvers: HCPro.

Connell, Barbara. 2011. Exploring the Media: Text, Industry, Audience. New York: Columbia University Press.

Crenshaw, Kimberle. 1989. “Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics.” University of Chicago Legal Forum 140: 139–167.

Darbyshire, Philip. 2014. “Heroines, Hookers and Harridans: Exploring Popular Images and Representations of Nurses and Nursing.” In Contexts of Nursing: An Introduction , edited by J. Daly, S. Speedy, D. Jackson, 51-64. Sydney: Elsevier.

Dean, Erin. 2014. “Lancet Commission to Tackle the Poor Perception of UK Nursing.” Nursing Standard 28 (21): 10.

Deep Throat . 1972. Directed by Jerry Gerald. Florida, United States. Bryanston Pictures.

Doyle, Gillian. 2002. Media Ownership: The Economics and Politics of Convergence and Concentration in the UK and European Media. London: SAGE Publications.

Dyer, Richard. 2013. The Matter of Images: Essays on Representation . 2 nd edition. London: Routledge.

Edgley, Alison, Theodore Stickley, Stephen Timmons, and Andy Meal. 2014. “Critical Realist Review: Exploring the Real, Beyond the Empirical.” Journal of Further and Higher Education 40 (3): 316–330.

Ehrenreich, Barbara and Deirdre English. 2010. Witches, Midwives, and Nurses: A History of Women Healers . 2 nd edition. New York: The Feminist Press.

Fagin, Claire M. 2000. Essays on Nursing Leadership . New York: Springer.

Finlayson, Belinda, Jennifer Dixon, Sandra Meadows, and George Blair. 2002. “Mind the Gap: The Extent of the NHS Nursing Shortage.” BMJ 325 (7363): 538-541.

Ford, Steve. 2017a . “Trust Backtracks on Job Ad Dubbed ‘Disrespectful’ to Nurses.” Nursing Times . Accessed 22 July 2017. https://www.nursingtimes.net/news/hospital/trust-backtracks-on-job-ad-dubbed-disrespectful-to-nurses/7017390.article .

_____. 2017b. “'Sexist’ Nurse Job Ads Were Used by Mistake, Says Trust . ” Nursing Times . Accessed 25 July 2017. https://www.nursingtimes.net/news/hospital/sexist-nurse-job-ads-were-used-by-mistake-says-trust/7018256.article .

Fraenkel, Jack R. and Wallen, Norman E. 2006. How to Design and Evaluate Research in Education. 6th edition. Boston: McGraw-Hill Education (ISE Editions).

Gill, R. 2007. Gender and the Media . Cambridge: Polity Press.

Gillett, Karen. 2014. “Nostalgic Constructions of Nurse Education in British National Newspapers.” Journal of Advanced Nursing 70 (11): 2495-2505.

Girvin, June. 2015. “Editorial: The Public Understanding of Nursing - Time for A Step Change?” Journal of Clinical Nursing 24 (23-24): 3341-3342.

_____. 2017. “#50shadesofHey (acknowledgement of the # to P Darbyshire.” Still Learning About Leadership. Accessed 4 June 2017. https://junegirvin2.wordpress.com/2017/05/17/50shadesofhey-acknowledgment-of-the-to-p-darbyshire/ .

Girvin, June, D. Jackson, and M. Hutchinson, M. 2016. “Contemporary Public Perceptions of Nursing: A Systematic Review and Narrative Synthesis of The International Research Evidence.” Journal of Nursing Management 24: 994–1006.

Grant, Maria J. and Andrew Booth. 2009. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal 26 (2): 91–108.

Hall, Stuart. 1973. Encoding and Decoding in the Television Discourse . Birmingham: University of Birmingham Centre for Contemporary Cultural Studies.

Hallam, Julia. 2000. Nursing the Image: Media, Culture and Professional Identity . London: Routledge.

Happer, Catherina and Greg Philo. 2013. “The Role of the Media in the Construction of Public Belief and Social Change.” Journal of Social and Political Psychology 1 (1): 321-336.

Hart, Andrew. 2013. Teaching the Media: International Perspectives . London: Routledge.

Heilemann, Mary-Sue V. 2012. “Media Images and Screen Representations of Nurses.” Nursing Outlook 60 (5): 1-3.

Hsieh, Hsiu-Fang and Sarah E. Shannon. 2005. “Three Approaches to Qualitative Content Analysis.” Qualitative Health Research 15 (9): 1277-1288.

Hudson-Jones, Anne. 1988. Images of Nurses: Perspectives from History, Art and Literature . Philadelphia: University of Pennsylvania Press.

Huston, Carol J. 2014. Professional Issues in Nursing . Third edition. Philadelphia: Lippincott Williams & Wilkins.

Jones-Berry, S. 2017. “Nursing Now! Campaign To Support and Empower Nurses.” Nursing Standard , 31 (51): 7-8.

Joseph, Ammu. 2013. “Media Pluralism and Gender: Not Just a Question of Numbers.” Waccglobal . Accessed 3 September 2017. http://www.waccglobal.org/articles/media-pluralism-and-gender-not-just-a-question-of-numbers .

Kalisch, Philip A. and Beatrice J. Kalisch. 1987. The Changing Image of the Nurse . California: Addison-Wesley Publishing Company.

Knapton, Sarah. 2015. “#ImInWorkJeremy: NHS Staff Post Weekend Working Pictures on Twitter.” Telegraph. Accessed 1 September 2017. http://www.telegraph.co.uk/news/health/news/11749289/ImInWorkJeremy-NHS-staff-post-weekend-working-pictures-on-Twitter.html .

Lippmann, Walter. 1956. Public Opinion . New York: Macmillan.

Long, Paul and Tim Wall. 2014. Media Studies: Texts, Production, Context . 2nd edition. London: Routledge.

Madison, Jean and Victor Minichiello. 2000. “Recognizing and Labelling Sex Based and Sexual Harassment in the Health Care Workplace.” Journal of Nursing Scholarships 4 (32): 405–410.

Matos, Carolina. 2017. Globalization, Gender Politics and the Media . London: Lexington Books.

McAllister, Margaret, Terri Downer, Julie Hanson, and Florin Oprescu. 2014. “Transformers: Changing the Face of Nursing and Midwifery in the Media.” Nurse Education in Practice 14 (2): 148-153.

McNab, Christine. 2009. “What Social Media Offers to Health Professionals and Citizens.” Bulletin of the World Health Organization 87 (8): 566.

Media Reform Coalition. 2015. “Who Owns the UK Media?” Accessed 3 September 2017. http://www.mediareform.org.uk/wp-content/uploads/2015/10/Who_owns_the_UK_media-report_plus_appendix1.pdf .

Monaghan, Thomas. 2015. “A Critical Analysis of the Literature and Theoretical Perspectives on Theory–Practice Gap amongst Newly Qualified Nurses Within the United Kingdom.” Nurse Education Today 35 (8): 1-7.

Muehlbauer, Paula M. 2012. “How Can We Improve the Way the Media Portrays the Nursing Profession?” ONS Voice . Accessed 1 September 2017. https://voice.ons.org/news-and-views/how-can-we-improve-the-way-the-media-portrays-the-nursing-profession .

Muff, Janet. 1988. Socialization, Sexism, and Stereotyping: Women's Issues in Nursing. Prospect Heights, Ill.: Waveland Press.

Mulvey, Laura. 1975. “Visual Pleasure and Narrative Cinema.” Screen 16 (3): 6-18.

_____. 1989. “Afterthoughts on ‘Visual Pleasure and Narrative Cinema’ inspired by King Vidor’s Duel in the Sun (1946).” In Visual and Other Pleasures , edited by Stephen Heath, Colin MacCabe, and Denise Riley, 29-38. Language, Discourse, Society. Palgrave Macmillan, London.

_____. 2013. Fetishism and Curiosity . Basingstoke: Palgrave Macmillan.

Munn, Flavia. 2016. “Nursing the Most Trusted Profession in the UK.” Nursing Standard . Accessed 21 May 2017. https://rcni.com/nursing-standard/newsroom/news/nursing-most-trusted-profession-uk-72621 .

NMC – Nursing & Midwifery Council. 2015. “Guidance on Using Social Media Responsibly.” Accessed 1 September 2017. https://www.nmc.org.uk/globalassets/sitedocuments/nmc-publications/social-media-guidance.pdf .

NMC – Nursing & Midwifery Council. 2016. “Annual Equality and Diversity Report . ” Accessed 11 August 2017. https://www.nmc.org.uk/globalassets/sitedocuments/annual_reports_and_accounts/equality-and-diversity-report-2015-16.pdf .

Norman, Jim. 2016. “Americans Rate Healthcare Providers High on Honesty, Ethics.” Gallup . Accessed 20 May 2017. http://www.gallup.com/poll/200057/americans-rate-healthcare-providers-high-honesty-ethics.aspx .

Ott, Brian L. and Robert L. Mack. 2014. Critical Media Studies: An Introduction . 2nd edition. Oxford: John Wiley & Sons.

Pawson, Ray. 2006. Evidence-Based Policy: A Realist Perspective . London: Sage.

Pawson, Ray and Nicholas Tilley. 1997. Realistic Evaluation . London: Sage.

Peate, Ian. 2015. “The Lancet—Commission or Condescension?” British Journal of Nursing 24 (8): 429.

Press Association, The. 2014. “Media Coverage ‘Idealises Nursing from the Past.”’ Nursing Times. Accessed 26 Jun 2017. https://www.nursingtimes.net/media-coverage-idealises-nursing-from-the-past/5071086.article .

Ross, Karen and Claudia Padovani. 2017. Gender Equality and the Media . Basingstoke: Taylor & Francis Limited.

RCN – Royal College of Nursing. 2015. “This is Nursing in the South West.” Accessed 27 May 2017. https://www.rcn.org.uk/-/media/royal-college-of-nursing/documents/publications/2015/july/pub-004873.pdf .

RCN – Royal College of Nursing. 2017. “Campaigns.” Accessed 27 May 2017. https://www2.rcn.org.uk/newsevents/campaigns .

Sayer, Andrew. 2000. Realism and Social Science . London: Sage.

Seago, Jean Anne, Joanne Spetz, Andrew Alvarado, Dennis Kean, et al. 2006. “The Nursing Shortage: Is It Really About Image?” Journal of Healthcare Management 51 (2): 96-110.

Sephel, Alysse. 2011. “Digging Deeper: Nurse Excess or Shortage?” Journal of Professional Nursing 27 (6): 390-393.

Shelton, Anthony. 1995. Fetishism: Visualising Power and Desire . London: Lund Humphries.

Silva, Michelle and Julie Freischlag. 2017. “Challenge Accepted: Social Media as a Stereotype Change Agent.” The Journal of Thoracic and Cardiovascular Surgery 154 (4): 1354-1355.

Simpkin, Victoria L. and Elias Mossialos. 2017. “Brexit and the NHS: Challenges, Uncertainties and Opportunities.” Health Policy 121 (5): 477-480.

Skeggs, Beverley. 1997. Formations of Class and Gender: Becoming Respectable . Thousand Oaks, CA: SAGE Publications.

Soothill, Keith, Christine Henry, and Kevin Kendrick. 1992. Themes and Perspectives in Nursing . London: Chapman & Hall.

Spector, Paul E., Zhiqing E. Zhou, and Xin Xuan Chen. 2014. “Nurse Exposure to Physical and Nonphysical Violence, Bullying, and Sexual Harassment: A Quantitative review.” International Journal of Nursing Studies 51 (1): 72-84.

Stange, Mary Zeiss, Carol Oyster, and Jane Sloan. 2013. The Multimedia Encyclopedia of Women in Today's World. London: SAGE Publications Inc.

Summers, Sandy. 2010. “The Image of Nursing: The Naughty Nurse.” Nursing Times . Accessed 26 April 2017. https://www.nursingtimes.net/roles/nurse-educators/the-image-of-nursing-the-naughty-nurse/5020678.article .

Summers, Sandy and Harry Jacobs Summers. 2015. Saving Lives: Why the Media’s Portrayal of Nursing Puts Us All at Risk. 2 nd edition. New York: Oxford University Press.

Takase, Miyuki, Esme Kershaw, and Lorraine Burt. 2002. “Does Public Image of Nurses Matter?” Journal of Professional Nursing 18 (4): 196-205.

Takase, Miyuki., Philip Maude, and Elizabeth Manias. 2006. “Impact of the Perceived Public Image of Nursing on Nurses' Work Behaviour.” Journal of Advanced Nursing 53 (3): 333-343.

The Prime Minister’s Commission on the Future of Nursing and Midwifery in England. 2010. “Front Line Care: The Future of Nursing and Midwifery in England.” Accessed 27 May 2017. http://webarchive.nationalarchives.gov.uk/20100331110400/http:/cnm.independent.gov.uk/wp-content/uploads/2010/03/front_line_care.pdf .

The Times . 2017. “Storm over ‘Sexist’ Poster Campaign to Recruit Nurses.” Accessed 20 July 2017. https://www.thetimes.co.uk/article/storm-over-sexist-poster-campaign-to-recruit-nurses-d7w5zd6b2 .

Thio, Alex and Jim Taylor. 2012. Social Problems . Sudbury: Jones & Bartlett Learning.

Thompson, Victoria. 2010. “Ad Campaign Attacked as 'Gratuitous Sexualisation of Nursing.” Nursing Times . Accessed 17 July 2017. https://www.nursingtimes.net/roles/nurse-managers/ad-campaign-attacked-as-gratuitous-sexualisation-of-nursing/5012644.article .

Thornham, Sue. 2007. Women, Feminism and media. Edinburgh: Edinburgh University Press.

Thorpe, Charles. 2016. Necroculture . New York: Palgrave Macmillan.

Tuchman, Gaye. 1979. “Women's Depiction by the Mass Media.” Signs: Journal of Women in Culture and Society 4 (3): 528-542.

Ulmer, Brenda. 2000. “The Image of Nursing.” AORN Journal 71 (6): 1124-1127.

Ramsay, G. 2019. “Who Owns the UK Media?” Accessed 12 May 2019. https://www.mediareform.org.uk/wp-content/uploads/2019/03/FINALonline2.pdf .

Walter, D. 2013. “Nurse Past, Present and Future: The Making of Modern Nursing.” AORN Journal 97 (3): 383-384.

Willis, Phil. 2012. “Quality with Compassion: The Future of Nursing Education: Report of the Willis Commission 2012.” Accessed 27 May 2017. http://www.williscommission.org.uk/__data/.../Willis_Commission_executive_summary.pdf .

Yar, Mohammed, David Dix, and Madhavi Bajekal. 2006. “Socio-demographic Characteristics of the Healthcare Workforce in England and Wales--Results from the 2001 Census.” Health Stat Q 32:44-56.

Download references

Author information

Authors and affiliations.

Royal Derby Hospital, Room 401, Uttoxeter Road, Derby, DE22 3DT, UK

Jasmine Gill & Charley Baker

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jasmine Gill .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

1 Blink 182. 1999. Enema of the State Album Cover. [photograph]. https://www.truthaboutnursing.org/media/music/blink182.html . Accessed 28 July 2017.

2 Ford, Steve. 2012a. RCN Launches Ad Campaign to Promote ‘Remarkable’ Nurses. [photograph]. https://www.nursingtimes.net/roles/nurse-managers/rcn-launches-ad-campaign-to-promote-remarkable-nurses-17-09-2012/ . Accessed: 28 July 2017.

3 Ford, Steve. 2012b. RCN Launches Ad Campaign to Promote ‘Remarkable’ Nurses. [photograph]. https://www.nursingtimes.net/roles/nurse-managers/rcn-launches-ad-campaign-to-promote-remarkable-nurses-17-09-2012/ . Accessed: 28 July 2017.

4 Thompson, Victoria. 2010. Ad Campaign Attacked as ‘Gratuitous Sexualisation of Nurses’. [photograph]. https://www.nursingtimes.net/roles/nurse-managers/ad-campaign-attacked-as-gratuitous-sexualisation-of-nursing-16-03-2010/ . Accessed: 27 July 2017.

5 Wigan Hospitals. 2017. Looking to work for a great employer…look no further? [photograph]. https://twitter.com/WWLNHS/status/834447744743960576 . Accessed 27 July 2017.

6 The Times. 2017. Storm Over ‘Sexist’ Poster Campaign to Recruit Nurses. [photograph]. https://www.thetimes.co.uk/article/storm-over-sexist-poster-campaign-to-recruit-nurses-d7w5zd6b2 . Accessed: 27 July 2017.

Rights and permissions

Reprints and permissions

About this article

Gill, J., Baker, C. The Power of Mass Media and Feminism in the Evolution of Nursing’s Image: A Critical Review of the Literature and Implications for Nursing Practice. J Med Humanit 42 , 371–386 (2021). https://doi.org/10.1007/s10912-019-09578-6

Download citation

Published : 12 November 2019

Issue Date : September 2021

DOI : https://doi.org/10.1007/s10912-019-09578-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Representation
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 19 March 2024

Interventions, methods and outcome measures used in teaching evidence-based practice to healthcare students: an overview of systematic reviews

• Lea D. Nielsen 1 ,
• Mette M. Løwe 2 ,
• Francisco Mansilla 3 ,
• Rene B. Jørgensen 4 ,
• Asviny Ramachandran 5 ,
• Bodil B. Noe 6 &
• Heidi K. Egebæk 7  

BMC Medical Education volume  24 , Article number:  306 ( 2024 ) Cite this article

150 Accesses

Metrics details

To fully implement the internationally acknowledged requirements for teaching in evidence-based practice, and support the student’s development of core competencies in evidence-based practice, educators at professional bachelor degree programs in healthcare need a systematic overview of evidence-based teaching and learning interventions. The purpose of this overview of systematic reviews was to summarize and synthesize the current evidence from systematic reviews on educational interventions being used by educators to teach evidence-based practice to professional bachelor-degree healthcare students and to identify the evidence-based practice-related learning outcomes used.

An overview of systematic reviews. Four databases (PubMed/Medline, CINAHL, ERIC and the Cochrane library) were searched from May 2013 to January 25th, 2024. Additional sources were checked for unpublished or ongoing systematic reviews. Eligibility criteria included systematic reviews of studies among undergraduate nursing, physiotherapist, occupational therapist, midwife, nutrition and health, and biomedical laboratory science students, evaluating educational interventions aimed at teaching evidence-based practice in classroom or clinical practice setting, or a combination. Two authors independently performed initial eligibility screening of title/abstracts. Four authors independently performed full-text screening and assessed the quality of selected systematic reviews using standardized instruments. Data was extracted and synthesized using a narrative approach.

A total of 524 references were retrieved, and 6 systematic reviews (with a total of 39 primary studies) were included. Overlap between the systematic reviews was minimal. All the systematic reviews were of low methodological quality. Synthesis and analysis revealed a variety of teaching modalities and approaches. The outcomes were to some extent assessed in accordance with the Sicily group`s categories; “skills”, “attitude” and “knowledge”. Whereas “behaviors”, “reaction to educational experience”, “self-efficacy” and “benefits for the patient” were rarely used.

Conclusions

Teaching evidence-based practice is widely used in undergraduate healthcare students and a variety of interventions are used and recognized. Not all categories of outcomes suggested by the Sicily group are used to evaluate outcomes of evidence-based practice teaching. There is a need for studies measuring the effect on outcomes in all the Sicily group categories, to enhance sustainability and transition of evidence-based practice competencies to the context of healthcare practice.

Peer Review reports

Evidence-based practice (EBP) enhances the quality of healthcare, reduces the cost, improves patient outcomes, empowers clinicians, and is recognized as a problem-solving approach [ 1 ] that integrates the best available evidence with clinical expertise and patient preferences and values [ 2 ]. A recent scoping review of EBP and patient outcomes indicates that EBPs improve patient outcomes and yield a positive return of investment for hospitals and healthcare systems. The top outcomes measured were length of stay, mortality, patient compliance/adherence, readmissions, pneumonia and other infections, falls, morbidity, patient satisfaction, patient anxiety/ depression, patient complications and pain. The authors conclude that healthcare professionals have a professional and ethical responsibility to provide expert care which requires an evidence-based approach. Furthermore, educators must become competent in EBP methodology [ 3 ].

According to the Sicily statement group, teaching and practicing EBP requires a 5-step approach: 1) pose an answerable clinical question (Ask), 2) search and retrieve relevant evidence (Search), 3) critically appraise the evidence for validity and clinical importance (Appraise), 4) applicate the results in practice by integrating the evidence with clinical expertise, patient preferences and values to make a clinical decision (Integrate), and 5) evaluate the change or outcome (Evaluate /Assess) [ 4 , 5 ]. Thus, according to the World Health Organization, educators, e.g., within undergraduate healthcare education, play a vital role by “integrating evidence-based teaching and learning processes, and helping learners interpret and apply evidence in their clinical learning experiences” [ 6 ].

A scoping review by Larsen et al. of 81 studies on interventions for teaching EBP within Professional bachelor-degree healthcare programs (PBHP) (in English undergraduate/ bachelor) shows that the majority of EBP teaching interventions include the first four steps, but the fifth step “evaluate/assess” is less often applied [ 5 ]. PBHP include bachelor-degree programs characterized by combined theoretical education and clinical training within nursing, physiotherapy, occupational therapy, radiography, and biomedical laboratory students., Furthermore, an overview of systematic reviews focusing on practicing healthcare professionals EBP competencies testifies that although graduates may have moderate to high level of self-reported EBP knowledge, skills, attitudes, and beliefs, this does not translate into their subsequent EBP implementation [ 7 ]. Although this cannot be seen as direct evidence of inadequate EBP teaching during undergraduate education, it is irrefutable that insufficient EBP competencies among clinicians across healthcare disciplines impedes their efforts to attain highest care quality and improved patient outcomes in clinical practice after graduation.

Research shows that teaching about EBP includes different types of modalities. An overview of systematic reviews, published by Young et al. in 2014 [ 8 ] and updated by Bala et al. in 2021 [ 9 ], synthesizes the effects of EBP teaching interventions including under- and post graduate health care professionals, the majority being medical students. They find that multifaceted interventions with a combination of lectures, computer lab sessions, small group discussion, journal clubs, use of current clinical issues, portfolios and assignments lead to improvement in students’ EBP knowledge, skills, attitudes, and behaviors compared to single interventions or no interventions [ 8 , 9 ]. Larsen et al. find that within PBHP, collaboration with clinical practice is the second most frequently used intervention for teaching EBP and most often involves four or all five steps of the EBP teaching approach [ 5 ]. The use of clinically integrated teaching in EBP is only sparsely identified in the overviews by Young et al. and Bala et al. [ 8 , 9 ]. Therefore, the evidence obtained within Bachelor of Medicine which is a theoretical education [ 10 ], may not be directly transferable for use in PBHP which combines theoretical and mandatory clinical education [ 11 ].

Since the overview by Young et al. [ 8 ], several reviews of interventions for teaching EBP used within PBHP have been published [ 5 , 12 , 13 , 14 ].

We therefore wanted to explore the newest evidence for teaching EBP focusing on PBHP as these programs are characterized by a large proportion of clinical teaching. These healthcare professions are certified through a PBHP at a level corresponding to a University Bachelor Degree, but with strong focus on professional practice by combining theoretical studies with mandatory clinical teaching. In Denmark, almost half of PBHP take place in clinical practice. These applied science programs qualify “the students to independently analyze, evaluate and reflect on problems in order to carry out practice-based, complex, and development-oriented job functions" [ 11 ]. Thus, both the purpose of these PBHP and the amount of clinical practice included in the educations contrast with for example medicine.

Thus, this overview, identifies the newest evidence for teaching EBP specifically within PBHP and by including reviews using quantitative and/or qualitative methods.

We believe that such an overview is important knowledge for educators to be able to take the EBP teaching for healthcare professions to a higher level. Also reviewing and describing EBP-related learning outcomes, categorizing them according to the seven assessment categories developed by the Sicily group [ 2 ], will be useful knowledge to educators in healthcare professions. These seven assessment categories for EBP learning including: Reaction to the educational experience, attitudes, self-efficacy, knowledge, skills, behaviors and benefits to patients, can be linked to the five-step EBP approach. E.g., reactions to the educational experience: did the educators teaching style enhance learners’ enthusiasm for asking questions? (Ask), self-efficacy: how well do learners think they critically appraise evidence? (Appraise), skills: can learners come to a reasonable interpretation of how to apply the evidence? (Integrate) [ 2 ]. Thus, this set of categories can be seen as a basic set of EBP-related learning outcomes to classify the impact from EBP educational interventions.

Purpose and review questions

A systematic overview of which evidence-based teaching interventions and which EBP-related learning outcomes that are used will give teachers access to important knowledge on what to implement and how to evaluate EBP teaching.

Thus, the purpose of this overview is to synthesize the latest evidence from systematic reviews about EBP teaching interventions in PBHP. This overview adds to the existing evidence by focusing on systematic reviews that a) include qualitative and/ or quantitative studies regardless of design, b) are conducted among PBHP within nursing, physiotherapy, occupational therapy, midwifery, nutrition and health and biomedical laboratory science, and c) incorporate the Sicily group's 5-step approach and seven assessment categories when analyzing the EBP teaching interventions and EBP-related learning outcomes.

The questions of this overview of systematic reviews are:

Which educational interventions are described and used by educators to teach EBP to Professional Bachelor-degree healthcare students?

What EBP-related learning outcomes have been used to evaluate teaching interventions?

The study protocol was guided by the Cochrane Handbook on Overviews of Reviews [ 15 ] and the review process was reported in accordance with The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement [ 16 ] when this was consistent with the Cochrane Handbook.

Inclusion criteria

Eligible reviews fulfilled the inclusion criteria for publication type, population, intervention, and context (see Table  1 ). Failing a single inclusion criterion implied exclusion.

Search strategy

On January 25th 2024 a systematic search was conducted in; PubMed/Medline, CINAHL (EBSCOhost), ERIC (EBSCOhost) and the Cochrane library from May 2013 to January 25th, 2024 to identify systematic reviews published after the overview by Young et al. [ 8 ]. In collaboration with a research librarian, a search strategy of controlled vocabulary and free text terms related to systematic reviews, the student population, teaching interventions, teaching context, and evidence-based practice was developed (see Additional file 1 ). For each database, the search strategy was peer reviewed, revised, modified and subsequently pilot tested. No language restrictions were imposed.

To identify further eligible reviews, the following methods were used: Setting email alerts from the databases to provide weekly updates on new publications; backward and forward citation searching based on the included reviews by screening of reference lists and using the “cited by” and “similar results” function in PubMed and CINAHL; broad searching in Google Scholar (Advanced search), Prospero, JBI Evidence Synthesis and the OPEN Grey database; contacting experts in the field via email to first authors of included reviews, and by making queries via Twitter and Research Gate on any information on unpublished or ongoing reviews of relevance.

Selection and quality appraisal process

Database search results were merged, duplicate records were removed, and title/abstract were initially screened via Covidence [ 17 ]. The assessment process was pilot tested by four authors independently assessing eligibility and methodological quality of one potential review followed by joint discussion to reach a common understanding of the criteria used. Two authors independently screened each title/abstract for compliance with the predefined eligibility criteria. Disagreements were resolved by a third author. Four authors were paired for full text screening, and each pair assessed independently 50% of the potentially relevant reviews for eligibility and methodological quality.

For quality appraisal, two independent authors used the AMSTAR-2 (A MeaSurement Tool to Assess systematic Reviews) for reviews including intervention studies [ 18 ] and the Joanna Briggs Institute Checklist for systematic reviews and research Synthesis (JBI checklist) [ 19 ] for reviews including both quantitative and qualitative or only qualitative studies. Uncertainties in assessments were resolved by requesting clarifying information from first authors of reviews and/or discussion with co-author to the present overview.

Overall methodological quality for included reviews was assessed using the overall confidence criteria of AMSTAR 2 based on scorings in seven critical domains [ 18 ] appraised as high (none or one non-critical flaw), moderate (more than one non-critical flaw), low (one critical weakness) or critically low (more than one critical weakness) [ 18 ]. For systematic reviews of qualitative studies [ 13 , 20 , 21 ] the critical domains of the AMSTAR 2, not specified in the JBI checklist, were added.

Data extraction and synthesis process

Data were initially extracted by the first author, confirmed or rejected by the last author and finally discussed with the whole author group until consensus was reached.

Data extraction included 1) Information about the search and selection process according to the PRISMA statement [ 16 , 22 ], 2) Characteristics of the systematic reviews inspired by a standard in the Cochrane Handbook (15), 3) A citation index inspired by Young et al. [ 8 ] used to illustrate overlap of primary studies in the included systematic reviews, and to ensure that data from each primary study were extracted only once [ 15 ], 4) Data on EBP teaching interventions and EBP-related outcomes. These data were extracted, reformatted (categorized inductively into two categories: “Collaboration interventions” and “  Educational interventions ”) and presented as narrative summaries [ 15 ]. Data on outcome were categorized according to the seven assessment categories, defined by the Sicily group, to classify the impact from EBP educational interventions: Reaction to the educational experience, attitudes, self-efficacy, knowledge, skills, behaviors and benefits to patients [ 2 ]. When information under points 3 and 4 was missing, data from the abstracts of the primary study articles were reviewed.

Results of the search

The database search yielded 691 references after duplicates were removed. Title and abstract screening deemed 525 references irrelevant. Searching via other methods yielded two additional references. Out of 28 study reports assessed for eligibility 22 were excluded, leaving a total of six systematic reviews. Screening resulted in 100% agreement among the authors. Figure  1 details the search and selection process. Reviews that might seem relevant but did not meet the eligibility criteria [ 15 ], are listed in Additional file 2 . One protocol for a potentially relevant review was identified as ongoing [ 23 ].

PRISMA flow diagram on search and selection of systematic reviews

Characteristics of included systematic reviews and overlap between them

The six systematic reviews originated from the Middle East, Asia, North America, Europe, Scandinavia, and Australia. Two out of six reviews did not identify themselves as systematic reviews but did fulfill this eligibility criteria [ 12 , 20 ]. All six represented a total of 64 primary studies and a total population of 6649 students (see Table  2 ). However, five of the six systematic reviews contained a total of 17 primary studies not eligible to our overview focus (e.g., postgraduate students) (see Additional file 3 ). Results from these primary studies were not extracted. Of the remaining primary studies, six were included in two, and one was included in three systematic reviews. Data from these studies were extracted only once to avoid double-counting. Thus, the six systematic reviews represented a total of 39 primary studies and a total population of 3394 students. Nursing students represented 3280 of these. One sample of 58 nutrition and health students and one sample of 56 mixed nursing and midwife students were included but none from physiotherapy, occupational therapy, or biomedical laboratory scientists. The majority ( n  = 28) of the 39 primary studies had a quantitative design whereof 18 were quasi-experimental (see Additional file 4 ).

Quality of systematic review

All the included systematic reviews were assessed as having critically low quality with 100% concordance between the two designed authors (see Fig.  2 ) [ 18 ]. The main reasons for the low quality of the reviews were a) not demonstrating a registered protocol prior to the review [ 13 , 20 , 24 , 25 ], b) not providing a list of excluded studies with justification for exclusion [ 12 , 13 , 21 , 24 , 25 ] and c) not accounting for the quality of the individual studies when interpreting the result of the review [ 12 , 20 , 21 , 25 ].

Overall methodological quality assessment for systematic reviews. Quantitative studies [ 12 , 24 , 25 ] were assessed following the AMSTAR 2 critical domain guidelines. Qualitative studies [ 13 , 20 , 21 ] were assessed following the JBI checklist. For overall classification, qualitative studies were also assessed with the following critical AMSTAR 2 domains not specified in the JBI checklist (item 2. is the protocol registered before commencement of the review, item 7. justification for excluding individual studies and item 13. consideration of risk of bias when interpreting the results of the review)

Missing reporting of sources of funding for primary studies and not describing the included studies in adequate detail were, most often, the two non-critical items of the AMSTAR 2 and the JBI checklist, not met.

Most of the included reviews did report research questions including components of PICO, performed study selection and data extraction in duplicate, used appropriate methods for combining studies and used satisfactory techniques for assessing risk of bias (see Fig.  2 ).

Main findings from the systematic reviews

As illustrated in Table  2 , this overview synthesizes evidence on a variety of approaches to promote EBP teaching in both classroom and clinical settings. The systematic reviews describe various interventions used for teaching in EBP, which can be summarized into two themes: Collaboration Interventions and Educational Interventions.

Collaboration interventions to teach EBP

In general, the reviews point that interdisciplinary collaboration among health professionals and/or others e.g., librarian and professionals within information technologies is relevant when planning and teaching in EBP [ 13 , 20 ].

Interdisciplinary collaboration was described as relevant when planning teaching in EBP [ 13 , 20 ]. Specifically, regarding literature search Wakibi et al. found that collaboration between librarians, computer laboratory technicians and nurse educators enhanced students’ skills [ 13 ]. Also, in terms of creating transfer between EBP teaching and clinical practice, collaboration between faculty, library, clinical institutions, and teaching institutions was used [ 13 , 20 ].

Regarding collaboration with clinical practice, Ghaffari et al. found that teaching EBP integrated in clinical education could promote students’ knowledge and skills [ 25 ]. Horntvedt et al. found that during a six-week course in clinical practice, students obtained better skills in reading research articles and orally presenting the findings to staff and fellow students [ 20 ]. Participation in clinical research projects combined with instructions in analyzing and discussing research findings also “led to a positive approach and EBP knowledge” [ 20 ]. Moreover, reading research articles during the clinical practice period enhances the students critical thinking skills. Furthermore, Horntvedt et al. mention, that students found it meaningful to conduct a “mini” – research project in clinical settings, as the identified evidence became relevant [ 20 ].

Educational interventions

Educational interventions can be described as “Framing Interventions” understood as different ways to set up a framework for teaching EBP, and “  Teaching methods ” understood as specific methods used when teaching EBP.

Various educational interventions were described in most reviews [ 12 , 13 , 20 , 21 ]. According to Patelarou et al., no specific educational intervention regardless of framing and methods was in favor to “ increase knowledge, skills and competency as well as improve the beliefs, attitudes and behaviors of nursing students”  [ 12 ].

Framing interventions

The approaches used to set up a framework for teaching EBP were labelled in different ways: programs, interactive teaching strategies, educational programs, courses etc. Approaches of various durations from hours to months were described as well as stepwise interventions [ 12 , 13 , 20 , 21 , 24 , 25 ].

Some frameworks [ 13 , 20 , 21 , 24 ] were based on the assessments categories described by the Sicily group [ 2 ] or based on theory [ 21 ] or as mentioned above clinically integrated [ 20 ]. Wakibi et al. identified interventions used to foster a spirit of inquiry and EBP culture reflecting the “5-step approach” of the Sicily group [ 4 ], asking PICOT questions, searching for best evidence, critical appraisal, integrating evidence with clinical expertise and patient preferences to make clinical decisions, evaluating outcomes of EBP practice, and disseminating outcomes useful [ 13 ]. Ramis et al. found that teaching interventions based on theory like Banduras self-efficacy or Roger’s theory of diffusion led to positive effects on students EBP knowledge and attitudes [ 21 ].

Teaching methods

A variety of teaching methods were used such as, lectures [ 12 , 13 , 20 ], problem-based learning [ 12 , 20 , 25 ], group work, discussions [ 12 , 13 ], and presentations [ 20 ] (see Table  2 ). The most effective method to achieve the skills required to practice EBP as described in the “5-step approach” by the Sicely group is a combination of different teaching methods like lectures, assignments, discussions, group works, and exams/tests.

Four systematic reviews identified such combinations or multifaceted approaches [ 12 , 13 , 20 , 21 ]. Patelarou et al. states that “EBP education approaches should be blended” [ 12 ]. Thus, combining the use of video, voice-over, PowerPoint, problem-based learning, lectures, team-based learning, projects, and small groups were found in different studies. This combination had shown “to be effective” [ 12 ]. Similarly, Horntvedt et al. found that nursing students reported that various teaching methods improved their EBP knowledge and skills [ 20 ].

According to Ghaffari et al., including problem-based learning in teaching plans “improved the clinical care and performance of the students”, while the problem-solving approach “promoted student knowledge” [ 25 ]. Other teaching methods identified, e.g., flipped classroom [ 20 ] and virtual simulation [ 12 , 20 ] were also characterized as useful interactive teaching interventions. Furthermore, face-to-face approaches seem “more effective” than online teaching interventions to enhance students’ research and appraisal skills and journal clubs enhance the students critically appraisal-skills [ 12 ].

As the reviews included in this overview primarily are based on qualitative, mixed methods as well as quasi-experimental studies and to a minor extent on randomized controlled trials (see Table  2 ) it is not possible to conclude of the most effective methods. However, a combination of methods and an innovative collaboration between librarians, information technology professionals and healthcare professionals seem the most effective approach to achieve EBP required skills.

EBP-related outcomes

Most of the systematic reviews presented a wide array of outcome assessments applied in EBP research (See Table  3 ). Analyzing the outcomes according to the Sicily group’s assessment categories revealed that assessing “knowledge” (used in 19 out of 39 primary studies), “skills” (used in 18 out of 39 primary studies) and “attitude” (used in 17 out of 39) were by far the most frequently used assessment categories, whereas outcomes within the category of “behaviors” (used in eight studies) “reaction to educational experience” (in five studies), “self-efficacy” (in two studies), and “benefits for the patient” (in one study), were used to a far lesser extent. Additionally, outcomes, that we were not able to categorize within the seven assessment categories, were “future use” and “Global EBP competence”.

The purpose of this overview of systematic reviews was to collect and summarize evidence of the diversity of EBP teaching interventions and outcomes measured among professional bachelor- degree healthcare students.

Our results give an overview of “the state of the art” of using and measuring EBP in PBHP education. However, the quality of included systematic reviews was rated critically low. Thus, the result cannot support guidelines of best practice.

The analysis of the interventions and outcomes described in the 39 primary studies included in this overview, reveals a wide variety of teaching methods and interventions being used and described in the scientific literature on EBP teaching of PBHP students. The results show some evidence of the five step EBP approach in accordance with the inclusion criteria “interventions aimed at teaching one or more of the five EBP steps; Ask, Search, Appraise, Integrate, Assess/evaluate”. Most authors state, that the students´ EBP skills, attitudes and knowledge improved by almost any of the described methods and interventions. However, descriptions of how the improvements were measured were less frequent.

We evaluated the described outcome measures and assessments according to the seven categories proposed by the Sicily group and found that most assessments were on “attitudes”, “skills” and “knowledge”, sometimes on “behaviors” and very seldom on” reaction to educational experience”, “self-efficacy” and “benefits to the patients”. To our knowledge no systematic review or overview has made this evaluation on outcome categories before, but Bala et al. [ 9 ] also stated that knowledge, skills, and attitudes are the most common evaluated effects.

Comparing the outcomes measured between mainly medical [ 9 ] and nursing students, the most prevalent outcomes in both groups are knowledge, skills and attitudes around EBP. In contrast, measuring on the students´ patient care or on the impact of the EBP teaching on benefits for the patients is less prevalent. In contrast Wu et al.’s systematic review shows that among clinical nurses, educational interventions supporting implementation of EBP projects can change patient outcomes positively. However, they also conclude that direct causal evidence of the educational interventions is difficult to measure because of the diversity of EBP projects implemented [ 26 ]. Regarding EBP behavior the Sicily group recommend this category to be assessed by monitoring the frequency of the five step EBP approach, e.g., ASK questions about patients, APPRAISE evidence related to patient care, EVALUATE their EBP behavior and identified areas for improvement [ 2 ]. The results also showed evidence of student-clinician transition. “Future use” was identified in two systematic reviews [ 12 , 13 ] and categorized as “others”. This outcome is not included in the seven Sicily categories. However, a systematic review of predictive modelling studies shows, that future use or the intention to use EBP after graduation are influenced by the students EBP familiarity, EBP capability beliefs, EBP attitudes and academic and clinical support [ 27 ].

Teaching and evaluating EBP needs to move beyond aiming at changes in knowledge, skills, and attitudes, but also start focusing on changing and assessing behavior, self-efficacy and benefit to the patients. We recommend doing this using validated tools for the assessment of outcomes and in prospective studies with longer follow-up periods, preferably evaluating the adoption of EBP in clinical settings bearing in mind, that best teaching practice happens across sectors and settings supported and supervised by multiple professions.

Based on a systematic review and international Delphi survey, a set of interprofessional EBP core competencies that details the competence content of each of the five steps has been published to inform curriculum development and benchmark EBP standards [ 28 ]. This consensus statement may be used by educators as a reference for both learning objectives and EBP content descriptions in future intervention research. The collaboration with clinical institutions and integration of EBP teaching components such as EBP assignments or participating in clinical research projects are important results. Specifically, in the light of the dialectic between theoretical and clinical education as a core characteristic of Professional bachelor-degree healthcare educations.

Our study has some limitations that need consideration when interpreting the results. A search in the EMBASE and Scopus databases was not added in the search strategy, although it might have been able to bring additional sources. Most of the 22 excluded reviews included primary studies among other levels/ healthcare groups of students or had not critically appraised their primary studies. This constitutes insufficient adherence to methodological guidelines for systematic reviews and limits the completeness of the reviews identified. Often, the result sections of the included reviews were poorly reported and made it necessary to extract some, but not always sufficient, information from the primary study abstracts. As the present study is an overview and not a new systematic review, we did not extract information from the result section in the primary studies. Thus, the comprehensiveness and applicability of the results of this overview are limited by the methodological limitations in the six included systematic reviews.

The existing evidence is based on different types of study designs. This heterogeneity is seen in all the included reviews. Thus, the present overview only conveys trends around the comparative effectiveness of the different ways to frame, or the methods used for teaching EBP. This can be seen as a weakness for the clarity and applicability of the overview results. Also, our protocol is unpublished, which may weaken the transparency of the overview approach, however our search strategies are available as additional material (see Additional file 1 ). In addition, the validity of data extraction can be discussed. We extracted data consecutively by the first and last author and if needed consensus was reached by discussion with the entire research group. This method might have been strengthened by using two blinded reviewers to extract data and present data with supporting kappa values.

The generalizability of the results of this overview is limited to undergraduate nursing students. Although, we consider it a strength that the results represent a broad international perspective on framing EBP teaching, as well as teaching methods and outcomes used among educators in EBP. Primary studies exist among occupational therapy and physiotherapy students [ 5 , 29 ] but have not been systematically synthesized. However, the evidence is almost non-existent among midwife, nutrition and health and biomedical laboratory science students. This has implications for further research efforts because evidence from within these student populations is paramount for future proofing the quality assurance of clinical evidence-based healthcare practice.

Another implication is the need to compare how to frame the EBP teaching, and the methods used both inter-and mono professionally among these professional bachelor-degree students. Lastly, we support the recommendations of Bala et al. of using validated tools to increase the focus on measuring behavior change in clinical practice and patient outcomes, and to report in accordance with the GREET guidelines for educational intervention studies [ 9 ].

This overview demonstrates a variety of approaches to promote EBP teaching among professional bachelor-degree healthcare students. Teaching EBP is based on collaboration with clinical practice and the use of different approaches to frame the teaching as well as different teaching methods. Furthermore, this overview has elucidated, that interventions often are evaluated according to changes in the student’s skills, knowledge and attitudes towards EBP, but very rarely on self-efficacy, behaviors, benefits to the patients or reaction to the educational experience as suggested by the Sicily group. This might indicate that educators need to move on to measure the effect of EBP on outcomes comprising all categories, which are important to enhance sustainable behavior and transition of knowledge into the context of practices where better healthcare education should have an impact. In our perspective these gaps in the EBP teaching are best met by focusing on more collaboration with clinical practice which is the context where the final endpoint of teaching EBP should be anchored and evaluated.

Availability of data and materials

The datasets used an/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Evidence-Based Practice

Professional bachelor-degree healthcare programs

Mazurek Melnyk B, Fineout-Overholt E. Making the Case for Evidence-Based Practice and Cultivalting a Spirit of Inquiry. I: Mazurek Melnyk B, Fineout-Overholt E, redaktører. Evidence-Based Practice in Nursing and Healthcare A Guide to Best Practice. 4. ed. Wolters Kluwer; 2019. p. 7–32.

Tilson JK, Kaplan SL, Harris JL, Hutchinson A, Ilic D, Niederman R, et al. Sicily statement on classification and development of evidence-based practice learning assessment tools. BMC Med Educ. 2011;11(78):1–10.

Google Scholar  

Connor L, Dean J, McNett M, Tydings DM, Shrout A, Gorsuch PF, et al. Evidence-based practice improves patient outcomes and healthcare system return on investment: Findings from a scoping review. Worldviews Evid Based Nurs. 2023;20(1):6–15.

Article   PubMed   Google Scholar  

Dawes M, Summerskill W, Glasziou P, Cartabellotta N, Martin J, Hopayian K, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5(1):1–7.

Article   PubMed   PubMed Central   Google Scholar  

Larsen CM, Terkelsen AS, Carlsen AF, Kristensen HK. Methods for teaching evidence-based practice: a scoping review. BMC Med Educ. 2019;19(1):1–33.

Article   CAS   Google Scholar  

World Health Organization. Nurse educator core competencies. 2016 https://apps.who.int/iris/handle/10665/258713 Accessed 21 Mar 2023.

Saunders H, Gallagher-Ford L, Kvist T, Vehviläinen-Julkunen K. Practicing healthcare professionals’ evidence-based practice competencies: an overview of systematic reviews. Worldviews Evid Based Nurs. 2019;16(3):176–85.

Young T, Rohwer A, Volmink J, Clarke M. What Are the Effects of Teaching Evidence-Based Health Care (EBHC)? Overview of Systematic Reviews PLoS ONE. 2014;9(1):1–13.

Bala MM, Poklepović Peričić T, Zajac J, Rohwer A, Klugarova J, Välimäki M, et al. What are the effects of teaching Evidence-Based Health Care (EBHC) at different levels of health professions education? An updated overview of systematic reviews. PLoS ONE. 2021;16(7):1–28.

Article   Google Scholar  

Copenhagen University. Bachelor in medicine. 2024 https://studier.ku.dk/bachelor/medicin/undervisning-og-opbygning/ Accessed 31 Jan 2024.

Ministery of Higher Education and Science. Professional bachelor programmes. 2022 https://ufm.dk/en/education/higher-education/university-colleges/university-college-educations Accessed 31 Jan 2024.

Patelarou AE, Mechili EA, Ruzafa-Martinez M, Dolezel J, Gotlib J, Skela-Savič B, et al. Educational Interventions for Teaching Evidence-Based Practice to Undergraduate Nursing Students: A Scoping Review. Int J Env Res Public Health. 2020;17(17):1–24.

Wakibi S, Ferguson L, Berry L, Leidl D, Belton S. Teaching evidence-based nursing practice: a systematic review and convergent qualitative synthesis. J Prof Nurs. 2021;37(1):135–48.

Fiset VJ, Graham ID, Davies BL. Evidence-Based Practice in Clinical Nursing Education: A Scoping Review. J Nurs Educ. 2017;56(9):534–41.

Pollock M, Fernandes R, Becker L, Pieper D, Hartling L. Chapter V: Overviews of Reviews. I: Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions version 62. 2021 https://training.cochrane.org/handbook Accessed 31 Jan 2024.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, m.fl. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:1-9

Covidence. Covidence - Better systematic review management. https://www.covidence.org/ Accessed 31 Jan 2024.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;21(358):1–9.

Joanna Briggs Institute. Critical Appraisal Tools. https://jbi.global/critical-appraisal-tools Accessed 31 Jan 2024.

Horntvedt MT, Nordsteien A, Fermann T, Severinsson E. Strategies for teaching evidence-based practice in nursing education: a thematic literature review. BMC Med Educ. 2018;18(1):1–11.

Ramis M-A, Chang A, Conway A, Lim D, Munday J, Nissen L. Theory-based strategies for teaching evidence-based practice to undergraduate health students: a systematic review. BMC Med Educ. 2019;19(1):1–13.

Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, et al. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev. 2021;10(1):1–19.

Song CE, Jang A. Simulation design for improvement of undergraduate nursing students’ experience of evidence-based practice: a scoping-review protocol. PLoS ONE. 2021;16(11):1–6.

Cui C, Li Y, Geng D, Zhang H, Jin C. The effectiveness of evidence-based nursing on development of nursing students’ critical thinking: A meta-analysis. Nurse Educ Today. 2018;65:46–53.

Ghaffari R, Shapoori S, Binazir MB, Heidari F, Behshid M. Effectiveness of teaching evidence-based nursing to undergraduate nursing students in Iran: a systematic review. Res Dev Med Educ. 2018;7(1):8–13.

Wu Y, Brettle A, Zhou C, Ou J, Wang Y, Wang S. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ Today. 2018;70:109–14.

Ramis MA, Chang A, Nissen L. Undergraduate health students’ intention to use evidence-based practice after graduation: a systematic review of predictive modeling studies. Worldviews Evid Based Nurs. 2018;15(2):140–8.

Albarqouni L, Hoffmann T, Straus S, Olsen NR, Young T, Ilic D, et al. Core competencies in evidence-based practice for health professionals: consensus statement based on a systematic review and Delphi survey. JAMA Netw Open. 2018;1(2):1–12.

Hitch D, Nicola-Richmond K. Instructional practices for evidence-based practice with pre-registration allied health students: a review of recent research and developments. Adv Health Sci Educ Theory Pr. 2017;22(4):1031–45.

Download references

Acknowledgements

The authors would like to acknowledge research librarian Rasmus Sand for competent support in the development of literature search strategies.

This work was supported by the University College of South Denmark, which was not involved in the conduct of this study.

Author information

Authors and affiliations.

Nursing Education & Department for Applied Health Science, University College South Denmark, Degnevej 17, 6705, Esbjerg Ø, Denmark

Lea D. Nielsen

Department of Oncology, Hospital of Lillebaelt, Beriderbakken 4, 7100, Vejle, Denmark

Mette M. Løwe

Biomedical Laboratory Science & Department for Applied Health Science, University College South Denmark, Degnevej 17, 6705, Esbjerg Ø, Denmark

Francisco Mansilla

Physiotherapy Education & Department for Applied Health Science, University College South Denmark, Degnevej 17, 6705, Esbjerg Ø, Denmark

Rene B. Jørgensen

Occupational Therapy Education & Department for Applied Health Science, University College South Denmark, Degnevej 17, 6705, Esbjerg Ø, Denmark

Asviny Ramachandran

Department for Applied Health Science, University College South Denmark, Degnevej 17, 6705, Esbjerg Ø, Denmark

Bodil B. Noe

Centre for Clinical Research and Prevention, Section for Health Promotion and Prevention, Bispebjerg and Frederiksberg Hospital, Nordre Fasanvej 57, 2000, Frederiksberg, Denmark

Heidi K. Egebæk

You can also search for this author in PubMed   Google Scholar

Contributions

All authors have made substantial contributions to the conception and design of the study, acquisition of data, analysis, and interpretation of data, writing the main manuscript, preparing figures and tables and revising the manuscripts.

Corresponding author

Correspondence to Lea D. Nielsen .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nielsen, L.D., Løwe, M.M., Mansilla, F. et al. Interventions, methods and outcome measures used in teaching evidence-based practice to healthcare students: an overview of systematic reviews. BMC Med Educ 24 , 306 (2024). https://doi.org/10.1186/s12909-024-05259-8

Download citation

Received : 29 May 2023

Accepted : 04 March 2024

Published : 19 March 2024

DOI : https://doi.org/10.1186/s12909-024-05259-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• MH "Students, Health occupations+"
• MH "Students, occupational therapy"
• MH "Students, physical therapy"
• MH "Students, Midwifery"
• “Students, Nursing"[Mesh]
• “Teaching"[Mesh]
• MH "Teaching methods+"
• "Evidence-based practice"[Mesh]

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 18 March 2024

Exploring advanced clinical practitioner perspectives on training, role identity and competence: a qualitative study

• Maxine Kuczawski   ORCID: orcid.org/0000-0002-0774-8113 1 ,
• Suzanne Ablard 1 ,
• Fiona Sampson 1 ,
• Susan Croft 1 , 2 ,
• Joanna Sutton-Klein 1 , 3 &
• Suzanne Mason 1  

BMC Nursing volume  23 , Article number:  185 ( 2024 ) Cite this article

260 Accesses

1 Altmetric

Metrics details

Advanced Clinical Practitioners (ACPs) are a new role that have been established to address gaps and support the existing medical workforce in an effort to help reduce increasing pressures on NHS services. ACPs have the potential to practice at a similar level to mid-grade medical staff, for example independently undertaking assessments, requesting and interpreting investigations, and diagnosing and discharging patients. These roles have been shown to improve both service outcomes and quality of patient care. However, there is currently no widespread formalised standard of training within the UK resulting in variations in the training experiences and clinical capabilities of ACPs. We sought to explore the training experiences of ACPs as well as their views on role identity and future development of the role.

Five online focus groups were conducted between March and May 2021 with trainee and qualified advanced clinical practitioners working in a range of healthcare settings, in the North of England. The focus groups aimed to explore the experiences of undertaking ACP training including supervision, gaining competence, role identity and career progression. Thematic analysis of the focus group transcripts was performed, informed by grounded theory principles.

Fourteen advanced clinical practitioners participated. Analysis revealed that training was influenced by internal and external perceptions of the role, often acting as barriers, with structural aspects being significant contributory factors. Key themes identified (1) clinical training lacked structure and support, negatively impacting progress, (2) existing knowledge and experience acted as both an enabler and inhibitor, with implications for confidence, (3) the role and responsibilities are poorly understood by both advanced clinical practitioners and the wider medical profession and (4) advanced clinical practitioners recognised the value and importance of the role but felt changes were necessary, to provide security and sustainability.

Conclusions

Appropriate structure and support are crucial throughout the training process to enable staff to have a smooth transition to advanced level, ensuring they obtain the necessary confidence and competence. Structural changes and knowledge brokering are essential, particularly in relation to role clarity and its responsibilities, sufficient allocated time to learn and practice, role accreditation and continuous appropriate supervision.

Peer Review reports

Introduction

Advanced Clinical Practitioners (ACPs) are a relatively new role in the National Health Service (NHS), introduced to address the increasing complexity of healthcare needs and the growing demand for skilled professionals. They are becoming increasingly embedded within a wide range of NHS healthcare settings spanning community services, mental health wards and hospitals. ACPs play a vital role in expanding the scope of practice within healthcare teams, take on more advanced and complex levels of clinical work, including tasks historically carried out by doctors, with an aim to help alleviate the strain on medical professionals and enhance the efficiency of healthcare delivery. Their integration into the workforce has been shown to enhance patient care by providing timely access to high-quality services while also fostering interdisciplinary collaboration [ 1 ]. Studies have shown that ACPs contribute to improved patient outcomes, increased patient satisfaction, and cost-effective healthcare delivery [ 2 , 3 ]. Additionally, their presence supports the development of junior staff by providing mentorship and guidance, thus ensuring a sustainable healthcare workforce for the future [ 1 , 4 ]. As non-medical healthcare professionals, ACPS are required to undertake further education (Masters degree) and extended training in specific clinical areas such as nursing, pharmacy, or allied health professions to qualify as an ACP. According to the multi-professional framework advanced clinical practice, this training is underpinned by four pillars: clinical practice, leadership and management, education, and research [ 5 ]. However, there is wide variability in this practice and training of ACPs across the UK [ 6 ].

Recent years have seen attempts to standardise the training and practice of ACPs. A framework for advanced clinical practice in England was authored in 2017 by Health Education England (HEE) (NHS England) [ 5 ], which set out standards for advanced clinical practice. Within this framework, advanced clinical practitioners should be able to deliver care with a high degree of autonomy and undertake complex decision making. The knowledge and skills should be underpinned by a Master’s level award (or equivalent) that incorporates the Four Pillars of Practice: Clinical Practice, Leadership and Management, Education and Research [ 5 ]. In 2020, The Centre for Advancing Practice ( https://advanced-practice.hee.nhs.uk/ ) began accrediting some of the many advanced clinical practice Masters programmes available in the UK, which it deemed to have met the standards laid out in HEE’s framework [ 7 ]. This process of defining common standards remains in its early stages, and there remains little research on ACP training programmes and their structures or governance. In a further effort to improve and standardise advanced clinical practice, The Centre for Advancing Practice additionally created guidance on workplace supervision for ACPs, noting the crucial need for high-quality supervision [ 8 ]. The Nursing and Midwifery Council (NMC) published their 2020-25 corporate strategy also in 2020, and committed to explore the need for regulation in a comprehensive review of advanced nurse practice [ 9 ]. The review is still in progress but research undertaken in the early stages by The Nuffield Trust and BritainThinks as part of the review reported inconsistency in definitions, outcomes, standards of education and proficiency in advance practice [ 10 ], and support for regulation by health professionals [ 11 ]. Despite calls for improvements in the supervision of ACPs, there has been limited research in this area.

Studies have shown that ACPs have historically struggled with the transition from their previous career to their advanced practice roles [ 12 , 13 , 14 , 15 ]. The challenges of the transition have been exacerbated by a lack of clear professional identity for ACPs, which has been noted to be a source of tension and confusion, impacting on training, development and ultimately patient safety [ 6 , 16 , 17 , 18 , 19 , 20 ]. Recognising the importance of successful integration into the workforce will help ACPs to realise their full potential [ 21 , 22 ], impacting on role satisfaction [ 23 ], staff retention [ 24 ] and ultimately, building a more sustainable workforce.

As efforts to standardise and develop the ACP role continue, ACPs are becoming more widespread within the NHS. The proliferation of ACPs brings a need for a better understanding of all aspects of ACP training, both during and after qualification. We sought to explore the training experiences of ACPs with the aim of informing future models of education and support.

Theoretical framework

The theoretical framework of this study is based on the theories of Bourdieu, particularly his concept of Habitus [ 25 ], as it offers a valuable lens for examining the multifaceted identities, roles, and positionalities of ACPs. Habitus, ingrained dispositions and cultural knowledge shaped by social experiences, acts as a bridge between individual practitioners and the complex healthcare field they navigate. It influences how ACPs perceive and enact their roles, shaped by their educational background, professional training, and prior clinical experiences. Furthermore, Habitus interacts with the “field,” the social space within which ACPs operate, characterized by power dynamics, established hierarchies, and competing ideologies. This interaction influences the capital, both symbolic and material, that ACPs possess and wield within the field. Through this lens, we can understand how ACPs negotiate complex power dynamics within the healthcare system, navigate tensions between professional autonomy and institutional constraints, and ultimately construct their own sense of meaning and purpose within their evolving roles. By analyzing these interactions between Habitus, field, and capital, Bourdieu’s framework offers a rich and nuanced understanding of the experiences and challenges faced by ACPs, paving the way for further research and dialogue on optimizing their practice and impact.

We sought to explore the training experiences of ACPs as well as their perceptions on role identity, gaining clinical competency and future development of the role.

This exploratory study used a qualitative design to conduct focus groups with a purposive sample of ACPs currently working in South Yorkshire and Bassetlaw in the North of England. In 2022 there were 585 trainee ACPs and approximately 1200 qualified ACPs working in this region.

Participants

Qualified ACPs or trainee ACPs that have completed at least 1 year’s full time equivalent of Advanced Care Practitioners clinical training, and currently work in this role within either Mental Health, Community or Secondary Care within the South Yorkshire and Bassetlaw region. It was felt 12-month minimum training experience would ensure trainees were sufficiently embedded in the clinical and educational programmes.

Recruitment

The NHS England Regional Faculty for Advancing Practice– North East and Yorkshire (FACP-NEY) acted as gatekeepers for the recruitment, contacting all qualified and trainee Advanced Care Practitioners working in the region with an invitation to participate by email. The email included a brief outline of the study, dates and times of the focus groups, details of an incentive payment of £30 for participation, a participant information sheet and, a web link to a short online questionnaire and contact details form. Additionally, the study was also advertised on social media platforms (Twitter, Facebook), with those who expressed an interest sent the same study invitation email, documentation and web link. Recruitment was open between February and May 2021, with one reminder email sent from the FACP-NEY during this time.

ACPs that wished to participate in the study were required to complete the short online questionnaire built using the survey tool, Qualtrics ( www.qualtrics.com ). After confirming eligibility, basic details were recorded about the participant and their ACP training including name, contact details, gender, age group, ethnicity, length of experience in the ACP role, supervision routine, portfolio status and supernumerary time. A variety of dates and times (morning, afternoon and evening) were provided for the focus groups to maximise recruitment, and participants were asked to indicate their preference. A total of 14 participants took part across five focus groups.

Data collection

Focus groups took place online using the Google Meets platform, with a maximum of 3 participants per group. To ensure participants were confident in using the Google Meets platform, the focus group began with an overview of the main functions and how to use them, for example clicking the ‘hand-up’ icon to indicate a wish to speak and chat facility. A focus group schedule was designed and used to guide the discussion similar to that used by Macnaghten and Jacobs (1997) [ 26 ] with an emphasis on each topic followed by discussion amongst the participants. The topics covered included experiences of undertaking ACP training (including gaining competence), role identity and career progression. Data collection was discontinued once it was felt there was no new contributions to the analysis, and there had been full investigation of the developed themes.

Participants provided written informed consent prior to attending the focus group, and consent was also acquired verbally at the start of each focus group. Each focus group was facilitated by one of the two authors (SA and MK), both of whom are experienced qualitative researchers with no clinical background or experience. Google Meets was used to video and audio-record the focus groups. The focus groups were transcribed verbatim by a third party, and quality checked against the recordings for accuracy. The duration of the focus groups was 2 h with a 15-minute comfort break. On completion of the focus groups, participants were sent a £30 shopping voucher to compensate them for their time.

Data analysis

The data was thematically analysed by three researchers (MK, SA and JSK) following the six-phrase process of Braun and Clarke, commencing with familiarisation of the data and then line by line coding to identify preliminary categories [ 27 , 28 ]. The data was then ordered and synthesised, combining similar categories and exploring the relationships between them [ 29 ]. This process was repeated for three of the five transcripts at which point the main themes and sub-themes were identified forming a test model, this was then applied to the final two transcripts. Following discussion amongst the research team, the main themes and sub-themes were agreed. NVIVO Release 1.3 (QSR International) [ 30 ] was used to help organise the data. The Standards for Reporting Qualitative Research (SRQR) checklist was used to report the findings (see Additional file 1).

The focus groups highlighted significant variability in the training experience of ACPs, dependent on their role and place of work. Table  1 provides an overview of the participant characteristics of each of the focus groups, and an overview of the overarching themes and sub-themes that were developed are displayed in Table  2 .

Overarching themes

A number of overarching themes were identified in our analysis that appeared to be strongly linked to role identity. We found the experiences of the ACP training were influenced by internal and external perceptions of the ACP role, often acting as barriers, with structural aspects being significant contributory factors. These findings were revealed in four key themes - lack of structure and support in the clinical training, existing experience and knowledge as enablers and inhibitors to progress with implications for confidence, the poorly understood nature of the ACP role and associated responsibilities, and a need for change to provide security to the ACP role in the future.

Clinical training lacked structure and support

The data revealed a stark contrast between the academic and clinical training, with clinical training found to be lacking in structure and support. Experiences of the clinical training were often expressed negatively due to the lack of structure which was heavily reliant on supervision and placements. As a result, ACPs often had to take the lead on their training and having to identify their own supervisor(s) and/ or placements was felt to be challenging. Consequently, some ACPs reported they had no dedicated medical supervisor at all. Where supervisors were in place, the quality of supervision varied, from being ad hoc (p41) and chaotic (p52) to great ( p53). Some of the supervision issues raised by the ACPs included lack of supervisor knowledge in relation to the ACP training and their required responsibilities, accessibility of supervisor (available time) and little direct clinical oversight. ACPs felt they needed an experienced medical professional as their supervisor, providing similar support and advice to that received by junior doctors.

We have nursing supervision from the lead community matron who is our line manager, but we do miss that sort of medical supervision (p22, Trainee ACP– Primary care) . I’m line managed by a nurse who is the operational lead for the service. He is the right person, but I don’t go to him for clinical support. It would be nice to have a medical supervisor (p. 41, Trainee ACP– Community care) .

Good supportive supervision appeared to enhance the ACP training, conversely poor, unsuitable or no supervision was perceived to have a serious negative impact on training and well-being, with suggestions that ACPs had left during training because of it.

I’ve had free reign over my own training, and planned everything myself, and that’s a positive for me (p41, Trainee ACP– Community care) . So the positives, um, I think the academic and educational supervision’s been, err, accessible and supportive. So we have, um, supervision from [regional] ACP lead,…and then there’s, um, the course unit lead, which she’s there and she’s supportive. So yeah, the academic, err, supervision is good (p52, Trainee ACP– Secondary care) . I think, um, something that I haven’t touched upon is, which I realised, so I’ve got a, um, clinical supervisor, she’s a consultant *****, and…the module I’ve just done which is minor illness, you had to do like a learning log, so they had to see you do….a load of things. And it made me laugh cos they turned around and said, look, I haven’t assessed anybody’s abdomen in ten years…. (p53, Trainee ACP– Secondary care)

Similar to supervision, clinical placements were highly valued by the ACPs and recognised as an important part of the training to achieve competence and consolidate their academic learning. All of the ACPs reported obstacles in organising and undertaking such placements, with those working in the community or mental health facing particular difficulties due to placements needing to be in a different clinical setting to where they worked. Competition with other trainees, the need to ‘ beg ’ (p7) and insufficient time from trainers were highlighted as ongoing problems. Conflict with junior doctors was also described as a competition for training opportunities.

Completely unsupported by the Trust because they just weren’t set up for it, there was no one leading on it, there were no one for us to contact really to talk. And then, like you said, I got my placements from begging on a, on a forum on Facebook and a nurse set me up (p16, Trainee ACP– Secondary care) . To kind of fulfil the module requirements, it was pretty much, for minor illness basically phoning up GP surgeries, practice nurses, beg stealing and borrowing, you know, begging people can you help me out, to try and get the amount of hours that you needed (p7, Trainee ACP– Secondary care) . But sometimes, it’s a little bit of a fight to get to what you need when you need because there’s so many junior doctors that also need that same training. So, there are occasions where you have to sort of step up and say we are training the same as these guys, we also need to be able to have these opportunities and you kind of have to have a little bit of a voice to say, we’re here (p17, Trainee ACP– Secondary care) .

In contrast to the clinical training, the academic learning followed a traditional format of taught lessons which ACPs felt covered a wide breadth of knowledge. There was some feeling that modules might have been more useful if they had been tailored towards individuals’ specialisms such as mental health or physiotherapy, however on the whole it was described as a positive learning experience with good supportive academic supervision.

I found the dissection labs quite alien but they have really helped to develop my practice (p24, Qualified ACP– Secondary care) . It feels a lot like there’s university, which is one day a week, and you do that, and it’s really supportive, and I’ve made some really good friends there, and everybody supports each other. But then at work, it’s a bit of a try and find your own way (p53, Trainee ACP– Secondary care) .

ACPs did describe the two learning environments (clinical and academic) as disconnected, separate and discrete, even though the ACP training is a combination of academic and clinical learning.

From the course point of view it’s pretty straightforward but it’s marrying that up with the expectations of the employer. Willingness of the employer to be able to give you the time you need to do what you need to do (p. 38, Qualified ACP– Secondary care) . They’d learn something at University (e.g. Cardiology) but there was no way this could be built on within the Trust. They just don’t deal with the physical health side of things (p. 7, Trainee ACP– Community mental health) .

Existing knowledge and experience appeared to act as both an enabler and inhibitor for ACPs, with implications for confidence

As existing experienced clinical practitioners, ACPs felt they were able to recognise their knowledge gaps and work quickly towards filling them, however the training approach also led to declines in confidence when deficiencies in knowledge and skills were highlighted. ACPs reported learning ‘backwards’ compared to junior doctors, using pattern recognition rather than pathology as a starting point, for example, being able to identify the treatment based on a diagnosis, but not necessarily knowing how the diagnosis was made originally. Not being able to adequately answer questions sufficiently on such subjects when tested by clinicians, and as experienced clinical practitioners, ACPs perceived themselves as lacking competence with a subsequent drop in confidence.

ACPs are trained ‘bottom-up’– we learn pattern recognition and then work our way back, whereas doctors know the diseases better (p41, Trainee ACP– Community care) . I think about cases backwards compared to doctors– as they think about pathology first and then build on that (p9, Trainee ACP– Primary care) .

A comparison between the clinical training processes of junior doctors and ACPs was a common discussion between ACPs with suggestions that it would be more beneficial if ACPs were recognised in a similar manner to junior doctors. For example, ACPs felt they should not be ‘counted in the nursing numbers’ when working on a ward, and as a consequence should not be expected to undertake a dual role of managing a nursing shift and practicing as an advanced practitioner:-

So say for example, you’re sat with somebody talking about their prescription and trying, you know, looking to see if there needs to be a change made, and then you’ve got other people banging on the door saying, I want to go out on leave, and I need this and I need that, and you’re the nurse in charge and need to be doing that. The people that usually do those jobs, so say for example the doctors in the week, when they’re having those sorts of consultations with people, they’ve not got that stress, the pressure, the disruption and the responsibility of running a nursing shift or a completely other shift. So, us as novices, it just doesn’t make sense to me (p. 52, Trainee ACP– Secondary care)

ACPs spoke of being unsure of when they had reached clinical competency, and how they would maintain this. They worried that if they were not given sufficient time to practise the new clinical skills, their confidence would decline and that they would ultimately feel unsafe in their clinical practice. ACPs emphasised the importance of having sufficient time to practice new skills and consolidate knowledge, enabling autonomy and confidence building. It was also felt this provided essential opportunities for colleagues to observe progress.

I’ve got most of my competencies but I still wouldn’t see myself as an expert practitioner (p41, Trainee ACP– Community care) .

The ACP role and associated responsibilities are poorly understood by ACPs and the wider medical profession

Exploring the experiences of training and the process of developing clinical competence with ACPs revealed there was a lack of clarity regarding the job role depending on where the ACP worked, and this applied to the ACPs themselves as well as their colleagues. This uncertainty impacted the responsibilities the ACP undertook within the clinical environment, and the expectations on them from the staff that they worked with.

ACPs that worked within the Emergency Department reported that colleagues understood the ACP role and utilised the advanced skillset the ACPs gained as the training progressed. They described feeling fully immersed within the department as an advanced practitioner, yet they were also recognised as being in a transitional stage with appropriately allocated time to undertake the necessary training.

ACPs working in other areas of healthcare such as acute wards, outpatients, mental health and community care discussed a general lack of awareness about the advanced practitioner role by both healthcare staff and patients. It was felt this led to a lack of utilisation of the advanced skills of the ACPs and expectations that the ACP should fulfil multiple job roles, creating feelings of intense pressure and demoralisation. ACPs reported hearing discouraging comments from colleagues about their abilities and felt a need to justify their role. Some ACPs described struggling with how to introduce themselves to both staff and patients, with their uniform described as an important part of their identity and how they were perceived by others. Adding to these external perceptions, ACPs revealed their job description was not necessarily updated to reflect their ACP role and where it was, the job description could be vague further undermining their role identity and leading to feelings of conflict between their original healthcare professional role (e.g., nurse) and working at an advanced level.

There’s been a lot of ambiguity around the job description for ACPs and trainee ACPs, so that’s left wriggle room for everybody making their own assumptions about what you’re supposed to do and what you should be doing, and therefore you’re pulled into all different things that don’t tie in to on paper in terms of national, regional frameworks……. there’s just pressure on the role being categorised as an extension of the nursing team, and taking on classic nursing tasks, it’s what people are familiar with, it’s what they assume (p52, Trainee ACP– Secondary care) . The challenge is with our role, is the ACP is tagged on to the end of our existing job. So, we have all of our normal nursing duties, we’re bed managers, we triage nurse, we run the hospital. And then you’ve got ACP tagged on the end. (p25, Qualified ACP– Secondary care)

Inconsistencies in awareness of the role, experience, training and clinical practice were felt to be a reflection of the different professions undertaking ACP training, a lack of standardised job role and unclear expectations. The variation in financial remuneration within and across different organisations for ACPs was also felt to be a contributing factor to these identity issues.

The ACP role is important, but changes are required to provide security to the role in the future

There was consensus that the combination of experience and advanced skills made the ACP a unique and valued role in the NHS, fulfilling an important gap in patient care. ACPs reported uncertainty about their future in the role, and the need for change structurally to ensure the ACP role has a future. Accreditation was felt to be necessary as this would legitimise the ACP role and apply some professional control in respect to the role title. ACPs viewed this as an existing issue with ‘advanced’ used by a multitude of health professions that have not undertaken the accredited training.

I kind of feel that, certainly as an ACP title, it should be some sort of standardised title, and then people would probably understand it a little bit more. I think our colleagues would understand it, and I think you won’t get so much resistance, from some medical colleagues, maybe, if people were sort, if it were a bit more regulated. I mean, if there were talking about credentialing and looking at a directory for ACPs anyway, it should be a registered regulated title (p54, Qualified ACP– Primary care) . I think everybody should be under the same governing body and there should be a bit of standardised, training placement (p41, Trainee ACP– Community care) .

As well as increased knowledge and skills, ACPs discussed the additional benefits of the training including the broad range of opportunities offered both during and after the training, and the potential boost in future prospects. A key attraction to the ACP training route that was repeatedly highlighted was the fact that it offers career progression whilst maintaining clinical responsibilities, progressing through more traditional routes into a managerial role appears to involve considerably less clinical duties and contact with patients. However, there was also some feelings of insecurity regarding the future of the ACP role because of the general lack of awareness of how ACPs fitted and could contribute to the NHS. It was felt that the deficiency in formal structure for the ACP role contributed to this; ensuring job descriptions existed and reflected the responsibilities of the role, and there was a structure for career progression was proposed as a good starting point to improve understanding amongst staff.

In terms of where I see myself in five to ten years’ time, I’m not sure, it depends how that organisation I work for pans out, because…. I won’t be sat here in five years’ time saying the same stuff. If it’s still the same I won’t be there, I will have gone somewhere else cos there are places that fulfil the role (p52, Trainee ACP– Secondary care) . I don’t see much career progression within ACPs other than to become a lead ACP and there is nothing to define progression within that role from a banding point of view (p1, Trainee ACP– Secondary care) .

On the whole, the ACPs felt the role had great future potential but this was often caveated, that changes were needed in formalisation of the training and particularly, wider recognition of the role and its responsibilities. Without these changes, a number of ACPs felt they would not be in the ACP role in 5 years’ time.

The one thing that I do know is that I love the job, I love the role (p38, Qualified ACP– Secondary care) .

This qualitative study collected the perspectives of 14 ACPs from different specialties and at different stages of their career. The findings suggest that ACPs continue to face significant barriers, undermining their development, transition and integration into the healthcare workforce.

ACPs described a number of challenges experienced in their training within the clinical environment, notably with placements and supervision. Both of these elements appeared to suffer from a lack of formal structure; where some ACPs experienced a supportive clinical environment making their training experience ‘phenomenal’, others reported unsuitable supervision and having to identify their own supervisors and/ or placements. This lack of support was felt to have a serious negative impact on ACP training and well-being, which has been reported nationally and internationally [ 17 , 31 , 32 ]. It is recognised that a supportive environment is a healthy environment, aiding not only ACPs in their competency, role transition and job satisfaction but also helping to optimise quality patient care, recruitment and retention [ 13 , 24 ]. Additionally, a disconnect between academic and clinical training was highlighted. This lack of ‘joined-up’ working between educators, healthcare staff and managers has been described previously with suggestions that it can impede the development of ACPs and their fulfilment of the role [ 22 , 33 ].

The knowledge and experience already held by ACPs from their original professional training was perceived as both a strength and weakness. Whilst the ACPs felt they could provide improved holistic patient care and identify gaps in their own training, it influenced their approach to learning which was described as ‘bottom-up’ and ‘backwards’ compared to how junior doctors learnt. This had implications for confidence as ACPs often felt they could not adequately answer questions posed during training. Furthermore, if they were not given sufficient time to consolidate their new knowledge, this led to an additional drop in confidence and doubts about their competence. This was reported by MacLellan, Higgins and Levett-Jones (2017) [ 34 ] and has been referred to as Imposter Syndrome [ 35 ]. It links closely with role transition and identity which has been widely researched within the advanced practitioner community [ 12 , 13 , 14 ]. Increasing autonomy and responsibility is part of the transition for ACPs and whilst some of the ACPs in this study found this experience exciting, the majority conveyed mixed emotions including feeling stressed, pressured and uncertain. This was more prominent for those ACPs in areas where the role appeared to be less established and a lack of awareness among healthcare staff of the ACP role. For a smooth and successful transition, Barnes (2015) [ 12 ] identified a number of defining attributes including a shift from provider of care to prescriber of care, straddling two identities and mixed emotions. The experiences of our ACPs covered all of these attributes and suggest they have not experienced a smooth transitional journey.

Inconsistencies in the ACP training and lack of structure in relation to the clinical job role were discussed as contributing factors to role identity issues, which impacted their daily working lives. It appears the ACPs in our study are still experiencing the consequences of a role which was introduced without clear definition, standardisation, skills and scope [ 20 ], even though there has been significant development in recent years within advanced practice [ 5 ] of the ACP training. As a role introduced to work alongside doctors, nurses, pharmacists, and other healthcare professionals to deliver comprehensive and patient-centred care, ACPs play a pivotal role in fostering interprofessional collaboration within healthcare teams. However, with blurred definitions regarding the ACP role and responsibilities, it is unsurprising our ACPs reported a lack of understanding of their expertise and respect from their colleagues. Such barriers to interprofessional collaboration not only prevents ACPs from working to the full extent of their education and training [ 36 ] but impacts patients, on their outcomes and access to specialist care [ 21 , 37 , 38 ]. A review of 64 studies undertaken by Schot, Tummers and Noordegraaf (2020) of interprofessional collaboration among healthcare professionals described this as being multifaceted, and that for change to occur, individuals needs to work daily on tasks such as bridging gaps, negotiating overlaps and creating spaces [ 39 ].

There was agreement between the ACPs that accreditation of the role would help address some of the issues around role identity. The use of ‘advanced practice’ is widely applied within healthcare with little relationship to education level, often leading to confusion [ 18 ]. Accreditation would help protect the role by providing professional identity as well as providing more clarity to ACPs and those in the wider healthcare setting about the role and scope of practice [ 6 , 17 , 18 ]. It may also alieve fears of insecurity which were raised by the ACPs in relation to the future of the role. Improving and promoting knowledge brokering at both the individual and collective (system) levels would improve the transition process [ 40 ], whilst also encouraging change in an environment that is traditionally intransigent.

Although the ACPs reported challenges in their training and felt changes were necessary to ensure wider recognition of the ACP role, there was consensus among the ACPS that participated in this study that the training ‘boosted’ opportunities and allowed career progression whilst maintaining clinical responsibilities, an important factor to many of the ACPs in this study. Surprisingly, there was little discussion regarding the impact of the COVID-19 pandemic on ACP training, even though the focus groups took place during the pandemic. When it was discussed, it was generally in the context of placements and how they had been further limited.

This qualitative insight into the training experience of ACPs has highlighted that there are many challenges still to be overcome to ensure ACPs feel supported through their role transition journey and are recognised appropriately for their skills and experience in the healthcare workforce. These findings are not new [ 13 , 14 , 20 , 22 ] but after the release of the 2017 HEE multi-professional framework for advanced clinical practice [ 5 ], it would be expected that there would have been more clarity and structure in the ACP training and role, benefitting ACPs, wider healthcare professionals and employers. Progress may improve as a result of the NMC review on regulation of advanced nursing practice that is due in the next 12 months [ 9 ], however, at the time of this study, the ACPs appeared to feel progress was slow and more work was needed.

Strengths and limitations

The opinions and experiences provided in this study were from a group of ACPs, either during (> 1 year FTE) or post training, working in the South Yorkshire and Bassetlaw region. It is reasonable to suggest therefore that the results are not generalisable to other populations. Qualified and trainee ACPs were contacted about the study by email through the regional FACP-NEY who acted as gate keepers, as well as the study being advertised on social media platforms. It is assumed that this broad recruitment strategy helped to reach a wider population, although most respondents appeared to be as a result of the direct email. This approach may have introduced some bias but using a purposive sampling approach, participants from different specialties, professions and career stages were included. Information about the local ACP workforce such as size and individual characteristics was requested from the regional FACP-NEY but this was not provided thus an exact response rate cannot be calculated nor can any inferences be made regarding how representative the sample of ACPs were that participated in the study. The number of males that registered an interest in the study was low (three) and only one male participated in the focus groups; this is a limitation as there may be different perspectives and experiences of ACP training related to gender. Due to the COVID-19 pandemic focus groups had to be undertaken online. Adaptions were made to accommodate for this such as reducing the number of participants per focus group and creating time to build rapport [ 41 ]. One participant did experience technical issues, however using a digital approach did not appear to impede the participant-researcher interaction and compared favourably with traditional face to face focus groups [ 41 , 42 ]. There is a risk that views from participants were oversimplified due to the limited number of ACPs involved in the focus groups but findings from this study appear to align with previously published literature [ 6 , 17 , 19 , 21 ] providing some confidence in the results.

Future work

This was a small exploratory study in a rapidly evolving field, providing insights on ACP training, role identity and competence at one point in time. ACPs did report differences in their experiences due to their specialty thus a much larger study would provide an opportunity to explore this further and allow for more in-depth comparisons. The multi-professional framework was relatively new when this study was undertaken and since its publication, there has been much development in the guidance and practice of ACPs including the Royal College of Emergency Care ACP training [ 43 ] and the merger of Health Education England with NHS England. It would be useful to explore what impact, if any, these developments may have had on ACPs and if similar issues around role identity and competence still exist.

The ACP role is now integrated across many specialties both nationally and internationally, however challenges continue to persist in training, impacting on transition into the role. At a collective level, there remains a lack of structure and clarity around the ACP role, and individually ACPs appear to experience issues with supervision and support. This study has highlighted that the journey to advanced level practice is often turbulent, and changes are required to further embed the ACP training and role into the workplace. Ensuring ACPs have appropriate continuous support, allocated sufficient time to learn and practice, and wider recognition of the ACP role through accreditation would aid the training experience and a successful role transition.

Data availability

The datasets generated and analysed during the current study are not publicly available due to the participants privacy being compromised but are available from the corresponding author on reasonable request.

Abbreviations

Advanced Clinical Practitioner

Faculty for Advanced Clinical Practice

Health Education England

National Health Service

Imison C, Castle-Clarke S, Watson R. Reshaping the workforce to deliver the care patients need. Research report 2016, Nuffield Trust: London.

Mann C, et al. Exploring the role of advanced clinical practitioners (ACPs) and their contribution to health services in England: a qualitative exploratory study. Nurse Educ Pract. 2023;67:103546.

Article   PubMed   PubMed Central   Google Scholar  

Evans C, et al. Characterising the outcomes, impacts and implementation challenges of advanced clinical practice roles in the UK: a scoping review. BMJ Open. 2021;11(8):e048171.

Stewart-Lord A, et al. The role and Development of Advanced Clinical Practice within Allied Health Professions: a mixed method study. J Multidisciplinary Healthc. 2020;13:1705–15.

Article   Google Scholar  

Health Education England. Multi-professional framework for advanced clinical practice in England . 2017.

Evans CR, et al. Characterising the evidence base for advanced clinical practice in the UK: a scoping review protocol. BMJ Open. 2020;10(5):e036192.

Health Education England: The Centre for Advancing Practice. Programme accreditation. 2023; Available from: https://advanced-practice.hee.nhs.uk/our-work/programme-accreditation/ .

Health Education England. The Centre for advancing Practice., workplace supervision for Advanced Clinical Practice: an integrated multi-professional approach for practitioner development . Health Education England; 2020.

Nursing and Midwifery Council (NMC). Regulate, support, influence: strategy 2020–2025. London: NMC; 2020.

Google Scholar  

Palmer W, Julian S, Vaughan L. Independent report on the regulation of advanced practice in nursing and midwifery. Research report. London: Nuffield Trust; 2023.

BritainThinks. Advanced Practice qualitative research . 2023, BritainThinks: London.

Barnes H. Nurse practitioner role transition: a Concept Analysis. Nurs Forum. 2015;50(3):137–46.

Article   PubMed   Google Scholar  

Moran G, Nairn S. How does role transition affect the experience of trainee Advanced Clinical practitioners: qualitative evidence synthesis. J Adv Nurs. 2018;74(2):251–62.

Fitzpatrick S, Gripshover J. Expert nurse to novice nurse practitioner: the Journey and how to improve the process. J Nurse Practitioners. 2016;12(10):e419–21.

Timmons S, et al. The Advanced Clinical Practitioner (ACP) in UK healthcare: dichotomies in a new ‘multi-professional’ profession. SSM - Qualitative Res Health. 2023;3:100211.

Reynolds J, Mortimore G. Transitioning to an ACP: a challenging journey with tribulations and rewards. Br J Nurs. 2021;30(3):166.

Fothergill L, et al. Nationwide evaluation of the advanced clinical practitioner role in England: a cross-sectional survey. BMJ Open. 2022;12(1):e055475.

Leary A, et al. Variation in job titles within the nursing workforce. J Clin Nurs. 2017;26(23–24):4945–50.

Lawler J, Maclaine K, Leary A. Workforce experience of the implementation of an advanced clinical practice framework in England: a mixed methods evaluation. Volume 18. Human Resources for Health; 2020. 1.

Nadaf C. Perspectives: reflections on a debate: when does Advanced Clinical Practice stop being nursing? J Res Nurs. 2018;23(1):91–7.

Evans C, et al. Advanced Clinical practitioners in Primary Care in the UK: a qualitative study of Workforce Transformation. Int J Environ Res Public Health. 2020;17(12):4500.

Jones A, et al. Realising their potential? Exploring interprofessional perceptions and potential of the advanced practitioner role: a qualitative analysis. BMJ Open. 2015;5(12):e009740.

Wood E, et al. Sources of satisfaction, dissatisfaction and well-being for UK advanced practice nurses: a qualitative study. J Nurs Manag. 2021;29(5):1073–80.

Price S, Reichert C. The importance of Continuing Professional Development to Career satisfaction and patient care: meeting the needs of novice to Mid- to late-Career nurses throughout their Career Span. Administrative Sci. 2017;7(2):17.

Bourdieu P. The logic of practice. Translated by R. Nice . 1990, London, UK: Polity Press.

Macnaghten P, Jacobs M. Public identification with sustainable development: investigating cultural barriers to participation. Glob Environ Change, 1997: p. 5–24.

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101.

Braun V, Clarke V. Successful qualitative research: a practical guide for beginners. Thousand Oaks, California: Sage; 2013.

Strauss A, Corbin JM. Basics of qualitative research: grounded theory procedures and techniques. California: Sage Publications Inc; 1990.

QSR International Pty Ltd. NVivo qualitative data analysis software . 2020.

Cusson R, Strange S. Neonatal nurse practitioner role transition: the process of reattaining expert status. J Perinat Neonatal Nurs. 2008;22(4):329–37.

Duke C. The lived experience of nurse practitioner graduates’ transition to hospital-based practice , in nursing . East Carolina University: US; 2010. p. 138.

Townsend A. Developing a portfolio for advanced practice. Nurs Times. 2015;111:23–6.

MacLellan L, Higgins I, Levett-Jones T. An exploration of the factors that influence nurse practitioner transition in Australia: a story of turmoil, tenacity, and triumph. J Am Assoc Nurse Pract. 2017;29(3):149–56.

Redfern-Jones J. Unmask your talent. Nurs Stand. 2009;23(38):64.

Moss E, Seifert CP, O’Sullivan A. Registered nurses as Interprofessional Collaborative Partners: creating Value-based outcomes. Online J Issues Nurs, 2016. 21(3).

Goldsberry JW. Advanced practice nurses leading the way: interprofessional collaboration. Nurse Educ Today. 2018;65:1–3.

Hardy M. Health and Care Professions Council. Advanced practice: research report . 2021.

Schot E, Tummers L, Noordegraaf M. Working on working together. A systematic review on how healthcare professionals contribute to interprofessional collaboration. J Interprof Care. 2020;34(3):332–42.

Putri AF, Tocher J, Chandler C. Emergency department nurses’ role transition towards emergency nurse practitioner: a realist-informed review. Int Emerg Nurs. 2022;60:101081.

Keen S, Lomeli-Rodriguez M, Joffe H. From challenge to opportunity: virtual qualitative research during COVID-19 and Beyond. Int J Qual Methods. 2022;21:16094069221105075.

Murray B. Online tools for qualitative research during the Covid-19 pandemic. Nurs Times [online]. 2022;118(6):1–3.

Royal College of Emergency Medicine. Emergency Medicine– Advanced Clinical Practitioner (EM-ACP). 2024; Available from: https://rcem.ac.uk/emergency-care-advanced-clinical-practitioners/ .

Download references

Acknowledgements

The authors would like to thank the Sheffield Emergency Care Forum (SECF) PPI group ( https://secf.org.uk/ ) for their helpful feedback on the recruitment materials (email invitation, information sheet and consent form) developed for this study to ensure they were suitable for a lay audience. For helping with the recruitment, we would also like to thank the Faculty of Advanced Practice, particularly Fran Mead. Lastly, we would like to thank our participants for giving up their valuable free time to share with us their experiences of developing clinical competence as an Advanced Clinical Practitioner, and for their opinions on role identity.

This manuscript is independent research funded by the National Institute for Health and Care Research, Yorkshire and Humber Applied Research Collaborations (NIHR200166). The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the National Institute for Health and Care Research or the Department of Health and Social Care.

Author information

Authors and affiliations.

Centre for Urgent and Emergency Care Research (CURE), Sheffield Centre for Health and Related Research (ScHARR), The University of Sheffield, S1 4DA, Sheffield, UK

Maxine Kuczawski, Suzanne Ablard, Fiona Sampson, Susan Croft, Joanna Sutton-Klein & Suzanne Mason

Emergency Department, Northern General Hospital, S5 7AU, Sheffield, UK

Susan Croft

Manchester Royal Infirmary, Oxford Rd, M13 9WL, Manchester, UK

Joanna Sutton-Klein

You can also search for this author in PubMed   Google Scholar

Contributions

Made substantial contributions to conception and design, or acquisition of data: MK, SA and JSK; Analysis and interpretation of data: MK, SA, SC and JSK; Manuscript draft: MK, SA and JSK; Manuscript critical revisions: MK; SA; SC; JSK; FS; SM. All authors approved the final version of the manuscript.

Corresponding author

Correspondence to Maxine Kuczawski .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval for the conduct of the research was obtained from the University of Sheffield Research Ethics Committee (REC reference. 032462). Written informed consent was obtained from all participants prior to the focus group, and all methods were performed in accordance with the relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Kuczawski, M., Ablard, S., Sampson, F. et al. Exploring advanced clinical practitioner perspectives on training, role identity and competence: a qualitative study. BMC Nurs 23 , 185 (2024). https://doi.org/10.1186/s12912-024-01843-x

Download citation

Received : 20 September 2023

Accepted : 04 March 2024

Published : 18 March 2024

DOI : https://doi.org/10.1186/s12912-024-01843-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Advanced clinical practitioner
• Qualitative
• Role identity

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

SYSTEMATIC REVIEW article

Promoting mental health in children and adolescents through digital technology: a systematic review and meta-analysis.

• Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan, China

Background: The increasing prevalence of mental health issues among children and adolescents has prompted a growing number of researchers and practitioners to explore digital technology interventions, which offer convenience, diversity, and proven effectiveness in addressing such problems. However, the existing literature reveals a significant gap in comprehensive reviews that consolidate findings and discuss the potential of digital technologies in enhancing mental health.

Methods: To clarify the latest research progress on digital technology to promote mental health in the past decade (2013–2023), we conducted two studies: a systematic review and meta-analysis. The systematic review is based on 59 empirical studies identified from three screening phases, with basic information, types of technologies, types of mental health issues as key points of analysis for synthesis and comparison. The meta-analysis is conducted with 10 qualified experimental studies to determine the overall effect size of digital technology interventions and possible moderating factors.

Results: The results revealed that (1) there is an upward trend in relevant research, comprising mostly experimental and quasi-experimental designs; (2) the common mental health issues include depression, anxiety, bullying, lack of social emotional competence, and mental issues related to COVID-19; (3) among the various technological interventions, mobile applications (apps) have been used most frequently in the diagnosis and treatment of mental issues, followed by virtual reality, serious games, and telemedicine services; and (4) the meta-analysis results indicated that digital technology interventions have a moderate and significant effect size ( g  = 0.43) for promoting mental health.

Conclusion: Based on these findings, this study provides guidance for future practice and research on the promotion of adolescent mental health through digital technology.

Systematic review registration: https://inplasy.com/inplasy-2023-12-0004/ , doi: 10.37766/inplasy2023.12.0004 .

1 Introduction

In recent years, the mental health status of children and adolescents (6–18 years old) has been a matter of wide societal concern. The World Health Organization noted that one in seven adolescents suffers from mental issues, accounting for 13% of the global burden of disease in this age group ( World Health Organization, 2021 ). In particular, the emergence of COVID-19 has led to an increase in depression, anxiety, and other psychological symptoms ( Jones et al., 2021 ; Shah et al., 2021 ). There is thus an urgent need to monitor and diagnose the mental health of teenagers.

The development of digital technology has brought about profound socio-economic changes; it also provides new opportunities for mental health diagnosis and intervention ( Goodyear and Armour, 2018 ; Giovanelli et al., 2020 ). First, digital technology breaks the constraints of time and space. It not only provides adolescents with mental health services at a distance but also enables real-time behavioral monitoring for the timely acquisition of dynamic data on adolescents’ mental health ( Naslund et al., 2017 ). Second, due to the still-developing stage of mental health resource building, traditional intervention methods may not be able to meet the increasing demand for mental health services among children and adolescents ( Villarreal, 2018 ; Aschbrenner et al., 2019 ). In addition, as digital natives in the information age, adolescents have the ability to use digital technology proficiently, and social media, such as the internet, has long been integrated into all aspects of adolescents’ lives ( Uhlhaas and Torous, 2019 ). However, it is worth noting that excessive reliance on digital technology (e.g., internet and smartphone addiction) are also common triggers of mental problems among youth ( Wacks and Weinstein, 2021 ). Therefore, we must be aware of the risks posed by digital technology to better utilize it for promoting the mental health of young people.

Mental health, sometimes referred to as psychological health in the literature, encompasses three different perspectives: pathological orientation, positive orientation, and complete orientation ( Keyes, 2009 ). Pathological orientation refers to whether patients exhibit symptoms of mental issues, including internalized mental disorders (e.g., depression and anxiety) and behavioral dysfunctions (e.g., aggression, self-harm) as well as other mental illnesses. Studies have indicated that both internalizing and externalizing disorders belong to different dimensions of mental disorders ( Scott et al., 2020 ), and internalizing symptoms often occur simultaneously with externalizing behaviors ( Essau and de la Torre-Luque, 2023 ). The positive orientation suggests that mental health is a positive mental state, characterized by a person’s ability to fully participate in various activities and to express positive and negative emotions ( Kenny et al., 2016 ). The complete orientation integrates pathological and positive orientation ( Antaramian et al., 2010 ), suggesting that mental health means the absence of mental issues and the presence of subjective well-being ( Suldo and Shaffer, 2008 ). The development of social emotional abilities helps to promote subjective well-being for adolescents during social, emotional, and cognitive development ( Cejudo et al., 2019 ). Adolescents with mental health issues may thus exhibit pathological symptoms or lack of subjective well-being due to a lack of social emotional abilities. In this study, mental health is defined as a psychological state advocated by the complete orientation.

Promoting mental health using digital technology involves providing help through digital tools such as computers, tablets, or phones with internet-based programs ( Hollis et al., 2017 ). Currently, various digital technologies have been tested to address mental health issues in young individuals, including apps, video games, telemedicine, chatbots, and virtual reality (VR). However, the impact of digital technology interventions is affected by various factors ( Piers et al., 2023 ). Efficacy varies based on the kind of mental health issues. Individuals with mental illness related to COVID-19 may profit more from digital interventions than those experiencing depression and anxiety. Moreover, studies reveal that several mental health conditions in young people deteriorate with age, particularly anxiety and suicide attempts ( Tang et al., 2019 ). The impact of digital technology interventions may therefore differ depending on the adolescent’s age. Having psychological problems usually indicates that people are in an unhealthy mental state for a long time, so an enduring intervention may have greater efficacy than a short-term one. Earlier studies have also suggested that the outcomes of treatment are linked to its duration, with patients receiving long-term treatment experiencing better results ( Grist et al., 2019 ).

Although more digital technologies are being used to treat mental health issues, the most important clinical findings have come from strict randomized controlled trials ( Mohr et al., 2018 ). It is still unclear how these interventions affect long-term care or how they would function in real-world settings ( Folker et al., 2018 ). There is much relevant empirical research, but it is scattered, and there is a need for systematic reviews in this area. In previous studies about technology for mental health, Grist et al. (2019) analyzed how digital interventions affect teenagers with depression and anxiety, but their study only considered mental disorders, without considering other mental health issues. Cheng et al. (2019) examined serious games and their application of gamification elements to enhance mental health; however, they overlooked various technological approaches beyond serious games and did not give adequate consideration to the diverse types and features of technology. Eisenstadt et al. (2021) reviewed how mobile apps can help adults between 18 and 45 years of age improve their emotional regulation, mental health, and overall well-being; however, they did not investigate the potential benefits of apps for teenagers.

The present study reviews research from the past decade on digital technology for promoting adolescent mental health. A systematic literature review and meta-analysis are used to explore which types and features of technology can enhance mental health. We believe that the present study makes a meaningful contribution to scholarship because it is among the earliest to report on the impact of technology-enhanced mental health interventions and has revealed crucial influencing factors that merit careful consideration during both research and practical implementation. The following three research questions guided our systematic review and meta-analysis:

1. What is the current status of global research on digital technology for promoting children and adolescent mental health?

2. What digital technology characteristics support the development of mental health among children and adolescents?

3. How effective is digital technology in promoting the mental health of children and adolescents? What factors have an impact on the effectiveness of digital technology interventions?

2 Study 1: systematic literature review

2.1.1 study design.

This study used the systematic literature review method to analyze the relevant literature on the promotion of mental health through digital technology. It followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis statement for the selection and use of research methods. The protocol for this study was registered with INPLASY (2023120004). Standardized systematic review protocol is used to strictly identify, screen, analyze, and integrate literature ( Bearman et al., 2012 ). To clarify the research issues, systematic literature reviews typically comprise the following six key procedures: planning, literature search, literature assessment, data extraction, data synthesis, and review composition ( Lacey and Matheson, 2011 ).

2.1.2 Literature search

To access high-quality empirical research literature from the past decade, this study selected SCIE and SSCI index datasets from the Web of Science core database and Springer Link. Abstracts containing the English search terms “mental health or psychological health or psychological wellbeing” AND “technology or technological or technologies or digital media” AND “K-12 or teenager or children or adolescents or youth” were retrieved. The search period spanned from January 1, 2013, to July 1, 2023, and 1,032 studies were obtained. To ensure the relevance of the studies to the research question, the relevant inclusion and exclusion criteria were developed based on the 1,032 studies retrieved. The specific criteria are listed in Table 1 .

Table 1 . Literature screening criteria.

In this study, we followed a systematic literature review approach and screened the retrieved studies based on the above selection criteria. We conducted three rounds of screening and supplemented new studies through snowballing, ultimately including 59 effective sample documents. The specific process is shown in Figure 1 .

Figure 1 . Screening process and results.

2.1.3 Coding protocol

To extract key information from the included papers, we systematically analyzed 59 studies on the basis of reading the full text. Our coding protocol encompassed the following aspects: (a) basic information about the study, including the first author, publication year, publication region, study type, study object, and intervention duration; (b) the type of technology used in the study, including apps, chatbots, serious games, VR/AR, short messaging service (SMS), telemedicine services, and others; (c) mental health issues, including depression and anxiety, mental illness, bullying, lack of social and emotional competence, mental health issues caused by COVID-19, and other mental health issues; and (d) experimental data (mean, sample size, standard deviation or p -value, t -value, etc.). By capturing basic study information, we establish a foundation for comparing and contextualizing the selected studies. The type of technology used is crucial as it reflects the innovative approaches and their technical affordances. Mental health issues are the core focus that dictates the objectives of the technological interventions as well as their suitability and relevance. Experimental data provides quantifiable evidence to support the effectiveness claims and lays a foundation for the meta-analysis. Together, these four coding aspects offer a holistic view for a comprehensive understanding and analysis of the existing literature. The document coding was completed jointly by the researchers after confirming the coding rules and details through multiple rounds of negotiation. Problems arising in the coding process were intensively discussed to ensure consistency and accuracy of the coding.

2.2 Results and discussion

2.2.1 study and sample characteristics.

As shown in Figure 2 , in terms of the time of publication, the number of studies has gradually increased from 2013 to 2021 along with the development of digital technology. The proportion of studies published in the past 5 years (2019–2023) accounted for 76.3% of the total (45/59), with a peak in 2021 with 15 papers. Social isolation, school suspension, and reduced extracurricular activities caused by COVID-19 may exacerbate mental health issues among children and adolescents, which has attracted more researchers to explore the application of digital technology to mental health treatment.

Figure 2 . Trend in the number of studies published in the past decade.

From the perspective of published journals, all of the studies were published in 41 kinds of journals, but two fields were clear leaders: 46 studies (77.97%) were published in medical journals, followed by psychological journals (13.56%). Table 2 shows the source distribution and types of the sample studies. Looking at the country of the first author, the largest number of articles came from the Americas, including the United States and Canada, accounting for 40.7%, followed by European countries, including the United Kingdom and Finland. Only one article came from the African region. In terms of the research types, experimental research was the main type, followed by mixed research, and the number of investigation- and design-based research was relatively small.

Table 2 . Coding results for sample studies.

Looking more specifically at the research objects, the age range varied from 6 to 18 years. Overall, adolescents aged 13–18 years received more attention, while only six articles considered the younger age group aged 6–12 years. In addition, by coding the sample size of the studies, we found that the quality and size of the studies varied, ranging from small pilot studies or case studies to large-scale cluster studies. For example, Orlowski et al. (2016) conducted a qualitative study on adolescents with experience of seeking help in mental health care institutions in rural Australia; in their study, 10 adolescents with an average age of 18 years were recruited for semi-structured interviews to determine their attitudes and views on the use of technology as a mental health care tool. Another large-scale, randomized controlled trial is planned to enroll 10,000 eighth graders to investigate whether cognitive behavioral therapy (CBT) provided by a smartphone app can prevent depression ( Werner-Seidler et al., 2020 ).

2.2.2 Mental health issues and technology interventions

Based on the coding results, we present the total number of studies that correspond to both mental health issues and technological interventions in Figure 3 . Our findings indicate that apps represent the most prevalent form of digital technology, particularly in addressing depression and anxiety. Telemedicine services also rank highly in terms of utilization. Contrarily, there are comparatively fewer apps involving virtual reality (VR), augmented reality (AR), chatbots, and serious games. Below, we delve into the specifics of digital technology application and its unique affordances, tailored to distinct mental health issues.

Figure 3 . Numbers of studies by mental health issues and technology interventions.

2.2.2.1 Depression and anxiety

Depression and anxiety in adolescents have become increasingly common, and their presence may signal the beginning of long-term mental health issues, with approximately one in five people experiencing a depressive episode before the age of 18 years ( Lewinsohn et al., 1993 ). This has a range of adverse consequences, including social dysfunction, substance abuse, and suicidal tendency. From the 59 articles considered here, 29 studies used digital technology to treat depression- and anxiety-related symptoms in adolescents. Among the many types of digital technology considered, 19 studies used apps or educational websites as intervention tools, accounting for 76%, followed by serious games, chatbots, and VR with two articles each.

Apps are a broad concept, but they typically refer to software that can be downloaded from app stores to mobile devices such as phones or tablets. Due to characteristics such as their clear structure, ease of use, accessibility, strong privacy, interactivity, and multi-modularity, apps and educational websites are commonly used as tools for technological interventions. For example, Gladstone et al. (2015) developed an interactive website called CATCH-IT to prevent depression in adolescents; the site includes 14 optional modules. The course design of each module applies educational design theories, such as attracting learners’ attention, reviewing content, enhancing memory, and maintaining transfer. Apps and websites can also combine CBT with digital technology. The theoretical framework of CBT is rooted in a core assumption that depression is caused and maintained by unhelpful cognitions and behaviors. Treatment thus focuses on improving the function of these areas by applying skill-based behavioral strategies ( Wenzel, 2017 ). Multiple studies have incorporated CBT’s emphasis on reducing cognitive errors and strengthening positive behavior into their designs by, for example, using fictional storylines to help participants correct irrational thought patterns during reflective tasks, thereby improving patients’ depression conditions ( Stasiak et al., 2014 ; Topooco et al., 2019 ; Neumer et al., 2021 ).

In addition to the intervention methods involving apps and websites, serious games have also become a prospect for treating depression due to their interesting and interactive characteristics. Low-intensity human support combined with smartphone games may potentially reduce the resource requirements of traditional face-to-face counseling. Games contain complete storylines and competitive and cooperative tasks between peers in the form of levels that encourage adolescents to reflect on quizzes at the end of each challenge ( Gonsalves et al., 2019 ). Game designs tend to use flow theory, which emphasizes the dynamic matching of game challenges and the user’s own skill level ( Csikszentmihalyi, 2014 ). During game design, it is necessary to provide users with an easy-to-use and interesting gaming experience, as well as appropriate difficulty challenges, clear rules and goals, and instant feedback, which will help them relax and relieve stress, concentrate on changing cognitive processes, and improve their mood.

Two articles also consider the use of chatbots in interventions. Chatbots act as a dialog agent ( Mariamo et al., 2021 ), which makes the intervention process more interactive. Establishing a relationship of trust between adolescents and chatbots may also help lead to better results in depression and anxiety treatment. Chatbot functions are typically integrated into apps ( Werner-Seidler et al., 2020 ) and tend to be developed as part of the program rather than as a separate technological tool.

In recent years, with the gradual marketization of head-mounted VR devices, VR technology has been increasingly applied to mental health interventions. Studies have shown that the effectiveness of VR apps is often attributed to the distraction created by immersive environments, which produce an illusion of being in a virtual world, thus reducing users’ awareness of painful stimuli in the real world ( Ahmadpour et al., 2020 ). In the treatment of depression and anxiety for adolescents, active distraction supported by VR can engage users in games or cognitive tasks to redirect their attention to virtual objects and away from negative stimuli. Studies have also shown that, in addition to providing immersion, VR should create a pleasant emotional experience (e.g., the thrill of riding a roller coaster) and embed narrative stories (e.g., adventure and exploration) to meet adolescents’ need for achievement ( Ahmadpour et al., 2019 ).

2.2.2.2 Mental illness

In this study, we define mental illness as neurological developmental problems other than depression and anxiety. Among the 59 reviewed articles, 10 were coded as mental illness, including obsessive-compulsive disorder, attention-deficit/hyperactivity disorder, conduct disorder, oppositional defiant disorder, personality disorder, drug addiction, bipolar disorder, and non-suicidal self-injury. For the treatment of mental illness, mobile apps based on CBT appeared twice in 10 articles, while other technology types included SMS intervention, serious games, remote video conferencing, and mobile sensing technology.

Similar to apps for treating depression and anxiety, adolescent patients believe that the apps have good usability and ease of use and can encourage them to share their thoughts, feelings, and behavioral information more openly and honestly while protecting their privacy ( Adams et al., 2021 ). However, due to the severe condition of patients with mental illness, the apps not only are used independently by patients but also serves as a bridge between therapists and patients. Therapists can thus closely monitor treatment progress through behavioral records, which can provide direct feedback to both patients and therapists ( Babiano-Espinosa et al., 2021 ).

SMS interventions send specific content text messages to patients. As a longitudinal intervention method, it is convenient, easy to operate, and low cost. For example, Owens and Charles (2016) sent text messages to adolescents with non-suicidal self-injury behaviors in an attempt to reduce their self-mutilation behaviors. The ultimate effect seemed to be unsatisfactory, as interventions for adolescents with self-mutilation behaviors may be better applied in schools and adolescents’ service agencies, which can help them control their self-mutilation behaviors in the early stages and prevent such behaviors from escalating.

There are also studies that have designed six serious games based on CBT frameworks to treat typical developmental disorders in adolescents, including attention-deficit/hyperactivity disorder, conduct disorder, and oppositional defiant disorder ( Ong et al., 2019 ). In the safe environment provided by the game world, the research subjects shape the behavior of the characters in the context through rule learning and task repetition, which allows them to master emotional management strategies and problem-solving skills. In addition to interventions, digital technology can also be used to evaluate treatment effectiveness and the type of disease. Orr et al. (2023) used mobile sensing technology and digital phenotyping to quantify people’s behavioral data in real time, thereby allowing diagnosis and evaluation of diseases.

2.2.2.3 Bullying

Bullying generally includes traditional bullying and cyberbullying. Traditional bullying usually manifests as direct physical violence or threats of abuse against victims, as well as indirect methods such as spreading rumors and social exclusion. Cyberbullying is defined as intentional harm to others through computers, mobile phones, and other electronic devices. Data show that, as of 2021, the proportion of adolescents who have experienced cyberbullying in the United States may be as high as 45.5% ( Patchin, 2021 ), which indicates that it has become a serious social problem. Among the nine articles on the topic of bullying and cyberbullying, three used SMS intervention methods, and two used mobile apps; chatbots, technology-supported courses, and CBT-based telemedicine services were also used in the mental health treatment for patients who had been bullied and cyberbullied.

The SMS intervention for bullying implemented personalized customization, and the automatic SMS content can be customized based on the subjects’ previous questionnaire or completed self-report status ( Ranney et al., 2019 ). The subjects are required to rate their feelings at the end of the day and report whether they were bullied that day. The psychotherapist then made adjustments based on their actual situation, and if necessary, the psychotherapist would also contact specific subjects to provide offline psychological counseling services ( Ranney et al., 2019 ). In addition to having similar functions as the SMS intervention ( Kutok et al., 2021 ), mobile apps can provide opportunities for personalized learning, where a variety of learning methods can be applied (e.g., providing therapist guidance, conducting meetings, and conducting family practice activities) to promote the acquisition of mental health skills ( Davidson et al., 2019 ). Furthermore, for adolescents, touchscreen learning, interactive games, and video demonstrations can enhance their enthusiasm for participating in the treatment process.

Chatbots with specific names and images were also used to guide research subjects through a series of online tasks in the form of conversations, including watching videos involving bullying and cyberbullying among adolescents, provoking self-reflection through questions and suggestions, and providing constructive strategic advice ( Gabrielli et al., 2020 ). Digital technology-supported courses and CBT-based telemedicine services both make full use of the convenience of technology, effectively addressing the time- and location-based limitations of traditional face-to-face treatment. Digital courses can be implemented on a large scale in schools through teacher training, and compared with professional medical services, such courses have a wider target audience and can play a scientific and preventive role in bullying and cyberbullying. Telemedicine services refer to the use of remote communication technology to provide psychological services ( Joint Task Force for the Development of Telepsychology Guidelines for Psychologists, 2013 ). For families with severely troubled adolescents, telemedicine allows parents and children to meet together, increasing the flexibility of timing, and one-on-one video services can help to build a closer relationship between patients and therapists.

2.2.2.4 Lack of social emotional competence

In research, social emotional competence typically refers to the development of emotional intelligence in adolescents ( De la Barrera et al., 2021 ), which also includes personal abilities (self-awareness and self-management), interpersonal relationships (social awareness and interpersonal skills), and cognitive abilities (responsible decision-making) ( Collaborative for Academic, Social, and Emotional Learning, 2020 ). It is an important indicator for measuring the mental health level of adolescents. People with positive social emotional intelligence are less likely to experience mental health issues such as depression, anxiety, and behavioral disorders. Using digital technology to promote social emotional development is becoming increasingly common, and in six intervention studies on social emotional competence, apps, serious games, VR technology, and SMS interventions were used.

The studies considered all emphasized the importance of interactive design in digital technology to enhance social and emotional skills, as interactive technology can increase students’ engagement, resulting in positive learning experiences. For example, Cherewick et al. (2021) designed a smartphone app that can be embedded with multimedia learning materials, allowing adolescents to watch social and emotional skill–related learning videos autonomously and complete topic reflection activities with family/peers after school. The app also has rich teaching interaction functions, allowing teachers to evaluate and share course and learning materials, which can provide pleasant learning experiences to students while also improving the flexibility of teaching. In addition to teacher–student interaction, another paper also mentioned the importance of human–computer interaction for developing social emotional competence. The fun and interactivity of the app are the key to attracting adolescents to download and use it, and it can also have a positive effect on improving students’ self-management and decision-making skills ( Kenny et al., 2016 ).

Unlike the treatment of depression and anxiety, the application of VR in the cultivation of social emotional competence not only relies on its highly immersive characteristics but also emphasizes the positive effects of multi-sensory experiences on emotional regulation. By utilizing various sensor devices and visualization devices, adolescents are provided with ideal visual, auditory, and tactile guidance and regulation, which can enhance their emotional regulation abilities and relieve psychological stress ( Wu et al., 2022 ). Existing studies have integrated dance and music into virtual scenes ( Liu et al., 2021 ), using virtual harmonic music therapy to allow users to relax physically and mentally while enjoying music, thereby reducing stress and anxiety. VR technology is also highly adaptable and generalizable, which can help in building diverse scenes that meet the psychological expectations of patients based on the characteristics of the different treatment objects.

2.2.2.5 Mental health issues caused by the COVID-19 pandemic

The global outbreak of COVID-19 created severe challenges for the mental health of adolescents. Factors such as lack of social contact, lack of personal space at home, separation from parents and relatives, and concerns about academics and the future have exacerbated mental health risks, leading to increased loneliness, pain, social isolation, mental disorders, and symptoms of anxiety, depression, and stress. The reports from five studies indicated that the COVID-19 pandemic has exacerbated mental health issues in adolescents. During the pandemic, technology—which is not limited by time and space—became the preferred method of treatment. Apps, remote health services, and online training courses were used in research. The apps were resource-oriented and evidence-based interventions that allowed patients to interact with therapists through remote conferencing and encouraged patients to self-reflect and express themselves after the conference to improve their mental condition ( Gómez-Restrepo et al., 2022 ). Remote health services combined CBT and dialectical behavior therapy with professional counselors engaging in online communication with patients for several weeks. This was in line with research that indicates that the establishment of a positive relationship between therapists and patients is the foundation for obtaining good effect ( Zepeda et al., 2021 ).

2.2.2.6 Other mental health issues

In addition to the common mental health issues mentioned above, there were also interventions mentioned in the literature for improving body image anxiety, mental issues caused by hospitalization, and reading disabilities through digital technology means. Due to its high-immersion and simulation characteristics, VR technology was selected for improving mental health issues such as loneliness, disconnection from peers, and academic anxiety caused by hospitalization ( Thabrew et al., 2022 ). Immersive VR experience technology used 360° panoramic live broadcast and VR headphones to enable hospitalized adolescents to indirectly participate in social activities through cameras in school or home environments, as well as to contact peers and teachers through methods such as text messages; such interventions are conducive to improving social inclusion, social connectivity, and happiness. Furthermore, two studies mentioned body image anxiety, especially targeting female audiences, and the research integrated body image CBT techniques into serious games and chatbots ( Mariamo et al., 2021 ; Matheson et al., 2021 ), using interesting interactive exploration and free dialog forms to help adolescents gain a correct understanding of body image and solve body image anxiety issues.

Another study used eye-tracking technology to treat children with reading disabilities ( Davidson et al., 2019 ). The researcher developed a reading evaluation platform called Lexplore, which used eye-tracking technology to monitor children’s eye movements when reading to determine the cognitive processes behind each child’s individual reading style and then design appropriate strategies to improve their reading difficulties.

3 Study 2: meta-analysis

To explore the effect of digital technology in promoting mental health, this study used a meta-analysis to assess 10 papers. It includes both experimental and quasi-experimental research studies. CMA3.0 (Comprehensive Meta-Analysis 3.0) was used, and the meta-analysis process consisted of five phases.

Phase 1: Literature screening, based on the prior stage of literature information coding. Relevant literature was filtered using the following criteria for meta-analysis: (a) the study must compare “technical intervention” and “traditional intervention”; (b) the study should report complete data that can generate the effect amount (e.g., average, sample size, standard deviation or t -values, p -values, etc.); and (c) the dependent variables in the study should contain at least one aspect of mental health.

Phase 2: Effect size calculation. In the case of a large sample size, there is little difference between Cohen’s d, Glass, and Hedges’ g values, but Cohen’s d can significantly overestimate the effect size for studies with a small sample ( Hedges, 1981 ). Therefore, Hedges’ g was used as the effect size indicator in this study.

Phase 3: Model selection. Meta-analyses include fixed- and random-effects models. Different models may produce different effect sizes. Due to the differences in sample size, experimental procedures, and methods among the initial studies included in the meta-analysis, the estimated average effect values may not be completely consistent with the true population effect values, which results in sample heterogeneity. This study used the method proposed by Borenstein et al. (2009) to establish fixed- and random-effects models to eliminate the influence of sample heterogeneity. When the heterogeneity test ( Q value) results were significant, the random-effects model was used; otherwise, the fixed-effects model was used.

Phase 4: Testing of main effects and moderating effects. Based on the selected model, a test of the main effects was conducted. Meanwhile, if heterogeneity was present, a test of moderating effects could be conducted.

Phase 5: Publication bias test. Publication bias is a common systematic error in meta-analyses and refers to a tendency for significantly significant research results to be more likely to be published than non-significant results. This study used a funnel plot to visually assess publication bias qualitatively and then further quantitatively assessed publication bias using Begg’s rank correlation method and the trim and fill method.

3.2 Results and discussion

3.2.1 inclusion and coding results.

For the studies that met the requirements of the meta-analysis, detailed classification was carried out based on the following variables one-by-one on the basis of the systematic review coding: (a) basic information (authors, year, sample size); (b) age stage, which is divided into three categories: primary school, junior high school, and senior high school; (c) mental health issues, including depression, bullying, and mental health issues caused by COVID-19; (d) technology type, including app, telemedicine, and chatbots; (e) intervention duration, coded as short-term for interventions less than a month and long-term for intervention that lasted more than a month; and (f) effect size. The coding results are shown in Table 3 .

Table 3 . Research coding results included in meta-analysis.

3.2.2 The overall effect of digital technology on mental health outcomes

According to the results of the heterogeneity test in Table 4 , the Q test is significant ( p  < 0.001), which indicates that there is significant heterogeneity among the samples. The random-effects model was therefore selected as the more reasonable option. The pooled effect size is 0.43. According to the criteria proposed by Cohen (1992) , 0.2, 0.5, and 0.8 are considered the boundaries of small, medium, and large effect sizes, respectively. It can be seen that the effect size for the promotion of mental health by digital technology is moderate and significant. At the same time, the lower limit of the 95% confidence interval is greater than 0 for each study, which indicates that the probability of the effect size being caused by chance is very small. In addition, the I 2 value is 78.164, which indicates that the heterogeneity between studies is high. Important moderating variables therefore may exist ( Higgins and Green, 2008 ), and additional moderating effect tests need to be conducted.

Table 4 . Overall effect of technology on mental health.

3.2.3 Moderating effect test

Moderating effect tests were conducted on four variables: age stage, mental health issues, technology type, and intervention duration. As shown in Table 5 , among the four moderating variables, only the age stage has a significant moderating effect ( p  < 0.05). In particular, the effect size is the largest for the primary school stage, followed by the senior high school stage with a moderate promoting effect. In addition, although the effect size for the junior high school stage is small, it is still significant, which may be related to the limited number of studies considering this population. The results also indicate that the moderating effects of mental health issues, technology type, and intervention duration are not significant. However, it can be seen that digital technology methods have the largest effect size for treating psychological problems caused by COVID-19, while compared with apps and chatbots, remote medical services can achieve better effects. In terms of treatment duration, the effect size for short-term interventions is greater than that for long-term interventions.

Table 5 . Regulatory effect test of technology (random-effect model).

3.2.4 Publication bias test

This study used funnel plots, Begg’s test, and the trim and fill method for the publication bias test. As shown in Figure 4 , the distribution of effect values in the study shows uneven and asymmetric distribution on both sides of the mean effect value, which initially suggests the possibility of publication bias. Begg’s test was thus used for further testing. Begg’s test is a method of quantitatively identifying bias using a rank correlation test, and it applies to studies with a small sample. The result of Begg’s test shows that t  = 0.267, p  = 0.283, Z  = 1.01 < 1.96, which indicates that there is no obvious publication bias. Finally, the censoring method was used to censor the literature on both sides of the effect value, and this revealed that the effect value was still significant. In summary, there is negligible publication bias.

Figure 4 . Distributions of effect sizes for mental health treatment outcomes.

4 Conclusion and implications

4.1 summary of key findings.

This study made a systematic review and meta-analysis of 59 studies on digital technology promoting adolescents’ mental health over the past decade. Based on the investigation of current research, the types and characteristics of the commonly used technology interventions for different mental health issues were analyzed, and the actual effects and potential regulatory variables of digital technology in promoting mental health were investigated in the meta-analysis. The main findings are outlined below.

• Over the past decade, especially between 2013 and 2021, the number of studies on digital technology promoting adolescents’ mental health has generally shown an upward trend, with nearly 80% of the literature being published in medical journals.

• Digital technology is most commonly used to intervene in the mental health issues of adolescents aged 13–18 years, and children in the younger age group (6–12 years old) receive relatively less attention.

• Depression and anxiety disorders are the mental health issues that received the most research attention, followed by obsessive-compulsive disorder, attention-deficit hyperactivity disorder, conduct disorder, and other mental illnesses. There were also studies on, in decreasing order of the number of studies, bullying, social emotional competence deficiency, and mental health issues caused by COVID-19, dyslexia, and adolescent body image anxiety.

• Apps with convenience, ease of use, interactivity, and remote communication were most commonly used to treat mental health issues. Serious games, remote health services, and text message intervention were less often used, and only three studies used VR, which is difficult to realize for mental health treatment.

• Digital technology plays a significant role in promoting the treatment of mental health issues of adolescents, especially in primary and senior high school.

4.2 Interpretation and insights

The findings of this study highlight the nuanced role played by digital technology in promoting mental health for children and adolescents. While technology has broadened the scope of mental health interventions with innovative apps and programs, it should be viewed as a complement to traditional face-to-face approaches, not a replacement ( Aguilera, 2015 ), as they cannot replicate the personal connection and empathy provided by a trained mental health professional. Moreover, different technologies vary in effectiveness for specific mental health issues, emphasizing the need for careful evaluation of their benefits and limitations. For instance, virtual reality, cognitive behavioral therapy apps, and online support platforms have shown promise for in addressing depression and anxiety, but their effects vary depending on individual needs and contexts, suggesting the non-uniform efficacy of digital technologies across mental health conditions.

Furthermore, this study also draws attention to the limited incorporation of digital technology in mental health education, especially among children aged 6 to 12. Given the significance of this developmental stage, where emotional management, relationships, and mental health knowledge are crucial, innovative digital approaches that draw upon the unique affordances of mobile apps, online courses, and virtual reality are warranted to deliver interactive and personalized learning experiences. Nevertheless, this innovation poses challenges and risks, including addiction to virtual environments and a reduction in social activities, which can also negatively impact the mental health of youth ( Taylor et al., 2020 ). Therefore, striking a balance between harnessing technology’s potential and mitigating its risks is essential, emphasizing the need for responsible and targeted use of digital tools in mental healthcare and education.

4.3 Implication for practice and future research

Based on the results of the systematic review and meta-analysis, this study puts forward the relevant implications for practice and research. First, for mental health education service personnel, we suggest that the first step is to fully utilize the characteristics of digital technology and select the most appropriate digital intervention tools for different mental health issues. For example, apps are more suitable for the treatment of depression, anxiety, and mental illnesses. When facing adolescents who have been bullied, text message interventions may be a good choice. In addition, serious games and VR could play a greater role in developing adolescents’ social emotional competence.

Second, for mental health counselors or school mental health workers, it is necessary to consider learner characteristics and intervention duration, among other factors. In contrast to previous research results ( Tang et al., 2019 ), we found that the regulating effect of age was significant, so therapists need to implement personalized technical interventions for adolescents at different age stages. Short-term interventions seem to induce a greater effect size, so lengthy interventions should be avoided, as they are more likely to cause marginal effect and develop technical immunity for the youth population.

Third, for technology intervention developers, it is important to recognize that not all practitioners (e.g., psychologists, therapists) are technology savvy. In the process of designing mental health apps and VR interventions, it is necessary to provide sufficient technical support, such as instructional manuals and tutorial videos, to reduce the potential digital divide. It is also essential to arrange for appropriate technical personnel to provide safeguard services and training continuously, ensuring the personal safety and cybersecurity of practitioners and patients during intervention sessions.

For researchers, we suggest that, first, more empirical studies are needed to report first-hand experimental results. Most of the existing studies only described the experimental scheme and lacked key research results. It is hoped that future research will report the results as comprehensively as possible to improve the credibility and reliability of meta-analytical results. Second, the number of studies on moderating effects in the meta-analysis was relatively small. For example, there was only one study on the primary school population. Future research needs to focus on people who have paid less attention to existing studies and thus enhance the understanding of technology interventions in mental health. Finally, there have been few studies that analyze cost-effectiveness, which is key to determining whether technical interventions can be normalized and sustainable. Future studies need to conduct sufficient investigation and report on the cost-effectiveness of digital technology interventions, including the development and maintenance costs of VR ( Kraft, 2020 ).

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

TC: Data curation, Formal analysis, Investigation, Visualization, Writing – original draft. JO: Formal analysis, Investigation, Writing – original draft. GL: Formal analysis, Writing – review & editing. HL: Conceptualization, Methodology, Supervision, Writing – review & editing.

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Adams, Z., Grant, M., Hupp, S., Scott, T., Feagans, A., Phillips, M. L., et al. (2021). Acceptability of an mHealth app for youth with substance use and mental health needs: iterative, mixed methods design. JMIR Form. Res. 5:e30268. doi: 10.2196/30268

PubMed Abstract | Crossref Full Text | Google Scholar

Aguilera, A. (2015). Digital technology and mental health interventions: opportunities and challenges. Arbor 191:a210. doi: 10.3989/arbor.2015.771n1012

Crossref Full Text | Google Scholar

Ahmadpour, N., Randall, H., Choksi, H., Gao, A., Vaughan, C., and Poronnik, P. (2019). Virtual reality interventions for acute and chronic pain management. Int. J. Biochem. Cell Biol. 114:105568. doi: 10.1016/j.biocel.2019.105568

Ahmadpour, N., Weatherall, A. D., Menezes, M., Yoo, S., Hong, H., and Wong, G. (2020). Synthesizing multiple stakeholder perspectives on using virtual reality to improve the periprocedural experience in children and adolescents: survey study. J. Med. Internet Res. 22:e19752. doi: 10.2196/19752

Antaramian, S. P., Huebner, E. S., Hills, K. J., and Valois, R. F. (2010). A dual-factor model of mental health: toward a more comprehensive understanding of youth functioning. Am. J. Orthopsychiatry 80, 462–472. doi: 10.1111/j.1939-0025.2010.01049.x

Aschbrenner, K. A., Naslund, J. A., Tomlinson, E. F., Kinney, A., Pratt, S. I., and Brunette, M. F. (2019). Adolescents' use of digital technologies and preferences for mobile health coaching in public mental health settings. Front. Public Health 7:178. doi: 10.3389/fpubh.2019.00178

Babiano-Espinosa, L., Wolters, L. H., Weidle, B., Compton, S. N., Lydersen, S., and Skokauskas, N. (2021). Acceptability and feasibility of enhanced cognitive behavioral therapy (eCBT) for children and adolescents with obsessive-compulsive disorder. Child Adolesc. Psychiatry Ment. Health 15:47. doi: 10.1186/s13034-021-00400-7

Bearman, M., Smith, C. D., Carbone, A., Slade, S., Baik, C., Hughes-Warrington, M., et al. (2012). Systematic review methodology in higher education. High. Educ. Res. Dev. 31, 625–640. doi: 10.1080/07294360.2012.702735

Borenstein, M., Hedges, L. V., Higgins, J. P. T., and Rothstein, H. R. (2009). Introduction to meta-analysis Wiley.

Google Scholar

Cejudo, J., López-Delgado, M. L., and Losada, L. (2019). Effectiveness of the videogame “Spock” for the improvement of the emotional intelligence on psychosocial adjustment in adolescents. Comput. Hum. Behav. 101, 380–386. doi: 10.1016/j.chb.2018.09.028

Cheng, V. W. S., Davenport, T., Johnson, D., Vella, K., and Hickie, I. B. (2019). Gamification in apps and technologies for improving mental health and well-being: systematic review. JMIR Mental Health 6:e13717. doi: 10.2196/13717

Cherewick, M., Lebu, S., Su, C., Richards, L., Njau, P. F., and Dahl, R. E. (2021). Study protocol of a distance learning intervention to support social emotional learning and identity development for adolescents using interactive mobile technology. Front. Public Health 9:623283. doi: 10.3389/fpubh.2021.623283

Cohen, J. (1992). A power primer. Psychol. Bull. 112, 155–159. doi: 10.1037/0033-2909.112.1.155

Collaborative for Academic, Social, and Emotional Learning (2020), What is the CASEL framework? A framework creates a foundation for applying evidence-based SEL strategies to your community. Available at: https://casel.org/fundamentals-of-sel/what-is-the-casel-framework/

Csikszentmihalyi, M. (2014). “Learning, “flow,” and happiness” in Applications of flow in human development and education (Dordrecht: Springer), 153–172.

Davidson, T. M., Bunnell, B. E., Saunders, B. E., Hanson, R. F., Danielson, C. K., Cook, D., et al. (2019). Pilot evaluation of a tablet-based application to improve quality of care in child mental health treatment. Behav. Ther. 50, 367–379. doi: 10.1016/j.beth.2018.07.005

De la Barrera, U., Postigo-Zegarra, S., Mónaco, E., Gil-Gómez, J.-A., and Montoya-Castilla, I. (2021). Serious game to promote socioemotional learning and mental health (emoTIC): a study protocol for randomised controlled trial. BMJ Open 11:e052491. doi: 10.1136/bmjopen-2021-052491

Eisenstadt, M., Liverpool, S., Infanti, E., Ciuvat, R. M., and Carlsson, C. (2021). Mobile apps that promote emotion regulation, positive mental health, and well-being in the general population: systematic review and meta-analysis. JMIR Mental Health 8:e31170. doi: 10.2196/31170

Essau, C. A., and de la Torre-Luque, A. (2023). Comorbidity between internalising and externalising disorders among adolescents: symptom connectivity features and psychosocial outcome. Child Psychiatry Hum. Dev. 54, 493–507. doi: 10.1007/s10578-021-01264-w

Folker, A. P., Mathiasen, K., Lauridsen, S. M., Stenderup, E., Dozeman, E., and Folker, M. P. (2018). Implementing internet-delivered cognitive behavior therapy for common mental health disorders: a comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv. 11, 60–70. doi: 10.1016/j.invent.2018.02.001

Gabrielli, S., Rizzi, S., Carbone, S., and Donisi, V. (2020). A chatbot-based coaching intervention for adolescents to promote life skills: pilot study. JMIR Hum. Factors 7:e16762. doi: 10.2196/16762

Giovanelli, A., Ozer, E. M., and Dahl, R. E. (2020). Leveraging technology to improve health in adolescence: a developmental science perspective. J. Adolesc. Health 67, S7–S13. doi: 10.1016/j.jadohealth.2020.02.020

Gladstone, T. G., Marko-Holguin, M., Rothberg, P., Nidetz, J., Diehl, A., DeFrino, D. T., et al. (2015). An internet-based adolescent depression preventive intervention: study protocol for a randomized control trial. Trials 16:203. doi: 10.1186/s13063-015-0705-2

Gómez-Restrepo, C., Sarmiento-Suárez, M. J., Alba-Saavedra, M., Bird, V. J., Priebe, S., and van Loggerenberg, F. (2022). Adapting DIALOG+ in a school setting-a tool to support well-being and resilience in adolescents living in postconflict areas during the COVID-19 pandemic: protocol for a cluster randomized exploratory study. JMIR Res. Protoc. 11:e40286. doi: 10.2196/40286

Gonsalves, P. P., Hodgson, E. S., Kumar, A., Aurora, T., Chandak, Y., Sharma, R., et al. (2019). Design and development of the "POD adventures" smartphone game: a blended problem-solving intervention for adolescent mental health in India. Front. Public Health 7:238. doi: 10.3389/fpubh.2019.00238

Goodyear, V. A., and Armour, K. M. (2018). Young people’s perspectives on and experiences of health-related social media, apps, and wearable health devices. Soc. Sci. 7:137. doi: 10.3390/socsci7080137

Grist, R., Croker, A., Denne, M., and Stallard, P. (2019). Technology delivered interventions for depression and anxiety in children and adolescents: a systematic review and meta-analysis. Clin. Child. Fam. Psychol. Rev. 22, 147–171. doi: 10.1007/s10567-018-0271-8

Hedges, L. V. (1981). Distribution theory for glass's estimator of effect size and related estimators. J. Educ. Stat. 6, 107–128. doi: 10.3102/10769986006002107

Higgins, J. P., and Green, S. (2008). Cochrane handbook for systematic reviews of interventions: Cochrane book series . Hoboken, New Jersey: Wiley.

Hollis, C., Falconer, C. J., Martin, J. L., Whittington, C., Stockton, S., Glazebrook, C., et al. (2017). Annual research review: digital health interventions for children and young people with mental health problems – a systematic and meta-review. J. Child Psychol. Psychiatry 58, 474–503. doi: 10.1111/jcpp.12663

Joint Task Force for the Development of Telepsychology Guidelines for Psychologists (2013). Guidelines for the practice of telepsychology. Am. Psychol. 68, 791–800. doi: 10.1037/a0035001

Jones, E. A. K., Mitra, A. K., and Bhuiyan, A. R. (2021). Impact of COVID-19 on mental health in adolescents: a systematic review. Int. J. Environ. Res. Public Health 18:2470. doi: 10.3390/ijerph18052470

Kenny, R., Dooley, B., and Fitzgerald, A. (2016). Developing mental health mobile apps: exploring adolescents’ perspectives. Health Informatics J. 22, 265–275. doi: 10.1177/1460458214555041

Keyes, C. L. M. (2009). “Toward a science of mental health” in The Oxford handbook of positive psychology (Oxford: Oxford University Press), 88–96.

Kraft, M. A. (2020). Interpreting effect sizes of education interventions. Educ. Res. 49, 241–253. doi: 10.3102/0013189x20912798

Kutok, E. R., Dunsiger, S., Patena, J. V., Nugent, N. R., Riese, A., Rosen, R. K., et al. (2021). A cyberbullying media-based prevention intervention for adolescents on instagram: pilot randomized controlled trial. JMIR Mental Health 8:e26029. doi: 10.2196/26029

Lacey, F. M., and Matheson, L. (2011). Doing your literature review: Traditional and systematic techniques . London: Sage.

Lewinsohn, P. M., Hops, H., Roberts, R. E., Seeley, J. R., and Andrews, J. A. (1993). Adolescent psychopathology: I. Prevalence and incidence of depression and other DSM-III—R disorders in high school students. J. Abnorm. Psychol. 102, 133–144. doi: 10.1037/0021-843x.102.1.133

Liu, T.-C., Lin, Y.-C., Wang, T.-N., Yeh, S.-C., and Kalyuga, S. (2021). Studying the effect of redundancy in a virtual reality classroom. Educ. Technol. Res. Dev. 69, 1183–1200. doi: 10.1007/s11423-021-09991-6

Mariamo, A., Temcheff, C. E., Léger, P.-M., Senecal, S., and Lau, M. A. (2021). Emotional reactions and likelihood of response to questions designed for a mental health chatbot among adolescents: experimental study. JMIR Hum. Factors 8:e24343. doi: 10.2196/24343

Matheson, E. L., Smith, H. G., Amaral, A. C. S., Meireles, J. F. F., Almeida, M. C., Mora, G., et al. (2021). Improving body image at scale among Brazilian adolescents: study protocol for the co-creation and randomised trial evaluation of a chatbot intervention. BMC Public Health 21:2135. doi: 10.1186/s12889-021-12129-1

Mohr, D. C., Riper, H., and Schueller, S. M. (2018). A solution-focused research approach to achieve an implementable revolution in digital mental health. JAMA Psychiatry 75, 113–114. doi: 10.1001/jamapsychiatry.2017.3838

Naslund, J. A., Aschbrenner, K. A., Kim, S. J., McHugo, G. J., Unützer, J., Bartels, S. J., et al. (2017). Health behavior models for informing digital technology interventions for individuals with mental illness. Psychiatr. Rehabil. J. 40, 325–335. doi: 10.1037/prj0000246

Neumer, S.-P., Patras, J., Holen, S., Lisøy, C., Askeland, A. L., Haug, I. M., et al. (2021). Study protocol of a factorial trial ECHO: optimizing a group-based school intervention for children with emotional problems. BMC Psychol. 9:97. doi: 10.1186/s40359-021-00581-y

Ong, J. G., Lim-Ashworth, N. S., Ooi, Y. P., Boon, J. S., Ang, R. P., Goh, D. H., et al. (2019). An interactive mobile app game to address aggression (regnatales): pilot quantitative study. JMIR Ser Games 7:e13242. doi: 10.2196/13242

Orlowski, S., Lawn, S., Antezana, G., Venning, A., Winsall, M., Bidargaddi, N., et al. (2016). A rural youth consumer perspective of technology to enhance face-to-face mental health services. J. Child Fam. Stud. 25, 3066–3075. doi: 10.1007/s10826-016-0472-z

Orr, M., MacLeod, L., Bagnell, A., McGrath, P., Wozney, L., and Meier, S. (2023). The comfort of adolescent patients and their parents with mobile sensing and digital phenotyping. Comput. Hum. Behav. 140:107603. doi: 10.1016/j.chb.2022.107603

Owens, C., and Charles, N. (2016). Implementation of a text-messaging intervention for adolescents who self-harm (TeenTEXT): a feasibility study using normalisation process theory. Child Adolesc. Psychiatry Ment. Health 10:14. doi: 10.1186/s13034-016-0101-z

Patchin, J. W. (2021). 2021 Cyberbullying Data. Available at: https://cyberbullying.org/2021-cyberbullying-data

Piers, R., Williams, J. M., and Sharpe, H. (2023). Review: can digital mental health interventions bridge the ‘digital divide’ for socioeconomically and digitally marginalised youth? A systematic review. Child Adolesc. Ment. Health 28, 90–104. doi: 10.1111/camh.12620

Ranney, M. L., Patena, J. V., Dunsiger, S., Spirito, A., Cunningham, R. M., Boyer, E., et al. (2019). A technology-augmented intervention to prevent peer violence and depressive symptoms among at-risk emergency department adolescents: protocol for a randomized control trial. Contemp. Clin. Trials 82, 106–114. doi: 10.1016/j.cct.2019.05.009

Scott, L. N., Victor, S. E., Kaufman, E. A., Beeney, J. E., Byrd, A. L., Vine, V., et al. (2020). Affective dynamics across internalizing and externalizing dimensions of psychopathology. Clin. Psychol. Sci. 8, 412–427. doi: 10.1177/2167702619898802

Shah, S. M. A., Mohammad, D., Qureshi, M. F. H., Abbas, M. Z., and Aleem, S. (2021). Prevalence, psychological responses and associated correlates of depression, anxiety and stress in a global population, during the coronavirus disease (COVID-19) pandemic. Community Ment. Health J. 57, 101–110. doi: 10.1007/s10597-020-00728-y

Stasiak, K., Hatcher, S., Frampton, C., and Merry, S. N. (2014). A pilot double blind randomized placebo controlled trial of a prototype computer-based cognitive behavioural therapy program for adolescents with symptoms of depression. Behav. Cogn. Psychother. 42, 385–401. doi: 10.1017/s1352465812001087

Suldo, S. M., and Shaffer, E. J. (2008). Looking beyond psychopathology: the dual-factor model of mental health in youth. School Psychol. Rev. 37, 52–68. doi: 10.1080/02796015.2008.12087908

Tang, X., Tang, S., Ren, Z., and Wong, D. F. K. (2019). Prevalence of depressive symptoms among adolescents in secondary school in mainland China: a systematic review and meta-analysis. J. Affect. Disord. 245, 498–507. doi: 10.1016/j.jad.2018.11.043

Taylor, C. B., Ruzek, J. I., Fitzsimmons-Craft, E. E., Sadeh-Sharvit, S., Topooco, N., Weissman, R. S., et al. (2020). Using digital technology to reduce the prevalence of mental health disorders in populations: time for a new approach. J. Med. Internet Res. 22:e17493. doi: 10.1177/2167702619859336

Thabrew, H., Chubb, L. A., Kumar, H., and Fouché, C. (2022). Immersive reality experience technology for reducing social isolation and improving social connectedness and well-being of children and young people who are hospitalized: open trial. JMIR Pediatr. Parent. 5:e29164. doi: 10.2196/29164

Topooco, N., Byléhn, S., Dahlström Nysäter, E., Holmlund, J., Lindegaard, J., Johansson, S., et al. (2019). Evaluating the efficacy of internet-delivered cognitive behavioral therapy blended with synchronous chat sessions to treat adolescent depression: randomized controlled trial. J. Med. Internet Res. 21:e13393. doi: 10.2196/13393

Uhlhaas, P., and Torous, J. (2019). Digital tools for youth mental health. NPJ Digit. Med. 2:104. doi: 10.1038/s41746-019-0181-2

Villarreal, V. (2018). Mental health collaboration: a survey of practicing school psychologists. J. Appl. Sch. Psychol. 34, 1–17. doi: 10.1080/15377903.2017.1328626

Wacks, Y., and Weinstein, A. M. (2021). Excessive smartphone use is associated with health problems in adolescents and young adults. Front. Psych. 12:669042. doi: 10.3389/fpsyt.2021.669042

Wenzel, A. (2017). Basic strategies of cognitive behavioral therapy. Psychiatr. Clin. North Am. 40, 597–609. doi: 10.1016/j.psc.2017.07.001

Werner-Seidler, A., Huckvale, K., Larsen, M. E., Calear, A. L., Maston, K., Johnston, L., et al. (2020). A trial protocol for the effectiveness of digital interventions for preventing depression in adolescents: the future proofing study. Trials 21:2. doi: 10.1186/s13063-019-3901-7

World Health Organization (2021). Mental health of adolescents. Available at: https://www.who.int/news-room/fact-sheets/detail/adolescent-mental-health (Accessed December 14, 2023).

Wu, B., Zheng, C., and Huang, B. (2022). Influence of science education on mental health of adolescents based on virtual reality. Front. Psychol. 13:895196. doi: 10.3389/fpsyg.2022.895196

Zepeda, M., Deighton, S., Markova, V., Madsen, J., and Racine, N. (2021). iCOPE with COVID-19: A brief telemental health intervention for children and adolescents during the COVID-19 pandemic. PsyArXiv . doi: 10.31234/osf.io/jk32s

Keywords: children and adolescents, digital technology, systematic literature review, meta-analysis, mental health issues

Citation: Chen T, Ou J, Li G and Luo H (2024) Promoting mental health in children and adolescents through digital technology: a systematic review and meta-analysis. Front. Psychol . 15:1356554. doi: 10.3389/fpsyg.2024.1356554

Received: 15 December 2023; Accepted: 29 February 2024; Published: 12 March 2024.

Reviewed by:

Copyright © 2024 Chen, Ou, Li and Luo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Heng Luo, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Burt J, Campbell J, Abel G, et al. Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience. Southampton (UK): NIHR Journals Library; 2017 Apr. (Programme Grants for Applied Research, No. 5.9.)

Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience.

Chapter 12 conclusions, implications for practice and recommendations for future research.

• Conclusions

In Chapter 1 we outlined how, following the introduction of a wide range of quality improvement strategies as part of an overarching ‘clinical governance’ strategy in the late 1990s, there had been step changes in the management of major chronic diseases in the NHS. However, the ways in which patients experienced health care had not been given such a priority and the need for a rebalancing was seen by increasing attention to patient experience in policy documents, the routine publication of patient experience data, benchmarking of hospitals in relation to patient experience and even an (ill-fated) attempt to attach payments to patients’ assessments of their GP ’s care.

There has therefore been widespread acceptance that good patient experience is an important outcome of care in its own right and our work 304 and that of others 305 has shown that patient experience is a domain of quality that is distinct from, but complementary to, the quality of clinical care. Although an increasing number of surveys have been developed to measure patient experience, there has been equally widespread acceptance that these measures have not been very effective at actually improving care. 45 This is the background to our programme of work. Entitled IMPROVE , we aimed to find better ways of both measuring and using information on patient experience that would lead to improvements in patient care in both in-hours and out-of-hours primary care settings.

In the introduction, we described a range of ways of obtaining patient feedback on their care, including surveys, focus groups and analysis of complaints. In this programme, we have focused on the use of patient surveys as they are the dominant method currently used in the UK. However, in Chapter 10 we describe an exploratory trial of RTF , which moves away from the paper-based questionnaires that still dominate the measurement of patient experience in the NHS.

This programme had seven aims, each of which was tied closely to one work package of research. These aims were to:

• understand how general practices respond to low patient survey scores, testing a range of approaches that could be used to improve patients’ experience of care
• estimate the extent to which aggregation of scores to practice level in the national study masks differences between individual doctors
• investigate how patients’ ratings on questions in the GP Patient Survey relate to actual behaviour by GPs in consultations
• understand better patients’ responses to questions on communication and seeing a doctor of their choice
• understand the reasons why minority ethnic groups, especially South Asian populations, give lower scores on patient surveys than the white British population
• carry out an exploratory RCT of an intervention to improve patient experience, using tools developed in earlier parts of the programme
• investigate how the results of the GP Patient Survey can be used to improve patients’ experience of out-of-hours care.

The aims of the programme did not change during the 5 years of our research, although some details of the research were modified as the work progressed (we have summarised any changes in each individual chapter). We presented the results of our research under three broad headings and also use these headings in this conclusions chapter, namely:

• understanding patient experience data (aims 3 and 4)
• understanding patient experience in minority ethnic groups (aim 5)
• using data on patient experience for quality improvement (aims 1, 2, 6 and 7).

Understanding patient experience data

Patient surveys are now widely used in many countries, yet still comparatively little is known about what experiences lead patients to respond in particular ways in these surveys. What drives them to tick particular boxes and how do those responses relate to the care that they have actually received? We approached this in two main studies, one in which we asked patients directly about how they chose certain items on the questionnaire while showing them a video of their consultation (see Chapter 2 ) and one in which we compared their responses with those of expert raters using two standard instruments for assessing videos of consultations (see Chapter 3 ). The results of these studies have important implications for the interpretation of survey data, particularly data focused on patient evaluations of specific encounters with health-care professionals.

The first study (see Chapter 2 ) showed that, although patients readily criticised their care when reviewing GP consultations on video, they had been reluctant to be critical when completing a questionnaire after the consultation. Reasons for this included the need to maintain a relationship with the GP (including uncertainty about how confidential survey results would be) and their gratitude for the care that they had received from the NHS in the past. In addition, perceived power asymmetries made people reluctant to criticise their doctor. Patients were also disinclined to be critical when completing a questionnaire if they had actually received the treatment that they wanted. Overall, we concluded that patients find that questionnaires administered at the point of care may be limited tools for being able to feed back concerns about primary care consultations.

The second study (see Chapter 3 ) reinforced our conclusion from Chapter 2 that patient evaluations of consultations in surveys may present an uncritical view of the actual consultations. In this study videotapes of GP –patient consultations were assessed by four independent clinical raters. The results were striking. When trained raters rated communication within a consultation to be of a high standard, patients did the same (with one single exception). However, when trained raters judged the communication during a consultation to be of a poor standard, patients’ assessments varied from poor to very good. This finding again points to the reluctance of patients to criticise their doctor in questionnaire surveys. In the previous study the ‘gold standard’ was the patient’s own account of the consultation and in this study the standard was that of a trained external GP rater.

We do not think that these results mean that patient surveys cannot be used to assess the quality of general practice care. However, they do point to clear limitations. One of the concerns that GPs have about surveys (see Chapters 7 and 8 ) is that they are selectively completed by critical or grumpy patients and that survey results will therefore give a negative and biased view of their care. The results of the two studies described here suggest that the opposite is the case. Patients’ reluctance to criticise their doctor means that survey responses using evaluative type of questions are likely to give an overly positive view of their doctor’s care. This is one reason why there has been a move towards using report items in some survey instruments (though we do not know whether or not these suffer from similar problems). Because of this tendency for patients to choose the most positive response options, we suggest that absolute scores should be treated with some caution, as they may present an overly optimistic view of their care. However, this does not mean that surveys cannot be used to look at relative scores: scores from a GP that are lower than those of his or her colleagues and from GPs in other practices are likely to indicate a problem, even though high scores from other doctors or practices may conceal deficiencies in care in those practices too.

We also looked at how GPs rated their own consultations. GPs completed a form immediately after each consultation, using the same scale as the patients. GPs were certainly more inclined to criticise themselves than the patients were to criticise the care that they had received. This is entirely consistent with the findings from our subsequent interviews with patients. However, we found absolutely no correlation between patient scores and GP scores. Neither did we find any correlation between GPs’ own scores and those of expert raters who reviewed the consultations on video. GPs are clearly using different parameters when assessing their own performance, but we were not able to investigate this in more detail in this study.

When we spoke to GPs about their survey results (see Chapters 7 and 8 ), through both focus groups and face-to-face interviews, they reported how, although positive about the concept of patient feedback, they struggled to engage with and make changes under the current approaches to measurement. They also commonly expressed concern that patients would be critical of their care if they did not get what they wanted (e.g. an antibiotic prescription). This concern was borne out to some extent by our results. In our analysis of the assessment of nurses (see Chapter 4 ), a strong predictor of survey scores was whether or not patients wanted to see a nurse when they first contacted the practice. If they had wanted to see a GP but saw a nurse, the scores given to those nurses were much lower. We have no reason to think that the nurses’ communication was worse in those consultations and the low scores may therefore indicate a more general dissatisfaction of patients because of not having their original expectations met.

It is important to understand that, in line with the overall aims of the programme, the work in these two chapters focused on the assessment of communication in the primary care consultation (such as giving the patient enough time and explaining tests and treatments). Our conclusion that survey scores have more value in assessing relative performance than absolute performance of doctors may or may not hold true for other aspects of practice performance commonly assessed in surveys, such as difficulty in getting appointments, getting through on the telephone and waiting times. Patients’ reasons for not wanting to criticise their doctor may be less important when they assess what they regard as management aspects of the practice.

A second aspect of care that we identified as part of our programme of work relates to patients’ ability to see a GP of their choice. Although most of our research focused on communication, the results that we report in Chapter 4 have some important findings in relation to patient choice. 142 The results show that most patients have a particular GP who they prefer to see. It is sometimes suggested that this matters only for some population groups (e.g. not young people) but we found that this is not the case. Even among those aged 18–24 years, > 50% of respondents to the GP Patient Survey have a particular doctor who they prefer to see, rising to > 80% in those aged > 75 years. Disturbingly, a large percentage of people who have such a preference are unable to see the doctor of their choice. This percentage has risen from 30% to 40% from 2010 to 2015. One possible impact of this change comes from our analysis of data from patients who saw a nurse when they had originally wanted to see a doctor; they expressed considerable dissatisfaction with their subsequent consultation with the nurse. However, these data do not reflect what would have happened if patients had seen another doctor, just not the one of their choice.

Overall, patients express more negative opinions about choice of doctor than in any other part of the GP Patient Survey, something that may in part have got worse as a result of government policies to improve access. There is a clear tension between the ability of practices to provide rapid access and the ability of practices to provide continuity of care and data from our studies suggest that patients’ inability to see a doctor of their choice is a significant quality issue for the NHS.

Understanding patient experience in minority ethnic groups

In this part of our research, we focused our main work on survey responses from minority ethnic groups and on South Asian groups in particular. The general interest in minority ethnic groups is because they tend to report worse experiences using surveys in most countries studied, including in the UK. Our research on out-of-hours care in this programme ( see Chapter 11 ) replicated this result, with Asian and mixed ethnic groups reporting worse experiences than the white majority.

Our specific focus in the major strand of this research was on South Asian respondents because of the size of this group in England and the consistently low scores generated by this group in English surveys across both primary and secondary care settings. We focused on questionnaires competed in English; although the GP Patient Survey is available in 15 languages, a tiny minority of surveys are completed in languages other than English (typically < 0.2% of returns).

A number of potential explanations have been suggested for the lower ratings given by South Asian and other minority ethnic groups. Broadly, these relate to whether these groups of patients (1) receive lower-quality care or (2) receive the same care but rate this more negatively. 75 For example, such respondents might rate the same care more negatively if they have higher expectations or because they interpret the survey items and response options in different ways (such as being culturally less likely to check extreme options).

The last of these options was potentially the simplest to explore. Taking advantage of the large numbers of respondents available in the GP Patient Survey to examine the responses of South Asian groups using item response theory and allowing for a wide range of other sociodemographic characteristics (see Chapter 5 , Workstream 3 ), we found no evidence that South Asian respondents used the scales in a different way from white British respondents. Although these results do not provide conclusive evidence of equivalence in the way in which different respondents use the survey scales, they increase the likelihood that the worse experience reported by South Asian respondents reflects either differences in expectations or genuinely worse care. Our previous work 75 suggested that, for one aspect of care (waiting times), South Asian respondents might have higher expectations of care, implying that their lower scores on surveys might not be associated with worse care. We were able to advance our understanding of this complex issue considerably as a result of the research in this programme.

First, we showed that South Asian respondents to the GP Patient Survey tend to be registered in practices with generally low scores. This explained about half of the difference in reported experience between South Asian and white British patients (see Chapter 5 , Workstream 1 ) and identified that some practice effects were related to the ethnicity of the doctor (with minority ethnic doctors receiving lower scores for doctor–patient communication; see Chapter 5 , Workstream 4 ). However, these practice effects did not account for the low scores among South Asian patients, even though the differences were reduced when practices offered consultations in a South Asian language 172 (PhD project allied to our programme). Next, we showed that, far from being uniform across all population groups, the lower scores from South Asian patients were much more marked among older female respondents. It was therefore important in our subsequent work to ensure that these patients were represented in our research (see Chapter 5 , Workstream 2 ).

In video elicitation interviews with South Asian patients (see Chapter 2 ), we identified the same issues driving evaluations of communication in South Asian as in white British patients: their relationship with their GP (and others within the practice), their expectations of the consultation and a reluctance to criticise their doctor’s performance. The finding that South Asian patients are assessing broadly similar issues when completing questionnaires therefore still leaves unanswered the question of why scores from South Asian patients are low.

The final and most original part of this work provides insight into this (see Chapter 6 ). Here, we filmed 16 simulated consultations based on transcripts of real consultations using various combinations of white and Asian doctors and patients, with half scripted to be ‘good’ and half scripted to be ‘poor’. We showed three randomly sampled videos to each of 1120 people (half of whom were white British and half of whom were Pakistani, equally split between those aged < 55 years and those aged ≥ 55 years) and asked them to score the consultations using the communication items from the GP Patient Survey.

If the low scores reported by South Asian patients in real-life settings were the result of higher expectations on their part, then we would expect them to give lower scores in the experimental vignette situation. However, quite the reverse happened. When viewing the same consultations, South Asian respondents gave scores that were higher, indeed much higher when adjusted for sociodemographic characteristics, than those of the white British respondents. This suggests that the low scores given by South Asian patients in surveys such as the GP Patient Survey reflect care that is genuinely worse, and possibly much worse, than that experienced by their white British counterparts. This is consistent with the only previous study of this type in which predominantly written consultations were shown to people from different ethnic groups in the USA, with the conclusion being that differences in ratings were more likely to represent differences in care than differences in expectations or scale use. 81

There is a clear practice implication of this result: low scores from South Asian patients should be investigated as possible indicators of poor care. This is relevant to all settings, not just primary care.

Using data on patient experience for quality improvement

The results that we have discussed so far indicate that the results of patient experience surveys such as the GP Patient Survey can identify areas where there are important gaps in care that the NHS provides, such as patients being able to see a doctor of their choice. However, although patients tend to give very high scores for doctor–patient communication, these conceal significant negative experiences that patients describe when shown, and which independent observers can see in, recorded primary care consultations. These issues extend to minority ethnic patients and our research suggests that the negative scores that South Asian patients record (compared with those of white British patients) do represent genuine problems with care. This therefore brings us to the important issue of how data from patient surveys can be used to improve care.

Current national approaches to measuring patient experience, including communication, rely on practice-level assessments of care. In Chapter 9 , we outline the results of a patient experience survey that we conducted across 25 general practices, asking patients specifically about their experience of a particular consultation with a named GP . We found that practice-level scores for communication mask considerable variation between GPs within each practice, notably for those practices receiving poorer communication scores overall. Such ‘poorly performing’ practices, which may be identified as such through the national GP Patient Survey, may in fact contain GPs with communication skills ranging from very poor to very good. This has important implications for the use of national survey data to identify primary care practices and practitioners in need of improvement.

In Chapters 7 and 8 we describe the two studies in which we sought the views of GPs and practice staff on survey results, seeking to understand how they could better be used as quality improvement tools. Chapter 7 describes focus groups with practice staff following feedback of practice-level scores for patient experience and Chapter 8 describes interviews with GPs after we had conducted a survey in which they received individual feedback from surveys returned by patients whom they had seen in the surgery. In Chapter 11 , we describe how out-of-hours providers use data from patient surveys.

Broadly, staff in different primary care settings neither believed nor trusted patient surveys. Concerns were expressed about the validity and reliability of surveys (some practices have very low rates of response) and about the likely representativeness of those who responded. Some practice groups mentioned recent negative experiences with pay linked to survey scores as part of the QOF (a technicality of the payment schedule meant that payments could be reduced even though practice performance had improved). There was also a view expressed that some patients had unreasonable expectations: staff worked as hard as they could and could not be expected to respond to all patients’ ‘wants’. Some practices did describe improvements that they had made as a result of survey results. Those that were easiest to engage with related to practices’ office functions such as appointment systems and telephone answering systems. Addressing an individual doctor’s performance (e.g. communication skills) was much more difficult. Out-of-hours service staff were also concerned that service users did not understand the complex care pathways within urgent care settings and that this might lead to unrealistic expectations of what individual services were expected to deliver. Staff viewed surveys as necessary, but not sufficient. Clear preferences for more qualitative feedback to supplement survey scores were expressed as this provided more actionable data on which to mount quality improvement initiatives.

The doctors who we interviewed expressed markedly ambivalent views in discussing feedback from surveys. Although they had a number of concerns about individual doctor surveys (credibility, reliability, concerns about patient motivation), they also expressed positive views about the importance of patient feedback in monitoring and improving services.

These results led us to consider how patient feedback might be obtained in a way that would engage doctors more actively with patient survey results to stimulate quality improvement. We conducted a preliminary evaluation of RTF , using touch screens that patients could use to leave feedback following a primary care consultation. RTF was selected to address some of the problems identified by our research, such as providing practice feedback on a much more regular basis (e.g. fortnightly) and allowing practices the opportunity to add questions of their own to the RTF survey to increase the relevance of the results to their service.

As RTF has not been widely used, an exploratory RCT and qualitative study were conducted to answer questions about the feasibility of using RTF in real-world general practice, estimate likely response rates, obtain patient and staff views on providing feedback in this way and estimate the costs to a practice of introducing RTF. We also included facilitated feedback in one arm of the exploratory trial.

In the exploratory trial, only 2.5% of consulting patients left any RFT without prompting; however, if encouraged to leave RTF by staff, as many as 60% of patients did so. Encouragement was rare, with such encouragement provided in only 5% of > 1100 patient–staff interactions that we observed in reception areas. Of patients who used a touch screen to leave RTF, 86% found it easy to use and were positive about it as a feedback method. Lack of awareness of the screens and lack of time were the most common reasons given for not providing feedback.

Staff were broadly positive about using RTF and practices valued the ability to include their own questions in the survey. Practices that had open communication between staff members tended to be more positive about using patient feedback. Practice staff identified clear benefits from having a facilitated session for discussion of patient feedback and having protected time to discuss the results.

Had practices not been taking part in a research study, the cost of RTF to practices would have been substantial at > £1000 for the 12 weeks, with the bulk of the cost relating to provision of the equipment and analysis and feedback of the data collected from the touch screens.

Although the absolute number of patients providing RTF to each practice (> 100) was comparable to the number of respondents per practice in the national GP Patient Survey, we do now know how the considerably lower response rate in our RTF study (2.5%) would have affected the outcome of the patient experience surveys (it was not part of our study design to find this out). We do not know how representative or valuable the views of a small proportion of patients who respond are, just as we do not know how representative are the views of the very small numbers of patients providing the narrative feedback that is recorded on NHS Choices.

Considering these results together, we have been able to identify some clear learning to take forward into a future clinical trial examining the potential utility and effectiveness of RTF in informing service delivery in primary care.

• Implications for practice

The work that we have carried out over the 5 years of the programme grant has clear implications for practice. We summarise these here.

The importance of patient experience

Our research supports the continuing emphasis on obtaining patient experience feedback as an important means of informing NHS care. Although continuing effort should be invested in refining the most effective and meaningful mechanism to capture high-quality patient feedback, the key challenge is to provide primary care staff with the support and means to enable them to act on patient feedback.

The need for action on the quality of care for minority ethnic groups

There has been much speculation whether the lower scores reported by minority ethnic groups on numerous patient experience surveys are ‘real’, reflecting poorer quality of care, or are an artefact of the questionnaires used or higher expectations of care. We have now conducted a series of studies to progressively examine this issue to understand with greater certainty the major drivers of reported variations in care. Examinations of survey responses, interviews with patients and an innovative experimental vignette study combine to strongly suggest that it is the former: patients from South Asian backgrounds experience considerably poorer communication with GPs than their white British counterparts. It is of concern that survey results may be dismissed as artefactual when, in fact, they are likely to point to real areas of concern. Effort should be invested to ensure that lower scores from such groups on patient experience surveys in both primary care and secondary care are investigated as markers of poorer quality of care.

Patients give overly positive responses when rating their care

Our results show the difficulty that patients have in feeding back negative experiences in questionnaire surveys. This suggests that there is more work to be done in improving patient experience than might be suggested by the high scores that are commonly seen in patient surveys. However, patients’ reluctance to criticise a doctor or provider with whom they have to maintain an ongoing relationship will not be addressed simply by changing the survey method. Efforts should be made to ensure that providers and managers understand that absolute scores paint an optimistic picture of patients’ true views.

Surveys are not sufficient to fully capture patient feedback

Across primary and out-of-hours care settings, staff view patient surveys as necessary, but not sufficient. Alternative methods for gaining more qualitative feedback were commonly used to supplement survey scores, with free text often viewed as providing more actionable data than responses to standard survey questions. Taken alongside our findings on patients’ reluctance to criticise doctors through surveys and staff challenges to the credibility of surveys, we suggest that additional approaches are therefore needed to better capture aspects of patient experience that can be used to improve the quality of care.

The need for valid, reliable individual-level feedback for doctors

Despite the comments above, we have shown that there is substantial variation in performance within practices for aspects of care related to individual doctors (e.g. doctor–patient communication). Reporting patient experience at practice level masks this variation and makes it more difficult for doctors to relate to feedback. However, we have also shown that, if a practice has overall high scores for doctor–patient communication, it is very unlikely that such a practice contains a low-scoring doctor. In contrast, when a practice is low scoring, individual doctors may be high or low scoring. Therefore, if there are additional requirements for individual-level surveys, they could be focused on practices with low overall scores. Additionally, robust mechanisms are required to help practices, particularly lower-scoring practices, identify and support individual doctors whose patient feedback identifies areas of potential improvement.

We note that, at present, data are provided at practice level for the GP Patient Survey, scores are produced at practice level for the Friends and Family Test and GPs have to provide individual-level surveys to meet GMC requirements for revalidation. This results in considerable overlap and duplication and adds to the sense that these are ‘boxes to be ticked’ rather than sources of information that are valuable for improving care.

Patient surveys need to become more meaningful to staff

Our research shows that primary care staff in different settings are ambivalent about the value of patient surveys. Although believing in general about the importance of issues such as doctor–patient communication, they use every opportunity to challenge the credibility and reliability of scores produced by national surveys. This is not helped by their recent experiences, for example of a poorly conceived attempt to tie financial incentives to patient reports of waiting times to get an appointment 306 and the imposition of the Friends and Family Test, which is even regarded by NHS England as being of limited value for comparing health-care organisations. 60

On the whole, practices found it easier to engage with items on surveys that related to practice management (e.g. availability of appointments, ability to get through on the telephone) than to items that related to issues around communication between patients and clinical staff. Staff viewed surveys as necessary, but not sufficient, and expressed a clear preference for qualitative feedback to supplement survey scores as this provided more actionable data on which to mount quality improvement initiatives.

Immediacy of feedback, regularity of feedback and having some control over the questions asked were all aspects of our experiment with RTF that were valued by practices and had the potential to make feedback more useful. However, a number of important questions remain before RTF could be recommended as a replacement for postal questionnaires. We outline these in the next section on research recommendations.

The value of surveys in monitoring national trends

Despite some reservations about the value of national surveys as vehicles for stimulating quality improvement in general practices and out-of-hours services, they can be important for monitoring national trends. For example, the GP Patient Survey is the only source of data which demonstrates that, year on year, from 2010 to 2015, patients report that they have had increasing difficulty in seeing a doctor of their choice. Indeed, for out-of-hours services the GP Patient Survey is the only way to monitor such trends as individual services use very different tools and approaches, precluding comparisons. Additionally, patient feedback – particularly in secondary care – is used for organisational risk assessment and regulatory monitoring. However, when national surveys are used to monitor trends in care it is important that the questions stay the same. In contrast to questions in the GP Patient Survey related to whether or not patients are able to see a doctor of their choice, questions in the survey on access have undergone major changes, making it difficult to follow long-term trends. However, it should be noted that much smaller sample sizes are required to monitor national trends and comparable national surveys often include tens of thousands of participants rather than millions. Our work on out-of-hours care suggests some ways in which the current questions in the GP Patient Survey could be improved.

Development of surveys in out-of-hours care

Our work on the use of patient experience surveys in out-of-hours care highlights a number of areas requiring consideration. National quality requirements (NQR5) state that all out-of-hours services must audit patient experience but provide no information on how to do this. 307 In the absence of clear guidance on tools and approaches, many services are taking different tacks to both collect and act on patient feedback. As well as being inefficient in approach, with little consistency or shared learning, this also precludes national comparisons being made between providers. We suggest that NQR5 should be reviewed and tightened to avoid the duplication of effort occurring in different services.

Second, out-of-hours items from the GP Patient Survey are now being used for the purposes of CQC and National Audit Office monitoring of out-of-hours care. Our research in this area commenced prior to the launch of the CQC and providers knew little about the GP Patient Survey and expressed concern about the relevance of the out-of-hours items. Our research suggests that, subject to minor amendments, the GP Patient Survey is suitable for this kind of national monitoring of out-of-hours care; indeed, it is the only current approach suitable for monitoring, given the variation in approaches to patient feedback currently taken by service providers. However, although the GP Patient Survey enables the use of benchmarking, it is not sufficiently detailed to support quality improvement and as such is unlikely to replace the in-house methods and tools being used by providers. We also note that current presentations of GP Patient Survey data for out-of-hours care are at ‘commissioner’ level; as providers often cover more than one commissioner level, such analyses may not highlight problems occurring at the larger organisational level. Finally, to look at the performance of different out-of-hours providers on key patient experience measures, it is important that NHS England maintains a list of such providers to ensure oversight, which it currently does not.

Overall, large-scale postal surveys are likely to remain the dominant approach for gathering patient feedback for the time being, although refinements to this approach as well as the development of other modes are required to address the weaknesses that we have identified. We are aware that providers are experimenting with a wide range of other approaches, one of which ( RTF ) has been part of our research. Other methods include interviews and focus groups, online feedback, analysis of complaints, practice participation groups and social media. In the following section, we outline recommendations for research and identify the criteria that any new methods will need to meet to become useful quality improvement tools.

• Recommendations for research

The world of patient feedback is becoming increasingly diverse and complex, with standard patient survey approaches being supplemented by the use of tablets, kiosks, online feedback, including that provided by the NHS and by commercial organisations, analysis of complaints, the use of interviews and focus groups and practice participation groups. In addition, social media may come to play an important part in how patients choose their doctor and how they feed back on their experiences. Some of these new approaches are being evaluated in terms of their ability to provide more detailed information on what is needed to improve services, for example using patient narratives 308 and through the analysis of internet-based feedback. 309 , 310 However, despite the plethora of approaches to gathering patient feedback, our research demonstrates that there is a major deficit in taking action as a result of such feedback. Enabling and supporting providers to engage with and plan changes may require complex whole-system approaches, and our knowledge of what is most effective in this area is currently sparse.

Research is therefore needed into how gathering and acting on patient feedback may be best supported, across five key areas:

• How patient experience can be captured so that it more effectively identifies areas of performance that could be improved – this should include investigation of diverse methods of obtaining patient feedback to support patients to highlight poor care when necessary. An additional important area of work is how some of the issues highlighted within this report, such as patients’ reluctance to criticise, apply to different approaches to assessing patient experience using either rating-type or report-type questionnaire items.
• The system, practitioner and patient factors that influence poorer reported experiences of care in South Asian patient groups and how these may be addressed – this should include a particular focus on the impact of cross-cultural consultations.
• How information from patients can be fed back to clinicians and services in a way that appears credible to them – this should include evaluations of approaches to increase the plausibility of patient surveys, such as greater use of benchmarking and innovative ways of presenting and interpreting findings, as well as assessment of varying, tailored ways of presenting feedback to the different health-care professionals who might receive feedback on their care. Of additional relevance here is how clinicians are encouraged to reflect on their own performance and others’ assessments of this, with the aim of understanding where and how gaps in evaluations may occur.
• How services can be organised and managed in such a way that patient feedback is seen as a positive opportunity for improving services.
• What interventions are most effective in improving care when deficiencies in care are identified – the area where there is the greatest gap here is in doctor–patient communication, with our results showing that clinicians have great difficulty in even discussing deficiencies among their colleagues and that few effective interventions exist.

Our finding in the research on out-of-hours care that commercial providers had lower ratings for patient experience than services provided by the NHS is consistent with previous work suggesting that practices working under Alternative Provider Medical Services contracts, which are sometimes provided by the private sector, may provide worse care. 311 However, the circumstances in which commercial providers gain contracts for primary care services may be very different from those in other areas. The way in which the primary care workforce is configured is changing rapidly, with an increase in the proportion of salaried GPs, the development of GP federations and super-practices and an increase in the number of large-scale provider groups (owned both by commercial companies and by GPs). It is important that these changes should be monitored so that we understand their impact on quality of care.

Included under terms of UK Non-commercial Government License .

• Cite this Page Burt J, Campbell J, Abel G, et al. Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience. Southampton (UK): NIHR Journals Library; 2017 Apr. (Programme Grants for Applied Research, No. 5.9.) Chapter 12, Conclusions, implications for practice and recommendations for future research.
• PDF version of this title (54M)
• Disable Glossary Links

In this Page

Other titles in this collection.

• Programme Grants for Applied Research

Recent Activity

• Conclusions, implications for practice and recommendations for future research -... Conclusions, implications for practice and recommendations for future research - Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

The Power of Mass Media and Feminism in the Evolution of Nursing's Image: A Critical Review of the Literature and Implications for Nursing Practice

Affiliations.

• 1 Royal Derby Hospital, Room 401, Uttoxeter Road, Derby, DE22 3DT, UK. [email protected].
• 2 Royal Derby Hospital, Room 401, Uttoxeter Road, Derby, DE22 3DT, UK.
• PMID: 31713004
• DOI: 10.1007/s10912-019-09578-6

Nursing has evolved, yet media representation has arguably failed to keep up. This work explores why representation has been slow in accurately depicting nurses' responsibilities, impacts on public perceptions and professional identity. A critical realist review was employed as this method enables in-depth exploration into why something exists. A multidisciplinary approach was adopted, drawing from feminist, psychological and sociological theories to provide insightful understanding and recommendations. One main feminist lens has been implemented, using Laura Mulvey's 'Male-Gaze' framework for content analysis of three nurse-related advertisements to explore how the profession's female status influences representation, public perception and how this might impact nursing. Nurse representation has important real-world consequences. It is essential to improve unnecessary negative portrayals and contest ingrained stereotypes as there are costs to public opinion and nursing's self-identity. Nursing's female status has an impact within a male-dominated media industry, with a leisurely approach adopted toward changing representation. Media images become societally ingrained, this reiterates the significance of accurate/positive depictions. Social media is an instant method of communication with the public to combat stereotypes and maintain engagement to provide better understanding of what nurses do.

Keywords: Image; Media; Nurse; Representation.

© 2019. Springer Science+Business Media, LLC, part of Springer Nature.

Publication types