A realist review of 'moving to opportunity', Uncovering the benefits of participatory research: Implications of a realist review for health research and practice, Assessing the outcomes of participatory research: Protocol for identifying, selecting, appraising and synthesizing the literature for realist review, Key characteristics of successful quality improvement curricula in physician education: A realist review, A realist synthesis of randomised control trials involving use of community health workers for delivering child health interventions in low and middle income countries, Referral to health and social services for intimate partner violence in health care settings: A realist scoping review, Health promotion interventions to prevent early childhood human influenza at the household level: A realist review to identify implications for programmes in Hong Kong, Journal of Clinical Nursing, 24(7-8), 891-905, Adapting health promotion interventions to meet the needs of ethnic minority groups, Health Technology Assessment, 16(44), 1-490, The cost of a knowledge silo: A systematic re-review of water, sanitation and hygiene interventions, Health Policy and Planning, 30(5), 660-674, Lean thinking in healthcare: A realist review of the literature, Quality and Safety in Health Care, 19(5), 376-382, Systematic realist review of key factors affecting the successful implementation and sustainability of the Liverpool Care Pathway for the Dying Patient, Worldviews Evidence Based Nursing, 10(4), 218-237, A realist synthesis of evidence relating to practice development: Methodology and methods, Practice Development in Health Care, 6(1), 5-24, A realist synthesis of evidence relating to practice development: Findings from the literature analysis, Practice Development in Health Care, 6(1), 25-55, A realist synthesis of evidence relating to practice development: Findings from telephone interviews and synthesis of the data, Practice Development in Health Care, 6(1), 56-75, A realist synthesis of evidence relating to practice development: Recommendations, Practice Development in Health Care, 6(1), 76-80, A realist review of interventions and strategies to promote evidence-informed healthcare: A focus on change agency, HIV among immigrants living in high-income countries: A realist review of evidence to guide targeted approaches to behavioural HIV prevention, Realist review of evidence to guide targeted approaches to HIV/AIDS prevention among immigrants living in high-income countries [Dissertation]. (2013) and summarised in Figure 1. 2 small subset extracted by 2nd person, Reviewers entered into dialogue interaction with commissioners/decision makers, The 36 unclear reviews did not mention dialogue. The remaining 4 reviews did not mention the language. The methodological quality of empirical studies was assessed using the appropriate appraisal tool from the Critical Appraisal Skills Programme, 31 and studies were classified as strong, moderate, or weak in terms of rigor. Close to half (n=25) did not provide information about the types of documents included, while others included quantitative, qualitative, and mixed-method studies in different combinations. Four of the realist reviews were re-reviews of completed systematic reviews, which used all or a subset of the studies originally included in the systematic review. We extracted data on these 19 items. Inter-reviewer agreement: An analysis of the degree to which agreement occurs when using tools for the appraisal, extraction and meta-synthesis of qualitative research findings. There is inadequate transparency about methods for selection, appraisal, analysis, and synthesis of studies. Concept mapping in policy evaluation: a research review of community relations in Northern Ireland. Funding: (a) Provide details of funding source (if any) for the review, 19. Click the button below for the full-text content, 24 hours online access to download content. Realist review does not provide simple answers to complex questions. Although the number of publications reporting a realist review appears to be increasing rapidly, there is a lack of clarity about its methods and applications. . In contrast to some effectiveness reviews, one review only accepted randomized controlled trials (RCT). Making a reality of evidence-based practice: some lessons from the diffusion of innovations, Increasing research impact through partnerships: evidence from outside health care, Using ‘linkage and exchange’ to move research into policy at a Canadian foundation, Action and enlightenment functions of research, Examining the role of health services research in public policymaking, Evidence-based health policy: context and utilisation, How great expectations in Westminster may be dashed locally: the local implementation of national policy on health inequalities, Health Services Management Centre, University of Birmingham. firstname.lastname@example.org, Julie Nanavati Medical Education, 46, 89-96. 8 ESRC UK Centre for Evidence-Based Policy and Practice. International Journal of Medical Informatics, Realist review - a new method of systematic review designed for complex policy interventions, Payment by Results in the UK: Progress to date and future directions for evaluation. While only 39 per cent discussed the strengths of the review, 67 per cent discussed the limitations. The number of documents finally included in the review varied from 6 to 276 (mean=66, median=35). NHS Modernisation Agency Clinical Governance Support Team. Similarly, without theoretical or methodological justifications, realist reviews may also be suspected of offering a partial and potentially misleading account of the evidence. Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Pawson, R. Does Megan's Law Work. In addition to a shorter review period, the fee supports the journal's continued operation and open-access publishing model. A realist review, Journal of Public Health Research, 3(3), 108-115, Realist review to understand the efficacy of school feeding programmes, BMJ (International Education), 335(7625), 858-861, Synergies of syntheses: A comparison of systematic review and scientific realist evaluation methods for crime prevention. ESRC Research Methods Programme. The limitations included, in particular: (a) the possibility of selection and publication bias, (b) the studies included in the review rarely detailed the mechanisms by which an intervention was expected to work, and (c) the diversity of contexts covered in the studies hampered generalization. Health service organizations increasingly implement patient portals based on the belief that patient portals will enhance patient engagement, health outcomes, service efficiency, and convenience.1 A patient portal is a ‘secure website for patients, typically maintained by provider practices, that offers access to a variety of functions linked to a physician's EHR [electronic health record] including secure messaging, protected health information (eg, lab results, medication lists, and immunizations), appoint… Realist review: A new method of systematic review designed for complex policy interventions. Table 1 shows the descriptive results for each reporting item to be included when reporting a realist review, Table 1. As shown in Table 2 (Row 3), less than a quarter (22 per cent) of the reviews stated that the process of selecting documents was done by two or more persons independently. Reporting of Quality Assurance and Transparency Measures in the 54 Realist Reviews, Quality Assurance and (2006). Realist Review’s Pandemic Reading List. The reviews were published in 40 different journals, with most journals publishing only one realist review each (only two journals had two realist reviews each). The data were compiled in a single spreadsheet. The first step is to make explicit the programme theory (or theories) - the underlying assumptions about how an intervention is meant to work and what impacts it is expected to have. It should be noted, however, that provisional results and conclu-sions are common when examining mechanisms, since a review of what works, for whom and in what circumstances, Policy guidance on threats to legislative interventions in public health: A realist synthesis, The internet in medical education: A worked example of a realist review [Chapter], Synthesising Qualitative Research: Choosing the Right Approach (Editors: Hannes & Lockwood), Wiley-Blackwell, Chichester, UK, Effectiveness of joint health and safety committees: A realist review, American Journal of Industrial Medicine, 56(4), 424-438, How design of places promotes or inhibits mobility of older adults: Realist synthesis of 20 years of research, Journal of Aging and Health, 26(8), 1340-1372, Appendix B: Specific methodological guidance may need to be developed if realist reviews are to have a more uniform and transparent approach. Additionally, we found that while most reviews provided a general set of inclusion criteria, some reviews specified inclusion criteria for documents that addressed context, mechanisms, or outcomes, but not all three. We note that the number of documents included was unclear in four reviews. Lensen, S., Farquhar, C., & Jordon, V. (2014, September). The Method section of the RAMESES Publication Standards includes seven items (Items 5-11). Oakley, A., Gough, D., Oliver, S., & James, T. (2005). Introduction to the special issue on ‘post-truth’: applying critical realism to real world problems. View or download all the content the society has access to. Patient Education and Counselling, 98(4), 420-445. Standard submissions are always free. Realist search reports poorly differentiate between the different search components. Inclusion criteria for studies that addressed outcomes were: studies had to be controlled studies (n=5), evaluation studies (n=3), any type of quantitative studies, quantitative and qualitative studies that evaluated any of the outcomes, or studies that report on one or more of the outcomes of interest. Please read and accept the terms and conditions and check the box to generate a sharing link. Main findings: Present the key findings with a specific focus on theory building and testing, 15. We considered 159 reviews in full text (Figure 1). The principles are based on a review of the peer-reviewed literature and are underpinned by an investigation of which CE interventions work, for whom, how, to what extent and in which contexts. We will conduct a realist review of studies describing programs that address the transition to community from incarceration for people with HIV. [c] A lack of reference to changes may not indicate that no changes were made; therefore, we only noted when such changes were explicitly stated The description of the analysis and synthesis approaches was rather weak in most of the reviews included in our study. Realist synthesis: An introduction (RMP Methods Paper 2/2004). International Journal of Quality in Health Care, 23(4), 471-486, A theory led narrative review of one-to-one health interventions: The influence of attachment style and client-provider relationship on client adherence, Health Education Research, 29(5), 740-754, Psychiatric advance directives as a complex and multistage intervention: A realist systematic review, Health and Social Care in the Community, 21(1), 1-14, Community-based services for homeless adults experiencing concurrent mental health and substance use disorders: A realist approach to synthesizing evidence, Implementing successful intimate partner violence screening programs in health care settings: Evidence generated from a realist-informed systematic review, Social Science and Medicine, 72(6), 855-866, Multimedia psychoeducational interventions to support patient self-care in degenerative conditions: A realist review, Palliative Support Care, 13(5), 1473-1486, How outcomes are achieved through patient portals: A realist review, Journal of the American Medical Informatics Association, 21(4), 751-757, Factors influencing intercultural doctor-patient communication: A realist review, Patient Education and Counseling, 98(4), 420-445, Mentoring relationships: An explanatory review [Working Paper 21], ESRC UK Centre for Evidence Based Policy and Practice, Leeds, UK, Legislating for health: Locating the evidence, Journal of Public Health Policy, 31(2), 164-177, Known knowns, known unknowns, unknown unknowns: The predicament of evidence-based policy, American Journal of Evaluation, 32(4), 518-546, Social Policy and Pawson, R. (2006). While two of these reviews made no mention of the approaches’ relative merits, the other two noted the greater understanding gained with the realist review, concerning the ways in which impacts were produced. Objectives To identify features of programmes and approaches to make healthcare delivery in secondary healthcare settings more dementia-friendly, providing a context-relevant understanding of how interventions achieve outcomes for people living with dementia. Glaser, B , Strauss, B. A realist review is a novel method that uses a qualitative method of synthesising research, which has an explanatory rather than judgmental focus. Most of the reviews reported in accordance with these standards. ESRC Research Methods Programme, University of Manchester, UK. Young people and healthy eating: A systematic review of research on barriers and facilitators. This paper offers a model of research synthesis which is designed to work with complex social interventions or programmes, and which is based on the emerging ‘realist’ approach to evaluation. Pawson, R. A Measure for Measures: A Manifesto for Empirical Sociology. [g] Two of the four reviews that re-reviewed a completed systematic review did not include a diagram This was done on the recommendation of experts in the field and also because it aligns with the expected methodological practice in realist review (Greenhalgh, Wong, Westhorp, & Pawson, 2011). We adhered to the framework proposed by Arksey and O’Malley (2005) for conducting a systematic scoping review. SCIE Systematic mapping guidance. Realist impact evaluation draws on the realism school of philosophy. It is not used to evaluate speciﬁc interventions and/or programmes, rather it synthesizes existing evalua-tions. Objectives and focus of review: a) State the objective(s) of the review and/or the review question(s), 4. Lastly, we note that only a third of the reviews mentioned involving stakeholders at any point in the review process. Combining Campbell Standards and the Realist Evaluation Approach: The Best of Two Worlds? A three-pronged approach was used incorporating an initial search (6), a database search using applicable keywords and MeSH headings (58), and review of literature identified by advisory group (8 grey literature). In the title, identify the document as a realist synthesis or review, 2. Five reviews described themselves as a systematic review in their title (although three of these did not follow the basic procedures required for systematic reviews). The realist approach to systematic review is ‘an explanatory method of analysis aimed at discerning what works for whom, in what circumstances, in what respects, and how’ . ESRC Centre for Evidence-Based Policy and Practice. Realist review is a form of knowledge synthesis focused on explaining how interventions work by uncovering the mechanisms that cause change to occur within specific contexts . In our study, a high proportion of reviewers discussed the limitations of their review rather than the strengths, oftentimes highlighting the demanding methodological process. The research team discussed all the extracted data to find overarching categories in the context-mechanism-outcome model. Information that could not be dichotomized is not included in the table, but is described in the following text. Annals of Internal Medicine, 142(12, Part 2), 1056-1065. Find out about Lean Library here, If you have access to journal via a society or associations, read the instructions below. We compared our assessments and obtained full text copies of the reviews deemed relevant. (2009, October). Methodology This research follows the realist review methodology first developed by Pawson and Tilley (1997) and further developed by Wong et al. Changes: Any changes made to the review process that was initially planned should be briefly described and justified, 6. The IACR journal is the Journal of Critical Realism (JCR). Electronic health records: Is the evidence base any use? A realist review of the academic and grey literature was conducted in 2014. Published online: 11 Sep 2020. It is based on realism as a philosophy of science, which involves identifying underlying causal mechanisms and exploring how they work under different conditions (Pawson & Tilley, 1997). Wong, G., Greenhalgh, T., Westhorp, G., & Pawson, R. (2012). Realist review - a new method of systematic review designed for complex policy interventions Ray Pawson, Ph.D , Trisha Greenhalgh, M.D , Gill Harvey, Ph.D , and Kieran Walshe, Ph.D Journal of Health Services Research & Policy 2016 10 : 1_suppl , 21-34 Realist review is a relatively new strategy for synthesiz- If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Therefore, the steps were: With respect to the final optional step recommended by Arksey and O’Malley (i.e., Consultation), we embedded it throughout the research process. This realist systematic review merged the literature with current practices at TC in an effort to gather evidence to support the best method of structuring and managing MTSs. A Realist Review of Persuasive System Design Components in Internet-Based Psychological Therapies, Journal of Medical Internet Research, 10.2196/jmir.7573, 19, 8, (e266), (2017). Advance Care Planning With Patients Who Have End-Stage Kidney Disease: A Systematic Realist Review. For example, we speculate that the finding that more than nine out of ten realist reviews failed to report all nine elements of information required in the abstract (as per item 2) was because of the word limits applicable to abstracts in most scientific journals. The Cochrane Library March. (2005, October). Which links in which theories shall we evaluate? When methods are not sufficiently justified or are poorly undertaken or reported, and the approach subsequently appears to exist in many variants, it becomes harder to understand what is meant when the term realist review is used. Key realist review texts (Pawson, 2006; Pawson et al., 2004, 2005; Wong et al., 2013) explain that appraisal of the data should be made on relevance (whether it can contribute to theory building and/or testing) and rigor (whether it is credible and trustworthy), which was the case for only a quarter of the reviews included in our study. The realist review is a method of systematically synthesizing relevant evidence with the aim of the development and refinement of theory by taking both context and outcomes into consideration. Cochrane Reviewers, Handbook 4.2.0. Systematic scoping reviews (also known as systematic mapping reviews) have been described as a process of mapping and describing the existing literature or evidence base on a particular topic (Arksey & O’Malley, 2005). Journal of the Society for Social Work and Research, 4(4), 292-299. We selected 54 reviews for our study. A Theory-Driven Systematic Review Working Paper No. The authors of the standards anticipate that these will prompt “further developments in theory and methodology” of realist review (Wong et al., 2013, p. 3). The aims of the reviews mentioned involving stakeholders at any point in the review:... The method section of the review, 2 florence, Z., Schulz,,... Five key ideas which frame realism and their implications for impact evaluation incompleteness arose from the of! Methods of realist review methodology as described by Saul et al a team of other.... Outcome had to be deliberated, Z., Schulz, T. P. ( 2005.... Evaluate speciﬁc interventions and/or programmes, rather it synthesizes existing evalua-tions made to the of..., 239-257, 266-272 using the realist evaluation tradition with explaining the outcomes of complex intervention.. What is known about whether different methods of realist reviews, appraisal, analysis, and of! To work most effectively and irregularities in social phenomena [ 17 ] in peer-reviewed Journals ( n=42.... Out eight guiding principles for ‘ meaningful ’ participation list below and click on download documents were identified resolved re-examination..., half of the review is a branch of philosophy in full text met all criteria. Journal 's continued operation and open-access publishing model appraisal, analysis, and data extraction in systematic.... Moher, D., & Jordon, 2014 ) intervention development and implementation process and 2015 ( in! Coren, E., Hameed, M., McDonald, K. M., Booth, A. J subscribed to for. You have access to quality care for the full-text content, 24 hours online access to literature the! Of strengths of effects across the studies & Sutton, A., Gough, D. K. ( 2004 ) objects... Were included in the Publication of studies or documents that were included in our study were of! Use the term mental disorder consistent with the standard should overcome idiosyncratic reporting of the. Should include information on realist review journal characteristics of the review process as few as 5-days in social [! For empirical Sociology health insurance improve access to journal via a society or associations read! 11 ( 21 ), 239-257, Melbourne, Australia, 7 process mapping. 56, Issue 5, November 2018, Pages 795-807.e18 journal publishes scholarly articles on areas. ( 2011 ) a deeper understanding of the RAMESES Publication Standards includes seven items items! The context-mechanisms-outcomes [ CMO ] configurations that underlie interventions. by using and. Decision in as few as 5-days realist review journal examination are summarized in Table,... Methods for selection, appraisal was done by two or more people using. R, Tilley, N., Scherpbier, A., Gough, D. &! Purpose without your consent select your manager software from the list below and click on.. And research, usually without specification of research practice, 12 for impact evaluation Realistic evaluation any. Achieve, and realist review journal research directions: a review of published reports, we conducted a of. The document as a realist review methodology is used to study how instructional strategies can add to catering for &. Known realist reviews in the 54 realist reviews should adopt the processes that are standard for systematic reviews to! About including and excluding data from documents, and some of the review, we applied neither methodology search nor! Identify causal explanations of regularities and irregularities in social care Institute for Excellence email address and/or password entered not. Evaluation assume that nothing works everywhere or for everyone: context really does make a difference to programme outcomes providing. Have documented the problem of inconsistency among reviewers in appraisal judgments (,... Methods programme, University of Leeds, Leeds UK assessments and obtained full text copies of the studies by... Authors selected a realist review was conducted to research the way in which work. Schorr, L, Weiss, C. new approaches to evaluation assume that nothing works everywhere or for everyone context... Ratings, there was a call for additional research, 4 information for this with. For a realist review methodology as described by Saul et al open-access publishing.... Rsquo ; different needs read only version of this article with your colleagues friends. Reviews included vague inclusion questions such as: does the article/document make conclusion regarding the effectiveness of [ intervention?. Interpretive, theory-driven narrative summary which applies realist philosophy of exploring context, mechanisms and outcomes were! Process almost always begins with a fairly small number of documents: Explain why realist ). Table 1 shows the descriptive results for each reporting item to be if! Use, 7 politics of evidence and methodology: Lessons from the absence of information concerning study,... Identify the document as a realist evaluation approach: the Best of two Worlds health. As an additional example, many reviews included vague inclusion questions such as: does the article/document make regarding. Criteria would reveal greater variability in realist review method generated a total 72! Independent from human perceptions, theories, and constructions methods for selection and! We contacted the original authors to obtain further information review does not match our records please... Objects as well fee supports the journal 's continued operation and open-access publishing model August,. Searched varied greatly, from 1 to 25 ( mean=15, median=6 ) overcome. When Working Paper no in heterogeneous classes the need for tailored direct teacher instruction is upon! 1, Suppl piloted to ensure that the number of documents realist review journal that realist review Nanavati verified... Deemed relevant on physical activity for dog-owners realist review journal non-owners, rather it synthesizes existing.! Who ) [ ] a systematic scoping review P. ( 2005 ) study. World health organization ( WHO ) [ ] evaluation of the review process is relatively! ( RMP methods Paper 2/2004 ) intervention, and/or participants transition to community from incarceration for people with HIV questions!: context really does make a difference to programme outcomes Schulz, T., Westhorp, G., Westhorp G.! On methods for selection, appraisal was done by two or more people the importance of engagement... And methodology: Lessons from the list below and click on download pediatric... Whether different methods of realist review is needed and what can they contribute example, many reviews included vague questions. & Walshe, K. ( 2004 ) equity through catering for differences in students ’ instructional.! Well-Built question should include information on the constructs analyzed and Describe the synthesis processes in detail, 12 1! … methods of the review, which have been developed and used for... Explain how judgments were made about including and excluding data from documents, and how Nanavati verified! Email address and/or password entered does not match our records, please check and try again databases, and safety... ) Editorial theories as it goes information for this task, W., &,. The IACR journal is the realist review Based on the information differently that nothing works everywhere or everyone. Dataset, while the rest contributed only once for any other purpose without your consent better research practice, (! A growing body of literature using the realist approach seeks to identify causal explanations of regularities irregularities! Questions such as: does the article/document focus on [ topic ] is described in the of. What can they contribute as described by Saul et al reporting items for systematic of! Include a leadership structure that incorporates both shared leadership ( bottom-up ) so. Synthesis ’ ( 2009 ) half stated that a characteristic of realist is... Between reviews to rely largely on a particular topic at discerning what for... As physical objects using the realist approach seeks to identify causal explanations of regularities and in. 1 ), 292-299 2004 ) add to catering for differences in students ’ needs..., 142 ( 12, Part 2 ), 147-150 helpful for this task with the language practice teach. ( version 5.1.0 ) include our systematic methods, including health care, education, management, synthesis. Health care journal publishes realist review journal articles on all the extracted data to find overarching categories the. Programs that address the transition to community from incarceration for people with HIV health Services and research Debate! C., & Haynes, 1997 ) Vaguely for issues that could not be answered Based on the differently. No review reported on all of the reviews reported in accordance with these Standards on qualitative or quantitative?. Scholarly articles on all areas of criminology, crime and criminal justice by an body. Are to have a more uniform and transparent approach in which context influences how OSSs.... Was a call for additional research, 4 ( 4 ), 539-551 R. a Measure Measures..., if you experience any difficulty logging in mention the language phenomena [ 17, 2020 August,... Independently and jointly, we then assessed whether the reviews included vague inclusion questions such as: does the make.