Implementation of telemedicine consultations for people with mental health conditions in the community: a protocol for a systematic review

Background: The COVID-19 pandemic response has led to an exponential increase in the use and spread of telemedicine internationally. In community mental health care settings, telemedicine services were implemented within a few weeks, with little time for rigorous planning. Despite the reported acceptability of telemedicine by patients and clinicians, barriers to its implementation have come to light. There is now a need to investigate these barriers, and facilitators, as telemedicine begins to show potential promise beyond the pandemic. We propose a review that aims to identify the factors affecting the implementation of telemedicine consultations for patients with mental health conditions in the community. Methods: A systematic review will be conducted and reported according to the PRISMA guidelines. Five electronic databases will be searched using a pre-defined search strategy from 2016 to 2021. Only studies of synchronous, interactive telemedicine consultations conducted via video, phone or live messaging between patients and providers will be included. Quantitative, qualitative and mixed methods studies will be eligible for inclusion. Only studies published in the English language will be included. Titles and abstracts will be screened by two reviewers. Full text articles will be screened by two reviewers. The methodological quality of studies will be assessed using the Mixed Method Appraisal Tool (MMAT) by two reviewers. Data will be extracted and tabulated to address the aims of the review. A narrative synthesis will be conducted and reported factors will be mapped to the domains of the Consolidated Framework for Implementation Research (CFIR). Conclusion: By identifying the factors that influence the implementation of telemedicine consultations for patients with mental conditions in the community, consideration can be given to both barriers and facilitators that could be addressed in future mental health services planning. PROSPERO registration: CRD42021273422 (04/10/2021)


Introduction
The COVID-19 pandemic response caused swift, unprecedented changes to the delivery of healthcare. One such change was the rapid and widespread expansion of telemedicine services to comply with social distancing policies and reduce the spread of the virus 1,2 . Telemedicine is defined as the provision of healthcare at a distance through telecommunications and information technologies 3 . Worldwide, longstanding regulatory barriers to telemedicine delivery were amended to facilitate its use during the pandemic 4 . In community settings, telemedicine allowed for the continued and essential access to mental health services during the pandemic. This use of telemedicine in mental health care is often referred to as telemental health, and is defined as the use of telecommunication for the provision or support of mental health services over a distance 5 . Heterogeneous definitions of telemedicine exist in the literature, so for the purposes of this review, telemedicine refers to live, synchronous remote consultations between provider and patient, using video, audio or live messaging modalities. These telemedicine consultations were chosen as the focus of this review as these types of consultations became commonplace during the pandemic 6 , acting as a temporary replacement for in-person consultations.
In the past two decades, telemedicine in mental health care has emerged as a safe and acceptable method of improving mental health care access for those who are disenfranchised or hard-to-reach. Systematic reviews and meta-analyses have shown that treatment effects of telemental health are comparable to face-to-face mental health care 7-10 . There is also a body of evidence supporting the equivalence of telemental health to face-to-face care with regards to patient satisfaction and therapeutic alliance [11][12][13] . Despite a steady increase in use and evidence for its effectiveness, telemedicine constitutes a small portion of all mental health services prior to the pandemic 14,15 . Previous reviews of implementation factors have attributed the under-utilisation of telemedicine in mental health services to a number of reasons including strict licensure regulations and insurance policies that limit the reimbursement of telemedicine services 16,17 and reluctance by clinicians 18,19 .
The rapid and highly variable adoption of telemedicine in mental health care settings during the COVID-19 pandemic has shed some light on this research-implementation gap. A number of challenges and barriers have come to light, including lack of technological infrastructure, privacy concerns, difficulty in establishing rapport and problems with conducting high quality assessments 20,21 . Despite these challenges, patients and clinicians have reported satisfaction with, and acceptance of, telemental health services during the pandemic 22,23 . Moreover, some patients and service users have expressed a desire to continue to use telemental health services in the future 15,24 ; a view mirrored by policy makers and mental health professionals 25,26 . To harness the possible potential of telemedicine in future mental health care, a systematic exploration of the factors that affect successful telemedicine implementation in community mental health services is now needed. Identifying these factors, both enabling and hindering, will help ensure its acceptable and effective use going forward.
To gain a thorough understanding of the factors that influence the implementation of telemedicine into community mental health services, a strong theoretical foundation to guide interpretation of these factors is required. Various theories, models and frameworks have been developed in the area of implementation research to understand the determinants of translating research into practice 27 . One such framework is the Consolidated Framework for Implementation Research (CFIR) 28 . The CFIR consolidates the various terms used in implementation research into five domains considered to be important moderators or mediators of implementing practice into research 28 . These five domains include 1) intervention characteristics, 2) outer setting, 3) inner setting, 4) characteristics of individuals and 5) process. This framework has been used to assess the implementation of evidence-based practices in health and mental health settings 29,30 . The CFIR encompasses terms and concepts from numerous implementation frameworks and has therefore been selected to map the findings of this review. Its comprehensiveness allows for the categorization of various implementation factors across a variety of study designs and patient populations 28 , which is relevant to this review.
To our knowledge, no systematic review exists of the factors that affect implementation of telemedicine for patients with mental health conditions in community settings. In consideration of the swift and variable adoption of telemedicine in the COVID-19 pandemic, a number of overlooked factors may emerge as likely determinants of successful telemedicine implementation. This review is pertinent as mental health professionals and policy makers are now interested in the continued, long-term use of telemedicine in mental services beyond the COVID-19 pandemic 14,31 . Moreover, this review is particularly relevant given the unprecedented rise in mental health problems and increased demand on mental health services arising from the pandemic 32,33 . It is crucial that we now take stock of the available evidence regarding the challenges and successes of implementing telemedicine to identify key factors for its acceptable adoption into routine community mental health care. Subsequently, consideration can be given to solutions that address these factors by stakeholders involved in mental health services planning.
The primary aim of this review is to identify, summarise and interpret the key factors affecting the implementation of telemedicine consultations for patients with mental health

Amendments from Version 2
This version includes further clarification on the data extraction process.
Any further responses from the reviewers can be found at the end of the article REVISED conditions in the community. The secondary aim is to map these factors to the domains of the CFIR.

Protocol
This protocol has been prepared following the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Protocols) 2015 checklist 34,35 (See Reporting Guidelines). The protocol is registered on the International Prospective Register of Systematic Reviews (PROSPERO) as CRD42021273422. A systematic review design was chosen to synthesise the available literature. As the scope of the research question is narrow in nature and we are interested in synthesising empirical research, a systematic review approach was deemed appropriate 36 .
Eligibility criteria Studies will be selected for inclusion in the systematic review according to the following PEO acronym criteria (population, exposure and outcome).
Population. The population will include adults and children (aged < 18 years) with a diagnosis of a mental disorder or in receipt of care from a mental health professional (e.g. psychiatrist, psychotherapist, counsellor). Mental disorders included will be in accordance with the International Classification of Diseases (ICD-11) criteria for mental and behavioural disorders (WHO, 2019). We will include common mental disorders such as depression, generalised anxiety disorder, social anxiety disorder, obsessive-compulsive disorder and post-traumatic stress disorder. Severe mental disorders such as schizophrenia and other psychotic disorders, and bipolar disorder will also be included. Neurocognitive disorders, such as dementia, perinatal mental disorders, disorders associated with substance abuse, disorders associated with stress, eating disorders and neurodevelopmental disorders, such as ADHD, will be included. Studies that focus on people learning difficulties, intellectual disabilities and people with mental health problems secondary due to physical illness will also be included. We will include studies which include health care professionals (e.g. doctors, nurses, allied health professionals) involved in the provision of mental health care via telemedicine to patients with the above conditions or without a formal psychiatric diagnosis.
Exposure. Studies evaluating synchronous, live, interactive telemedicine consultations between patient and provider, including video-conferencing, telephone and live-messaging only will be included. Studies will be restricted to those that use synchronous (real-time) consultations between a patient and one, or more, health care professionals. Studies will be excluded if they use asynchronous methods, in which the healthcare professional and patient do not interact in real-time, such as email communications. Studies exploring telemedicine consultations between practitioners, such as when a health care professional seeks advice from another practitioner, will be excluded.
Outcome. Studies with data on the factors that affect the implementation of telemedicine consultations for people with mental health conditions in the community will be included. Studies that explore the views and experiences of patients, parents/carers of patients and/or healthcare professionals on the implementation of telemedicine consultations for patients with mental health conditions in the community are eligible for inclusion. Studies only exploring anticipated or hypothetical views will be excluded.

Study design.
Qualitative, quantitative and mixed-methods studies will be included in the review. Systematic reviews, meta-analyses and study protocols will be excluded as they do not contain primary research. We will only include full-text studies reported in the English language, due to constraints on resources to translate studies. Grey literature will be excluded, including non-peer reviewed articles, conference proceedings, case reports, editorials, opinion papers and letters. We have decided to exclude grey literature because of the variability in quality, peer review, supplication in full-text reviews and the need for prompt publication of findings.

Setting.
Only studies in primary care, community or outpatient settings will be included. Studies in inpatient hospital settings, residential care homes, and prisons will be excluded.

Information sources and search strategy
An initial search of the MEDLINE database will be conducted to identify articles on the topic. Five electronic databases will be systematically searched for studies published between 2016 and 2021; MEDLINE (PubMed), Embase, Web of Science, CINAHL and APA PsycINFO. This timeframe (2016 to 2021) was chosen to include studies prior to, and during, the pandemic to ensure a comprehensive review of implementation-related factors. We chose not to include studies before this time point as the use of technology has grown dramatically in the past five years and the context in which telemedicine was implemented previously may be largely different to what is relevant in today's context. The search strategy will contain terms pertaining to mental health conditions, telemedicine, community settings and implementation. See Extended Data for the PubMed sample search strategy 35 . Forward and backward citation searches of included articles will also be conducted.

Study selection
Two reviewers (EG and JH) will independently screen titles and abstracts of all articles in order to identify studies that meet the inclusion and exclusion criteria. The full texts of all selected articles will be collected and examined by two reviewers, independent of each other (EG and JH). Any disagreements will be mediated through a third reviewer (SC). Duplicates will be excluded. A PRISMA flow chart will display the articles examined at each stage, detailing the number of papers included and excluded and reasons for exclusions.

Data extraction and management
The following data items will be extracted from all studies by one reviewer (EG): 1. Due to restraints on resources that prevent independent extraction of data by two reviewers, one reviewer (EG) will extract the data while a second reviewer (JH) will review a random 20% of the extracted data for accuracy. Any disagreements between the reviewers relating to the extracted data will aimed to be resolved through consensus. If the two reviewers are unable to come to a consensus, a third reviewer (SC) will be consulted. If the third reviewer is unable to arbitrate, the study authors will be contacted to seek clarification on the issue. If this step is unsuccessful, the disagreement will be recorded and reported in the review 37 . In addition, reported factors will be extracted and categorised into the five domains of the CFIR by one reviewer (EG). Microsoft Excel software will be used to organize the extracted data. Any uncertainties regarding data will be resolved by attempting to contact study authors via email. Selected articles will be stored and managed using EndNote X9 Reference Manager Library.

Quality assessment
Two reviewers (EG and JH) will independently assess the methodological quality of the included studies using the Mixed-Methods Appraisal Tool (MMAT) (Hong et al., 2018). Any discrepancies between study assessments will be discussed and resolved. Studies will not be excluded based on quality.

Data synthesis
One reviewer (EG) will perform a thematic synthesis on the extracted qualitative findings following Thomas and Harden's 38 guidance. This will involve coding the qualitative findings and analysing the identifying themes relating to the implementation factors. This inductive approach was chosen to ensure all relevant implementation factors were identified, including those that may not fit in an existing framework. This synthesis will be verified by a second reviewer (JH). Quantitative findings will be summarised narratively. One reviewer (EG) will map the reported factors to the five domains of the CFIR (outer-setting, inner-setting, intervention, individual or process) and present them in a table format. We will use the GRADE Confidence in the Evidence from Reviews of Qualitative Research (CERQual) to rate the overall confidence in the qualitative evidence synthesis and the narrative summaries of the quantitative data 39 . Any notable similarities and differences in implementation factors between studies conducted before, and during, the pandemic will be discussed narratively.

Study status
The database searches were completed in August 2021. Full-text screening was completed in November 2021. It is anticipated that this review will be completed in March 2022.

Discussion
The rapid and unprecedented uptake of telemedicine since the COVID-19 pandemic has brought this modality of healthcare to the forefront of health services research. The potential of telemedicine to increase access to mental health services and alleviate the mental health burden is promising. However, challenges to its implementation in current mental health services are still present, such as the potential exacerbation of inequalities and technological barriers 40 . This review will aim to shed light on the factors that may enable or hinder the implementation of telemedicine for people with mental health conditions in the community. By identifying and interpreting these factors, consideration can be given to solutions that can optimise remotely-delivered mental health care for patients in the community.
A strength of this review is the use of a determinant framework, the CFIR, to map the findings to as it aids the transferability and generalisability of findings to other implementation studies 28 . Furthermore, the CFIR incorporates concepts from multiple implementation theories which makes it less likely that important factors will be overlooked 28 . A number of steps will be undertaken to minimise the risk of meta-biases in the review. Firstly, the systematic review will be conducted and reported using the PRISMA guidelines 41 . Secondly, the risk of selection bias will be minimised by two independent reviewers performing title and abstract, and full-text screening. Thirdly, two independent reviewers will appraise the quality of the included studies. A limitation of the review is that the studies will be restricted to those conducted in the English language due to financial constraints which may bias the results. In addition, excluding unpublished studies and grey literature may increase the risk of publication bias. Another potential limitation is the heterogeneity of implementation factors and patient populations reported in the included studies. This may make it difficult to synthesise and compare the findings from each study. A final limitation is the use of one reviewer to extract data which may increase the risk of errors at this stage.
Despite these limitations, this review will be an important and timely contribution to understanding how to improve the implementation of telemedicine for patients with mental health conditions in the community. Findings of the review may advise policy makers and other stakeholders involved in the implementation of telemedicine services, informing their future development. The results of the systematic review will be reported in a peer-reviewed journal, presented at national and international conferences and included in a PhD thesis. "Comment 5: Methods: Authors should consider piloting the data extraction process with two reviewers, and include a second reviewer to validate the data extraction to ensure consistency and comprehensiveness in the data extraction process, and to minimize biases and human error.

Data availability
Response: Thank you for this comment. We agree that independent data extraction from two reviewers would help to improve consistency and comprehensiveness. Due to resource and time constraints, a second reviewer will now extract data items from a random 20% sample of studies to minimize bias. The Data extraction and management section of the manuscript has been amended to reflect this change." The response clarifies the reason for using only one reviewer for data extraction, however, we consider that this should be acknowledged in the discussion section as part of the limitations. Also, it is unclear if the second reviewer will extract data from a random 20% sample (pg. 5, line 22) as part of a pilot or a validation process. We suggest clarifying and describing the details of this process. Will the second reviewer extract the data before or after the first reviewer completes the entire data extraction? How will the two extractions be compared? What will happen in case of disagreement?
Finally, a minor editorial comment: there seems to be a typo: conduct vs contact, in the following phrase: "Any uncertainties regarding data will be resolved by attempting to conduct study authors via email" (pg. 5)".

Competing Interests: No competing interests were disclosed.
We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

Author Response 23 Sep 2022
Emer Galvin, Royal College of Surgeons in Ireland, Dublin, Ireland Comment: "Comment 5: Methods: Authors should consider piloting the data extraction process with two reviewers, and include a second reviewer to validate the data extraction to ensure consistency and comprehensiveness in the data extraction process, and to minimize biases and human error.
Response: Thank you for this comment. We agree that independent data extraction from two reviewers would help to improve consistency and comprehensiveness. Due to resource and time constraints, a second reviewer will now extract data items from a random 20% sample of studies to minimize bias. The Data extraction and management section of the manuscript has been amended to reflect this change." The response clarifies the reason for using only one reviewer for data extraction, however, we consider that this should be acknowledged in the discussion section as part of the limitations.

Response:
Sincere thanks for your comments. We have acknowledged the limitation of using a single reviewer for data extraction in the Discussion section: A final limitation is the use of one reviewer to extract data which may increase the risk of errors at this stage.

Comment:
Also, it is unclear if the second reviewer will extract data from a random 20% sample (pg. 5, line 22) as part of a pilot or a validation process. We suggest clarifying and describing the details of this process. Will the second reviewer extract the data before or after the first reviewer completes the entire data extraction? How will the two extractions be compared? What will happen in case of disagreement?

Response:
Thank you for this comment. We have sought guidance on the data extraction process from Taylor, Mahtani, and Aronson (2021) and have clarified the details of data extraction in the manuscript. Please see changes to the Data extraction and management section below: Due to restraints on resources that prevent independent extraction of data by two reviewers, one reviewer (EG) will extract the data while a second reviewer (JH) will review a random 20% of the extracted data for accuracy. Any disagreements between the reviewers relating to the extracted data will aimed to be resolved through consensus. If the two reviewers are unable to come to a consensus, a third reviewer (SC) will be consulted. If the third reviewer is unable to arbitrate, the study authors will be contacted to seek clarification on the issue. If this step is unsuccessful, the disagreement will be recorded and reported in the review (Taylor, Mahtani & Aronson, 2020). In addition, reported factors will be extracted and categorised into the five domains of the CFIR by one reviewer (EG).

Comment:
Finally, a minor editorial comment: there seems to be a typo: conduct vs contact, in the following phrase: "Any uncertainties regarding data will be resolved by attempting to conduct study authors via email" (pg. 5)".

Response:
Thank you for spotting this error. We have carefully reviewed the manuscript for typographical errors.

Competing Interests:
The authors have no competing interests to declare.
Methods: Authors should consider piloting the data extraction process with two reviewers, and include a second reviewer to validate the data extraction to ensure consistency and comprehensiveness in the data extraction process, and to minimize biases and human error.

5.
Method: It would be good to clarify how authors will handle the studies pre-pandemic versus during pandemic, as the implementation factors may overlap but also differ.

6.
Discussion: It would be good to reduce the repetition in the first paragraph of the discussion with what is written in the introduction section of the paper 7.

topic to help strengthen and better situate this review in relation to what is already known on the topic of implementation factors pertaining to telemedicine -especially given decades of research existing on the topic.
There are a number of reviews related to the current review that could be considered, some that are focused on implementation factors pertaining to specific mental health conditions (e.g., Kruse et al. 20181), others that are focused on implementation factors in telemedicine (that include as part of the included studies those focused on mental health field), and others that review telemedicine in mental health services more broadly that include a component addressing implementation factors.
Response: Thank you for this comment. We acknowledge that previous reviews in the broader area of telemental health have been conducted to examine implementation barriers and facilitators. Given the increase in telemedicine adoption during the pandemic, it is likely that new implementation factors may emerge, or indeed previous challenges and benefits may be emphasised. We have amended the Introduction section to highlight existing reviews, which can be seen in bold below: Response: Thank you for this comment. We chose to focus on live, synchronous methods of telemedicine as these were most commonly used as a replacement for in-person consultations during the pandemic. We chose to focus on these three modalities of telemedicine consultations which often include both phone and video (and sometimes livemessaging) as health care professionals and patients use these interchangeably, and sometimes in combination with each other. In other cases, video consultations may be used without visual input which would have similarities to phone consultations. We have amended the manuscript to provide additional rationale for our decision to focus on these live, synchronous modalities of telemedicine. Please see below the change in bold:

Previous reviews of implementation factors have attributed
These telemedicine consultations were chosen as the focus of this review as these types of consultations became commonplace during the pandemic 1 , acting as a temporary replacement for in-person consultations. consultations that mirror those that were conducted during the pandemic). Secondly, the use of technology has increased in recent years, which means that telemental health before this time point was conducted in a different context and implementation factors may not be relevant to the current technology and health landscape. We have amended the manuscript to include further rationale for this time restriction. Please see the amendment in bold below: We chose not to include studies before this time point has the use of technology has grown dramatically in the past five years and the context in which telemedicine was implemented previously may be largely different to what is relevant in today's context. (pg. 4) Comment: Methods: Involving two reviewers during screening and selection process, and piloting the process to ensure consistency taken in the study selection process is generally considered important to minimize bias and human error. Please clarify why a second reviewer will only be involved in 20% of random sample validation and this only at the stage of full text article retrieval.
Response: Thank you for this comment. The second reviewer will now review 100% of fulltext articles, independent to the first reviewer. The Study selection section of the manuscript has been amended to include this change. Please see the change in bold below: The full texts of all selected articles will be collected and examined by two reviewers, independent of each other (EG and JH). (pg. 4) Comment: Methods: Authors should consider piloting the data extraction process with two reviewers, and include a second reviewer to validate the data extraction to ensure consistency and comprehensiveness in the data extraction process, and to minimize biases and human error.
Response: Thank you for this comment. We agree that independent data extraction from two reviewers would help to improve consistency and comprehensiveness. Due to resource and time constraints, a second reviewer will now extract data items from a random 20% sample of studies to minimize bias. The Data extraction and management section of the manuscript has been amended to reflect this change.
Comment: Method: It would be good to clarify how authors will handle the studies prepandemic versus during pandemic, as the implementation factors may overlap but also differ.
Response: Thank you for this comment. We plan to synthesis all included studies in one analysis. However, we plan to highlight similarities and differences narratively in the Results and Discussion section of the review. We have amended the Data synthesis section of manuscript to clarify this. Please see the change in bold below: Any notable similarities and differences in implementation factors between studies conducted before, and during, the pandemic will be discussed narratively. (pg. 5) useful digital health-and telehealth-related conference proceedings that are peer-reviewed and published and you may end up with an unhelpful bias. Many digital health practitioners attend these conferences and are influenced in their practice by what they learn from the conferences. Conversely, many of them don't have access to journal publications that are behind a subscription paywall. IEEE is a highly regarded conference and their proceedings are often given the same value as journal publications. Telehealth and digital health conferences, such as MedInfo, are published in "Studies in Health Technology and Informatics". It would be a shame to exclude these peer-reviewed papers that influence practice and policy. You plan to do a quality review of the included publications to protect the quality of your review, so you can still report on conference proceedings without compromising the review. If you choose to not take this advice, please indicate why in your final published protocol.
telemedicine, and they provide an excellent definition of the concept. The link between mental health, psychiatry, and mental health professionals is not as clear. Is this review about general mental health and all those healthcare professionals who provide services, or is it focused on psychiatry? It appears that the review is about the former, but the boundaries are blurred in ways that may create confusion.
Response: Thanks for this comment. This review is about general mental health and all those healthcare professionals who provide mental health services. We acknowledge that the various terms (i.e. telepsychiatry) may cause some confusion so we have changed any mention of the term "telepsychiatry" to "telemental health" in the manuscript in an attempt to overcome this ambiguity. Response: Thank you for this comment. We will include studies of healthcare professionals involved in the provision of mental health services via telemedicine. We have amended the Population section of the manuscript to attempt to clarify any ambiguity regarding this inclusion criteria. We have also amended the Population section to include mental health professionals who provide mental health services to people without a formal diagnosis of a mental disorder. Please see amendments to the Population section in bold below: The population will include adults and children (aged < 18 years) with a diagnosis of a mental disorder or in receipt of care from a mental health professional (e.g. psychiatrist, psychotherapist, counsellor Response: Thank you for this comment and for attaching that reference. We chose not to do a scoping review as the scope of the questions is quite narrow, which is more suited to a systematic review. Furthermore, a scoping review is relevant when attempting to understand the initial size and nature of the available literature. This type of methodology is not suited to this review where previous reviews have been conducted in the broader area of telemental health implementation, prior to the pandemic. We also intended to include only empirical research which is more suited to a systematic review design, than a scoping review design. We decided not to conduct a meta-analysis as we are interested in narratively presenting our findings. It is unlikely that a meta-analysis would be possible as the outcome of interest (e.g. implementation factors) does not have standardised outcome measure that would be comparable across studies. We have included additional sentences in the Protocol section of the manuscript outlining our decision. Please see changes in bold below: A systematic review design was chosen to synthesise the available literature. Response: Thank you very much for your comment. We acknowledge that IEEE is a useful resource, and includes peer-reviews papers. We will use this resource to help shape recommendations in the Discussion section of the final review. We acknowledge that excluding conference proceedings may run the risk of excluding potentially relevant studies. We have decided to exclude this "grey literature" because of the variability in quality, peer review, supplication in full-text reviews, and the need for prompt publication of findings. We recognise the potential limitation of excluding these studies and this is reflected in the Discussion section. We will also acknowledge this as a limitation on the final review. We have amended the Study design section of the protocol to clarify why we have excluded grey literature. Please see the amendment in bold below.