The Effectiveness of Telepsychology With Veterans: A Meta-Analysis of Services Delivered by Videoconference and Phone

Abstract

Veterans face a variety of stressors due to their military service and are more likely to develop psychological problems as a result. Research suggests that as many as half of veterans with mental health conditions go untreated due to barriers including lack of accessibility to services and stigma. The present study builds on previous research by using meta-analytic techniques to determine the effectiveness of telepsychology-delivered therapy with veterans. Empirical studies were included if they reported veteran-related outcome data on a psychological intervention used to treat a mental health condition remotely using either videoconferencing or telephone. Twenty-seven studies including 2,648 total participants (1,667 in treatment conditions and 981 in control conditions) met our inclusion criteria and were incorporated into our analysis. Twenty-five studies provided pre-post data to evaluate various therapy outcomes, and 18 studies used a randomized clinical trials (RCTs) design that allowed a comparison between telehealth and traditional in-person therapy. Publication bias was evaluated using correlations between sample and effect sizes for posttraumatic stress disorder (PTSD) and depression for pretest-posttest and RCT designs; risk was determined to be minimal. Weighted average pre-post effect sizes were moderate-to-strong for depression and trauma, and videoconferencing was more effective than telephone for depression (d = 0.86 and 0.46, respectively) and trauma (d = 1.00 and 0.51, respectively). Weighted average effect sizes computed from RCT studies suggest telepsychology is similarly effective as services provided face-to-face. More research is needed for telepsychology-delivered treatments for other mental health conditions faced by veterans. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Full text

Psychological Services
The Effectiveness of Telepsychology With Veterans: A Meta-Analysis of
Services Delivered by Videoconference and Phone
Michael J. McClellan, Richard Osbaldiston, Rongxiu Wu, Rachael Yeager, Addison D. Monroe, Tyler McQueen, and Mattie
Helen Dunlap
Online First Publication, February 4, 2021. http://dx.doi.org/10.1037/ser0000522
CITATION
McClellan, M. J., Osbaldiston, R., Wu, R., Yeager, R., Monroe, A. D., McQueen, T., & Dunlap, M. H. (2021, February 4). The
Effectiveness of Telepsychology With Veterans: A Meta-Analysis of Services Delivered by Videoconference and Phone.
Psychological Services. Advance online publication. http://dx.doi.org/10.1037/ser0000522

The Effectiveness of Telepsychology With Veterans: A Meta-Analysis
of Services Delivered by Videoconference and Phone
Michael J. McClellan
1
, Richard Osbaldiston
1
, Rongxiu Wu
2
, Rachael Yeager
1
,
Addison D. Monroe
3
, Tyler McQueen
1
, and Mattie Helen Dunlap
1
1
Department of Psychology, Eastern Kentucky University
2
Department of Educational, School, and Counseling Psychology, Quantitative and Psychometric
Methods Program, University of Kentucky
3
Department of Counseling and Human Development, University of Georgia
Veterans face a variety of stressors due to their military service and are more likely to develop psychological
problems as a result. Research suggests that as many as half of veterans with mental health conditions go
untreated due to barriers including lack of accessibility to services and stigma. The present study builds on
previous research by using meta-analytic techniques to determine the effectiveness of telepsychology-
delivered therapy with veterans. Empirical studies were included if they reported veteran-related outcome
data on a psychological intervention used to treat a mental health condition remotely using either
videoconferencing or telephone. Twenty-seven studies including 2,648 total participants (1,667 in treatment
conditions and 981 in control conditions) met our inclusion criteria and were incorporated into our analysis.
Twenty-five studies provided pre–post data to evaluate various therapy outcomes, and 18 studies used a
randomized clinical trials (RCTs) design that allowed a comparison between telehealth and traditional in-
person therapy. Publication bias was evaluated using correlations between sample and effect sizes for
posttraumatic stress disorder (PTSD) and depression for pretest–posttest and RCT designs; risk was
determined to be minimal. Weighted average pre–post effect sizes were moderate-to-strong for depression
and trauma, and videoconferencing was more effective than telephone for depression (d=0.86 and 0.46,
respectively) and trauma (d=1.00 and 0.51, respectively). Weighted average effect sizes computed from
RCT studies suggest telepsychology is similarly effective as services provided face-to-face. More research
is needed for telepsychology-delivered treatments for other mental health conditions faced by veterans.
Impact Statement
Only about half of veterans who meet criteria for a mental health condition obtain treatment due to
barriers including service accessibility and embarrassment associated with waiting in line at provider
offices. Telepsychology, specifically videoconferencing and telephone, can help reduce these barriers to
treatment by eliminating some challenges associated with geographic distances to providers, and by
allowing patients to skip office visits. Services delivered by technology are an effective way to alleviate
mental health symptoms in this population and are comparable to services provided face-to-face.
Keywords: telepsychology, meta-analysis, veteran, videoconference, phone
Veterans face a variety of stressors related to their military service
(Institute of Medicine, 2014;Tanielian et al., 2008). Most veterans
who return from military conflict do not have a mental health
condition, but all veterans “experience a period of readjustment
as they reintegrate into life with family, friends, and community”
(Substance Abuse and Mental Health Services Administration
[SAMHSA], 2012,p.1;Tanielian et al., 2008). Those individuals
must juggle a variety of sometimes conflicting family and military-
related responsibilities, learn to readjust to environments where
behaviors associated with a constant state of readiness are no longer
needed, and process combat-related exposure that may collectively
contribute to problems for both veterans and their families (Institute
of Medicine, 2014;SAMHSA, 2012).
Veterans who are exposed to combat are more likely to report
normal reactions to stress including depression, hopelessness,
insomnia, nightmares, feelings of rejection, aggressive behavior,
and substance misuse (SAMHSA, 2012). They also report higher
levels of posttraumatic stress disorder (PTSD), major depression,
generalized anxiety, intermittent explosive disorder, and substance
misuse after returning from conflict (Lazar, 2014;SAMHSA, 2012;
Tanielian et al., 2008). Veterans with PTSD and major depression
Michael J. McClellan https://orcid.org/0000-0003-4094-0999
Richard Osbaldiston https://orcid.org/0000-0002-2212-1781
Addison D. Monroe https://orcid.org/0000-0002-5385-016X
The authors have no conflicts of interest to disclose. The authors thank
Colbey Adair, Logan Burris, Myresha Hinton, and Leslie Richie for their
assistance locating and screening articles for this article.
Correspondence concerning this article should be addressed to Michael
McClellan, 521 Lancaster Avenue, 127 Cammack Building, Department
of Psychology, Eastern Kentucky University, Richmond, KY 40475,
United States. Email: m.mcclellan@eku.edu
1
Psychological Services
© 2021 American Psychological Association
ISSN: 1541-1559 https://doi.org/10.1037/ser0000522
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

are also frequently diagnosed with other comorbid mental health-
related diagnoses, experience elevated death rates from suicide and
homicide, report more work-related problems such as unemploy-
ment, and are more likely to be homeless than the average person
(Lazar, 2014).
Many veterans are not receiving adequate mental health treatment
for these conditions (Lazar, 2014;Tanielian et al., 2008). Lazar
(2014) surveyed veterans who returned from conflict in Iraq and
Afghanistan and found that only 23%–40% with a diagnosable
mental health condition reported receiving help from a provider
in the previous year. Although the percentages of those who
meet criteria for a diagnosis and subsequently seek treatment vary
(23%–40% in the Lazar study vs. 53% in the Tanielian study), it is
clear that many veterans who are returning from conflict are not
receiving adequate treatment for their mental health-related pro-
blems. Collectively, the Lazar (2014) and the Tanielian et al. (2008)
studies suggest that the problem is potentially related to veterans
either not choosing to seek services or a shortage of providers
located near the veteran.
A number of barriers that prevent veterans who are in distress
from seeking mental health services have been presented in the
literature (United States Government Accountability Office [USGAO],
2011;Tanielian et al., 2008). Those barriers include the following:
(a) lack of access to mental health services (e.g., lack of transporta-
tion to a provider, the distance that veterans must travel to obtain
services is too far, etc.), (b) negative stigma associated with utilizing
mental health services (e.g., concerns about how other individuals
will view them if they found out about the veteran seeking treat-
ment), (c) lack of awareness of mental health conditions or treat-
ments such as whether their symptoms rise to the level of a diagnosis
or believe that no treatments are available to address their symp-
toms, and (d) concerns about whether the veteran would qualify for
services through the Veterans Health Administration ([VHA];
Tanielian et al., 2008;USGAO, 2011). Given the consequences
of these barriers (e.g., they serve to prevent veterans from seeking
or obtaining treatment), more work needs to be done to reduce their
impact on veterans.
Telepsychology, or the use of technology to provide mental
health services (American Psychological Association, 2013),
broadly encompasses a variety of direct formats including video-
conferencing, phone, and instant messaging as well as a variety of
indirect formats such as email, self-help apps, or websites (Luxton,
Nelson, et al., 2016). In this article, telepsychology is more nar-
rowly defined as the use of videoconferencing and telephone
technologies to provide mental health services in order to reflect
that the bulk of the available research is conducted using one of these
two mediums. There are also a number of terms presented in the
literature that are interchangeable with the term telepsychology such
as telehealth,telemental health,telebehavioral health,teletherapy,
telepsychotherapy, etc. (Luxton, Nelson, et al., 2016).
Telepsychology is often viewed as a potential solution for
increasing access to treatment, increasing privacy, and reducing
exposure to the negative stigma associated with seeking treatment
for veterans (Chen et al., 2019 [videoconferencing]; Turgoose et al.,
2018 [videoconferencing and phone]). For example, telepsychology
can allow veterans to use an internet-connected device, such as a
computer, in the comfort of their own home or in a local office
setting to meet with their provider who may be located across the
country by videoconference or phone. Participating intelepsychology
increases privacy associated with seeking mental health services by
eliminating the need to sit in a lobby withother veterans while waiting
for a mental health provider appointment. Furthermore, this increased
privacy associated with telepsychology visits can help prevent some
of the negative stigma or embarrassment that veterans might experi-
ence during face-to-face (FTF) visits by allowing them to avoid those
in-office contacts.
Effectiveness of Telepsychology
One question that must be presented to mental health providers
and researchers is whether veteran-focused telepsychology is at least
as effective as treatment that is delivered FTF (Bolton & Dorstyn,
2015 [videoconferencing]; Turgoose et al., 2018 [videoconferenc-
ing and phone]). A growing number of studies have examined this
question with some studies finding that telepsychology is effective
and equivalent to services delivered FTF, whereas other studies have
found mixed results (Turgoose et al., 2018 [videoconferencing and
phone]). Telepsychology has generally been found to be an effective
treatment delivery method for common mental health conditions
experienced by veterans (Bolton & Dorstyn, 2015 [videoconferenc-
ing]; Turgoose et al., 2018 [videoconferencing and phone]) including
the following: PTSD and other trauma and stressor-related disorders
(Wierwille et al., 2016 [videoconferencing]), anxiety (Burke et al.,
2019 [internet-delivered]; Yuen et al., 2013 [videoconferencing]),
depression (Sztein et al., 2018 [internet-only]), intermittent explosive
disorder (Osma et al., 2016 [videoconferencing]), substance use
disorders (Wilks et al., 2017 [internet-only]), etc.
For example, Wierwille et al. (2016) reported clinically signifi-
cant reductions in trauma-related symptom in a sample of 221
veterans with PTSD after videoconferencing-delivered exposure
therapy at posttest. In another study by Luxton, Pruitt, et al. (2016),
121 military service members and veterans were randomly assigned
to receive 8 sessions of behavioral activation therapy for depression
either through home-based videoconferencing or FTF visits. At
posttest, both the home-based telepsychology and FTF participants
demonstrated strong reductions in symptoms of depression and hope-
lessness, with the home-based group demonstrating a slight overall
benefit. Veterans reported statistically significant, high satisfaction
ratings suggesting that they were satisfied with treatment, regardless
of whether the services were provided through home-based video-
conferencing or FTF (Luxton, Nelson, et al., 2016).
As the volume of telepsychology research has grown, review
studies have been better able to answer the question of whether
telepsychology is as effective as FTF services by examining findings
from groups of studies that looked broadly at civilian samples (see
Bolton & Dorstyn, 2015 [videoconferencing]; Sztein et al., 2018
[internet-only]; Turgoose et al., 2018 [videoconferencing and
phone]). For example, Sztein et al. (2018) conducted a civilian-
focused systematic review and meta-analysis with 14 studies that
examined treatment outcomes for telepsychology-delivered cogni-
tive behavioral therapy for depression. The authors reported a
medium effect (d=0.74; p<.001) at posttest with sustained
reductions in symptoms of depression at 3- and 6-month follow-up
in adults, which the authors noted were comparable to effect sizes
commonly observed in FTF samples (Sztein et al., 2018 [internet-
only]). Barak et al.’s (2008) was another civilian-focused meta-analytic
study that broadly looked at mental health services provided through
telepsychology. The authors reviewed 64 articles that included 9,764
2MCCLELLAN ET AL.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

total participants andreported a mean overall medium weighted effect
size of d=0.53 for services delivered through web-based interven-
tions, which the authors noted is also comparable to effect sizes
observed in FTF-delivered talk therapies (Barak et al., 2008). Bolton
and Dorstyn (2015) looked specifically at the impact of videoconfer-
encing-delivered cognitive behavioral therapy on the symptoms of
depression, generalized anxiety, and PTSD in another civilian-
focused systematic review. The authors reported significant medium
to strong improvements (drange =0.66–3.22) in symptoms across
all conditions in both telepsychology and FTF conditions, but was
unable to determine whether telepsychology and FTF-delivered
treatments were equivalent (Bolton & Dorstyn, 2015).
Our research only noted one review study that focused specifically
on veterans. Turgoose et al. (2018) conducted a systematic review that
included 41 studies that focused on the use of telepsychology (i.e.,
videoconferencing and phone) to treat veterans with PTSD. The
authors noted that a majority of the studies reviewed found that
PTSD treatment delivered by telepsychology was just as effective
as PTSD treatments delivered FTF. The authors also noted that
veterans participating in telepsychology reported high levels of
acceptability and treatment satisfaction overall with services delivered
through technology, with some studies suggesting that veterans were
less comfortable with services delivered through technology. The
authors also noted similar dropout rates for telepsychology and
FTF treatments (Turgoose et al., 2018). One limitation of the system-
atic review, however, is that it is descriptive in nature and looks for
trends in the studies reviewed, so it does not allow the author to answer
statistical questions such as whether or not services delivered through
technology are effective or equivalent to services delivered FTF.
The Present Study
Research on the effectiveness of telepsychology is growing, and
both the quantity and quality of veteran-specific studies are expand-
ing (Turgoose et al., 2018). A number of veteran-focused review
studies are noted in the literature and those studies have generally
found telepsychology to be as effective as services delivered FTF
(see Bolton & Dorstyn, 2015;Sztein et al., 2018). Given the unique
lived experiences of veterans and the increasing reliance on services
delivered via telepsychology, more research is needed to determine
whether telepsychology is as effective as services delivered FTF.
The Turgoose et al. (2018) systematic review, which focused on
PTSD, suggests that services delivered through telepsychology are
as effective as services delivered FTF. A new review study,
however, is needed to statistically evaluate the effectiveness of
services provided by telepsychology for veterans. The present study
seeks to add to the literature on the efficacy of telepsychology with
veterans by using meta-analytic techniques to test whether tele-
psychology-based services are effective and whether they are
equivalent to services delivered FTF. Furthermore, as technological
methods of delivering therapy have evolved, the present study also
seeks to examine the differences in the effectiveness of therapy
delivered by videoconferencing versus phone.
Method
Research Question
The overall research question for this meta-analysis is “Is tele-
psychology effective for veteran populations?”There are three
approaches to answering this question. First, is telepsychology
effective across time when comparing pretest assessments to post-
test and follow-up assessments? Second, is telepsychology effective
compared to treatments as usual (TAU)? Furthermore, are there any
differences between the two primary modalities of telepsychology,
videoconferencing and phone?
Inclusion and Exclusion Criteria
Initially, we broadly defined telepsychology as including mental
health services delivered using technology. After reviewing the
available literature, we elected to limit the telepsychology modality
considered for inclusion in this study to synchronous videoconfer-
encing and telephone. The literature on areas such as instant
messaging, email, web apps, etc. with our focus population was
too sparse and did not allow for a robust enough sample to be
considered for inclusion.
We defined veterans as former service members at the time the
study was completed. We included studies that contained veterans
and that met our other search criteria. Other inclusion criteria
included empirical studies that reported findings of a therapeutic
intervention delivered by videoconference or telephone. The inter-
vention had to focus on delivering therapy to treat a psychological
condition, as opposed to other purposes like screening, testing,
assessing, or monitoring. The intervention also had to have a
therapeutic component (e.g., reducing symptoms of depression)
as opposed to being solely educational in nature. The most common
conditions assessed in these studies were PTSD, depression, anger,
and anxiety, although therapy for pain, quality of life, relationship
satisfaction, sleep quality, hopelessness, overall mental health,
psychological functioning, and smoking cessation were also pres-
ent. Finally, the study had to report clinical outcomes of a psycho-
logical nature that included both a pretest and posttest data collection
period. We excluded outcomes concerning feasibility, perceptions,
satisfaction, and costs.
Literature Search
The research team initially looked for meta-analytic, systematic,
and literature review studies for veteran samples to determine the
types of review studies that had previously been completed. Then,
we searched the reference lists of those articles for empirical studies
that met our inclusion criteria. After we exhausted all review studies
and their reference lists, the research team conducted a broader
literature search using various combinations of the following search
terms in the PsycInfo, MedLine, and CINAHL databases: telepsy-
chology, telehealth, telemental health, telebehavioral health, tele-
therapy, telecounseling, tele, phone, videoconferencing, therapy,
counseling, psychotherapy, veterans, treatment, disorder, and men-
tal health. We continued this process of searching databases and
back-searching reference lists until we were confident all of the
relevant published studies had been located.
Computing Effect Sizes
Practically, all outcome measures in this set of studies were
scored such that higher values indicated worse psychological
conditions and lower values indicated better psychological condi-
tions. For example, high scores on the PTSD Checklist (PCL), Beck
TELEPSYCHOLOGY 3
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Depression Inventory (BDI), Clinician-Administered PTSD Scale
(CAPS), Symptom Checklist-90-Revised (SCL), Patient Health
Questionnaire-9 (PHQ), etc. represent higher levels of psychologi-
cal distress, so therapies that lower scores on these measures have
desirable effects. Typically, these situations result in negative effect
sizes (i.e., the posttest scores are lower than the pretest scores).
However, we have reported positive effect sizes to reflect that
psychological conditions improved as a result of the therapy.
There were a few measures wherein higher values indicate better
psychological conditions (e.g., QOLI, relationship quality, and
satisfaction). We used positive effect sizes to indicate improvement
in psychological conditions for these measures.
Although some very good work has been done in developing
appropriate metrics for meta-analyses, the data presented in the
studies we reviewed do not permit that good work to be fully
capitalized upon. Nearly all of the studies that were located reported
pre–post comparisons, and the effect size for pre–post comparisons
is the mean change score divided by the standard deviation of the
change scores. Only one article reported these data (Fortney
et al., 2015).
More commonly, the articles reported the mean pre and post-
scores for the treatment group (and often a comparison group), and
they reported the standard deviation for the pretest scores and the
standard deviation for the posttest scores. Note that the standard
deviation of the change scores is not simply the difference between
the standard deviations of the pretest and posttest scores, nor is it the
average of the standard deviations of the pretest and posttest scores.
Pre–Post Effect Size
Because this was the only data provided, we made use of it by
computing the pre–post effect size (which can also be called the
“within group effect size”or the “uncontrolled effect size”)by
dividing the difference in pretest and posttest mean scores by the
average standard deviation of the pretest and posttest scores.
Randomized Clinical Trials Effect Size
Some studies used a true randomized clinical trial (RCT) design
to allow for the testing of noninferiority or equivalence of tele-
psychology to the same therapy-administered FTF. In these designs,
one group received therapy administered through telepsychology
and a comparison group received the same therapy administered in a
FTF format (the TAU group).
To compute the effect size for these research designs, we com-
puted a pre–post effect size for both the telepsychology group and
the TAU group, and then, we subtracted the TAU group pre–post
effect size from the telepsychology group pre–post effect size.
Multiple Effect Sizes From Studies
Of the 27 studies, 22 of them reported results for more than one
psychological outcome (e.g., Acierno et al., 2016, reported results
using both the Beck Depression Inventory, which we categorized as
depression, and the Patient Checklist, which we categorized as
trauma). Because we analyzed the data by psychological outcome,
each study could be included in the results more than once (e.g., the
Acierno et al., 2016, study contributed an effect size for both trauma
and depression). However, eight studies included two measures for
the same psychological outcome (e.g., Aburizik et al., 2013,
included both the Patient Health Questionnaire and the Beck
Depression Inventory, both of which we categorized as measures
of depression). In these cases, the effect sizes for the two measures of
the same construct were averaged together before being included in
the weighted average calculations. By doing this, we avoided the
problem of double-counting these studies.
Computing Weighted Average Effect Sizes
The goal of meta-analysis is to determine the weighted average
effect size for groups of studies that meet certain conditions.
However, the process for computing the weighted average effect
size differs slightly for the pre–post effect size compared to RCT
designs.
For pre–post effect sizes, the effect size from each study was
weighted by the number of participants who completed the posttest.
For the RCT designs, the effect size of each study was weighted
by the inverse of the variance of the estimate. The inverse of the
variance is highly dependent on the sample size, but it has better
psychometric properties than simply using the sample size of
each study.
Pre–Post and Follow-Up Designs
All of the studies included in this meta-analysis used at least a
pre–post design such that measures of psychological states were
collected twice. Many of the studies also included some sort of
follow-up measurement after the posttest administration. The most
common follow-up measurements were at 3 and 6 months after the
posttest.
In addition to computing the effect sizes for the pre–post com-
parisons, we also computed effect sizes for follow-ups when the data
were provided in the article. The procedure for computing the
follow-up effect sizes was exactly the same as the pre–post effect
size, except that instead of using the posttest mean values and
standard deviations, the follow-up mean values and standard devia-
tions were used.
Results
Descriptive Results of Literature Search
A total of 27 published studies were located that met the inclusion
criteria, and they contained 27 samples (see Table 1 for a summary
of the 27 studies included in the analysis organized by author last
name and dependent variable studied). Of these 27 studies,
25 provided pre–post data, and 18 were RCT designs. (Of the seven
pre–post studies, three also included a no-treatment wait-list control
group, but the wait-list data were not considered.) Of the 27 studies,
19 of them examined more than one psychological outcome
measure.
Computing the number of participants in these studies was
challenging. Not surprisingly, when therapy is being administered
over the course of weeks and months, dropout from the studies is a
problem. Sample sizes for pretest data were often greater than that
for the posttest and follow-up data. Where reported, we used the
sample size for the number of participants who were assessed at the
posttest or follow-up measurements (see Table 1). The 30 samples
used a total of 1,667 participants who received telepsychology
4MCCLELLAN ET AL.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Table 1
Description of Studies Included in the Meta-Analysis Organized by Author and Dependent Variable Studied
Author (year) Design Modality Condition DV
NPre–post dRCT d
ProviderTreat Control Post
3
months
6
months Post
3
months
6
months
Aburizik et al. (2013) PP Phone Depression PHQ 29 0.80 Counselor
Aburizik et al. (2013) PP Phone Depression BDI 29 0.37 Counselor
Acierno et al. (2016) RCT VC Depression BDI 112 121 0.89 Counselor
Acierno et al. (2016) RCT VC Trauma PCL 112 121 −0.11 Counselor
Acierno et al. (2017) RCT VC Trauma PCL 55 43 0.81 0.89 0.89 −0.39 −0.03 0.15 Counselor
Acierno et al. (2017) RCT VC Depression BDI 55 43 0.35 0.42 0.50 −0.43 −0.36 −0.19 Counselor
Carmody et al. (2013) PP Phone Depression BDI 48 0.19 0.50 0.27 Therapist
Carmody et al. (2013) PP Phone Pain PBCL 48 0.26 0.50 0.32 Therapist
Clapp et al. (2016) RCT Phone Trauma PCL 35 74 0.00 0.00 Psychologist, social workers, predoctoral
psychology interns
Egede et al. (2015) RCT VC Depression BDI 120 121 −0.34 −0.34 Counselor
Egede et al. (2015) RCT VC Depression GDS 120 121 −0.20 −0.20 Counselor
Fortney et al. (2015) PP VC Trauma PDS 133 0.49 Psychologist, psychiatrist, nurse
Fortney et al. (2015) PP VC Depression SCL 133 0.69 Psychologist, psychiatrist, nurse
Fortney et al. (2015) PP VC Mental Health MCS 133 0.22 Psychologist, psychiatrist, nurse
Frueh et al. (2007) RCT VC Trauma PCL 17 21 −0.11 0.46 −0.62 0.30 Therapist, psychiatrist
Frueh et al. (2007) RCT VC Depression BDI 17 21 −0.27 −0.36 −0.18 0.04 Therapist, psychiatrist
Frueh et al. (2007) RCT VC PF SCL 17 21 −0.24 −0.01 −0.48 −0.16 Therapist, psychiatrist
Gehrman et al. (2016) PP VC Sleep ISI 214 0.88 No data provided
Glassman et al. (2019) RCT VC QOL QOLI 60 60 0.29 0.26 0.16 −0.02 −0.27 0.01 Therapists
Gros et al. (2011) RCT VC Trauma PCL 62 27 1.21 −1.79 Therapists
Gros et al. (2011) RCT VC Depression BDI 62 27 1.14 −1.01 Therapists
Herbert et al. (2017) RCT VC Pain BPI 42 42 0.69 0.77 −0.05 0.08 Counselor
Maieritsch et al. (2016) RCT VC Trauma CAPS 25 26 1.38 −0.12 Psychologists, social workers
Maieritsch et al. (2016) RCT VC Trauma PCL 25 26 1.02 −0.42 Psychologists, social workers
Maieritsch et al. (2016) RCT VC Depression BDI 25 26 0.81 −0.37 Psychologists, social workers
Mohr et al. (2011) RCT Phone Depression HAMD 38 35 1.14 1.40 0.67 0.38 Psychologist
Mohr et al. (2011) RCT Phone Depression PHQ 38 35 1.03 1.00 0.14 0.01 Psychologist
Morland et al. (2010) RCT VC Anger STAXI 61 64 0.97 0.83 0.84 0.19 0.39 0.52 Psychologist
Morland et al. (2010) RCT VC Anger NAST 61 64 0.86 0.75 0.64 0.18 0.47 0.16 Psychologist
Morland et al. (2010) RCT VC Trauma PCL 61 64 0.40 −0.23 Psychologist
Morland et al. (2011) RCT VC Trauma CAPS 7 6 3.38 1.88 Therapist
Morland et al. (2014) RCT VC Trauma CAPS 48 52 0.98 1.09 0.97 0.38 0.40 0.29 Psychologist, social worker
Niles et al. (2012) PP Phone Trauma PCL 13 0.85 Psychologists
Rogers et al. (2016) PP Phone Smoking Cigs/day 270 0.20 Counselor
Silberbogen et al. (2012) PP Phone Depression BDI 8 0.35 Psychologists
Silberbogen et al. (2012) PP Phone Anxiety STAI 8 0.21 Psychologists
Silberbogen et al. (2012) PP Phone Mental health SF-36 V 8 0.23 Psychologists
Strachan et al. (2012) RCT VC Trauma PCL 18 13 0.98 0.30 Therapist
Strachan et al. (2012) RCT VC Depression BDI 18 13 0.51 −0.01 Therapist
Strachan et al. (2012) RCT VC Anxiety BAI 12 11 0.52 0.22 Therapist
Tuerk et al. (2010) RCT VC Trauma PCL 9 29 2.87 −1.39 Psychologist
Tuerk et al. (2010) RCT VC Depression BDI 9 29 2.29 0.14 Psychologist
Weiss et al. (2018) PP VC Trauma PCL 10 1.21 Psychologist, social worker
Weiss et al. (2018) PP VC Depression 10 0.93 Psychologist, social worker
Whealin et al. (2017) PP VC Relationship Quality 28 0.54 Therapist
(table continues)
TELEPSYCHOLOGY 5
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

(e.g., videoconferencing or telephone) and 981 participants who
received FTF therapy in RCT designs.
Publication Bias Assessment
Publication bias is a concern in meta-analysis because it is
possible that only studies that show strong effects get published.
When the conclusions of meta-analytic studies are based only on the
studies with strong effect sizes, the outcome of the meta-analysis is
biased.
There is no definitive method for assessing publication bias, and
there is also no definitive procedure for correcting for it. Of the
assessments that have been developed, all have limitations (van Aert
et al., 2019). However, one general assessment of publication bias is
to compute the correlation between sample size and effect size. If
this correlation is significant and negative, a tentative inference is
that smaller studies are only published when they have stronger
effects, resulting in bias in the meta-analytic conclusions.
We computed the correlations between sample size and effect size
for RCT designs measuring trauma and depression at posttest. These
correlations are r=−.17 and .11 for k=13 and 12 studies, p=.58
and.73, respectively.
Similarly, we computed the correlations between sample size and
effect size for all of the pre–post effect sizes for both trauma and
depression. These correlations are r=−.34 and −.20 for k=17
and 17 studies, p=.19 and .44, respectively.
Although these nonsignificant correlations do not establish that
there is no bias in our results, they are an indication that publication
bias is not likely to be a strong influence. For psychological
conditions other than trauma and depression, and for follow-up
assessment months after the posttest, there were simply not enough
effect sizes to make it meaningful to compute the correlations.
Results of Meta-Analysis
Recall that we examined two different designs: pre–post and
RCT. Most of the RCT designs also provided data that could be
analyzed for pre–post analyses. To interpret the weighted average
effect sizes, we used the guidelines suggested by Cohen (1988)
wherein effect sizes in the range of 0.20, 0.50, and 0.80 are
considered weak, moderate, and strong effects, respectively.
For the pre–post designs, the results for each type of psychologi-
cal problem are reported in Table 2, which shows the number of
studies, the total number of participants in those studies, the
weighted average effect size, and whether the study involved
videoconferencing or phone. These results show the effect of
telepsychology over time, without regards to any comparison group.
Most of the effect sizes are in the moderate-to-strong range, and
given that most of them are positive numbers, this result indicates
that participants’psychological well-being improved as a result of
receiving services through telepsychology.
For example, the two psychological conditions studied most
frequently were trauma and depression. We used the guidelines
suggested by Cohen (1988) to interpret the weighted average effect
sizes. The average weighted effect sizes for trauma (d=1.00 for
videoconferencing and d=0.51 for phone) and depression
(d=0.86 for videoconferencing and d=0.44 for phone) across
these studies fell into the moderate-to-strong range, and the positive
values of the effect sizes indicate that participants’psychological
Table 1 (continued)
Author (year) Design Modality Condition DV
NPre–post dRCT d
ProviderTreat Control Post
3
months
6
months Post
3
months
6
months
Whealin et al. (2017) PP VC Relationship Satisfact 28 0.40 Therapist
Wierwille et al. (2016) RCT Phone Trauma PCL 85 136 0.67 −0.49 Certified providers
Wierwille et al. (2016) RCT Phone Depression BDI 85 136 0.25 −0.26 Certified providers
Yuen et al. (2015) RCT VC Trauma CAPS 29 23 1.63 −0.37 Therapist
Yuen et al. (2015) RCT VC Trauma PCL 29 23 1.27 0.10 Therapist
Yuen et al. (2015) RCT VC Depression BDI 29 23 1.15 0.60 Therapist
Yuen et al. (2015) RCT VC Anxiety BAI 29 23 0.87 0.25 Therapist
Note. BDI =Beck Depression Inventory; BAI =Beck Anxiety Inventory; CAPS =Clinician-Administered PTSD Scale; Cigs/Day =cigarettes per day; GDS =Geriatric Depression Scale; ISI =Insomnia
Severity Index; MCS =Mental Component Summary; NAST =Novaco Anger Scale Total Score; PBCL =Problem Behavior Checklist; PCL =PTSD Checklist; PDS =Posttraumatic Diagnostic Scale; PF =
Patient Functioning; PHQ =Patient Health Questionnaire-9; PP =pre-post; QOL=Quality of Life; QOLI =Quality of Life Inventory; RCT =randomized controlled trial; SCL =Symptom Checklist –90 –
Revised; SF-36V =Short Form-36V, Mental Health Subscale; STAI =State-Trait Anxiety Inventory; STAXI =State-Trait Anger Expression Inventory-2; VC =videoconferencing.
6MCCLELLAN ET AL.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

symptoms of trauma and depression improved from pretest to
posttest. It is important to note that this is just improvement over
time in the participants, and in this type of research design, there is
no control or comparison group. Note that there is no way to
compute the confidence intervals for these values because the
formula for the confidence intervals depends on having the standard
deviation of the difference scores, and this information has not been
reported in the studies we located for this meta-analysis.
For RCT designs, the results for each type of psychological
problem are reported in Table 3, which shows the number of studies,
the total number of participants in those studies, the weighted
average effect size, the 95% confidence interval for the weighted
average effect size, and whether the study involved videoconfer-
encing or phone. These results show the effect of telepsychology
over time when compared to an equivalent FTF therapy. Strictly
speaking, these are the results of the highest scientific value.
Paradoxically, values close to 0.00 are a good thing; they indicate
that telepsychology was as effective as services provided FTF.
Positive values indicate that telepsychology was more effective
than FTF therapy, and correspondingly, negative values indicate
that FTF therapy was more effective than telepsychology.
For example, consider the values for RCT studies for trauma and
depression shown in Table 3. The weighted average effect sizes for
trauma are d=−0.25 for videoconferencing and d=−0.34 for
phone. The negative sign indicates that participants in the FTF
condition improved over time more than those in the telepsychology
condition. However, this is a small effect, and thus, it could be
practically interpreted as telepsychology is comparably as effective
as FTF treatment. Similarly, for depression, the weighted average
effect sizes are d=0.00 for videoconferencing and d=−0.09 for
phone. Again, the negative sign indicates that the FTF group fared
better than the telepsychology group. But, here again, the effect is
very small; it is quite close to 0.00. For practical purposes, tele-
psychology was equivalent to FTF treatment.
Discussion
Overall, the results of this meta-analysis suggest that telepsychol-
ogy is comparable to FTF therapy for veterans dealing with a variety
of psychological conditions. The pre–post comparisons suggest that
videoconferencing-based telepsychology has moderate-to-strong
effect sizes and telephone-based telepsychology has moderate effect
sizes for the two psychological conditions studied most frequently
(e.g., trauma and depression).
The RCT designs allow for a more powerful conclusion to be
drawn. In general, videoconferencing- and telephone-based tele-
psychology is practically equivalent to and FTF-delivered therapies.
The effect sizes comparing telepsychology to FTF therapies are
generally weak, indicating that there is little difference in the two
modalities.
Implications
The present study is also unique in that it compared the two
most frequently utilized modes of telepsychology, videoconferenc-
ing, and phone. It also extends the work of Turgoose et al. (2018)
who described the literature on the effectiveness and equivalence
Table 2
Summary Statistics for Various Mental Health Conditions in Meta-Analysis at Posttest, and 3- and 6-Month Following Treatment for
Pre–Post Studies
Condition Modality
Posttest 3 months 6 months
Nk d Nk d Nkd
Anger VC 61 1 0.91 61 1 0.79 61 1 0.74
Phone
Anxiety VC 41 2 0.77
Phone 1 8 0.21
Depression VC 270 9 0.86 114 3 0.56 188 2 0.64
Phone 208 5 0.44 48 1 0.50 86 2 0.68
Hopelessness VC 45 1 0.84 42 1 0.74
Phone
Mental health VC 133 1 0.22
Phone 8 1 0.23
Pain VC 42 1 0.69 42 1 0.77
Phone 48 1 0.26 48 1 0.50 48 1 0.32
Psy. Funct. VC 17 1 −0.24 17 1 −0.01
Phone
QoL VC 60 1 0.29 60 1 0.26 60 1 0.16
Phone
Relationship VC 28 1 0.47
Phone
Sleep VC 214 1 0.88
Phone
Smoking VC
Phone 270 1 0.20
Trauma VC 341 11 1.00 118 3 0.91 232 3 0.67
Phone 133 3 0.51
Note.N=Total number of participants in included studies; k=Number of samples; d=Weighted average Cohen’sdvalue; Psy. Funct. =Psychological
Functioning; QoL =Quality of Life; VC =Videoconferencing.
TELEPSYCHOLOGY 7
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

of telepsychology on veterans and adds to the larger body of
meta-analytic studies on the effectiveness and equivalence of tele-
psychology when compared to FTF treatment by applying the meta-
analytic approach to veteran-related studies. Turgoose et al.’s (2018)
systematic review noted that a majority of veteran-focused studies
have found telepsychology to be an effective way of reducing
psychological symptoms. That study also indicated that most studies
have found that talk therapies delivered to veterans by telepsychology
are as effective at reducing mental health symptoms as services that
are delivered FTF (Turgoose et al., 2018). Our study extends Tur-
goose et al.’sfindings by using meta-analytic techniques to quantita-
tively assess and subsequently support their conclusions related to the
effectiveness and equivalence of telepsychology. In addition, to our
knowledge, our study represents the first meta-analytic study to
evaluate the effectiveness and equivalence of telepsychology at
reducing mental health symptoms exclusively in veteran samples.
In our study, we found that services delivered by telepsychology
had a moderate-to-strong effect of reducing symptoms of PTSD and
depression in veterans. This finding is similar to the effect sizes
reported by other authors who have looked more broadly at the
effect sizes of telepsychology for PTSD and depression (Bolton &
Dorstyn, 2015;Coughtrey & Pistrang, 2018;Olthuis et al., 2016;
Osenbach et al., 2013). For example, Bolton and Dorstyn (2015)
reported findings that were similar to ours. Those authors found
moderate-to-strong effects (drange =0.66–3.22) for reductions in
cognitive and behavioral symptoms of PTSD, depression, and
generalized anxiety in a meta-analysis that included 11 studies
and 474 total participants (Bolton & Dorstyn, 2015). Olthuis
et al. (2016) reported similar effects for telepsychology for
PTSD, d=0.81. Collectively, these studies suggest that telepsy-
chology-delivered services are an effective way of reducing symp-
toms of PTSD and depression.
Our findings related to the equivalence of telepsychology com-
pared to services delivered FTF with veterans was also comparable
to that noted in previous research (Norwood et al., 2018;Osenbach
et al., 2013). In the present study, we found services delivered
via telepsychology to be equivalent to services delivered FTF
(drange =−0.48 to 0.24). Norwood et al. (2018) reported a sim-
ilar value of dof −0.03 when examining the noninferiority of
telepsychology at reducing target mental health symptoms. And
Osenbach et al. (2013) reported a dof −0.11 when comparing
telepsychology to FTF treatments and a dof 0.29 when comparing
telepsychology to TAU comparison groups.
Limitations
Two limitations are noted for the present study, as well as three
limitations for the larger body of literature we reviewed. First, in the
present study, we were able to locate a sufficient number of studies
to assess the effectiveness as well as the equivalence of telepsychol-
ogy versus FTF for PTSD and depression, especially at posttest for
the pre–post and RCT studies (ranged from 12 to 17 studies per
analysis). However, the number of studies available to us that
examined other diagnostic conditions (e.g., anxiety, pain, sleep
disorders, etc.) was minimal. We considered removing any analysis
of those diagnostic conditions from our study; however, we elected
to include them for documentation purposes to highlight what is
currently known. Ideally, we would have a minimum of five studies
for each diagnostic condition; we were simply unable to find enough
studies for all of the conditions.
Second, in the present study, just as we were only able to find a
limited number of studies for all of the diagnostic conditions, studies
with longitudinal data were limited. For example, we were able to
locate 13 RCT studies that provided results at posttest for PTSD;
however, only 4 of those studies provided 3-month follow-up data,
and only 2 of those studies provided 6-month follow-up data. This
lack of longitudinal data on the effectiveness of telepsychology for
each of the diagnostic conditions means that our findings reported
Table 3
Summary Statistics for Various Mental Health Conditions in Meta-Analysis at Posttest and at 3- and 6-Month Following Treatment for
Randomized Controlled Trial Studies
Condition
Posttest 3 months 6 months
95% CI 95% CI 95% CI
Mod Nk d LL UL Nk d LL UL Nk d LL UL
Anger VC 61 1 0.18 −0.17 0.54 61 1 0.43 0.07 0.78 61 1 0.34 −0.01 0.69
Phone
Anxiety VC 41 2 0.24 −0.21 0.70
Phone
Depression VC 372 10 0.00 −0.14 0.13 226 5 0.20 0.04 0.35 55 1 −0.19 −0.59 0.21
Phone 123 2 −0.09 −0.33 0.14 38 1 0.20 −0.26 0.66
Hopelessness VC 45 1 −0.23 −0.65 0.19 42 1 −0.06 −0.50 0.39
Phone
Pain VC 42 1 -0.05 -0.48 0.38 42 1 0.08 -0.35 0.50
Phone
Psy. Funct. VC 17 1 −0.48 −1.13 0.17 17 1 −0.16 −0.80 0.48
Phone
QoL VC 60 1 −0.02 −0.38 0.34 60 1 −0.27 −0.63 0.09 60 1 0.01 −0.35 0.36
Phone
Trauma VC 443 11 −0.25 −0.39 −0.11 230 4 −0.66 −0.86 −0.46 99 2 0.21 −0.08 0.50
Phone 120 2 −0.34 −0.56 −0.11
Note. CI =Confidence Interval; N=Total number of participants in included studies; k=Number of samples; d=Weighted average Cohen’sdvalue;
LL =Lower Limit; UL =Upper Limit; VC =videoconferencing.
8MCCLELLAN ET AL.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

effect sizes for 3- and 6-month follow-up that are based on limited
data. Ideally, our analysis would include a minimum of five studies
for each diagnostic condition at 3- and 6-month follow-up.
We also noted three limitations related to our review of the larger
body of veteran-focused telepsychology literature. First, more RCTs
are needed for all diagnostic conditions, especially for diagnostic
conditions other than PTSD and depression. For example, we noted
no RCTs that targeted substance use disorders and only one or two
studies that addressed symptoms of anxiety or pain. To some extent,
this makes sense because PTSD and depression are two of the most
common mental health problems that veterans experience. How-
ever, veterans face a number of mental health problems that tele-
psychology might be helpful in addressing.
Second, very few of the veteran-focused telepsychology studies
we identified provided demographic or outcome data on the race,
gender, or geographic makeup of their participants. These data could
be helpful in determining whether traditionally underserved veteran
subpopulations, such as rural, women, and/or veterans of color
populations, for example, are experiencing similar levels of efficacy
compared to the average participant.
Finally, we noted few veteran-focused telepsychology studies
that used samples outside of North America, and especially the
United States. Are telepsychology-delivered services as effective as,
and are those services equivalent to FTF-delivered treatments that
are provided to veterans in other areas of the world?
Future Directions
Based on the limitations noted above, both for the present study as
well as the larger body of telepsychology-focused research, we offer
six suggestions for the future research. First, more studies, especially
RCTs that look at diagnostic conditions other than PTSD and
depression, are needed. Individual studies in the larger body of
telepsychology literature (i.e., veteran and all other populations)
have looked at the efficacy of telepsychology for psychological
disorders ranging from substance use disorders, anxiety, and obses-
sive compulsive disorder (OCD) to more chronic forms of mental
illness including bipolar disorder and schizophrenia. Similar or
enhanced versions of these studies should be considered for veter-
ans. Second, future studies should include more follow-up data on
research participants. In the present study, follow-up data were most
commonly presented at posttest and then at 3- and 6-month post-
treatment. Future studies that include this type of follow-up data
would allow us to better understand the long-term efficacy of
telepsychology with veterans. Third, future studies should include
technology modality as a moderator variable to allow us to better
understand whether specific types of telepsychology-based services
(e.g., videoconferencing vs. telephone) are more effective with
veterans than other modalities.
Fourth, too few veteran-focused studies related to web apps,
email, computer-only, and text messaging or instant chat to consider
adding them to the present study. As this body of literature expands,
more work needs to be done to better understand the role of these
other, less often researched modalities of telepsychology to verify
whether and/or under what circumstances these other technologies
can be effective methods of providing services using meta-analytic
techniques. Fifth, more effort needs to be made to describe sample
characteristics such as the race, gender, and geographic location of
veteran research participants as well as outcome data on specific
veteran subpopulations (e.g., veterans of color). Including these
sample characteristics and providing outcome data on these special
populations will help future researchers better assess whether the
benefits of telepsychology are being experienced by all veterans.
Finally, the future research should focus on studying the efficacy of
veteran populations outside of North America. Generally, more
work must be done to close the gap between veterans who need
mental health services and those who receive these services. The
present study suggests that telepsychology, especially videoconfer-
encing, and phone whenever videoconferencing is not available, can
play an important role in addressing mental health service disparities
for veterans.
References
References marked with an asterisk indicate studies included in the meta-
analysis.
*Aburizik, A., Dindo, L., Kaboli, P., Charlton, M., Dawn, K., & Turvey, C.
(2013). A pilot randomized controlled trial of depression and disease
management program delivered by phone. Journal of Affective Disorders,
151(2), 769–774. https://doi.org/10.1016/j.jad.2013.06.028
*Acierno, R., Gros, D. F., Ruggiero, K. J., Hernandez-Tejada, M. A., Knapp,
R. G., Lejuez, C. W., Muzzy, W., Frueh, C. B., Egede, L. E., & Tuerk, P.
W. (2016). Behavioral activation and therapeutic exposure for posttrau-
matic stress disorder: A noninferiority trial of treatment delivered in person
versus home-based telehealth. Depression and Anxiety,33(5), 415–423.
https://doi.org/10.1002/da.22476
*Acierno, R., Knapp, R., Tuerk, P., Gilmore, A. K., Lejuez, C., Ruggiero, K.,
Muzzy, W., Edede, L., Hernandez-Tejada, M. A., & Foa, E. B. (2017). A
non-inferiority trial of prolonged exposure for posttraumatic stress disor-
der: In person versus home-based telehealth. Behaviour Research and
Therapy,89(2), 57–65. https://doi.org/10.1016/j.brat.2016.11.009
American Psychological Association. (2013). Guidelines for the practice of
telepsychology. American Psychologist,68(9), 791–800. https://doi.org/
10.1037/a0035001
Barak, A., Hen, L., Boniel-Nissim, M., & Shapira, N. (2008). A compre-
hensive review and a meta-analysis of the effectiveness of internet-based
psychotherapeutic interventions. Journal of Technology in Human Ser-
vices,26(2-4), 109–160. https://doi.org/10.1080/15228830802094429
Bolton, A., & Dorstyn, D. (2015). Telepsychology for posttraumatic stress
disorder: A systematic review. Journal of Telemedicine and Telecare,
21(5), 254–267. https://doi.org/10.1177/1357633X15571996
Burke, J., Richards, D., & Timulak, L. (2019). Helpful and hindering events
in internet-delivered cognitive behavioural treatment for generalized
anxiety. Behavioural and Cognitive Psychotherapy,47(3), 386–399.
https://doi.org/10.1017/S1352465818000504
*Carmody, T. P., Duncan, C. L., Huggins, J., Solkowitz, S. N., Lee, S. K.,
Reyes, N., Mozgai, S., & Simon, J. A. (2013). Telephone-delivered
cognitive-behavioral therapy for pain management among older military
veterans: A randomized trial. Psychological Services,10(3), 265–275.
https://doi.org/10.1037/a0030944
Chen, C. K., Palfrey, A., Shreck, E., Silvestri, B., Wash, L., Nehrig, N., Baer,
A. L., Schneider, J. A., Ashkenazi, S., Sherman, S. E., & Chodosh, J.
(2019). Implementation of telemental health (TMH) psychological ser-
vices for rural veterans at the VA New York Harbor healthcare system.
Psychological Services. Advance online publication. https://doi.org/10
.1037/ser0000323
*Clapp, J. D., Kemp, J. J., Cox, K. S., & Tuerk, P. W. (2016). Patterns of
change in response to prolonged exposure: Implications for treatment
outcome. Depression and Anxiety,33(9), 807–815. https://doi.org/10
.1002/da.22534
Cohen, J. (1988). Statistical power analysis for the behavioral sciences
(2nd ed.). Lawrence Erlbaum Associates.
TELEPSYCHOLOGY 9
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Coughtrey, A. E., & Pistrang, N. (2018). The effectiveness of telephone-
delivered psychological therapies for depression and anxiety: A systematic
review. Journal of Telemedicine and Telecare,24(2), 65–74. https://
doi.org/10.1177/1357633X16686547
*Egede, L. E., Acierno, R., Knapp, R. G., Lejuez, C., Hernandez-Tejada, M.,
Payne, E. H., & Frueh, B. C. (2015). Psychotherapy for depression in older
veterans via telemedicine: A randomized, open-label, non-inferiority trial.
The Lancet Psychiatry,2(8), 693–701. https://doi.org/10.1016/S2215-
0366(15)00122-4
*Fortney, J. C., Pyne, J. M., Kimbrell, T. A., Hudson, T. J., Robinson, D. E.,
Schneider, R., Moore, W. M., Custer, P. J., Grubbs, K. M., & Schnurr, P. P.
(2015). Telemedicine-based collaborative care for posttraumatic stress
disorder: A randomized clinical trial. JAMA Psychiatry,72(1), 58–67.
https://doi.org/10.1001/jamapsychiatry.2014.1575
*Frueh, B. C., Monnier, J., Yim, E., Grubaugh, A. L., Hamner, M. B., &
Knapp, R. G. (2007). A randomized trial of telepsychiatry for post-
traumatic stress disorder. Journal of Telemedicine and Telecare,13(3),
142–147. https://doi.org/10.1258/135763307780677604
*Gehrman, P., Shah, M. T., Miles, A., Kuna, S., & Godleski, L. (2016).
Feasibility of group cognitive-behavioral treatment of insomnia delivered
by clinical video telehealth. Telemedicine and e-Health,22(12),
1041–1046. https://doi.org/10.1089/tmj.2016.0032
*Glassman, L. H., Mackintosh, M.-A., Talkovsky, A., Wells, S. Y., Walter,
K. H., Wickramasinghe, I., & Morland, L. A. (2019). Quality of life
following treatment for PTSD: Comparison of videoconferencing and in-
person modalities. Journal of Telemedicine and Telecare,25(2), 123–127.
https://doi.org/10.1177/1357633X17740610
*Gros, D. F., Yoder, M., Tuerk, P. W., Lozano, B. E., & Acierno, R. (2011).
Exposure therapy for PTSD delivered to veterans via telehealth: Predictors
of treatment completion and outcome and comparison to treatment
delivered in person. Behavior Therapy,42(2), 276–283. https://doi.org/
10.1016/j.beth.2010.07.005
*Herbert, M. S., Afari, N., Liu, L., Heppner, P., Rutledge, T., Williams, K.,
Eraly, S., VanBuskirk, K., Nguyen, C., Bondi, M., Atkinson, J. H.,
Golshan, S., & Wetherell, J. L. (2017). Telehealth versus in-person
acceptance and commitment therapy for chronic pain: A randomized
noninferiority trial. The Journal of Pain,18(2), 200–211. https://
doi.org/10.1016/j.jpain.2016.10.014
Institute of Medicine. (2014). Understanding psychological health in the
military. In L. A. Denning, M. Meisnere, & K. E. Warner (Eds.),
Preventing psychological disorders in service members and their families:
An assessment of programs. The National Academies Press. https://
doi.org/10.17226/18597
Lazar, S. G. (2014). The mental health needs of military service members and
veterans. Psychodynamic Psychiatry,42(3), 459–478. https://doi.org/10
.1521/pdps.2014.42.3.459
Luxton, D. D., Nelson, E. L., & Maheu, M. M. (2016). A practictioner’s
guide to telemental health: How to conduct legal, ethical, and evidence-
based telepractice. American Psychological Association. https://doi.org/
10.1037/14938-000
*Luxton, D. D., Pruitt, L. D., Wagner, A., Smolenski, D. J., Jenkins-
Guarnieri, M. A., & Gahm, G. (2016). Home-based telebehavioral health
for U.S. military personnel and veterans with depression: A randomized
controlled trial. Journal of Consulting and Clinical Psychology,84(11),
923–934. https://doi.org/10.1037/ccp0000135
*Maieritsch, K. P., Smith, T. L., Hessinger, J. D., Ahearn, E. P., Eickhoff, J.
C., & Zhao, Q. (2016). Randomized controlled equivalence trial com-
paring videoconference and in person delivery of cognitive processing
therapy for PTSD. Journal of Telemedicine and Telecare,22(4),
238–243. https://doi.org/10.1177/1357633X15596109
*Mohr, D. C., Carmody, T., Erickson, L., Jin, L., & Leader, J. (2011).
Telephone-administered cognitive behavioral therapy for veterans served
by community-based outpatient clinics. Journal of Consulting and Clini-
cal Psychology,79(2), 261–265. https://doi.org/10.1037/a0022395
*Morland, L. A., Greene, C. J., Rosen, C. S., Foy, D., Reilly, P., Shore, J.,
He, Q., & Frueh, C. (2010). Telemedicine for anger management therapy
in a rural population of combat veterans with posttraumatic stress disorder:
A randomized noninferiority trial. The Journal of Clinical Psychiatry,
71(7), 855–863. https://doi.org/10.4088/JCP.09m05604blu
*Morland, L. A., Hynes, A. K., Mackintosh, M.-A., Resick, P. A., & Chard,
K. M. (2011). Group cognitive processing therapy delivered to veterans
via telehealth: A pilot cohort. Journal of Traumatic Stress,24(4), 465–
469. https://doi.org/10.1002/jts.20661
*Morland, L. A., Mackintosh, M.-A., Greene, C. J., Rosen, C. S., Chard, K.
M., Resick, P., & Frueh, C. (2014). Cognitive processing therapy for
posttraumatic stress disorder delivered to rural veterans via telemental
health: A randomized noninferiority clinical trial. The Journal of Clinical
Psychiatry,75(5), 470–476. https://doi.org/10.4088/JCP.13m08842
*Niles, B. L., Klunk-Gillis, J., Ryngala, D. J., Silberbogen, A. K., Paysnick,
A., & Wolf, E. J. (2012). Comparing mindfulness and psychoeducation
treatments for combat-related PTSD using a telehealth approach. Psycho-
logical Trauma: Theory, Research, Practice, and Policy,4(5), 538–547.
https://doi.org/10.1037/a0026161
Norwood, C., Moghaddam, N. G., Malins, S., & Sabin-Farrell, R. (2018).
Working alliance and outcome effectiveness in videoconferencing psy-
chotherapy: A systematic review and noninferiority meta-analysis. Clinical
Psychology & Psychotherapy,25(6), 797–808. https://doi.org/10.1002/
cpp.2315
Olthuis, J. V., Watt, M. C., Bailey, K., Hayden, J. A., & Stewart, S. H. (2016).
Therapist-supported internet cognitive behavioural therapy for anxiety
disorders in adults. Cochrane Database of Systematic Reviews. Advance
online publication. https://doi.org/10.1002/14651858.CD011565.pub2
Osenbach, J. E., O’Brien, K. M., Mishkind, M., & Smolenski, D. J. (2013).
Synchronous telehealth technologies in psychotherapy for depression: A
meta-analysis. Depression and Anxiety,30(11), 1058–1067. https://
doi.org/10.1002/da.22165
Osma, J., Crespo, E., & Castellano, C. (2016). Multicomponent cognitive-
behavioral therapy for intermittent explosive disorder by videoconferenc-
ing: A case study. Anales de Psicología,32(2), 424–432. https://doi.org/10
.6018/analesps.32.2.211351
*Rogers, E. S., Smelson, D. A., Gillespie, C. C., Elbel, B., Poole, S.,
Hagedorn, H. J., Kalman, D., Krebs, P., Fang, Y., Wang, B., & Sherman,
S. E. (2016). Telephone smoking-cessation counseling for smokers in
mental health clinics: A patient-randomized controlled trial. American
Journal of Preventive Medicine,50(4), 518–527. https://doi.org/10.1016/j
.amepre.2015.10.004
*Silberbogen, A. K., Ulloa, E., Mori, D. L., & Brown, K. (2012). A telehealth
intervention for veterans on antiviral treatment for the hepatitis c virus.
Psychological Services,9(2), 163–173. https://doi.org/10.1037/a0026821
*Strachan, M., Gros, D. F., Ruggiero, K. J., Lejuez, C. W., & Acierno, R.
(2012). An integrated approach to delivering exposure-based treatment for
symptoms of PTSD and depression in OIF/OEF veterans: Preliminary
findings. Behavior Therapy,43(3), 560–569. https://doi.org/10.1016/j
.beth.2011.03.003
Substance Abuse and Mental Health Services Administration. (2012,
August). Behavioral health issues among Afghanistan and Iraq U.S.
war veterans (Report No. SMA12-4670). United States Department of
Health and Human Services. https://store.samhsa.gov/system/files/sma12-
4670.pdf
Sztein, D. M., Koransky, C. E., Fegan, L., & Himelhoch, S. (2018). Efficacy
of cognitive behavioural therapy delivered over the internet for depressive
symptoms: A systematic review and meta-analysis. Journal of Telemedi-
cine and Telecare,24(8), 527–539. https://doi.org/10.1177/
1357633X17717402
Tanielian, T., Jaycox, L. H., Schell, T. L., Marshall, G. N., Burnam, A.,
Eibner, C., Karney, B. R., Ringel, J. S., & Vaiana, M. E. (2008). Invisible
wounds: Mental health and cognitive care needs of America’s returning
veterans. RAND Corporation. https://doi.org/10.7249/RB9336
10 MCCLELLAN ET AL.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

*Tuerk, P. W., Yoder, M., Ruggiero, K. J., Gros, D. F., & Acierno, R. (2010).
A pilot study of prolonged exposure therapy for posttraumatic stress
disorder delivered via telehealth technology. Journal of Traumatic Stress,
23(1), 116–123. https://doi.org/10.1002/jts.20494
Turgoose, D., Ashwick, R., & Murphy, D. (2018). Systematic review
of lessons learned from delivering tele-therapy to veterans with post-
traumatic stress disorder. Journal of Telemedicine and Telecare,24(9),
575–585. https://doi.org/10.1177/1357633X17730443
United States Government Accountability Office. (2011, October). VA
mental health: Number of veterans receiving care, barriers faced, and
efforts to increase access. (Publication No. GAO-12-12). https://www.gao
.gov/products/GAO-12-12
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2019).
Publication bias examined in meta-analyses from psychology and medi-
cine: A meta-meta-analysis. PLoS One,14(4), Article e0215052. https://
doi.org/10.1371/journal.pone.0215052
*Weiss, B. J., Azevedo, K., Webb, K., Gimeno, J., & Cloitre, M. (2018).
Telemental health delivery of skills training in affective and interpersonal
regulation (STAIR) for rural women veterans who have experienced
military sexual trauma. Journal of Traumatic Stress,31(4), 620–625.
https://doi.org/10.1002/jts.22305
*Whealin, J., Yoneda, A. C., Nelson, D., Hilmes, T., Kawasaki, M. M., &
Yan, O. H. (2017). A culturally adapted family intervention for rural
Pacific Island veterans with PTSD. Psychological Services,14(3),
295–306. https://doi.org/10.1037/ser0000186
*Wierwille, J. L., Pukay-Martin, N. D., Chard, K. M., & Klump, M. C.
(2016). Effectiveness of PTSD telehealth treatment in a VA clinical
sample. Psychological Services,13(4), 373–379. https://doi.org/10
.1037/ser0000106
Wilks, C., Yin, Q., Ang, S. Y., Matsumiya, B., Lungu, A., & Linehan, M.
(2017). Internet-delivered dialectical behavioral therapy skills training for
suicidal and heavy episodic drinkers: Protocol and preliminary results of a
randomized controlled trial. JMIR Research Protocols,6(10), e207.
https://doi.org/10.2196/resprot.7767
*Yuen, E. K., Gros, D. F., Price, M., Zeigler, S., Tuerk, P. W., Foa, E. B., &
Acierno, R. (2015). Randomized controlled trial of home-based telehealth
versus in-person prolonged exposure for combat-related PTSD in veter-
ans: Preliminary results. Journal of Clinical Psychology,71(6), 500–512.
https://doi.org/10.1002/jclp.22168
Yuen, E. K., Herbert, J. D., Forman, E. M., Goetter, E. M., Juarascio, A. S.,
Rabin, S., Goodwin, C., & Bouchard, S. (2013). Acceptance based
behavior therapy for social anxiety disorder through video conferencing.
Journal of Anxiety Disorders,27(4), 389–397. https://doi.org/10.1016/j
.janxdis.2013.03.002
Received February 04, 2020
Revision received October 03, 2020
Accepted October 06, 2020 ▪
TELEPSYCHOLOGY 11
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.