The Multidimensional Psychological Flexibility Inventory (MPFI): Discriminant Validity of Psychological Flexibility with Distress

Abstract

The psychological flexibility model of behavior change underpins acceptance and commitment therapy (ACT). However, the measurement of psychological flexibility over the past 20 years has mostly hinged on the use of measures with poor discriminant validity and fuzzy boundaries with distress and negative emotionality. The current study analyzed the structure of psychological flexibility as assessed with the recently developed Multidimensional Psychological Flexibility Inventory (MPFI) and examined its link with distress. We used data from an online cross-sectional survey (N = 1,542) which assessed psychological flexibility with the MPFI and the Acceptance and Action Questionnaire-II (AAQ-II), as well as anxiety, and depression. Exploratory and confirmatory factor analyses highlighted that the six psychological flexibility processes of the MPFI loaded on a psychological flexibility factor, while anxiety, depression and the AAQ-II loaded on a distress factor, supporting the discriminant validity of the MPFI in assessing psychological flexibility. Findings were consistent with those from prior studies which suggest that the AAQ-II is more of a measure of global distress than psychological inflexibility or by proxy, psychological flexibility. The 30-item MPFI psychological flexibility scale demonstrated good construct and discriminant validity. Overall, findings support psychological flexibility as a construct that is empirically distinct from distress.

Full text

Journal of Contextual Behavioral Science 21 (2021) 22–29
Available online 20 May 2021
2212-1447/© 2021 Association for Contextual Behavioral Science. Published by Elsevier Inc. All rights reserved.
Empirical Research
The Multidimensional Psychological Flexibility Inventory (MPFI):
Discriminant validity of psychological exibility with distress
Giulia Landi
a
,
b
,
*
, Kenneth I. Pakenham
c
, Elisabetta Crocetti
a
, Silvana Grandi
a
,
b
,
Eliana Tossani
a
,
b
a
Department of Psychology, University of Bologna, Viale Berti Pichat 5, 40127, Bologna, Italy
b
Laboratory of Psychosomatics and Clinimetrics, Department of Psychology, University of Bologna, Viale Europa 115, 47023, Cesena, Italy
c
The University of Queensland; School of Psychology, The University of Queensland, Brisbane, QLD, 4072, Australia
ARTICLE INFO
Keywords:
MPFI
Psychological exibility
Discriminant validity
Distress
Acceptance and commitment therapy
ABSTRACT
The psychological exibility model of behavior change underpins acceptance and commitment therapy (ACT).
However, the measurement of psychological exibility over the past 20 years has mostly hinged on the use of
measures with poor discriminant validity and fuzzy boundaries with distress and negative emotionality. The
current study analyzed the structure of psychological exibility as assessed with the recently developed Multi-
dimensional Psychological Flexibility Inventory (MPFI) and examined its link with distress. We used data from an
online cross-sectional survey (N =1542) which assessed psychological exibility with the MPFI and the
Acceptance and Action Questionnaire-II (AAQ-II), as well as anxiety, and depression. Exploratory and conr-
matory factor analyses highlighted that the six psychological exibility processes of the MPFI loaded on a
psychological exibility factor, while anxiety, depression and the AAQ-II loaded on a distress factor, supporting
the discriminant validity of the MPFI in assessing psychological exibility. Findings were consistent with those
from prior studies which suggest that the AAQ-II is more of a measure of global distress than psychological
inexibility or by proxy, psychological exibility. The 30-item MPFI psychological exibility scale demonstrated
good construct and discriminant validity. Overall, ndings support psychological exibility as a construct that is
empirically distinct from distress.
1. Introduction
Psychological exibility is a transdiagnostic concept that entails a
range of inter- and intra-personal skills and is considered the corner-
stone of mental health as it is closely related to resiliency (Kashdan &
Rottenberg, 2010). According to the psychological exibility model that
underpins acceptance and commitment therapy (ACT), psychological
exibility involves being open to inner experiencing in the present and
adjusting behaviors in response to changing situational demands that
are also aligned with personal values (Hayes et al., 2006). As such,
psychological exibility enables an individual to shift behavioral rep-
ertoires when they compromise pursuit of personal values, while also
adapting to changing circumstances (Kashdan & Rottenberg, 2010). ACT
uses six interrelated core processes to increase psychological exibility:
(1) acceptance – openness to inner experiencing, (2) defusion –
observing feelings and thoughts without attachment, (3) present
moment awareness – mindful awareness of the present, (4)
self-as-context – exible self-awareness and perspective taking, (5)
values – connection to personal values, (6) committed action –
values-guided effective action (Hayes et al., 2012). In contrast, higher
levels of the inverse of these processes foster psychological inexibly
which entails rigid and reactionary behavioral responses to uncom-
fortable and unwanted stimuli. The six psychological inexibility pro-
cesses involve: (1) experiential avoidance – avoidance of unwanted
inner experiencing, (2) lack of present moment awareness, (3)
self-as-content – rigid attachment to concepts of self, (4) fusion – ab-
sorption in unwanted thoughts and feelings rather than observing them
and allowing them to ow freely, (5) lack of contact with values, (6)
inaction and impulsiveness – derailment of functional behavior in
response to unwanted inner experiencing (Hayes et al., 2012).
ACT aims to increase psychological exibility and reduce psycho-
logical inexibility. It is an empirically supported treatment for a range
of mental health problems including anxiety, depression, substance use,
pain, and transdiagnostic groups (see reviews of meta-analyses, Gloster
* Corresponding author. Department of Psychology, University of Bologna, viale Berti Pichat 5, 40127, Bologna, Italy.
E-mail address: giulia.landi7@unibo.it (G. Landi).
Contents lists available at ScienceDirect
Journal of Contextual Behavioral Science
journal homepage: www.elsevier.com/locate/jcbs
https://doi.org/10.1016/j.jcbs.2021.05.004
Received 21 December 2020; Received in revised form 6 May 2021; Accepted 11 May 2021

Journal of Contextual Behavioral Science 21 (2021) 22–29
23
et al., 2020). Twenty meta-analyses indicate that, across 133 studies and
12,477 participants, ACT is more effective than waitlist and placebo
conditions and at least as effective as the traditional cognitive behav-
ioral therapies (Gloster et al., 2020). Higher psychological exibility is
associated with optimal responses to distress in the context of mean-
ingful goal pursuit and better mental health outcomes across a wide
range of contexts (Gloster et al., 2017; Hayes et al., 2006; Kashdan &
Rottenberg, 2010; Stabbe et al., 2019; Yadavaia et al., 2014). For
example, higher psychological exibility has been associated with
increased self-compassion and well-being (e.g., Hayes et al., 2012;
Kashdan & Rottenberg, 2010; Marshall & Brockman, 2016). Other
studies in the general population showed psychological exibility to be a
moderator in the relationship between daily stress, physical and mental
health, and wellbeing (Gloster et al., 2017). Psychological exibility was
also reported as a moderator of the adverse effects of major life events
and learned helplessness on depressive symptoms (Fonseca et al., 2020;
Trindade et al., 2020). Most importantly, research indicates that ACT
interventions produce therapeutic effects through improvements in
psychological exibility, which is the theoretically proposed change
mechanism (Levin et al., 2012).
1.1. Discriminant validity of psychological exibility measures
Although according to the ACT framework psychological exibility
is the core mechanism of change, almost all of the work on psychological
exibility over the past 20 years has hinged on studies that have
explored its opposite: psychological inexibility and in particular its
sub-process experiential avoidance (Doorley et al., 2020), dened as
unwillingness to remain in contact with aversive internal experiencing
(e.g., thoughts, memories, bodily sensations), which in turn is related to
inexible responses to these difcult experiences (Hayes et al., 2012).
Most of the research on the effectiveness of ACT interventions and the
causes and consequences of psychological exibility is based on the use
of two self-report measures of psychological inexibility, the Acceptance
and Action Questionnaire (AAQ-I and AAQ-II; Bond et al., 2011; Hayes
et al., 2004) and the Brief Experiential Avoidance Questionnaire (BEAQ;
G´
amez et al., 2014), which are used as a proxy for psychological exi-
bility (e.g., Kashdan et al., 2020; Rochefort et al., 2018; Tyndall et al.,
2019; Wolgast, 2014). Nevertheless, these instruments suffer from poor
discriminant validity as they overlap highly with several measures of
psychopathology including, depression, anxiety, stress and substance
abuse (Bond et al., 2011; Doorley et al., 2020). In fact, a small but
growing body of research evidence shows that the AAQ-II and the BEAQ
assess distress and negative emotionality rather than psychological
inexibility or psychological exibility by proxy (Kashdan et al., 2020;
Rochefort et al., 2018; Tyndall et al., 2019; Wolgast, 2014). Based on
these ndings, it has been proposed that rather than measuring the
ability to respond effectively to unwanted inner experiencing, the
AAQ-II and the BEAQ instead assess the inner experiences to which
participants are responding (e.g., Kashdan et al., 2020; Ong et al., 2020;
Tyndal et al., 2019; Wolgast, 2014). This differentiation is crucial,
especially in clinical settings where ACT researchers and practitioners
are primarily interested in how a person responds to their difcult
thoughts and feelings rather than the distress the person is experiencing.
In contrast to the abovementioned research on the AAQ-II and the
BEAQ, the discriminant validity ndings of two recently developed
psychological exibility measures show promise (Kashdan et al., 2020;
Ong et al., 2020). Results of these studies indicate that the Personalized
Psychological Flexibility Index (PPFI; Kashdan et al., 2020) and the
Comprehensive assessment of Acceptance and Commitment Therapy
processes (CompACT; Francis et al., 2016) evinced good discriminant
validity with negative emotionality and distress, although the CompACT
had weak structural validity (Kashdan et al., 2020; Ong et al., 2020).
However, there is no published data on the discriminant validity of the
recently developed Multidimensional Psychological Flexibility In-
ventory (MPFI; Rolffs et al., 2016). Hence, we investigated whether this
measure of psychological exibility is confounded with distress.
1.2. The Multidimensional Psychological Flexibility Inventory (MPFI) as
a measure of psychological exibility
The most widely used measures of psychological inexibility or
psychological exibility by proxy, the AAQ-II and the BEAQ, assess this
construct at the global level rather than at the level of the six interrelated
core processes that foster this overarching dimension (Rolffs et al.,
2016). The MPFI was developed to assess global psychological exibility
and inexibility and each of their respective six sub-processes. It was
created from a pool of 554 items which were derived from 22 of the most
widely used scales in the ACT and mindfulness literature as well as 84
items that were developed by the authors (Rolffs et al., 2016) based on
published conceptual denitions of the psychological exibility and
inexibility dimensions in the ACT literature. Over three studies and a
combined sample of 3040 participants, exploratory and conrmatory
factor analyses were used in combination with item response theory to
generate, rene and isolate the structure of the instrument. The nal
MPFI is composed of 60 items, of which 30 assess psychological exi-
bility and 30 assess inexibility, and their respective core processes.
Studies have provided supportive evidence for the validity of the in-
strument (Cherry et al., 2021), including factorial stability (Gr´
egoire
et al., 2020; Lin et al., 2020; Rolffs et al., 2016; Seidler et al., 2020),
responsiveness to change over time (Rolffs et al., 2016), identication of
individuals currently in counseling (Stabbe et al., 2019), and the pre-
diction of distress and wellbeing (Rolffs et al., 2016; Stabbe et al., 2019).
1.3. The present study
The present study examined whether the MPFI as a measure of
psychological exibility is confounded with distress. Only data on the
psychological exibility dimension is presented for several reasons.
First, consistent with the primary aim of prior studies which have
assessed the discriminant validity of two other psychological exibility
measures, the PPFI (Kashdan et al., 2020) and the CompACT (Ong et al.,
2020), the primary aim of the present study was to extend and evaluate
the discriminant validity of the 30 psychological exibility items of the
MPFI. Second, given the lack of research into psychological exibility
scales relative to the volume of studies examining psychological
inexibility scales, and often as a proxy for psychological exibility
(Kashdan et al., 2020; Rochefort et al., 2018; Tyndall et al., 2019;
Wolgast, 2014), the sole focus on psychological exibility in the present
study is warranted. Third, we have used the Italian version of the MPFI
which is still under review (Landi et al., 2021), and preliminary data on
the structure of the Italian MPFI has been published for only the psy-
chological exibility dimension (conrmatory factor analysis of the 30
items of the Italian MPFI, (Landi et al., 2020). We investigated the
discriminant validity of the 30 psychological exibility items of the
MPFI (Rolffs et al., 2016) with standardized measures of distress and a
widely used measure of psychological inexibility or by proxy psycho-
logical exibility, the AAQ-II. We expected the MPFI psychological
exibility items to load on an independent factor, whereas based on
previous research, we expected the AAQ-II items to load on a distress
factor together with items from anxiety and depression measures.
G. Landi et al.

Journal of Contextual Behavioral Science 21 (2021) 22–29
24
2. Methods
2.1. Participants and recruitment procedure
Data were collected from a total of 1587 participants. After removal
of responders who nished the online survey in less than 9 min
1
(n =
45), the analyses were conducted on a sample of 1542 participants who
completed the survey between May and August 2020 in Italy. People
≥18 years of age were eligible. Exclusion criteria were <18 years of age
and not being uent in Italian. Recruitment was conducted through
social networking platforms (e.g. WhatsApp and Facebook) and a
snowballing procedure, whereby participants invited friends and ac-
quaintances to participate in the study. The recruitment materials pre-
sented the study as “The Psychological Resources Project” and informed
potential respondents that participation was voluntary. The survey was
developed using Qualtrics software and took approximately 20–25 min
to complete. An accurate response rate was not possible to obtain, as
recruitment was primarily conducted through social networks. The
study received ethical clearance by an institutional human research
ethics committee.
A total of 70.6% of the sample were female. The age range was 18–83
(M =38.6, SD =15.0). Almost all (98.4%) responders were of Italian
nationality. Regarding highest education levels, 43.0% of the sample
had a bachelor’s degree, 42.2% completed high school and 8.3% post-
graduate courses. Almost half (45.2%) of the sample were either married
or living with a partner, 47.4% were single, while 7.4% were widowed
or divorced. Most (75.0%) participants were employed, 21.3% were
students, 5.2% unemployed, and 4.7% were retired. Regarding SES,
11.5% endorsed a mean income below average, 79.7% reported being in
the middle socioeconomic class, and 8.8% wealthier than the average.
2.2. Measures
2.2.1. Multidimensional Psychological Flexibility Inventory (MPFI)
The MPFI (Rolffs et al., 2016) was used to assess global psychological
exibility and its constituent six core processes (acceptance, present
moment awareness, self-as-context, defusion, values, committed action).
Participants were asked to refer to the past two weeks and respond on a
6-point Likert scale from 1 “never true” to 6 “always true.” Scores were
averaged and higher scores on the global dimension and the respective
sub-processes indicate greater psychological exibility. The Italian
version of this scale is currently under validation. In order to translate
the scale into Italian, a multistep strategy was used in which two inde-
pendent forward translations of the original version of the MPFI were
produced rst (one by the authors of the Italian validation and one by a
bilingual translator whose mother tongue is Italian and who is uent
with US English). Secondly, these forward translations were reviewed by
a translation panel consisting of the authors, the translator, two ACT
researchers and a lay person. After identifying ambiguities amongst
these two versions, a reconciled forward version was produced. In order
to examine the extent to which this preliminary Italian version of the
MPFI was clear and understandable, it was administered to a group of 30
respondents from the general population. Further modications were
applied based on this pilot testing, and the preliminary version was then
back-translated by a bilingual translator whose mother tongue was US
English with uency in Italian. A preliminary conrmatory factor
analysis was conducted highlighting excellent t indices for the original
six-factor model of psychological exibility (Landi et al., 2020). The
MPFI has demonstrated good reliability and validity in clinical and
nonclinical samples (Lin et al., 2019; Rogge et al., 2019; Seidler et al.,
2020; Stabbe et al., 2019). The observed Cronbach’s alpha range
(0.85–0.94) was similar to that obtained in the derivation study
(0.87–0.97).
2.2.2. General anxiety disorder scale (GAD-7)
The GAD-7 questionnaire (Spitzer et al., 2006) measures anxiety
symptoms over the past two weeks. Items are rated on a 4-point Likert
scale from 0 “not at all” to 3 “nearly every day.” Item scores were sum-
med, with higher scores reecting higher anxiety. We used the Italian
version of the GAD-7 (Kroenke & Spitzer, 2010). The instrument has
been shown to be psychometrically sound (L¨
owe et al., 2008; Plummer
et al., 2016). The observed Cronbach’s alpha was .88.
2.2.3. Patient Health Questionnaire-9 (PHQ-9)
The PHQ-9 (Spitzer et al., 1999) was used to measure depressive
symptomatology over the past two weeks. Items are rated on a 4-point
Likert scale from 0 “not at all” to 3 “nearly every day.” All item scores
were summed, with higher scores indicating higher depression. We used
the Italian validated version of the PHQ-9 (Kroenke & Spitzer, 2010).
The measure has demonstrated sound psychometric properties (Kroenke
et al., 2001). The observed Cronbach’s alpha was .83.
2.2.4. Acceptance and Action Questionnaire-II (AAQ-II)
The AAQ-II (Bond et al., 2011) is a self-report questionnaire
composed of 7-items measuring psychological inexibility, dened as
“rigid dominance of psychological reactions over chosen values and
contingencies in guiding actions” (p. 678). Items are rated on a 7-point
Likert from 1 “not at all true” to 7 “completely true.” All item scores were
summed, with higher scoring indicating higher psychological inexi-
bility. The AAQ-II has showed good internal consistency and test-retest
reliability in community samples (Bond et al., 2011). The observed
Cronbach’s alpha was .90.
2.3. Data analysis
Preliminary analyses (i.e., descriptive statistics, reliabilities, and
correlations among study variables) were performed in IBM SPSS 24.
Exploratory Structural Equation Modeling (ESEM) and Conrmatory
Factor Analysis (CFA) were conducted in Mplus 8.3 with the robust
maximum likelihood estimator (MLR; Muth´
en & Muth´
en, 1998–2018).
Because the percentage of missing data across all study items was 0.33%,
we used the Full Information Maximum Likelihood estimator in Mplus to
address missing data. We rst ran an ESEM with an oblique Geomin
rotation examining a two-factor solution with all psychological exi-
bility subscales (acceptance, present moment awareness, self-as-context,
defusion, values, committed action) as well as anxiety, depression, and
AAQ-II scores as observed variables. ESEM has been developed in order
to overcome CFA and exploratory factor analysis (EFA) limitations by
integrating benets of both. Like EFA, ESEM allows items with
cross-loadings to load on other factors and like CFA, ESEM examines
goodness-of-t statistics, offering a more accurate representation of the
data (Marsh et al., 2009; Marsh et al., 2014). The loading cut-off on the
main factor was ≥0.40 with a p value ≤.05 (Hair et al., 2010). If our
hypothesis was supported, we planned to conrm the differentiation of
the psychological exibility and distress factors by conducting a CFA
testing a second order two-factor model in which all psychological
exibility processes (acceptance, present moment awareness,
self-as-context, defusion, values and committed action) and all distress
scales (anxiety, depression and AAQ-II) were considered as latent vari-
ables. Specically, psychological exibility was included as a
second-order latent variable composed of acceptance, present moment
awareness, self-as-context, defusion, values and committed action as
rst-order latent variables, while distress was included as a second-order
latent variable composed of GAD-7, PHQ-9 and AAQ-II as rst-order
latent variables. The model t in ESEM and CFA analyses was
1
The 9-min survey completion cut-off was based on pilot testing. Five par-
ticipants were instructed to complete the online survey as quickly as possible
while ensuring they read and comprehended all written material. None of the
ve participants could complete the questionnaire in less than 9 min, therefore
we selected this criterion as the cut-off for inclusion in the study.
G. Landi et al.

Journal of Contextual Behavioral Science 21 (2021) 22–29
25
evaluated with the following goodness of t indices: Comparative Fit
Index (CFI), the Root Mean Square Error of Approximation (RMSEA) and
its 90% Condence Interval (CI), and the Standardized Root Mean
Square Residual (SRMR). CFI values >0.90, RMSEA values ≤0.08, and
SRMR values ≤0.09 as well as the upper bound of the 90% CI of RMSEA
lower than 0.10 were considered indices of a good model t (Marsh
et al., 2005). In both ESEM and CFA analyses, we also compared whether
our hypothesized two-factor model was superior to an alternative
one-factor model (in which all psychological exibility processes and all
distress scales loaded together onto a single factor). To identify the best
tting solution, the Akaike’s Information Criterion (AIC) and Bayes In-
formation Criterion (BIC) indices were also inspected (i.e., the model
with the smallest AIC and BIC values is the best tting one). Statistical
scripts for all analyses and relevant outputs are presented in Supple-
mentary Materials A.
3. Results
3.1. Preliminary analyses
Means, standard deviations, Cronbach’s alphas and correlations
among all variables are presented in Table 1. Global psychological
exibility and all psychological exibility processes were related to
lower anxiety, depression, and AAQ-II scores. The six psychological
exibility processes were signicantly positively and strongly associated
with global psychological exibility. The six psychological exibility
processes were also positively and signicantly related with each other
and most were of a moderate to large magnitude. Anxiety, depression
and AAQ-II scores were positively and strongly correlated with each
other.
3.2. Exploratory Structural Equation Modeling (ESEM): differentiating
psychological exibility from distress
The primary aim of this study was to examine the discriminant val-
idity of the psychological exibility items of the MPFI with measures of
anxiety and depression, and the AAQ-II. We rst conducted an ESEM
with an oblique Geomin rotation examining a two-factor solution with
all psychological exibility subscales (acceptance, present moment
awareness, self-as-context, defusion, values, committed action) as well
as the GAD-7, PHQ-9, and AAQ-II scores as observed variables. As pre-
dicted, results of this analysis indicated a two-factor model with very
good model t:
χ
2
(17) =163.468, p <.001; CFI =0.971; TLI =0.939;
RMSEA =0.075; RMSEA CI =[0.065, 0.085]; SRMR =0.026 (stan-
dardized factor loadings and 95% condence intervals for the two-factor
solution are reported in Table 2). The results are summarized in Fig. 1.
The MPFI psychological exibility sub-processes loaded on one factor,
whereas anxiety, depression and the AAQ-II all strongly loaded together
on a second factor. The two extracted factors were inversely related
(standardized correlation = − 0.446; 95% CI = − 0.497, −0.394).
As reported in Table 2, the psychological exibility sub-processes of
acceptance and present moment awareness displayed some positive
cross-loadings with the latent distress factor, while defusion and
committed action evidenced some negative cross-loadings with the
Table 1
Descriptive data and Person’s correlations among study variables (N =1542).
M (SD) Range
α
1 1a 1b 1c 1d 1e 1f 2 3
1. Global Psychological Flexibility 3.89 (0.69) 1.60–6 .94 –
1a. Acceptance 3.44 (0.86) 1–6 .85 .59** –
1b. Present Moment Awareness 3.96 (0.99) 1–6 .89 .70** .52** –
1c. Self-as-context 3.92 (0.99) 1–6 .89 .80** .33** .41** –
1d. Defusion 3.46 (0.96) 1–6 .88 .75** .29** .31** .67** –
1e. Values 4.38 (0.93) 1.40–6 .87 .76** .28** .45** .49** .45** –
1f. Committed Action 4.16 (0.95) 1–6 .88 .78** .25** .38** .58** .53** .66** –
2. GAD-7 6.52 (4.48) 0–21 .88 -.30** -.06* -.01 -.31** -.45** -.21** -.27** –
3. PHQ-9 7.27 (4.67) 0–27 .83 -.35** -.07** -.06* -.33** -.43** -.28** -.35** .75** –
4. AAQ-II 19.81 (8.61) 7–49 .90 -.42** -.08** -.09** -.41** -.50** -.34** -.41** .65** .67**
Note. * p <.05, ** p <.01.
α
=Cronbach’s Alpha. GAD-7 =Generalized Anxiety Disorder-7; PHQ-9 =Patient Health Questionnaire-9; AAQ-II =Acceptance and Action
Questionnaire- II.
Table 2
Factor loadings and 95% condence intervals for exploratory structural equa-
tion modeling (ESEM) differentiating psychological exibility from distress.
Factors Factor 1 95% CI Factor 2 95% CI
Acceptance .447** .393, .502 .127** .076, .179
Present Moment Awareness .619** .569, .669 .248** .198, .297
Self-as-Context .837** .808, .866 .002 -.020, .025
Defusion .674** .634, .715 -.210** -.254, -.167
Values .623** .576, .670 -.002 -.042, .038
Committed Actions .664** .625, .703 -.074* -.120, –.027
GAD-7 .036 -.001, .073 .887** .855, .918
PHQ-9 -.013* -.023, -.003 .856** .832, .880
AAQ-II -.196** -.239, -.152 .680** .641, .718
Note. Factor 1 =psychological exibility; Factor 2 =distress; Bold indicates the
highest factor loading. 95% CI =95% condence interval. GAD-7 =Generalized
Anxiety Disorder-7; PHQ-9 =Patient Health Questionnaire-9; AAQ-II =
Acceptance and Action Questionnaire- II. * p <.05, ** p <.01.
Fig. 1. Exploratory Structural Equation Modeling Differentiating Psychological
Flexibility from Distress. Factor 1 =Psychological Flexibility; Factor 2 =
Distress. GAD-7 =Generalized Anxiety Disorder-7; PHQ-9 =Patient Health
Questionnaire-9; AAQ-II =Acceptance and Action Questionnaire- II. Note.
Whereas the statistical model includes both major and minor loadings for all
items, for clarity only major loadings are displayed, while secondary loadings
are reported in Table 2.
G. Landi et al.

Journal of Contextual Behavioral Science 21 (2021) 22–29
26
latent distress factor; however, they had substantially larger primary
loadings on the psychological exibility factor (composed of only MFPI
items). The AAQ-II also showed a negative cross-loading with the latent
psychological exibility factor but had substantially larger primary
loadings on the distress factor (standardized factor loading of −0.196**
and 0.680**, respectively). Therefore, as expected, the AAQ-II loaded on
the distress factor together with the measures of anxiety and depression.
As displayed in Table 3, t indices clearly indicated that our hy-
pothesized two-factor model was superior to the alternative one-factor
model in which all psychological exibility sub-processes and all
distress scales loaded together as observed variables onto a single factor.
In fact, the two-factor model in which psychological exibility and
distress are considered as two distinct factors resulted in a substantial
improvement in t (ΔCFI =0.306, ΔTLI =0.421, ΔRMSEA = − 0.135,
ΔSRMR = − 0.099, ΔAIC = − 1425.561, ΔBIC = − 1.337,834).
3.3. Second-order conrmatory factor analysis (CFA): conrming the
differentiation of the psychological exibility and distress factors
As our hypothesis was supported, we conrmed the differentiation of
psychological exibility and distress factors by conducting a CFA testing
a second order two-factor model in which all psychological exibility
sub-processes (acceptance, present moment awareness, self-as-context,
defusion, values, committed action) and all distress scales (anxiety,
depression, and AAQ-II) were considered as latent variables. Specif-
ically, psychological exibility was included as a second-order latent
variable composed of acceptance, present moment awareness, self-as-
context, defusion, values and committed action as rst-order latent
variables, while distress was included as a second-order latent variable
composed of GAD-7, PHQ-9, and AAQ-II as rst-order latent variables.
Descriptive statistics and the correlation matrix of all items of each
questionnaire used in this analysis are presented in Supplementary
Materials B. Fit indices for the second-order CFA were excellent:
χ
2
(1309) =4204.809, p <.001; CFI =0.924; TLI =0.920; RMSEA =
0.038; RMSEA CI =[0.037, 0.039]; SRMR =0.064. Standardized factor
loadings are reported in Fig. 2 and show that the psychological exi-
bility and distress factors were inversely related (standardized correla-
tion = − 0.568; 95% CI = − 0.616, −0.519).
As reported in Table 3, compared to an alternative model in which all
psychological exibility sub-processes and all distress scales loaded
together into one overarching second-order factor, t indices showed
that our hypothesized model in which psychological exibility and
distress were considered as two distinct second-order factors resulted in
an improvement in t (ΔCFI =0.026, ΔTLI =0.027, ΔRMSEA =
−0.006, ΔSRMR = − 0.031, ΔAIC = − 1115.894, ΔBIC = − 1110.553). In
sum, results clearly indicate psychological exibility as an independent
factor and empirically distinct in relation to distress.
4. Discussion
The aim of the present study was to examine the discriminant val-
idity of psychological exibility assessed with the MPFI. We used ESEM
and second-order CFA to evaluate if the MPFI was effective in dis-
tinguishing psychological exible responding from the experience of
distress. Results revealed psychological exibility to be an independent
factor and separate from distress. Hence, the psychological exibility
items of the MPFI measure a distinct latent construct relative to anxiety,
depression, and AAQ-II psychological inexibility items in our sample.
Results of this study showed the 30 MPFI psychological exibility items
are effective in distinguishing psychological exibility from distress.
Findings are noteworthy as they are the rst published data showing
that all six psychological exibility processes as well as global exibility,
measured by the MPFI, are not confounded with distress and can
therefore be used as valid indicators of psychological exibility.
Results of this study also revealed that some psychological exibility
processes loaded on the distress factor even though they had substan-
tially larger primary loadings on the psychological exibility factor. In
particular, defusion and committed action evidenced negative cross-
loadings with distress. Unexpectedly, acceptance and present moment
awareness displayed positive cross-loadings with distress. These positive
cross-loading might be explained by the fact that data were collected in
the midst of the COVID-19 pandemic and during the aftermath of a
government imposed lockdown. The stress and fear associated with the
pandemic itself, accompanied by the lockdown effects of social isolation
and activity restrictions as well as anxiety about future health and
economic uncertainties are associated with detrimental mental health
impacts (Brooks et al., 2020; Xiong et al., 2020). Acceptance and present
moment awareness involve being open to difcult thoughts and feelings
and allowing them to organically unfold and pass. Therefore, of the six
psychological exibility processes, they are likely to be the most chal-
lenging in the context of a pandemic because they entail embracing
potentially intense psychological discomfort related to pandemic
induced fear, uncertanty, and social isolation and, hence, their positive
cross-loadings on the distress factor. Consistent with this proposal,
research on the psychological impacts of COVID-19 has shown that
greater acceptance is related to higher distress (Landi et al., 2020;
Pakenham et al., 2020). In addition, a non-COVID context study found
that acceptance cross-loaded with negative emotionality (Kashdan et al.,
2020) and highlighted the difculties in measuring constructs like
psychological exibility, which require evaluation with reference to a
specic context (Hayes et al., 2012; Kashdan et al., 2020).
Findings of the present study are consistent with the growing body of
evidence which suggests the AAQ-II assesses distress and negative
emotionality rather than psychological inexibility or psychological
exibility by proxy (Doorley et al., 2020; Kashdan et al., 2020; Ong
et al., 2020; Rochefort et al., 2018; Tyndall et al., 2019; Wolgast, 2014).
Results of the present study showed that the AAQ-II had poor discrim-
inant validity in relation to distress and risks circularity of measurement
and an overestimation of the association between psychological inex-
ibility (or its sub-process experiential avoidance) and measures of psy-
chopathology (Kashdan et al., 2020; Wolgast, 2014). Evidence from this
and other studies suggests that the AAQ-II does not adequately
discriminate between psychological inexibility as a behavioral pattern
of responding (e.g., the independent variable, context-insensitive
Table 3
Fit indices for exploratory structural equation modeling (ESEM) and conrmatory factor analysis (CFA) of two-factor and one-factor models of psychological exibility
and distress.
χ
2
df CFI TLI SRMR RMSEA [90% CI] AIC BIC
ESEM Models
Two-factors 163.468** 17 .971 .939 .026 .075 [.065, .085] 46,791.867 46,989.478
One-factor 1719.833** 25 .665 .518 .125 .210 [.201, .218] 48,217.428 48,327.312
Second-order CFA Models
Two-factors 4204.810** 1309 .924 .920 .064 .038 [.037, .039] 198,048.672 198,983.318
One-factor 5158.591** 1310 .898 .893 .095 .044 [.042, .045] 199,164.566 200,093.871
Note. ** p <.01;
χ
2
=Chi-square; df =degrees of freedom; CFI =Comparative Fit Index; TLI =; RMSEA [90% CI] =Root Mean Square Error of Approximation and 90%
condence interval; SRMR =Standardized Root Mean Square Residual; AIC =Akaike’s Information Criterion; BIC =Bayes Information Criterion. Bold indicates the
best tting factor solution.
G. Landi et al.

Journal of Contextual Behavioral Science 21 (2021) 22–29
27
avoidance of distressing stimuli) and the experience of emotional
distress (e.g., the dependent variables, anxiety, depression and other
measures of psychopathology) (Kashdan et al., 2020; Ong et al., 2020;
Rochefort et al., 2018; Tyndall et al., 2019; Wolgast, 2014).
This study has several methodological limitations. First, all data were
collected via an online survey and self-report measures, which increases
the risk of common method variance. Second, convenience sampling and
the bias towards female participants limits the generalizability of nd-
ings. Future studies should replicate these results in other community as
well as clinical samples. Third, this study did not use the 30-item MPFI
psychological inexibility scale for the reasons mentioned earlier.
However, future research should investigate the discriminant validity of
both the psychological exibility and the inexibility scales of the MFPI.
Investigation of whether psychological exibility and inexibility
should be considered separately and examination of their discriminant
validity in relation to distress and negative emotionality will have
important theoretical and practice implications. In particular, future
studies should clarify the structure of the psychological inexibility/
exibility model and shed light on whether the MPFI or other measures
could validly identify distinct subgroups among psychologically exible
or inexible individuals based on their proles of psychological exi-
bility and inexibility (e.g., Cherry et al., 2021; Christodoulou et al.,
2019; Rogge et al., 2019; Tyndal et al., 2020). Notwithstanding the
above limitations, this study is the rst to provide empirical support for
the discriminant validity of psychological exibility assessed with the
MPFI with respect to distress.
Fig. 2. Standardized solution of the second-order factor structure depicting the independence of the psychological exibility factor from distress. GAD-7 =
generalized anxiety Disorder-7; PHQ-9 =patient health Questionnaire-9; AAQ-II =acceptance and action questionnaire- II.
G. Landi et al.

Journal of Contextual Behavioral Science 21 (2021) 22–29
28
5. Conclusion
The 30-item MPFI psychological exibility scale showed good
discriminant validity in relation to distress. However, results revealed
some overlap between distress and two psychological exibility pro-
cesses, present moment awareness and acceptance, which may have
emerged due to their sensitivity to the context specic challenges of the
COVID-19 pandemic. Consistent with prior research, ndings showed
that the AAQ-II assesses distress rather than psychological inexibility
or by proxy, psychological exibility. Overall, ndings support psy-
chological exibility as a construct that is empirically distinct from
distress.
Declaration of competing interest
The authors declare no conict of interest.
Acknowledgement
The authors would like to thank Prof. Todd B. Kashdan for his valued
contribution to a preliminary discussion on the ideas underpinning this
article.
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.jcbs.2021.05.004.
References
Bond, F. W., Hayes, S. C., Baer, R. A., Carpenter, K. M., Guenole, N., Orcutt, H. K., &
Zettle, R. D. (2011). Preliminary psychometric properties of the acceptance and
action questionnaire–II: A revised measure of psychological inexibility and
experiential avoidance. Behavior Therapy, 42(4), 676–688. https://doi.org/10.1016/
j.beth.2011.03.007
Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., &
Rubin, G. J. (2020). The psychological impact of quarantine and how to reduce it:
Rapid review of the evidence. The Lancet, 395, 912–920. https://doi.org/10.1016/
s0140-6736(20)30460-8
Cherry, K. M., Vander Hoeven, E., Patterson, T. S., & Lumley, M. N. (2021). Dening and
measuring “psychological exibility”: A narrative scoping review of diverse
exibility and rigidity constructs and perspectives. Clinical Psychology Review,
101973. https://doi.org/10.1016/j.cpr.2021.101973
Christodoulou, A., Michaelides, M., & Karekla, M. (2019). Network analysis: A new
psychometric approach to examine the underlying ACT model components. Journal
of Contextual Behavioral Science, 12, 285–289. https://doi.org/10.1016/j.
jcbs.2018.10.002
Doorley, J. D., Goodman, F. R., Kelso, K. C., & Kashdan, T. B. (2020). Psychological
exibility: What we know, what we do not know, and what we think we know. Social
and Personality Psychology Compass. , Article e12566. https://doi.org/10.1111/
spc3.12566
Fonseca, S., Trindade, I. A., Mendes, A. L., & Ferreira, C. (2020). The buffer role of
psychological exibility against the impact of major life events on depression
symptoms. Clinical Psychologist, 24(1), 82–90. https://doi.org/10.1111/cp.12194
Francis, A. W., Dawson, D. L., & Golijani-Moghaddam, N. (2016). The development and
validation of the comprehensive assessment of acceptance and commitment therapy
processes (CompACT). Journal of Contextual Behavioral Science, 5(3), 134–145.
https://doi.org/10.1016/j.jcbs.2016.05.003
G´
amez, W., Chmielewski, M., Kotov, R., Ruggero, C., Suzuki, N., & Watson, D. (2014).
The brief experiential avoidance questionnaire: Development and initial validation.
Psychological Assessment, 26(1), 35–45. https://doi.org/10.1037/a0034473
Gloster, A. T., Meyer, A. H., & Lieb, R. (2017). Psychological exibility as a malleable
public health target: Evidence from a representative sample. Journal of Contextual
Behavioral Science, 6(2), 166–171. https://doi.org/10.1016/j.jcbs.2017.02.003
Gloster, A. T., Walder, N., Levin, M., Twohig, M., & Karekla, M. (2020). The empirical
status of acceptance and commitment therapy: A review of meta-analyses. Journal of
Contextual Behavioral Science. https://doi.org/10.1016/j.jcbs.2020.09.009
Gr´
egoire, S., Gagnon, J., Lachance, L., Shankland, R., Dionne, F., Kotsou, I., &
Rogge, R. D. (2020). Validation of the English and French versions of the
multidimensional psychological exibility inventory short form (MPFI-24). Journal
of Contextual Behavioral Science, 18, 99–110. https://doi.org/10.1016/j.
jcbs.2020.06.004
Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, J. (2006). Acceptance and
commitment therapy: Model, processes and outcomes. Behaviour Research and
Therapy, 44(1), 1–25. https://doi.org/10.1016/j.brat.2005.06.006
Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (2012). Acceptance and commitment therapy.
An experiential approach to behavior change (2nd ed.). The Guildford Press.
Hayes, S. C., Strosahl, K., Wilson, K. G., Bissett, R. T., Pistorello, J., Toarmino, D.,
Polusny, M. A., Dykstra, T. A., Batten, S. V., Bergan, J., Stewart, S. H.,
Zvolensky, M. J., Eifert, G. E., Bond, F. W., Forsyth, J. P., Karekla, M., &
McCurry, S. M. (2004). Measuring experiential avoidance: A preliminary test of a
working model. Psychological Record, 54, 553–578. https://doi.org/10.1007/
BF03395492
Kashdan, T. B., Disabato, D. J., Goodman, F. R., Doorley, J. D., & McKnight, P. E. (2020).
Understanding psychological exibility: A multimethod exploration of pursuing
valued goals despite the presence of distress. Psychological Assessment, 32(9),
829–850. https://doi.org/10.1037/pas0000834
Kashdan, T. B., & Rottenberg, J. (2010). Psychological exibility as a fundamental aspect
of health. Clinical Psychology Review, 30(7), 865–878. https://doi.org/10.1016/j.
cpr.2010.03.001
Kroenke, K., & Spitzer, R. L. (2010). Instruction manual: Instructions for patient health
questionnaire (PHQ) and GAD-7 measures. Retrived from www.phqscreeners.com.
Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief
depression severity measure. Journal of General Internal Medicine, 16(9), 606–613.
https://doi.org/10.1046/j.1525-1497.2001.016009606.x
Landi, G., Pakenham, K. I., Boccolini, G., Grandi, S., & Tossani, E. (2020). Health anxiety
and mental health outcome during COVID-19 lockdown in Italy: The mediating and
moderating roles of psychological exibility. Frontiers in Psychology, 11, 2195.
https://doi.org/10.3389/fpsyg.2020.02195
Landi, G., Pakenham, K. I., Presti, G., Boccolini, G., Cola, A., Grandi, S., & Tossani, E.
(2021). Italian validation of the Multidimensional Psychological Flexibility Inventory
(MPFI) [Manuscript submitted for publication]. Department of Psychology,
University of Bologna.
Levin, M. E., Hildebrandt, M. J., Lillis, J., & Hayes, S. C. (2012). The impact of treatment
components suggested by the psychological exibility model: A meta-analysis of
laboratory-based component studies. Behavior Therapy, 43(4), 741–756. https://doi.
org/10.1016/j.beth.2012.05.003
Lin, Y. Y., Rogge, R. D., & Swanson, D. P. (2020). Cross-cultural exibility: Validation of
the traditional Mandarin, simplied Mandarin, and Japanese translations of the
multidimensional psychological exibility inventory. Journal of Contextual
Behavioral Science, 15, 73–84. https://doi.org/10.1016/j.jcbs.2019.11.008
L¨
owe, B., Decker, O., Müller, S., Br¨
ahler, E., Schellberg, D., Herzog, W., & Herzberg, P. Y.
(2008). Validation and standardization of the generalized anxiety disorder screener
(GAD-7) in the general population. Medical Care, 46(3), 266–274. https://doi.org/
10.1097/MLR.0b013e318160d093
Marshall, E. J., & Brockman, R. N. (2016). The relationships between psychological
exibility, self-compassion, and emotional well-being. Journal of Cognitive
Psychotherapy, 30(1), 60–72. https://doi.org/10.1891/0889-8391.30.1.60
Marsh, H. W., Hau, K. T., & Grayson, D. (2005). Goodness of t evaluation in structural
equation modeling. In A. Maydeu-Olivares, & J. McArdle (Eds.), Contemporary
psychometrics. A festschrift for roderick P. McDonald (pp. 275–340). Erlbaum.
Muth´
en, L. K., & Muth´
en, B. O. (1998-2018). Mplus user’s guide (7th ed.). Muth´
en &
Muth´
en.
Ong, C. W., Pierce, B. G., Petersen, J. M., Barney, J. L., Fruge, J. E., Levin, M. E., &
Twohig, M. P. (2020). A psychometric comparison of psychological inexibility
measures: Discriminant validity and item performance. Journal of Contextual
Behavioral Science, 18, 34–47. https://doi.org/10.1016/j.jcbs.2020.08.007
Pakenham, K. I., Landi, G., Boccolini, G., Furlani, A., Grandi, S., & Tossani, E. (2020). The
moderating roles of psychological exibility and inexibility on the mental health
impacts of COVID-19 pandemic and lockdown in Italy. Journal of Contextual
Behavioral Science, 17, 109–118. https://doi.org/10.1016/j.jcbs.2020.07.003
Plummer, F., Manea, L., Trepel, D., & McMillan, D. (2016). Screening for anxiety
disorders with the GAD-7 and GAD-2: A systematic review and diagnostic
metaanalysis. General Hospital Psychiatry, 39, 24–31. https://doi.org/10.1016/j.
genhosppsych.2015.11.005
Rochefort, C., Baldwin, A. S., & Chmielewski, M. (2018). Experiential avoidance: An
examination of the construct validity of the AAQ-II and MEAQ. Behavior Therapy, 49
(3), 435–449. https://doi.org/10.1016/j.beth.2017.08.008
Rogge, R. D., Daks, J. S., Dubler, B. A., & Saint, K. J. (2019). It’s all about the process:
Examining the convergent validity, conceptual coverage, unique predictive validity,
and clinical utility of ACT process measures. Journal of Contextual Behavioral Science,
14, 90–102. https://doi.org/10.1016/j.jcbs.2019.10.001
Rolffs, J. L., Rogge, R. D., & Wilson, K. G. (2016). Disentangling components of exibility
via the hexaex model: Development and validation of the multidimensional
psychological exibility inventory (MPFI). Assessment, 25(4), 458–482. https://doi.
org/10.1177/1073191116645905
Seidler, D., Stone, B., Clark, B. E., Koran, J., & Drake, C. E. (2020). Evaluating the factor
structure of the Multidimensional Psychological Flexibility Inventory: An
independent replication and extension. Journal of Contextual Behavioral Science, 17,
23–31. https://doi.org/10.1016/j.jcbs.2020.04.007
Spitzer, R. L., Kroenke, K., & Williams, J. B. (1999). Validation and utility of a self-report
version of PRIME-MD: The PHQ primary care study. Jama, 282(18), 1737–1744.
https://doi.org/10.1001/jama.282.18.1737
Spitzer, R. L., Kroenke, K., Williams, J. B., & L¨
owe, B. (2006). A brief measure for
assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166
(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092
Stabbe, O. K., Rolffs, J. L., & Rogge, R. D. (2019). Flexibly and/or inexibly embracing
life: Identifying fundamental approaches to life with latent prole analyses on the
dimensions of the Hexaex model. Journal of Contextual Behavioral Science, 12,
106–118. https://doi.org/10.1016/j.jcbs.2019.03.003
Trindade, I. A., Mendes, A. L., & Ferreira, N. B. (2020). The moderating effect of
psychological exibility on the link between learned helplessness and depression
G. Landi et al.

Journal of Contextual Behavioral Science 21 (2021) 22–29
29
symptomatology: A preliminary study. Journal of Contextual Behavioral Science, 15,
68–72. https://doi.org/10.1016/j.jcbs.2019.12.001
Tyndall, I., Waldeck, D., Pancani, L., Whelan, R., Roche, B., & Dawson, D. L. (2019). The
Acceptance and Action Questionnaire-II (AAQ-II) as a measure of experiential
avoidance: Concerns over discriminant validity. Journal of Contextual Behavioral
Science, 12, 278–284. https://doi.org/10.1016/j.jcbs.2018.09.005
Tyndall, I., Waldeck, D., Pancani, L., Whelan, R., Roche, B., & Pereira, A. (2020). Proles
of psychological exibility: A latent class analysis of the acceptance and commitment
therapy model. Behavior Modication, 44(3), 365–393. https://doi.org/10.1177/
0145445518820036
Wolgast, M. (2014). What does the Acceptance and Action Questionnaire (AAQ-II) really
measure? Behavior Therapy, 45(6), 831–839. https://doi.org/10.1016/j.
beth.2014.07.002
Xiong, J., Lipsitz, O., Nasri, F., Lui, L. M., Gill, H., Phan, L., & McIntyre, R. S. (2020).
Impact of COVID-19 pandemic on mental health in the general population: A
systematic review. Journal of Affective Disorders, 277(1), 55–64. https://doi.org/
10.1016/j.jad.2020.08.001
Yadavaia, J. E., Hayes, S. C., & Vilardaga, R. (2014). Using acceptance and commitment
therapy to increase self-compassion: A randomized controlled trial. Journal of
Contextual Behavioral Science, 3(4), 248–257. https://doi.org/10.1016/j.
jcbs.2014.09.002
G. Landi et al.